Nix: The wounded siren
A few months ago I switched from Fedora to NixOS as my daily driver for my laptop. So far I’ve liked it enough I recently replaced Ubuntu on my home server with NixOS as well. It called to me like a siren, and I have been captivated by that call.
Yet, the call is not perfect. It’s remarkably imperfect. So much so that unlike many “hype” technologies, this one struggles to be recommended by those who it ensnares - me included.
This realization didn’t dawn on me until after a few different conversations I had recently. For the first, I was talking to someone who was switching from Windows to Linux. They asked what I used, and I replied that I use NixOS - but that I don’t recommend it. I instead recommended Fedora or Mint or one of the other distros.
Later, I talked to someone who was an experienced Linux user about which distros we were both using. Again, I said I use NixOS - but that I don’t recommend it.
Which posses the question - if I love NixOS so much I’m switching my daily driver and my servers to it, why am I not recommending it?
The Software “Problem”
Nix is built around a solution to a problem that you may - or may not - have. The entire philosophy around how everything works is built around this solution. And that brings a lot of headaches. And that problem is how software is installed and managed on your machine.
This problem ranges from simply installing different versions of the same software, to installing software with many shared - and sometimes conflicting - libraries. There are lots of solutions to this problem - package managers, homebrew, installers, flatpaks, etc. However, Nix takes the problem deeper than simply “installing” software or “isolating” software.
Nix strives to make software installation reproducible, declarative, and reliable.
In other words, you should be able to say “I want this software with these settings” and it should just happen. You can even say “for this project I want this version of this software with these settings, and for this other project I want a different version and different settings” and Nix will do it.
This is a very specific problem, and one many people have not had. Often because people usually don’t care about the version or settings - so long as the software works.
However, I do care about the version and settings.
Legacy Code, Interpreters, and Compilers
One of the big things I’ve done through my career - at least so far - is work on legacy systems. These are systems with not just old code, but old versions of stuff. Things like outdated build systems, language versions, interpreters, run times, native extensions, etc.
Often, the new version has some series of major breaking changes - or it simply doesn’t exist. The language extension the whole system was built around just simply stopped being maintained 6 years ago, and now the new language runtime made some internal change so now the extension won’t work.
When this happens, a rewrite or refactor or new system is usually started with the latest software - but it too quickly becomes legacy. Rinse and repeat a few times, and soon there’s a mess of legacy systems with very different version dependencies and incompatibilities all somehow working together to get a job done.
And this is the type of situation that I don’t just work in, but I thrive in.
These old systems are a canvas of opportunities. They also tend to be what makes the business the most money - way more than that shiny new microservice someone just built. That means these services are important to care for, extend, maintain, grow, and develop.
But, we very quickly run into a problem. Getting just one of these systems running in on a development machine is difficult. Getting two is excruciating. Three is almost impossible. Very quickly package systems start fighting.
I’ve had homebrew completely break my PHP runtime because it updated my NodeJS install which then bumped some shared library and deleted the old version. I’ve had NVM install x86 libraries on an M2 mac which then prevented Homebrew from installing an Arm version of Ruby since it tried to link to those x86 libraries. I’ve had Node 18 be unable to build a Node 14 project, and Node 20 be unable to run npm install for a Node 18 project. I’ve seen apt-get delete old versions of a compiler, homebrew installs get unlinked by the OS, nvm lose it’s mind, asdf get removed from the path by homebrew, snap just completely stop working, flatpak breaking auto updates, dotnet CLI completely brick itself trying to update, and so much more. Not to mention managing database versions, port conflicts, VPN, XCode developer tools, MSVC compiler toolchain, Windows vs Linux vs OSX incompatibilities, etc.
And then there’s the whole “breaking changes” in programming languages. I’ve gone through the JVM 8 to 9 to 11 trek, the PHP 5 to 7 to 8 landslide, and the Python 2 to 3 transition. And then there’s TypeScript which broke so many things when it introduced the “unknown” type.
And things just continue for my side projects. I love learning what’s new in languages. But when some languages have multiple implementations (e.g. C++), this quickly becomes an issue. Especially when I ever try sharing my code with someone and they have a different compiler version which has different syntax rules.
All this to say, I very much do care about what versions and settings I have. Having an easy way to manage a version is the difference between me having a productive day or spending a week rebuilding my developer machine.
And I’ve tried a lot of options. Vagrant, Docker, cloud workspaces, separate machines for different configurations, etc. None of them seemed to stick.
The Nix “Solution”
And then came along Nix. Promising to fix all of my problems.
Well, it did do a really good job addressing that one problem - and then gave me quite a few (more on that in a bit). But it really did a good job addressing my main problem - installing things intelligently and in an isolated way without breaking other things.
How it does it is similar to Docker or Vagrant or cloud workspaces at first. There’s a file with some custom DSL that describes how to build a machine, and then some program uses that to build the machine. The difference though, is that this isn’t running in some container or virtualization layer (which always brings headaches when trying to interface with said layer). Instead, it’s running natively on your machine. So much so that in my case, it is my machine.
What’s even better is it can archive a configuration so that you can rollback to a previous version. This is awesome when I’m trying to “improve” my Linux distribution. If I totally screw something up (like install a new login window and forget to disable the old one) then I can revert to the previous install that wasn’t broken.
This ability to “revert” goes into user-space as well. We can create a temporary “machine” with a nix shell, and that machine can now operate with different versions of compilers, new programs, or even remove programs we have installed “globally.” Then when we’re done, we can exit the shell thereby “reverting” to our machine’s global configuration.
And, since Nix is configuration driven, we can write config files describing those temporary machines and add them to our source control. This allows us to then share those temporary machines with other nix users so they can then run those machines locally. And again, with NixOS it’s not a VM - it’s their actual machine. With full access to their file system, with full access to the GUI, hardware, etc. No SSH, no remote debugging, no slowness from a VM hogging all the memory/CPU. It’s just a temporary “update” for their machine.
And, all of these shells are isolated. So you could have two, or three, or four “versions” of your machine running simultaneously and reading from the same data on your system.
We can then take it a step further and build VMs with nix for things we couldn’t do natively on our machine - like testing x86, ARM, RISC-V, PowerPC, and MIPS versions of our code all at once - something I recently did for one of my projects.
This is where I fell in love with Nix. It gave me something that solves the problem I have, and which wastes so much of my time.
Of course, no lunch comes for free, and Nix has a lot of drawbacks.
Why I don’t recommend Nix
Again, most people don’t usually have the problem I have. They just want to install a default option to have things work by default. They don’t care about comparing options, or having the ability to switch versions, or anything like that. They’re usually not installing a lot of things anyway. So, does Nix serve these types of people?
No, not really.
One of Nix’s biggest drawbacks is that everything is driven by a global config file. Your installed programs, user list, firewall, hosts file, timezone, printer discovery settings, etc. are all determined by /etc/nix/configuration.nix. Changing this file requires root access. Once the file is changed, it needs to be reloaded with a special command (also requiring root access). And, this file is in a unique “nix” configuration language. It’s not YAML, or JSON, or Lua, or TOML, or C, or JavaScript, or Bash, anything but Nix. This makes it a steep learning curve to just install a new program.
Not only that, but not everything works out of the box. It turns out, every time I installed NixOS, Bluetooth was disabled by default! And when I installed it on my laptop, I didn’t have proper WiFi drivers since I had to download and install a special hardware configuration for my laptop first (luckily I had an Ethernet adapter which worked, but this was terrifying for a bit!).
And then there’s a “fun” little quirk where you can’t just run executables built for generic Linux. I tried doing that, and I got a little pop-up saying that NixOs can’t open executables made for generic linux. So, things need to be specially packaged. This makes getting printer drivers installed a real pain (way more painful than other Linux distros).
Plus, there’s just some general annoyances that get in the way. Nix isn’t necessarily an immutable distro, but it does make a lot of standard configuration files immutable (like your bash/zsh configuration file, hosts file, etc).
Also, nix documentation is horrendously fragmented, and the parts you do find often comes across as some random person’s notes they wrote on the back of a napkin before wiping ketchup all over it and throwing it in the trash. Like, there’s definitely signs of a “eureka” moment someone had, but there’s no context at all as to where it should go in the configuration file, or what dependencies it needs, or really what anything means, and that page is really old and most likely will never be updated.
And then there’s the little things that are just broken in my day-to-day life, but I’ve become so used to them and developed so many workarounds by now that I’d only ever know about them if someone sat down watching me use my computer and then immediately point out everything that doesn’t work well. I know that stuff is there, but I love the fact that the many versions of each random compiler or interpreter I use works way more reliably that I just don’t notice the stuff that’s broken anymore.
These pain points, the wounds of nix, limit the reach of it’s call. The tune is filled with both promise and despair. A beautiful melody is maimed by an ugly harmony. At some point, perhaps the siren will no longer be encumbered by it’s short comings. And then, at that day, perchance those who it’s enraptured will be able to joyfully sing it’s praise. But until then, it’s tune only charms those who can focus on only the benefits and become blind to the defects.
P.S. For those who do want to try out nix, I don’t recommend starting with nix as a package manager/shell on another distro (e.g. debian). I know it’s an option, but it really sucks since at that point it’s just another opinionated package manager conflicting with everything else you’re doing. It was an awful experience that made me almost give up on nix as just another gimmicky package manager. A lot of the shortcomings of the standalone nix shell are addressed in NixOS.