In my previous blog post, I talked about some of the shortcomings of Nix as a project and what I wish to see exist in its stead. Since I published the blog post, there have been some promising developments: a constitutional assembly has formed, a fork of the Nix interpreter has arisen, and an alternative Nixpkgs ecosystem is beginning to form. I am enheartened that Nix has a potential future, but it is too early to tell how these new efforts will play out.

In the meantime, I prototyped an idea I had kicking around while I wrote the blog post. zb is an experiment in using Lua as an build configuration language while still using the model described in The Purely Functional Software Deployment Model. I am quite pleased with the results. In my view, this is an existence proof for using a more conventional programming language to reduce the onboarding cost of a technology like Nix.

The README has a more in-depth description of what I built, but in short:

• zb uses a fairly stock Lua 5.4 interpreter, with the notable exception that it includes dependency information in strings, much like the Nix language. Lua’s simple interpreter made it straightforward to add this feature.
• zb produces .drv files and uses nix-store --realise to build them. This means that zb is fully interoperable with Nix derivations (e.g. using nixpkgs derivations in zb derivations), but also proves that an alternative build backend could be created in the future.
• zb uses content-addressed builds to enable the shallow build optimization described in Build systems à la carte.
• I built a source bootstrap that is mostly equivalent to the nixpkgs standard environment by following the live-bootstrap steps. This includes a working GCC, GNU Make, Coreutils, and Automake, among many other common programs. I discovered in this process that this would be the largest time and effort investment. Even after a month, I still haven’t gotten a build that gets to the end of the steps.

I want to share this research prototype in the interest of showing others what is possible in the space, but I’m not planning on developing it to production-readiness. If you have feedback on zb, let me know on the Auxolotl forum, on the zb repository discussions, or privately.

(This is a long post. In the words of Blaise Pascal, “I have made this longer than usual because I have not had time to make it shorter.”)

I see the Nix project to have passed a critical moment. I think the Nix project has made great strides in solving some very hard problems, but my fear is that due to mismanagement, Nix will not succeed. In this post, I hope to share my perspective on what a better future for declarative, reproducible builds could look like, but sadly, I don’t think it involves Nix as it exists today. It’s hard to know what the future holds, and I am certain I do not hold the solutions to the problems that exist with Nix. However, I feel it would be a shame for the progress to be lost.

As background, on 2024-04-21, a group of concerned contributors and community members published an open letter to the NixOS foundation detailing Eelco Dolstra’s ill effects on the Nix project’s governance. On 2024-04-26, Eelco published a blog post that has caused many package maintainers to leave, including me. While I don’t have direct experience with the problems in the letter, I very much trust the folks who signed the letter. I am disappointed with how Eelco has chosen to handle the situation. I personally think this is a watershed moment for Nix, and not for the better. Such actions set the precedent for how an organization will act in the future, and given the power structure, there’s not much that can be done. In my mind, this endangers the health of Nixpkgs moving forward, and without healthy package maintenance, the Nix project is doomed. Consider this post a eulogy for a dream.

While the solution-that-is-Nix may evaporate, the problem it solves remains, at least for me. The rest of this blog post is my attempt at sketching what a suitable replacement would look like, in the hopes that it inspires someone to succeed where others have not. Perhaps a replacement is a fork, perhaps it is a wholly new project, perhaps it is the Nix project under better governance. I don’t know.

I write a lot of shell scripts. Many are one-offs or specific to a project, but every so often, I’ll have a script that transcends to become a part of my toolbelt. For example, nix-op-key is a script I wrote to generate new Nix signing keys and place them in 1Password. It’s not a task that requires a dedicated program, it just needs to glue two existing programs together: nix key generate-secret and op (the 1Password CLI). These sorts of scripts are great, but if you want to share them with someone else (or even just use it on a different computer), how do you do it? Scripts like these depend on specific programs (or maybe even specific versions) being installed and Bash does not have a package manager like pip or the go tool.

As it turns out, Nix is such a package manager. And with flakes, there’s built-in support for installing and running scripts with well-specified dependencies in a single command. For example, you can run my nix-op-key script I mentioned earlier (pinned to a specific version) with:

COMMIT=25e9bd52e977cca415df84ea91028efd92d3da92
nix run "github:zombiezen/dotfiles?dir=nix&rev=$COMMIT#nix-op-key" -- --help

Furthermore, you can install the script using the nix profile install command:

nix profile install "github:zombiezen/dotfiles?dir=nix&rev=$COMMIT#nix-op-key"

(If you try this out yourself, you can uninstall the script with nix profile remove '.*.nix-op-key').

In this blog post, I’ll show you how you can package your own shell scripts with Nix to make them more reliable and easier to share. This article assumes familiarity with Unix command line and Bash shell scripting. I’m using Nix 2.17.1. All of the source code in this post is released under the Unlicense and is available on GitHub.

I’ve been writing about Nix for a few years now, and it has become a staple for all my personal projects. It solves the thorny problem of setting up reproducible development environments in a flexible and general way, while still allowing me to use my tooling the way I want to. I no longer have to worry about my GitHub Actions having different versions of tools from my local dev environment, which is a huge relief.

I’ve started making my CI jobs use nix flake check, which lets me write and run my CI checks using Nix syntax, further removing any differences between local development and CI. However, I sometimes have tests that want to use the network, which usually isn’t allowed. I recently discovered that you can use __impure = true; to remove the networking sandbox:¹

# flake.nix
{
  inputs = {
    nixpkgs.url = "nixpkgs";
  };

  outputs = { nixpkgs, ... }: {
    checks.x86_64-linux.foo =
      let
        pkgs = import nixpkgs { system = "x86_64-linux"; };
      in pkgs.stdenvNoCC.mkDerivation {
        name = "foo-check";
        src = ./.;
        __impure = true;
        nativeBuildInputs = [
          pkgs.cacert
          pkgs.curl
        ];
        buildPhase = ''
          runHook preBuild
          curl -fsSL https://example.com/ > /dev/null
          runHook postBuild
        '';
        installPhase = ''
          runHook preInstall
          touch "$out"
          runHook postInstall
        '';
      };
  };
}

For this to work, you must also set extra-experimental-features = impure-derivations ca-derivations in /etc/nix/nix.conf.² Just passing a command-line argument doesn’t seem to work in my usage with Nix 2.13.3 in a multi-user installation.

Since this feature is still experimental, it’s not widely advertised or documented. Hopefully this helps you use it for your own tests.

Julia Evans wrote a toot asking:

are there any guides to nix that start from the bottom up (for example starting with this bash script https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh) and then working up the layers of abstraction) instead of from the top down?

I realized that despite the title, my blog post Nix From the Ground Up misses the mark on providing this type of explanation. While I do think the Nix language is the lowest abstraction layer to learn Nix, I wanted to zoom in on the core derivation abstraction through a tutorial. This way, we can better understand how Nix derivations relate to Bash scripts.

I recently spent some time learning Nix after watching this talk by Xe. Nix is a package manager/build system for Linux and macOS. It does a number of things I really like:

• Transparent handling of source and binary packages.
• Includes a rich central package registry, but you can host your package descriptions or binaries anywhere.
• Does not require root and runs alongside any Linux distribution.
• Easy to pin or customize versions of individual packages.
• Straightforward support for project-specific dependencies.

Nix is a cool piece of tech, but in my opinion, it’s pretty hard to learn (at least at time of writing). I think this is accidental complexity: I was able to be productive with Nix in my personal projects in a few days, but it took a fair amount of research from many different sources. I took a lot of notes, then realized I wanted to publish them to share this knowledge.

So here’s my guide! “Nix From the Ground Up” aims to help explain the concepts behind Nix with a hands-on approach.