Prologue

You can skip this section if you’d like.

For the last few days, I’ve been stuck inside of my room due to some kind of cold or flu, which or [note: The results of the PCR test are pending at the time of writing. ] In seeming correspondence with the progression of my cold, a thought occurred in the back of my mind: “Your blog deployment is kind of a mess”. On the first day, when I felt only a small tingling in my throat, I waved that thought away pretty easily. On the second day, feeling unwell and staying in bed, I couldn’t help but start to look up Nix documentation. And finally, on the third day, between coughing fits and overconsumption of oral analgesic, I got to work.

In short, this post is the closest thing I’ve written to a fever dream.

The Constraints

I run several versions of this site. The first is, of course, the “production” version, hosted at the time of writing on danilafe.com and containing the articles that I would like to share with the world. The second is a version of this site on which drafts are displayed - this way, I can share posts with my friends before they are published, get feedback, and even just re-read what I wrote from any device that has an internet connection. The third is the Russian version of my blog. It’s rather empty, because translation is hard work, so it only exists so far as another “draft” website.

Currently, only my main site is behind HTTPS. However, I would like for it to be possible to adjust this, and possibly even switch my hosts without changing any of the code that actually builds my blog.

I wanted to be able to represent all of this complexity in my NixOS configuration file, and that’s what this post is about!

Why Flakes

I decided to use Nix flakes to manage my configuration. But what is it that made me do so? Well, two things:

The Final Result

Here’s the relevant section of my configuration:

From configuration.nix, lines 42 through 59
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
  services.danilafe-blog = {
    enable = true;
    challengePath = "/var/www/challenges";
    sites = [
      (builders.english {
        ssl = true;
        host = "danilafe.com";
      })
      (builders.english {
        drafts = true;
        host = "drafts.danilafe.com";
      })
      (builders.russian {
        drafts = true;
        host = "drafts.ru.danilafe.com";
      })
    ];
  };

I really like how this turned out for three reasons. First, it’s very clear from the configuration what I want from my server: three virtual hosts, one with HTTPS, one with drafts, and one with drafts and in Russian. Second, there’s plenty of code reuse. I’m using two builder functions, english and russian, but under the hood, the exact same code is being used to run Hugo and perform all the necessary post-processing. Finally, all of this can be used pretty much immediately given my blog flake, which reduces the amount of glue code I have to write.

Getting There

A Derivation Builder

As I mentioned earlier, I need to generate multiple versions of my blog. All of these use pretty much the same build process – run Hugo on the Markdown files, then do some post-processing (in particular, convert the LaTeX in the resulting pages into MathML and nice-looking HTML). I didn’t want to write this logic multiple times, so I settled for a function that takes some settings, and returns a derivation:

From lib.nix, lines 6 through 21
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
  website = settings: stdenv.mkDerivation {
    inherit (settings) src ssl host;
    name = "blog-static";
    version = settings.src.rev;
    urlSub =
      let
        regexEscape = lib.escape [ "/" "(" ")" "[" "]" "+" "*" "\\" ];
      in
        with settings.replaceUrl; "s/${regexEscape from}/${regexEscape to}/g";
    publicPath = settings.path;
    extraFlags = (if settings.drafts then " -D " else "") + settings.extraFlags;
    builder = ./build/builder.sh;
    buildInputs = [
      hugo katex-html
    ];
  };

There are a few things here:

This new website function is general enough to represent all my blog versions, but it’s too low-level. Do I really want to specify the publicPath each time I want to describe a version of the site? What about settings.replaceUrl, or the source code? Just like I would in any garden variety language, I defined two helper functions:

From lib.nix, lines 25 through 48
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
    english = settings: website {
      inherit (settings) host;
      ssl = settings.ssl or false;
      drafts = settings.drafts or false;
      src = blog-source;
      path = ".";
      extraFlags = "--config=config.toml,config-gen.toml";
      replaceUrl = {
        from = "https://danilafe.com";
        to = wrapHost (settings.ssl or false) settings.host;
      };
    };
    russian = settings: website {
      inherit (settings) host;
      ssl = settings.ssl or false;
      drafts = settings.drafts or false;
      src = blog-source-localized;
      path = "ru";
      extraFlags = "";
      replaceUrl = {
        from = "https://ru.danilafe.com";
        to = wrapHost (settings.ssl or false) settings.host;
      };
    };

Both of these simply make a call to the website function (and thus return derivations), but they make some decisions for the caller, and provide a nicer interface by allowing attributes to be omitted. Specifically, by default, a site version is assumed to be HTTP-only, and to contain non-draft articles. Furthermore, since each function corresponds to a language, there’s no need for the caller to provide a blog version, and thus also the output path, or even to specify the “from” part of replaceUrl. The wrapHost function, not included in the snippet, simply adds http or https to the host parameter, which does not otherwise include this information. These functions can now be called to describe different versions of my site:

# Default version, hosted on the main site and using HTTPS
english {
    ssl = true;
    host = "danilafe.com";
}

# English draft version, hosted on draft domain and not using HTTPS.
english {
    drafts = true;
    host = "drafts.danilafe.com";
}

# Russian draft version, hosted on draft (russian) domain, and not using HTTPS.
russian {
    drafts = true;
    host = "drafts.ru.danilafe.com";
}

Configuring Nginx

The above functions are already a pretty big win (in my opinion) when it comes to describing my blog. However, by themselves, they aren’t quite enough to clean up my system configuration: for each of these blog versions, I’d need to add an Nginx virtualHosts entry where I’d pass in the corresponding host (like danilafe.com or drafts.danilafe.com), configure SSL, and so on. At one point, too, all paths in /var were by default mounted as read-only by NixOS, which meant that it was necessary to tell systemd that /var/www/challenges should be writeable so that the SSL certificate for the site could be properly renewed. Overall, this was a lot of detail that I didn’t want front-and-center in my server configuration.

However, with the additional “ghost” attributes, my derivations already contain most of the information required to configure Nginx. The virtual host, for instance, is the same as replaceUrl.to (since I’d want the Nginx virtual host for a blog version to handle links within that version). The ssl ghost parameter corresponds precisely to whether or not a virtual host will need SSL (and thus ACME, and thus the systemd setting). For each derivation built using website, I can access the attributes like ssl or host to generate the corresponding piece of the Nginx configuration.

To make this really nice, I wanted all of this to be “just another section of my configuration file”. That is, I wanted to control my site deployment via regular old attributes in configuration.nix. To this end, I needed a module. Xe recently wrote about NixOS modules in flakes, and what I do here is very similar. In essence, a module has two bits:

In short, a module describes the sort of options it will accept, and then provides a way to convert these newly-described options into changes to the system configuration. It may help if I showed you the concrete options that my newly-created blog module provides:

From module.nix, lines 32 through 43
32
33
34
35
36
37
38
39
40
41
42
43
    options.services.danilafe-blog = {
      enable = mkEnableOption "Daniel's blog service";
      sites = mkOption {
        type = types.listOf types.package;
        default = {};
        description = "List of versions of this blog that should be enabled.";
      };
      challengePath = mkOption {
        type = types.str;
        description = "The location for ACME challenges.";
      };
    };

There are three options here:

Now, while these are the only three options the user will need to set, the changes to the system configuration are quite involved. For instance, for each site (derivation) in the sites list, the resulting configuration needs to have a virtualHost in the services.nginx namespace. To this end, I defined a function that accepts a site derivation and produces the necessary settings:

From module.nix, lines 7 through 19
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
  virtualHost = package:
    {
      virtualHosts."${package.host}" = mkMerge [
        {
          root = package;
        }
        (mkIf (sslForSite package) {
          addSSL = true;
          enableACME = true;
          acmeRoot = cfg.challengePath;
        })
      ];
    };

Each virtual host always has a root option (where Nginx should look for HTML files), but only those sites for which SSL is enabled need to specify addSSL, enableACME, and acmeRoot. All the virtual hosts are assembled into a single array (below, cfg refers to the options that the user provided to the module, as specified above).

From module.nix, line 28
28
  virtualHosts = map virtualHost cfg.sites;

If the enable option is set, we enable Nginx, and provide it with a list of all of the virtual hosts we generated. Below, config (not to be confused with cfg) is the namespace for the module’s configuration.

From module.nix, lines 45 through 51
45
46
47
48
49
50
51
    config.services.nginx = mkIf cfg.enable (mkMerge (virtualHosts ++ [
      {
        # Always enable nginx.
        enable = true;
        recommendedGzipSettings = true;
      }
    ]));

In a similar manner to this, I generate a list of systemd services which are used to configure the challenge path to be writeable. Click the module.nix link above to check out the full file.

Creating a Flake

We now have two “things” that handle the deployment of the blog: the builder functions english and russian which help describe various blog versions, and the NixOS module that configures the server’s Nginx to serve said versions. We now want to expose these to the NixOS system configuration, which describes the entire server. This is where flakes finally come in. Yanik Sander wrote up a pretty comprehensive explanation of how their blog is deployed using flakes, which I often consulted while getting started – check it out if you are looking for more details.

In brief, a Nix flake has inputs and outputs. Inputs can be other flakes or source files that the flake needs access to, and outputs are simply Nix expressions that the flake provides.

The nice thing about flakes’ inputs is that they can reference other flakes via Git. This means that, should I write a flake for my blog (as I am about to do) I will be able to reference its git URL in another flake, and Nix will automatically clone and import it. This helps achieve the adding custom packages goal, since I can now easily write Nix expressions and reference them from my system configuration.

Importantly, flakes track the versions of their inputs in a flake.lock file; this means that, unless explicitly told to do otherwise, they will use the same version of their inputs. This achieves the versioning goal for my blog, too, since now it will pull the pre-defined commit from Git until I tell it to fetch the updated site. In addition to pinning the version of my blog, though, the flake also locks down the version of nixpkgs itself. This means that the same packages will be used in the build process, instead of those found on the host system at the time. This has the nice effect of preventing updates to dependencies from breaking the build; it’s a nice step towards purity and reproducibility.

Let’s take a look at the inputs of my blog flake:

From flake.nix, lines 2 through 19
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs";
    flake-utils.url = "github:numtide/flake-utils";
    katex-html.url = "git+https://dev.danilafe.com/Nix-Configs/katex-html";
    blog-source = {
      flake = false;
      url = "https://dev.danilafe.com/Web-Projects/blog-static.git";
      type = "git";
      submodules = true;
    };
    blog-source-localized = {
      flake = false;
      url = "https://dev.danilafe.com/Web-Projects/blog-static.git";
      ref = "localization";
      type = "git";
      submodules = true;
    };
  };

Two of these inputs are my blog source code, pulled from its usual Git host. They are marked as flake = false (my blog is just a Hugo project!), and both require submodules to be fetched. One of them is set to the localization branch, once again because localization is not yet stabilized and thus not merged into my blog’s master branch. The other three inputs are flakes, one of which is just nixpkgs. The flake-utils flake provides some convenient functions for writing other flakes, and katex-html is my own creation, a KaTeX-to-HTML conversion script that I use to post-process the blog.

So what outputs should this flake provide? Well, we’ve already defined a NixOS module for the blog, and we’d like our flake to expose this module to the world. But the module alone is not enough; its configuration requires a list of packages created using our builders. Where does one procure such a list? The caller will need access to the builders themselves. To make all of this work, I ended up with the following expression for my outputs:

From flake.nix, lines 21 through 34
21
22
23
24
25
26
27
28
29
30
31
32
33
34
  outputs = { self, blog-source, blog-source-localized, nixpkgs, flake-utils, katex-html }:
    let
      buildersFor = system: import ./lib.nix {
        inherit blog-source blog-source-localized;
        pkgs = import nixpkgs { inherit system; };
	katex-html = katex-html.defaultPackage.${system};
      };
    in
      {
	inherit buildersFor;
        nixosModule = (import ./module.nix);
      } // flake-utils.lib.eachDefaultSystem (system: {
          defaultPackage = (buildersFor system).english { host = "danilafe.com"; };
      });

The flake output schema provides a standard option for exposing modules, nixosModule. Then, exposing my module.nix file from the flake is simply a matter of importing it, as on line 31. There is, however, no standard way for exposing a function. The good news is that any attribute defined on a flake is accessible from code that imports that flake. Thus, I simply added a buildersFor function, which fetches the nixpkgs collection and LaTeX builder script for a given system, and feeds them to the file that defines the english and russian builders. This buildersFor function also provides the builders with the two different blog sources they reference, blog-source and blog-source-localized.

The system parameter to buildersFor is necessary because the set of packages from nixpkgs depends on it. Thus, if the builders use any packages from the collection (they do), they must know which system to pull packages for. This is a common pattern in flakes: the packages attribute is typically a system-to-package mapping, too.

Finally, the last little bit on lines 32 through 34 defines a default package for the flake. This is the package that is built if a user runs nix build .#. This isn’t strictly necessary for my purposes, but it’s nice to be able to test that the builders still work by running a test build. The eachDefaultSystem function generates a defaultPackage attribute for each of the “default” systems, so that the package is buildable on more than just my server architecture.

And that’s it for the blog flake! I simply push it to Git, and move on to actually using it from elsewhere.

Using the Module

In my server configuration (which is, itself, a flake), I simply list my blog-static-flake as one of the inputs:

From flake.nix, line 4
4
    blog.url = "git+https://dev.danilafe.com/DanilaFe/blog-static-flake";

Then, in the modules attribute, I include blog.nixosModule, making NixOS aware of its options and configuration. The final little piece is to provide the english and russian builders to the system configuration; this can be done using the specialArgs attribute. The whole flake.nix file is pretty short:

From flake.nix, entire file
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
{
  inputs = {
    nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
    blog.url = "git+https://dev.danilafe.com/DanilaFe/blog-static-flake";
  };
  outputs = { self, nixpkgs, blog }:
    let
      system = "x86_64-linux";
      builders = blog.buildersFor system;
    in  
      {
        nixosConfigurations.nixos-droplet-v2 = nixpkgs.lib.nixosSystem {
	  inherit system;
          specialArgs = { inherit system builders; };
          modules = [ ./configuration.nix blog.nixosModule ];
        };
      };
}

Finally, in configuration.nix, taking builders as one of the inputs, I write what you saw above:

From configuration.nix, lines 42 through 59
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
  services.danilafe-blog = {
    enable = true;
    challengePath = "/var/www/challenges";
    sites = [
      (builders.english {
        ssl = true;
        host = "danilafe.com";
      })
      (builders.english {
        drafts = true;
        host = "drafts.danilafe.com";
      })
      (builders.russian {
        drafts = true;
        host = "drafts.ru.danilafe.com";
      })
    ];
  };

Wrapping Up

So there you have it, a flake-based multi-version blog deployment written in a declarative style. You can check out both my system configuration flake and my blog flake on my Git server. If you want more, check out the articles by Xe and Yannik linked above. Thanks for reading!