Nix Setup and Usage with Flakes

Org-mode setup

This document is written in a project-agnostic way to be copied to other projects that use Nix.

Variables

We set variables in internal/params.el and access those settings with the following macros and source code blocks (using Noweb).

(alist-get (intern arg) (car (read-from-string (f-read "internal/params.el"))))

Formatting help

This snippet can be used as a post-processing step to crop down the results from an evaluation of a source code block.

(let* ((ls         (split-string text "\n"))
       (first-ls   (-take first-n ls))
       (rest-first (-drop first-n ls))
       (rest-last  (-drop-last (+ 1 last-n) rest-first))
       (last-ls    (-take-last (+ 1 last-n) rest-first)))
  (string-join
   (if rest-last
       (append first-ls '("…") last-ls)
     (append first-ls last-ls))
   "\n"))

Setup action

Next, we perform some side effects to set up the evaluation of the whole document.

rm --force result*
rm --force /tmp/nix-profile*

About this document

This document explains how to take advantage of software provided by Nix for people new to the Nix package manager. This guide uses this project for examples but focuses on introducing general Nix usage, which also applies to other projects using Nix.

How this project uses Nix

This project uses Nix to download all necessary dependencies and build everything from source. In this regard, Nix is helpful as not just a package manager but also a build tool. Nix helps us get from raw source files to built executables in a package we can install with Nix.

Within this project, the various files with a .nix extension are Nix files, each of which contains an expression written in the Nix expression language used by the Nix package manager to specify packages. If you get proficient with this language, you can use these expressions as a starting point to compose your own packages beyond what’s provided in this project.

Prerequisites

If you’re new to Nix consider reading the provided introduction.

This project supports {{{platforms}}}.

That may affect your ability to follow along with examples.

Otherwise, see the provided Nix installation and configuration guide if you have not yet set Nix up.

To continue following this usage guide, you will need Nix’s experimental flakes feature. You can enable this globally or use an alias such as the following:

alias nix-flakes = nix --extra-experimental-features 'nix-command flakes'

Working with Nix

Though comprehensively covering Nix is beyond the scope of this document, we’ll go over a few commands illustrating some usage of Nix with this project.

Referencing flake projects

Most of this document illustrates usage of the nix command, which provides a number of subcommands and centralizes Nix usage.

Many of the nix subcommands accept references to flake-enabled projects. A flake is specified with a Nix expression saved in a file named flake.nix. This file should be at the root of a project. We can reference both local and remote flake projects.

Here are some common forms we can use to reference flake projects:

Syntax	Location
`.`	flake in the current directory
`<path>`	flake in some other file path (must have a slash)
`<registry>`	reference to flake from the registry (see `nix registry list`)
`git+<url>`	latest flake in the default branch of a Git repository
`git+<url>?ref=<branch>`	latest flake in a branch of a Git repository
`git+<url>?rev=<commit>`	flake in a specific commit of a Git repository
`github:<owner>/<repo>`	latest flake in the default branch of a GitHub repository
`github:<owner>/<repo>/<branch>`	latest flake in a branch of a GitHub repository
`github:<owner>/<repo>/<commit>`	flake in a specific commit of a GitHub repository

This table introduces an angle-bracket notation for syntactic forms with components that change with context. This notation is used throughout this document.

Referencing local flake projects is easy enough with file paths. But the URL-like notation for remote flake projects can get verbose. Furthermore, some of these references are not fixed. For example, Git branches point to different commits over time.

To manage flake references, Nix provides a flakes registry. Upon installation this registry is prepopulated with some global entries:

nix registry list

…
global flake:nixos-homepage github:NixOS/nixos-homepage
global flake:nixos-search github:NixOS/nixos-search
global flake:nixpkgs github:NixOS/nixpkgs/nixpkgs-unstable
global flake:nur github:nix-community/NUR
global flake:patchelf github:NixOS/patchelf
global flake:poetry2nix github:nix-community/poetry2nix
global flake:pridefetch github:SpyHoodle/pridefetch
global flake:sops-nix github:Mic92/sops-nix
global flake:systems github:nix-systems/default
global flake:templates github:NixOS/templates

For example, rather than referencing the flake on the nixpkgs-unstable branch of the Nixpkgs GitHub repository with github:NixOS/nixpkgs/nixpkgs-unstable, we can use the simpler identifier nixpkgs.

If we want to point to a different branch but still use an identifier from the registry, we can do so by extending it with the branch. For example, the flakes identifier nixpkgs is the same as nixpkgs/nixpkgs-ustable, but we can also use {{{get(nixos-latest,~nixpkgs/nixos-,~)}}} to override the branch and point to the NixOS {{{nixos-latest}}} release branch.

Note that registries have mutable references, but Nix knows how to rebuild the snapshot referenced for some of these references deterministically. For example, when referencing a GitHub repository via a registry reference, Nix will take note of the commit ID of the snapshot retrieved. Nix typically stores this information required for reproducibility in a lock file called flake.lock adjacent to flake.nix.

Inspecting flake outputs

A flake can provide a variety of outputs that can be used in different contexts. A few of these outputs include the packages we can build and install.

We can use nix flake show to see the outputs provided by any flake, local or remote, by providing a flake reference discussed in the previous section. Here’s an example of inspecting the flake of this project locally:

nix flake show .

git+file:///home/shajra/src/nix-project
├───apps
│   ├───aarch64-darwin
…
├───flakeModules: unknown
├───legacyPackages
│   ├───aarch64-darwin omitted (use '--legacy' to show)
│   ├───x86_64-darwin omitted (use '--legacy' to show)
│   └───x86_64-linux omitted (use '--legacy' to show)
├───lib: unknown
├───overlays
│   └───default: Nixpkgs overlay
├───packages
│   ├───aarch64-darwin
│   │   ├───nix-scaffold omitted (use '--all-systems' to show)
│   │   └───org2gfm omitted (use '--all-systems' to show)
│   ├───x86_64-darwin
│   │   ├───nix-scaffold omitted (use '--all-systems' to show)
│   │   └───org2gfm omitted (use '--all-systems' to show)
│   └───x86_64-linux
│       ├───nix-scaffold: package 'nix-scaffold'
│       └───org2gfm: package 'org2gfm'
└───templates
    └───default: template: A starter project using shajra/nix-project.

Flake outputs are organized in a tree of attributes. References to paths of attributes are dot-delimited. There is a standard schema for the output attribute tree of a flake. Nix permits outputs outside this schema.

This document focuses on packages provided by the packages output attribute. Notice that a flake offers packages for different (but rarely all) system architectures.

For commands like nix flake show that expect a flake reference as an argument, . is assumed as default if an argument isn’t provided. So nix flake show is equivalent to nix flake show ..

Referencing flake outputs

Many of the nix subcommands work with references to flakes outputs. These references are called installables. There are many types of installables (hence the general name). In this document, we’ll hone in on the following forms:

<flake> to select a default output from
<flake>#<output reference> for a reference to packages(s) within a flake.

In this notation, <flake> is the same reference to a local or remote flake project discussed in the prior section.

When the installable is just a flake reference, the called nix subcommand will generally look for the packages.<system>.default attribute path within the flake. So <flake> will generally be the same as <flake>#packages.<system>.default. One exception to this rule, nix run will first look for <flake>#apps.<system>.default, and then for <flake>#packages.<system>.default.

Installables can also reference an output of a flake (<output reference> above) directly in a couple of ways:

Output reference	Example installable
`<output attribute>.<system>.<name>`	{{{get(package-attr-long,~.#,~)}}}
`<name>`	{{{get(package-attr-short,~.#,~)}}}

The first way is the most explicit, by providing the full attribute path we can see with nix flake show. But this requires specifying the package’s system architecture.

With the second form, with the shorter <name> package reference, Nix will detect the system we’re on and also search some attributes in a precedence order for the provided name:

apps.<system>.<name> (nix run only)
packages.<system>.<name> (all subcommands accepting installables)
legacyPackages.<system>.<name> (all subcommands accepting installables)

Which attributes are searched depends on the nix subcommand called.

For commands accepting installables as an argument, if none are provided, then . is assumed. Nix will attempt to read a flake.nix file in the current directory. If not found, Nix will recursively search parent directories to find a flake.nix file.

Searching flakes for packages

We can use the nix search command to see what package derivations a flake contains. For example, from the root directory of this project, we can execute:

nix search . ^

* packages.x86_64-linux.nix-scaffold
  Script to scaffold a Nix project

* packages.x86_64-linux.org2gfm
  Script to export Org-mode files to GitHub Flavored Markdown (GFM)

We’re required to pass regexes as final arguments to prune down the search. Above we’ve passed ^ to match everything and return all results.

We can also search a remote repository for packages to install. For example, Nixpkgs is a central repository for Nix, providing several thousand packages. We can search the “nixpkgs-unstable” branch of Nixpkgs’ GitHub repository for packages that match both “gpu|opengl|accel” and “terminal” as follows:

nix search github:NixOS/nixpkgs/nixpkgs-unstable 'gpu|opengl|accel' terminal

As discussed in a previous section, we can use the flakes registry identifier of nixpkgs instead of the longer github:NixOS/nixpkgs/nixpkgs-unstable to save some typing:

nix search nixpkgs 'gpu|opengl|accel' terminal
+end_src

#+name: nix-search-remote-concise
nix search nixpkgs 'gpu|opengl|accel' terminal | ansifilter

If we’re curious about what version of WezTerm is available in NixOS’s latest release, we can specialize the installable we’re searching as follows:

nix search nixpkgs/nixos-<<get("nixos-latest")>>#wezterm ^

* legacyPackages.x86_64-linux.wezterm (20240203-110809-5046fc22)
  GPU-accelerated cross-platform terminal emulator and multiplexer written by @wez and implemented in Rust

Here {{{get(nixos-latest,~/nixos-,~)}}} overrides the default nixpkgs-unstable branch of the registry entry, and the #wezterm suffix searches not just the flake, but a specific package named wezterm, which will either be found or not (there’s no need for regexes to filter further).

You may also notice that the Nixpkgs flake outputs packages under the legacyPackages attribute instead of the packages. The primary difference is that packages are flatly organized under packages, while legacyPackages can be an arbitrary tree. legacyPackages exists specifically for the Nixpkgs project, a central project to the Nix ecosystem that has existed long before flakes. Beyond Nixpkgs, you don’t have to think much about legacyPackages. Packages from all other flakes should generally be found under packages.

Building installables

The following result is one returned by our prior execution of nix search .:

* packages.x86_64-linux.org2gfm
  Script to export Org-mode files to GitHub Flavored Markdown (GFM)

We can see that a package can be accessed with the {{{package-attr-long}}} output attribute path of the project’s flake. Not shown in the search results above, this package happens to provide the {{{package-type}}} {{{package-target-long}}}.

We can build this package with nix build from the project root:

nix build .#<<get("package-attr-short")>>

As discussed in prior sections, the positional arguments to nix build are installables. Here, the . indicates that our flake should be found from the current directory. Within this flake, we look for a package with an attribute name of {{{package-attr-short}}}. We didn’t have to use the full attribute path {{{package-attr-long}}} because nix build will automatically look in the packages attribute for the system it detects we’re on.

If we omit the attribute path of our installable, Nix tries to build a default package, which it expects to find under the flake’s packages.<system>.default. For example, if we ran just nix build ., Nix would expect to find a flake.nix in the current directory with an output providing a packages.<system>.default attribute with a package to build.

Furthermore, if we didn’t specify an installable at all, Nix would assume we’re trying to build the default package of the flake found from the current directory. So, the following invocations are all equivalent:

nix build
nix build .
nix build .#packages.<system>.default

All packages built by Nix are stored in /nix/store. Nix won’t rebuild packages found there. Once a package is built, its content in /nix/store is read-only (until the package is garbage collected, discussed later).

After a successful call of nix build, you’ll see one or more symlinks for each package requested in the current working directory. These symlinks, by default, have a name prefixed with “result” and point back to the respective build in /nix/store:

readlink result*

/nix/store/islqrpxqp250lq2f8mda3jfngbcm280l-org2gfm

Following these symlinks, we can see the files the project provides:

tree -l result*

result
└── bin
    └── org2gfm

2 directories, 1 file

It’s common to configure these “result” symlinks as ignored in source control tools (for instance, for Git within a .gitignore file).

nix build has a --no-link switch in case you want to build packages without creating “result” symlinks. To get the paths where your packages are located, you can use nix path-info after a successful build:

nix path-info .#<<get("package-attr-short")>>

/nix/store/islqrpxqp250lq2f8mda3jfngbcm280l-org2gfm

Running commands in a shell

We can run commands in Nix-curated environments with nix shell. Nix will take executables found in packages, put them in an environment’s PATH, and then execute a user-specified command.

With nix shell, you don’t have to build the package first with nix build or mess around with “result” symlinks. nix shell will build any necessary packages required.

For example, to get the help message for the {{{run-target-short}}} executable provided by the package selected by the {{{run-attr-short}}} attribute path output by this project’s flake, we can call the following:

nix shell \
    .#<<get("run-attr-short")>> \
    --command <<get("run-target-short")>> --help

USAGE: org2gfm [OPTION]...  [FILE]...

DESCRIPTION:

    Uses Emacs to export Org-mode files to GitHub Flavored
…

Similarly to nix build, nix shell accepts installables as positional arguments to select packages to put on the PATH.

The command to run within the shell is specified after the --command switch. nix shell runs the command in a shell set up with a PATH environment variable that includes all the bin directories provided by the selected packages.

If you only want to enter an interactive shell with the configured PATH, you can drop the --command switch and following arguments.

nix shell also supports an --ignore-environment flag that restricts PATH to only packages selected, rather than extending the PATH of the caller’s environment. With --ignore-environment, the invocation is more sandboxed.

As with nix build, nix shell will select default packages for any installable that is only a flake reference. If no installable is provided to nix shell, the invocation will look for the default package under the packages.<system>.default attribute output by a flake assumed to be in the current directory. So, the following invocations are all equivalent:

nix shell
nix shell .
nix shell .#packages.<system>.default

Running installables

The nix run command allows us to run executables from packages with a more concise syntax than nix shell with a --command switch.

The main difference from nix shell is that nix run detects which executable to run from a package. If we want something other than what can be detected, we must continue using nix shell with --command.

As with other nix subcommands, nix run accepts an installable as an argument (but only one). If none is provided, then . is assumed.

If the provided installable is only a flake reference with no package selected, then nix run searches the following flake output attribute paths, in order, for something to run:

apps.<system>.default
packages.<system>.default

In flakes, applications are just packages that have been expanded with metadata. This metadata includes explicitly specifying which executable to use with nix run. Otherwise, for plain packages, nix run looks at metadata on the package to guess the binary to execute within the package.

If only a name is provided, then nix run searches in order through the following output attribute paths:

apps.<system>.<name>
packages.<system>.<name>
legacyPackages.<system>.<name>

And as always, we can specify a full output attribute path explicitly if nix run’s search doesn’t find what we want to run.

Here’s the nix run equivalent of the nix shell invocation from the previous section:

nix run .#<<get("run-attr-short")>>  -- --help

USAGE: org2gfm [OPTION]...  [FILE]...

DESCRIPTION:

    Uses Emacs to export Org-mode files to GitHub Flavored
…

We can see some of the metadata of this package with the --json switch of nix search:

nix search --json .#<<get("run-attr-short")>> ^ | jq .

{
  "packages.x86_64-linux.org2gfm": {
    "description": "Script to export Org-mode files to GitHub Flavored Markdown (GFM)",
    "pname": "org2gfm",
    "version": ""
…

The “pname” field in the JSON above indicates the package’s name. In practice, the package’s name may or may not differ from flake output name of the installable.

nix run works because the package selected by the output attribute name {{{run-attr-short}}} selects a package with a package name {{{run-name}}} that is the same as the executable provided at {{{run-target-long}}}.

If we want something other than what can be detected, then we have to continue using nix shell with --command.

`nix run` and `nix shell` with remote flakes

In the examples above, we’ve used selected packages from this project’s flake, like {{{get(run-attr-short,=.#,=)}}}. But one benefit of flakes is that we can refer to remote flakes just as easily, like nixpkgs#hello. Referencing remote flakes helps us quickly build environments with nix shell or run commands with nix run without committing to install software.

Here’s a small example.

nix run nixpkgs#hello

Hello, world!

When using nix shell, we can even mix local flake references with remote ones, all in the same invocation:

nix shell --ignore-environment \
    nixpkgs#which \
    .#<<get("run-attr-short")>> \
    --command which <<get("run-target-short")>>

/nix/store/islqrpxqp250lq2f8mda3jfngbcm280l-org2gfm/bin/org2gfm

What we do with local flake references can work just as well with remote flake references.

Installing and uninstalling programs

We’ve seen that we can build programs with nix build and execute them using the “result” symlink (result/bin/*). Additionally, we’ve seen that you can run programs with nix shell and nix run. But these additional steps and switches/arguments can feel extraneous. It would be nice to have the programs on our PATH. This is what nix profile is for.

nix profile maintains a symlink tree of installed programs called a profile. The default profile is at ~/.nix-profile. For non-root users, if this doesn’t exist, nix profile will create it as a symlink pointing to /nix/var/nix/profiles/per-user/$USER/profile. But you can point nix profile to another profile at any writable location with the --profile switch.

This way, you can just put ~/.nix-profile/bin on your PATH, and any programs installed in your default profile will be available for interactive use or scripts.

To install the {{{package-target-short}}} {{{package-type}}}, which is provided by the {{{package-attr-short}}} output of our flake, we’d run the following:

nix profile install .#<<get("package-attr-short")>>

We can see this installation by querying what’s been installed:

nix profile list

Name:               org2gfm
Flake attribute:    packages.x86_64-linux.org2gfm
Original flake URL: git+file:///home/shajra/src/nix-project
Locked flake URL:   git+file:///home/shajra/src/nix-project
Store paths:        /nix/store/islqrpxqp250lq2f8mda3jfngbcm280l-org2gfm

If we want to uninstall a program from our profile, we can reference it by name:

nix profile remove <<get("package-attr-short")>>

Also, if you look at the symlink-resolved location for your profile, you’ll see that Nix retains the symlink trees of previous generations of your profile. You can even roll back to an earlier profile with the nix profile rollback subcommand. You can delete old generations of your profile with the nix profile wipe-history subcommand.

Garbage collection

Every time you build a new version of your code, it’s stored in /nix/store. You can call nix store gc to purge unneeded packages. Programs that should not be removed by nix store gc can be found by starting with symlinks stored as garbage collection (GC) roots under three locations:

/nix/var/nix/gcroots
/nix/var/nix/profiles
/nix/var/nix/manifests.

For each package, Nix is aware of all files that reference back to other packages in /nix/store, whether in text files or binaries. This dependency tracking helps Nix ensure that dependencies of packages reachable from GC roots won’t be deleted by garbage collection.

Each “result” symlink created by a nix build invocation has a symlink in /nix/var/nix/gcroots/auto pointing back to it. So we’ve got symlinks in /nix/var/nix/gcroots/auto pointing to “result” symlinks in our projects, which then reference the actual built project in /nix/store. These chains of symlinks prevent packages built by nix build from being garbage collected.

If you want a package you’ve built with nix build to be garbage collected, delete the “result” symlink created before calling nix store gc. Breaking symlink chains under /nix/var/nix/gcroots removes protection from garbage collection. nix store gc will clean up broken symlinks when it runs.

Everything under /nix/var/nix/profiles is also considered a GC root. This is why users, by convention, use this location to store their Nix profiles with nix profile.

Also, note if you delete a “result*” link and call nix store gc, though some garbage may be reclaimed, you may find that an old profile is keeping the program alive. Use the nix profile wipe-history command to delete old profiles before calling nix store gc.

It’s also good to know that nix store gc won’t delete packages referenced by any running processes. In the case of nix run no garbage collection root symlink is created under /nix/var/nix/gcroots, but while nix run is running nix store gc won’t delete packages needed by the running command. However, once the nix run call exits, any packages pulled from a substituter or built locally are candidates for deletion by nix store gc. If you called nix run again after garbage collecting, those packages may be pulled or built again.

Next steps

This document has covered a fraction of Nix usage, hopefully enough to introduce Nix in the context of this project.

An obvious place to start learning more about Nix is the official documentation.

Bundled with this project is a small tutorial on the Nix language.

All the commands we’ve covered have more switches and options. See the respective man pages for more.

We didn’t cover much of Nixpkgs, the gigantic repository of community-curated Nix expressions.

The Nix ecosystem is vast. This project and documentation illustrate only a small sample of what Nix can do.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

nix-usage-flakes.org

nix-usage-flakes.org

Nix Setup and Usage with Flakes

Org-mode setup

Variables

Formatting help

Setup action

About this document

How this project uses Nix

Prerequisites

Working with Nix

Referencing flake projects

Inspecting flake outputs

Referencing flake outputs

Searching flakes for packages

Building installables

Running commands in a shell

Running installables

`nix run` and `nix shell` with remote flakes

Installing and uninstalling programs

Garbage collection

Next steps

Org-mode teardown

Files

nix-usage-flakes.org

Latest commit

History

nix-usage-flakes.org

File metadata and controls

Nix Setup and Usage with Flakes

Org-mode setup

Variables

Formatting help

Setup action

About this document

How this project uses Nix

Prerequisites

Working with Nix

Referencing flake projects

Inspecting flake outputs

Referencing flake outputs

Searching flakes for packages

Building installables

Running commands in a shell

Running installables

nix run and nix shell with remote flakes

Installing and uninstalling programs

Garbage collection

Next steps

Org-mode teardown

`nix run` and `nix shell` with remote flakes