Nix on a Kobo: The Cookbook

By: on May 30, 2026
A white Kobo e-reader on a table showing its home screen of books, a stylus beside it and a plant in the foreground

This Is the Reference, Not the Story

This is the technical companion to I Put Nix on My E-Reader Because I Can't Help It. That post is the story. This is the recipes.

If you want to know why anyone would do this, read the other one. If you have a Kobo, a NixOS build host, and a weekend, start here.

What You'll End Up With

  • A cyberdeckConfigurations.kobo flake output that builds a home-manager closure for armv7l-linux.
  • A NixOS build host that cross-compiles armv7l under qemu-user without you doing anything per-package after the initial overlay.
  • A Kobo with an ext4-in-a-file mounted at /nix, a populated /nix/store, and a ~/.nix-profile you can switch atomically.
  • A KOReader Terminal plugin that launches into fish + tmux + neovim with your full plugin set — same kit your workstation uses, on a 6-inch e-ink screen.

The build loop on every change after that is three commands:

nix build .#cyberdeckConfigurations.kobo.activationPackage
nix copy --to ssh://root@kobo.lan ./result
ssh root@kobo.lan ./result/activate

Prerequisites

  • A NixOS build host you control. Flakes and home-manager. The bigger and faster, the better — every package compiles from source under emulation, so build host wallclock dominates the first-run experience.
  • A Kobo with KFMon and KOReader installed. Both are sideloaded via the standard .kobo/ folder — no jailbreak. KOReader is the third-party reader app; KFMon is the launcher that wraps it. The cookbook below piggybacks on both.
  • SSH enabled on the Kobo. Drop an empty file named SSH into /mnt/onboard/.kobo via USB and reboot. The Kobo's dropbear sshd will be listening on port 22, root with no password until you drop a key into authorized_keys.
  • The device on the same network as the build host (or on a Tailscale tailnet).

Hardware Spec Sheet (Kobo Clara 2E)

The reference target. The recipes work on most armv7l Kobos (Clara HD, Libra H2O, Forma) with the same family of constraints; aarch64 Kobos (Libra Colour, Clara Colour) want a different cross.targets entry and likely a fresh overlay round.

ReleasedSeptember 2022
CPUAllwinner B300 (i.MX6SLL-class), ARM Cortex-A9, single-core, 1 GHz
Architecturearmv7l (32-bit, with NEON + vfpv3-d32 + Thumb-2)
RAM512 MB (~495 MB usable)
Storage16 GB eMMC; ~12 GB user-visible at /mnt/onboard
Display6" E-Ink Carta 1200, 300 ppi, frontlight + ComfortLight PRO
Connectivity802.11b/g/n Wi-Fi, USB-C
KernelLinux 4.1.15 (vendor-frozen)
Userspacebusybox 1.31.1, /bin/sh (no bash)
Missing/dev/net/tun, curl, CONFIG_USER_NS, glibc in any modern sense
Presentwget, dropbear-style sshd, lsmod, mount

Why the Build Is Expensive: Two Facts That Compose

This shapes every overlay decision below, so it's worth naming up front.

1. There's no substituter for armv7l. cache.nixos.org publishes binaries for x86_64-linux, aarch64-linux, and a handful of others — armv7l-linux isn't in that set. Every package the Kobo closure depends on — glibc, gcc, perl, python, openssl, busybox, every header and shared library, the whole stdenv bootstrap chain — compiles from source.

2. The compilation runs through qemu-user, not a textbook cross-compiler. boot.binfmt.emulatedSystems = [ "armv7l-linux" ]; doesn't set up an x86 gcc that emits ARM — it tells the kernel "when you see an ARM ELF binary, route it through qemu-user." So when Nix builds an armv7l package, the ARM gcc actually runs, inside qemu, translating instructions on the fly and emulating ARM syscalls against the host kernel. Output is bit-identical to a real Cortex-A9 gcc. Just slower.

Compose those two facts and you get the project's real cost shape: any change to a low-level package's hash rebuilds the world. Changing meson's hash means gfortran building itself from source under qemu (~12 hours), every numpy/scipy variant under qemu, every meson-using build script in the closure under qemu. The Gentoo crowd had a term for this twenty years ago: rebuild world. On armv7l Nix, it's the default condition, and every overlay decision has to be priced against it.

Standing the toolchain up the first time takes about a week of build-host wallclock. After that, normal nixpkgs bumps are partial, but a hash change to anything low-level (meson, gettext, perl, openssl) sends you back through a large fraction of the closure. The fan-out heuristic at the bottom of Layer 2 is where this becomes actionable.

Prior Art Worth Reading Alongside

  • nix-on-droid — the structural reference. Same single-user Nix profile + atomic ~/.nix-profile symlink swap pattern. Strip the Termux-specific bits and the activation-package machinery is exactly what you want.
  • Collin Dewey's Nix-on-Kindle writeup — the canonical reference if you're chasing a Kindle target instead of a Kobo. Bluetooth-HID on Kindle is locked down (Amazon's lipc layer), which is why I pivoted to Kobo.
  • FBInk + KOReader — five years of e-ink refresh tuning that the cyberdeck-shell launcher reuses by piggybacking on KOReader's Terminal plugin. Do not write your own framebuffer terminal.
  • DavHau/nix-on-armv7l — reference patterns for running Nix on armv7l without NixOS underneath.
  • mitanshu7/tailscale_kual — Tailscale on Kindle via a KUAL extension. Same userspace-mode pattern we use, done by hand instead of via the closure.

Reconnaissance: What an armv7 Kobo Actually Looks Like

Before you write a line of Nix, SSH in and look. The device's own state is the source of truth, not your assumptions.

$ uname -a
Linux kobo 4.1.15-00924-g7c8c7f01457 #39 SMP PREEMPT Thu Nov 17 ... armv7l GNU/Linux

$ cat /mnt/onboard/.kobo/version
N50632C230162,4.1.15,4.38.23038,...   # N506 = Clara 2E

$ free -m
              total        used        free
Mem:            495         174          93

$ ls /dev/net/tun
ls: /dev/net/tun: No such file or directory

$ command -v curl
(empty)

$ /bin/sh --version
BusyBox v1.31.1.kobo (2020-04-22)

Things to internalise from this:

  • armv7l, not aarch64. cache.nixos.org doesn't serve binaries for armv7l. Every package compiles from source.
  • Kernel 4.1.15. No CONFIG_USER_NS — Nix's build sandbox can't work, so sandbox = false in the device's nix.conf. We don't build on the device anyway.
  • No /dev/net/tun. Tailscale runs in userspace networking mode. Proxy on localhost:1055; outbound apps that don't auto-detect need ALL_PROXY=socks5://localhost:1055.
  • No curl, busybox shell only. The bootstrap must be POSIX sh and can't assume any modern tooling. Everything "modern" arrives via the Nix closure.
  • ~500 MB RAM. The kit (fish + tmux + neovim + mosh + tailscaled simultaneously) fits comfortably under 100 MB, but rules out any "ship Python or Node on-device" reflex.
  • /mnt/onboard is the USB-mass-storage partition. Persists across firmware updates, visible as a normal drive from a Linux laptop. The Nix store lives at /mnt/onboard/.adds/nix/ because of exactly these properties — survives factory resets, rescuable from another machine.

The pattern this exercise is teaching is discovery-by-observation, not configuration-by-interview. Same idea behind hardware-configuration.nix in NixOS — the system describes itself; you don't ask the user to type out their disk layout. The bootstrap script below uses the same instinct: uname -m to pick the Nix arch, /mnt/onboard/.kobo/version's existence to detect the device family.

The Mental Model: Six Layers

The single most reusable insight from this project is that "cross-compile to a different arch" is not one problem. It's at least six, and the symptoms at each layer look nothing like the symptoms at the others. Label your error to its layer first; the fix file is then obvious.

LayerQuestionWhere the fix goes
0. SubstituterIs there a cache for this arch?cache.nixos.org doesn't serve armv7l. Accept it, or stand up your own (harmonia).
1. SchedulingWill Nix even pick a builder?nixos/cross-compile.nix — declare both extra-platforms and extra-system-features.
2. Emulation fidelityBuild runs, a test fails?overlays/qemu-armv7l.nixdoCheck = false per package.
3. Emulated-native vs crossBuilds slow because gcc runs through qemu?Accept the overhead, or move to pkgsCross.armv7l-hf-multiplatform.
4. ABI / sub-archBinary built, but SIGILLs on device?Verify target /proc/cpuinfo matches what gcc was told to emit.
5. TransportCan't push closure to device?tar over SSH for the first push (no nix-store on device yet); nix copy after.
6. ActivationClosure landed, mount fails?Probe for symlink support; loop-mount ext4-in-a-file if FAT.

Recipes below are grouped by layer.

Layer 1: Scheduling — The binfmt Module Isn't Enough

The standard advice is one line:

boot.binfmt.emulatedSystems = [ "armv7l-linux" ];

That registers a qemu-user binfmt handler in the kernel and adds armv7l-linux to extra-platforms. After nh os switch, your host claims it can build armv7l-linux. No reboot needed — the binfmt registration happens at activation.

What it does not do is advertise the gccarch-* system features that low-level bootstrap derivations check for. So Nix sees a builder claiming armv7l support and a derivation saying "I need armv7l + gccarch-armv7-a" and gives up:

Failed to find a machine for remote build!
required (system, features): (armv7l-linux, [gccarch-armv7-a])

The fix is to map each emulated target to its gccarch features and advertise both. Wrap it in a module so future targets land in one place.

Recipe: nixos/cross-compile.nix

{ config, lib, ... }: let
  cfg = config.cross;
  gccarchFeatures = {
    "armv7l-linux"  = [ "gccarch-armv5te" "gccarch-armv6" "gccarch-armv7-a" ];
    "aarch64-linux" = [ "gccarch-armv8-a" ];
    "i686-linux"    = [ "gccarch-i686" ];
  };
in {
  options.cross.targets = lib.mkOption {
    type = lib.types.listOf lib.types.str;
    default = [];
    example = [ "armv7l-linux" "aarch64-linux" ];
  };

  config = lib.mkIf (cfg.targets != []) {
    boot.binfmt.emulatedSystems = cfg.targets;
    nix.settings.extra-system-features =
      lib.concatMap (t: gccarchFeatures.${t} or []) cfg.targets;

    assertions = [{
      assertion = !(builtins.elem config.nixpkgs.hostPlatform.system cfg.targets);
      message = "cross.targets must not include the host's own architecture.";
    }];
  };
}

Then on the build host:

cross.targets = [ "armv7l-linux" ];

The mapping is mechanical — it lives in nixpkgs/lib/systems/architectures.nix — but the binfmt module doesn't read it. One-time setup. Forget about it after that.

Common Trap: nh os switch Is the Verb That Activates binfmt

Editing the flake alone doesn't change the running system — the kernel still has no idea you want it to dispatch ARM binaries to qemu-user. You have to sudo nh os switch (or nixos-rebuild switch) so systemd-binfmt.service picks up the new /etc/binfmt.d/*.conf, and so nix-daemon.service restarts with the new extra-platforms/extra-system-features. Both happen as part of activation, no reboot required.

Two nix build attempts with the same "Failed to find a machine for remote build" error is a classic beginner trap: the flake says one thing, the running system says another, and Nix listens to the running system. Run the switch first.

Confirm registration with ls /proc/sys/fs/binfmt_misc/ | grep qemu-arm and nix show-config | grep -E 'extra-platforms|system-features'.

Layer 2: qemu-user Emulation Gaps

Once Layer 1 is fixed, the build chugs forward and then one package's test suite fails. Then another. Then another. After a couple of rounds you start spotting the pattern: it's almost never a real bug in the package — it's qemu-user approximating a real kernel imperfectly.

The fix is the same shape every time:

package = prev.package.overrideAttrs (_: { doCheck = false; });

The Classification Rule

If the package compiles and links, and only a test fails, it's an emulation gap, not a real bug. Skip and move on.

After the third one I stopped trying to understand each failure individually and started bucketing them by flavor. The flavors are useful because they tell you how worried to be about the skip — a wallclock timeout is a "definitely fine" skip; a signal-abort is a "spot-check on real hardware" skip.

Flavor Catalog

FlavorWhat qemu-user is doing wrongExampleSkip safety
Pipe truncationLoses bytes on nested fork/exec stdout capturerhash test 17, ripgrep compressed_*High
Signal abortsiginfo_t doesn't match real kernel for synthesized faultsboehmgc gctestMedium
Errno mismatchENOPROTOOPT / ENOTSUP for syscalls the real ARM kernel implementslibuv udp_multicast_interface6High
EEXIST on deletereaddir-while-unlink ordering wrongdejagnu file delete -forceHigh
Filesystem visibilityreaddir/stat doesn't see writes a test just madespdlog rotate, fish wildcardsHigh
File I/O orderingmmap+write rewrite ordering wrongelfutils elf_updateMedium
Wallclock timeoutPure slowdown trips upstream's hardcoded clocksmeson #254, pytest-xdist, mypyHighest
Harness bookkeepingPost-test driver crashes on uninit timestampsgit+prove $end_timeHighest
ENOSYS unimplementedqemu doesn't implement the syscall at allelfutils ptraceHighest
TTY/pty interactiveTermios + isatty() differfish pexpect, bat empty stdoutMedium
Sandbox capabilityNix sandbox blocks regardless of qemupsutil SIOCETHTOOL, tailscale BPFHighest
Memory accountingfree() doesn't return pages to host; allocator stats stay flatonetbb test_malloc_whiteboxHighest

Two triage shortcuts:

  • All tests inside the suite passed; the failure is in the harness wrapper. Almost always safe to skip. git is the canonical example: 26,738/26,738 tests passed, then prove crashed in its own cleanup.
  • The failing test is named after a kernel feature (xdp, ptrace, afalg, bpf, siocethtool). Almost always sandbox-capability or ENOSYS, never a real package bug.

Recipe: overlays/qemu-armv7l.nix (Full Annotated Slice)

The full file is ~430 lines for ~21 overrides. Each entry is one line of code plus a comment block explaining which qemu-user / sandbox / ABI flavor it's classified as, what the cascade unlock is, and any non-obvious gotcha. The comment blocks are load-bearing — six months from now, when nixpkgs bumps and one of these stops being necessary, the "why" is what tells you whether the skip can be deleted safely.

final: prev: {
  # rhash 1.4.4 — tests 17, 20, 22: recursive subprocess output comes back
  # empty/truncated under qemu-arm. Layer 2 pipe-truncation flavor.
  rhash = prev.rhash.overrideAttrs (_: { doCheck = false; });

  # libuv 1.51.0 — test #390 udp_multicast_interface6 aborts with ENOPROTOOPT
  # (-92) on an IPv6 multicast setsockopt that qemu-user doesn't implement.
  # 419/420 tests pass; binary works on real ARM.
  libuv = prev.libuv.overrideAttrs (_: { doCheck = false; });

  # boehm-gc 8.2.8 — gctest SIGABRTs. The GC relies on SIGSEGV/SIGBUS handlers
  # for dirty-page tracking; qemu's siginfo_t doesn't match what the test
  # expects. 16/17 tests pass; works on real ARM.
  boehmgc = prev.boehmgc.overrideAttrs (_: { doCheck = false; });

  # dejagnu 1.6.3 — Tcl `file delete -force` returns EEXIST on directory
  # cleanup. qemu-user's readdir-while-unlinking ordering doesn't match Tcl's
  # expectations. dejagnu is a test framework, so its own broken tests are
  # particularly safe to skip. MASSIVE cascade unlock: libffi → glib → dbus →
  # python3, ruby, rustc, llvm, guile, wayland, meson, ninja, zig.
  dejagnu = prev.dejagnu.overrideAttrs (_: { doCheck = false; });

  # source-highlight 3.1.9 — test_utils fails "assertion failed! expected
  # java.lang" on language-name lookup. 31/32 tests pass; likely locale (qemu
  # doesn't pass LC_* through cleanly). Blocks nmap, htop, libnl, libpcap,
  # iptables. Note the camelCase attr — nixpkgs renames dashes inconsistently.
  sourceHighlight = prev.sourceHighlight.overrideAttrs (_: { doCheck = false; });

  # openssl 3.6.2 — 26/4537 tests fail at OS-integration boundaries:
  # AF_ALG kernel socket, x509 store path semantics, file:// URI handling,
  # pthread cancellation timing. The crypto math (EC curves over all NIST +
  # Brainpool, AES, SHA, bignum, RSA) passes 100%. Saves ~99 min × ~3 openssl
  # variants in the closure. Cascade unlock: bind, cmake, curl, git, openssh,
  # python3, ruby, rustc, nmap, mosh, unbound, libevent, glib, meson, ninja,
  # yara, fish — i.e. most of the kit.
  openssl = prev.openssl.overrideAttrs (_: { doCheck = false; });

  # python3-psutil 7.1.2 — ioctl(SIOCETHTOOL) needs a real NIC, /sys/class/
  # power_supply isn't in the Nix sandbox. Sandbox-capability flavor, not a
  # pure qemu issue. Must use overridePythonAttrs (Python wraps via
  # buildPythonPackage; overrideAttrs silently no-ops). Must repoint
  # python3/python3Packages aliases too or the override misses one path.
  # MASSIVE cascade unlock: psutil → python3-env → fetch-cargo-vendor-util →
  # every Rust tool (bat, delta, fd, hexyl, hyfetch, ripgrep, tealdeer,
  # tree-sitter, zoxide, newsboat) + sphinx, mypy, requests, fish.
  python313 = prev.python313.override {
    packageOverrides = pyfinal: pyprev: {
      psutil = pyprev.psutil.overridePythonAttrs (_: { doCheck = false; });
      # pytest-xdist 3.8.0 — execnet IPC handshake exceeds hardcoded 10s
      # WAIT_TIMEOUT under qemu. Layer 2 timeout flavor.
      pytest-xdist = pyprev.pytest-xdist.overridePythonAttrs (_: { doCheck = false; });
      # mypy 1.17.1 — testIntOps subprocess.TimeoutExpired after 30s
      # (mypyc compiles a native ext then execs it; doesn't finish under qemu).
      # Test takes ~110 min wallclock — big build-time win regardless of fail.
      mypy = pyprev.mypy.overridePythonAttrs (_: { doCheck = false; });
    };
  };
  # Without these three alias lines, the override applies to one path and
  # silently misses another. Burned an hour discovering this — don't skip.
  python3 = final.python313;
  python3Packages = final.python313.pkgs;
  python313Packages = final.python313.pkgs;

  # elfutils 0.194 — 8/291 tests fail in three sub-flavors: elf_update can't
  # write data (mmap+write ordering), PTRACE_TRACEME returns ENOSYS (qemu
  # doesn't implement ptrace for emulated self-introspection), SIGABRT in
  # backtrace-native. Tests live under `installcheck` Makefile target, so
  # plain doCheck = false is a no-op — must also doInstallCheck = false.
  # Cascade: glib (debuginfo dep) → tmux, mosh, tailscale, fish, fontforge.
  elfutils = prev.elfutils.overrideAttrs (_: {
    doCheck = false; doInstallCheck = false;
  });

  # git 2.51.2 — all 26,738 tests across 985 files PASS, then Perl's `prove`
  # crashes in its own cleanup with "Use of uninitialized value $end_time in
  # subtraction". Harness-bookkeeping flavor: every test passed, the test
  # *runner* crashed on uninit timestamps under qemu's timing semantics.
  # Tests are under `installcheck`, not `check`. gitMinimal = git.override
  # {...} re-invokes the derivation function, dropping overrideAttrs changes
  # applied to `git` — override it explicitly to be safe.
  git = prev.git.overrideAttrs (_: { doInstallCheck = false; });
  gitMinimal = prev.gitMinimal.overrideAttrs (_: { doInstallCheck = false; });

  # spdlog 1.15.3 — `daily_logger rotate` count_files() returns 0 instead of 1.
  # Filesystem-visibility flavor: dirent ordering after a rotating-file write
  # doesn't match real-hardware semantics. 167/168 tests pass. Cascade:
  # doxygen → libnl → libxml2 → htop, libpcap → iptables → nmap.
  spdlog = prev.spdlog.overrideAttrs (_: { doCheck = false; });

  # meson 1.9.1 — `common: 254 long output` times out at ~65s (limit 60s)
  # under qemu's slower I/O loop. Wallclock-timeout flavor. 485/486 tests pass.
  # MUST use overrideAttrs, NOT overridePythonAttrs — the latter strips
  # .override from the result. python-packages.nix's toPythonModule wrapper
  # calls pkgs.meson.override; if .override is missing, the entire python3
  # eval path explodes with "attribute 'override' missing".
  meson = prev.meson.overrideAttrs (_: { doCheck = false; });

  # vim-go's postPatch references delve. Delve's source has an explicit
  # `your_linux_architecture_is_not_supported_by_delve` sentinel for armv7l.
  # Strip the postPatch; :Go* commands won't find helpers (fine, no Go on Kobo).
  vimPlugins = prev.vimPlugins // {
    vim-go = prev.vimPlugins.vim-go.overrideAttrs (_: { postPatch = ""; });
  };

  # libseccomp 2.6.0 — 5089/5090 regression tests pass. Root cause is
  # structural: seccomp filters at the kernel syscall boundary, but qemu-user
  # translates syscalls *before* they reach the kernel. A BPF rule against
  # guest syscall numbers can mismatch what hits the host. Cascade:
  # libseccomp → nix-store → nix → home-manager-generation.
  libseccomp = prev.libseccomp.overrideAttrs (_: { doCheck = false; });

  # bmake 20250804 — two qemu-user failures, both rooted in bmake's recursive
  # bootstrap: check phase + install phase both can't find sys.mk because
  # $out/share/mk isn't populated yet (chicken-and-egg). Fix: set MAKESYSPATH
  # to the source tree's mk/ directory. Cascade: bmake → lowdown → nix-manual.
  bmake = prev.bmake.overrideAttrs (_: {
    doCheck = false;
    preInstall = ''export MAKESYSPATH="$PWD/mk"'';
  });

  # onetbb 2022.3.0 — test_malloc_whitebox "Decreasing reallocation": expects
  # sysMemUsageBefore > sysMemUsageAfter, but under qemu-user free() doesn't
  # actually munmap pages back to the host kernel — the guest's mapping stays.
  # Memory-accounting flavor. 139/140 tests pass; test runs for 11+ hours
  # wallclock — skipping is a huge build-time win too. Cascade: libblake3 →
  # nix-store → nix.
  onetbb = prev.onetbb.overrideAttrs (_: { doCheck = false; });

  # nix 2.31.5 — nix's own functional tests pull mercurial → python-google-re2
  # → pybind11 → numpy → openblas. openblas fails to build on armv7l under
  # qemu (cpuid.S assembler error). We don't need nix's tests on armv7l.
  # This was the surprise consequence of adding pkgs.nix to the closure for
  # `nix copy --to ssh://` convenience.
  nix = prev.nix.overrideAttrs (_: {
    doCheck = false; doInstallCheck = false;
  });

  # tailscale 1.90.9 — TestXDP in derp/xdp: bpf(BPF_MAP_CREATE) needs CAP_BPF
  # / CAP_SYS_ADMIN, blocked by the Nix sandbox. Sandbox-capability flavor.
  # XDP isn't relevant on Kobo (no eBPF, ARM kernel pre-BTF).
  tailscale = prev.tailscale.overrideAttrs (_: { doCheck = false; });

  # ncdu 2.x is written in Zig. zig's bootstrap stage (zig1, wasm-compiled
  # running under qemu-user) hits OutOfMemory emitting zig2.c — the 32-bit
  # emulated address space plus qemu's translation overhead can't hold a
  # self-compile of zig. Different flavor: the *build itself* is too
  # memory-hungry. Substitute the legacy C version (smaller binary, no zig
  # runtime, identical disk-walking behavior).
  ncdu = prev.ncdu_1;

  # ripgrep 15.1.0 — 3/330 tests fail (compressed_brotli, compressed_lz4,
  # compressed_zstd). Spawns external decompressor as subprocess, pipes
  # in compressed bytes — qemu's subprocess pipe semantics break the test.
  # Same flavor as rhash and meson #254. 327/330 tests pass.
  ripgrep = prev.ripgrep.overrideAttrs (_: { doCheck = false; });

  # bind 9.20.21 — NOT a test failure; a real build break. `gen` (a build
  # helper) runs readdir() and gets EOVERFLOW: armv7l's 32-bit ino_t can't
  # hold modern tmpfs inode numbers. On real ARM with ext4-on-SD this never
  # triggers. On qemu-user vs host tmpfs, it always does. Fix:
  # -D_FILE_OFFSET_BITS=64 picks glibc's 64-bit readdir. The bind Makefile
  # compiles gen.c with raw `gcc` bypassing configure's CFLAGS, so we push
  # via NIX_CFLAGS_COMPILE which the cc-wrapper splices into every gcc call.
  # bind is in the closure because dogdns (Haskell, no armv7l bootstrap) was
  # swapped for bind.dnsutils to keep `dig` available.
  bind = prev.bind.overrideAttrs (old: {
    env = (old.env or {}) // {
      NIX_CFLAGS_COMPILE =
        (old.env.NIX_CFLAGS_COMPILE or "") + " -D_FILE_OFFSET_BITS=64";
    };
  });

  # fish 4.2.1 — 40/183 tests fail in two TTY/pty families: pexpect tests
  # time out at expect_prompt() (termios/pty signal delivery differs under
  # qemu), and .fish CHECK tests fail with "No matches for wildcard '*'" in
  # directories the setup just populated (filesystem-visibility, same as
  # spdlog).
  # GOTCHA: nixpkgs's fish.nix puts sphinx in nativeCheckInputs, which
  # doCheck = false drops. But fish also needs sphinx at build time to
  # render man pages — without it the build dies on `sphinx-build failed`.
  # Pull sphinx back into nativeBuildInputs explicitly.
  fish = prev.fish.overrideAttrs (old: {
    doCheck = false;
    nativeBuildInputs =
      (old.nativeBuildInputs or []) ++ [ final.python3Packages.sphinx ];
  });

  # bat 0.26.1 — 7/183 tests fail with empty stdout where help/list/file
  # output was expected. Queries isatty() through qemu's pty emulation and
  # picks a code path that suppresses output. TTY flavor.
  bat = prev.bat.overrideAttrs (_: { doCheck = false; });

  # Test2-Harness — 1/N tests fail in t/integration/preload.t. yath's
  # preload mode forks a long-lived parent + child stages; the test arms a
  # SIGALRM with a hardcoded deadline that real hardware beats and qemu
  # blows. Layer 2 signal-timing flavor.
  # GOTCHA: must override at the scope level via overrideScope. The
  # perl-interpreter-level .override = (overrides = pkgs: {...}) path
  # creates infinite recursion because perl's passthruFun computes its scope
  # as `perlPackages // (overrides pkgs)`.
  perl540Packages = prev.perl540Packages.overrideScope (pfinal: pprev: {
    Test2Harness = pprev.Test2Harness.overrideAttrs (_: { doCheck = false; });
  });
  perlPackages = final.perl540Packages;

  # libgit2 1.9.2 — many test families fail, all reduced to the same root
  # cause: git_futils_rmdir_r → "Directory not empty" because under qemu the
  # readdir/unlink ordering leaves a dirent visible after its inode is gone.
  # Filesystem-visibility flavor, same as spdlog and dejagnu. ONE racy
  # iteration in shared teardown cascades into every later test reporting F.
  # Cascade: libgit2 → lazygit, tig, delta paths in the closure.
  libgit2 = prev.libgit2.overrideAttrs (_: { doCheck = false; });

  # wl-clipboard provides wl-copy / wl-paste for Wayland clipboard integration.
  # nixpkgs's neovim wrapper unconditionally includes it — every neovim
  # closure drags in wayland + wayland-protocols + meson build of both. On a
  # Kobo (no compositor, e-ink, no clipboard manager) it's pure dead weight.
  # Replace with a no-op stub so nvim's wrapper finds wl-copy / wl-paste on
  # PATH but they do nothing. Internal nvim registers (`"a`, etc) keep working.
  wl-clipboard = final.symlinkJoin {
    name = "wl-clipboard-stub-armv7l";
    paths = [
      (final.writeShellScriptBin "wl-copy" "cat >/dev/null")
      (final.writeShellScriptBin "wl-paste" "")
    ];
    meta.description = "Stub wl-clipboard (no Wayland on Kobo)";
  };
}

The full file is in the justin-nix flake. Every entry has a single-paragraph "Why:" comment of this shape — the comment block costs you 30 seconds to write and saves the next person (often future-you) an hour of staring at the same test failure.

Three Meta-Gotchas That Bite Twice

  • doCheck = false vs doInstallCheck = false are not the same switch. git and elfutils put their test suites under the installcheck target. Grep the package's Makefile for installcheck: before guessing.
  • gitMinimal = git.override {...} drops overrideAttrs changes you applied to git. .override re-invokes the derivation function with new args. If you override git you have to override gitMinimal explicitly too.
  • doCheck = false can silently drop build-time deps via nativeCheckInputs. fish is the canonical victim — sphinx is in nativeCheckInputs, which doCheck = false drops, but fish also needs sphinx at build time to render man pages. When skipping tests causes a build break (not a test break), check whether the derivation routes a build dep through nativeCheckInputs and pull it back into nativeBuildInputs manually.

Sibling Pattern: meta.platforms Excludes armv7l

Some packages declare via meta.platforms that they don't support armv7l, and overrideAttrs on the wrapper doesn't propagate to the inner derivation. tmuxPlugins.fingers is the canonical victim — the wrapper accepts armv7l on paper, but the inner Rust fingers crate's meta.platforms excludes it. Conditional drop is cleaner than fighting the override:

# home/tmux.nix
plugins = with pkgs.tmuxPlugins; [
  resurrect
  continuum
] ++ lib.optionals (pkgs.stdenv.hostPlatform.system != "armv7l-linux") [
  fingers
];

Same shape: gate behind lib.optionals at the consumer site rather than monkey-patching the package's meta. Easier to read, doesn't lie about what the package supports.

Categories Above Layer 2 — When the Build Itself Breaks

CategorySymptomFix
GHC bootstrap absentcannot bootstrap GHC on this platformFind a non-Haskell substitute (e.g. dogdnsbind.dnsutils)
Upstream refuses armv7lSentinel file like your_linux_architecture_is_not_supported_by_delveOverride the consumer's derivation to not reference the unsupported tool
qemu-user OOM on memory-hungry compilerszig1.wasm self-compile OOMsSubstitute the legacy version (ncduncdu_1) or accept the platform doesn't support that tool
Build-helper ABI mismatchEOVERFLOW from readdir() mid-build-D_FILE_OFFSET_BITS=64 via NIX_CFLAGS_COMPILE
Unwanted closure inhabitantWayland in the closure of an e-readerStub the dep so the consumer's wrapper still finds it on PATH

The last one — "the build was about to succeed and ship me wayland on an e-reader" — has no error message. You have to notice. Read the build log for unexpected package names; every "wait, why is that being built?" instinct I had during this exercise turned out to be correct.

The Expensive Heuristic: Price the Rebuild Fan-out Before Changing a Low-Level Hash

On x86_64 with a working cache, changing meson's hash means a fast partial rebuild from a few misses. On armv7l with no cache, it means gfortran 14.3.0 building itself from source under qemu (~12 hours), every numpy/scipy variant compiling from source under qemu, every meson-using build script in the closure re-running under qemu.

Before changing the hash of a low-level dep (meson, gettext, perl, openssl) on a no-cache platform, run nix-store --query --referrers-closure against its store path. If the fan-out is in the thousands, that's a multi-day commit. The fix shape doesn't change. The timing decision does.

(For me this cost was triggered by a one-character swap — overridePythonAttrs strips .override from its result, which only mattered once pkgs.nix was in the closure and forced python3Packages.meson evaluation. Use overrideAttrs for plain stdenv attrs like doCheck even on Python packages, unless you genuinely need overridePythonAttrs.)

Layer 6: Getting the Closure Onto the Device

The Kobo's on-device install hides three sub-problems, each with a different fix shape. None of them are exotic — anyone shipping a Nix closure to a non-NixOS handheld will hit them.

6a — Bootstrap Transport: tar Before nix-store

nix copy --to ssh://kobo works by SSH'ing in and running nix-store --serve on the remote. The Kobo doesn't have nix-store — that's what you're trying to install. Chicken, egg.

The first push uses tar instead, then every subsequent push uses nix copy:

# build host — first push
tar -czf - -P -T <(nix-store -qR ./result) | \
  ssh root@kobo.lan 'cd / && tar -xzf -'

tar -P on the build side preserves absolute paths. The Kobo's busybox tar doesn't have -P on extract, but its default behavior strips leading slashes — so cd / first and the paths land back where they started.

6b — Filesystem Capability: FAT Can't Store Symlinks

Nix's store is symlinks all the way down. Atomic generation swaps work by renaming a symlink. Take symlinks away and Nix stops working entirely.

The Kobo's 14 GB /mnt/onboard partition is FAT32 — it has to be, because Kobo exposes it as USB Mass Storage when plugged in. FAT has no concept of symlinks. symlink(2) returns EPERM because the driver has nowhere to write the link. Symptom from a naïve install:

tar: can't create symlink '...libunistring.so.5' to '...': Operation not permitted

Fix: put a regular file on the FAT partition, format it as ext4, loop-mount it. Wubi-on-Windows from the 2000s. From FAT's perspective it's an opaque blob. From everything above the loop driver, it's a real ext4 with full POSIX semantics.

# build host
truncate -s 3500M /tmp/nix.img       # <4 GiB; see 6d
mkfs.ext4 -F /tmp/nix.img
scp /tmp/nix.img root@kobo:/mnt/onboard/.adds/nix.img

# kobo
mount -o loop /mnt/onboard/.adds/nix.img /nix

6c — Toolchain Gaps on busybox

The Kobo runs busybox 1.31.1. Three small papercuts you have to know about:

  • No -P on tar extract. Busybox tar always strips leading slashes. Workaround: cd / first.
  • No truncate. Can't build the ext4 image on-device. Build it on the host with truncate + mkfs.ext4 and scp it over.
  • mount -o loop may not auto-allocate a loop device. Depending on how busybox was compiled, you may need losetup -f --show $img first, then mount /dev/loopN /nix. Bootstrap script tries -o loop first and falls back.

6d — FAT32 Has a 4 GiB Single-File Ceiling

Max single-file size on FAT32 is 4,294,967,295 bytes — one byte under 4 GiB. A 6 GB ext4 image dies at ~66% through scp:

scp: write remote "/mnt/onboard/.adds/nix.img": Bad message

Use 3500 MB instead of 6 GB. For a closure around 2.5 GB that leaves ~1 GB of generation headroom — which only works if you actively manage it (see "Disk Hygiene" below).

Recipe: The Full bootstrap.sh

The full bootstrap script is ~220 lines of POSIX sh (busybox 1.31.1 — no bash). It's broken into seven numbered sections, each with a single clear responsibility. The shape is "discover what the device is, decide on a storage strategy, lay down the minimum Nix expects, print what to do next." Don't ask the user anything — the device tells you what it is.

#!/bin/sh
set -eu
log() { printf '[cyberdeck] %s\n' "$*"; }
die() { printf '[cyberdeck] ERROR: %s\n' "$*" >&2; exit 1; }

# -------------------------------------------------------------------------
# 1. Detect arch + device family — discovery-by-observation
# -------------------------------------------------------------------------
ARCH=$(uname -m)
case "$ARCH" in
  aarch64|arm64) NIX_ARCH=aarch64-linux ;;
  armv7l|armv7)  NIX_ARCH=armv7l-linux  ;;
  x86_64)        NIX_ARCH=x86_64-linux  ;;
  *) die "unsupported arch: $ARCH" ;;
esac

DEVICE=generic
PERSIST_DIR="$HOME"
if [ -f /mnt/onboard/.kobo/version ]; then
  DEVICE=kobo
  PERSIST_DIR=/mnt/onboard/.adds
elif [ -d /mnt/us ] && [ -f /etc/prettyversion.txt ]; then
  DEVICE=kindle
  PERSIST_DIR=/mnt/us/.adds
fi
mkdir -p "$PERSIST_DIR"
log "arch=$NIX_ARCH device=$DEVICE persist_dir=$PERSIST_DIR"

# -------------------------------------------------------------------------
# 2. Probe: does the persistent partition support symlinks?
# -------------------------------------------------------------------------
probe_supports_symlinks() {
  _t="$1/.cyberdeck_probe_target_$$"
  _l="$1/.cyberdeck_probe_link_$$"
  touch "$_t" 2>/dev/null || return 2
  if ln -s "$_t" "$_l" 2>/dev/null; then
    rm -f "$_l" "$_t"; return 0
  fi
  rm -f "$_t"; return 1
}

NIX_IMG="$PERSIST_DIR/nix.img"
NIX_DIR="$PERSIST_DIR/nix"
# Default 3500M — has to stay under FAT32's 4 GiB-per-file ceiling.
NIX_IMG_SIZE=${CYBERDECK_NIX_IMG_SIZE:-3500M}

if probe_supports_symlinks "$PERSIST_DIR"; then
  STRATEGY=direct
  log "$PERSIST_DIR supports symlinks — using a plain directory for /nix"
else
  STRATEGY=loopback
  log "$PERSIST_DIR is FAT-like (no symlinks) — using an ext4 loopback image"
fi

# -------------------------------------------------------------------------
# 3a. Direct strategy — bind-mount or symlink a real directory onto /nix
# -------------------------------------------------------------------------
if [ "$STRATEGY" = direct ]; then
  mkdir -p "$NIX_DIR/store" "$NIX_DIR/var/nix" "$NIX_DIR/etc"
  if [ ! -d /nix/store ]; then
    if mountpoint -q /nix 2>/dev/null; then
      :  # already mounted
    elif mkdir -p /nix 2>/dev/null && mount --bind "$NIX_DIR" /nix 2>/dev/null; then
      log "bind-mounted $NIX_DIR -> /nix"
    else
      ln -sfn "$NIX_DIR" /nix
      log "symlinked $NIX_DIR -> /nix (no mount privilege)"
    fi
  fi
fi

# -------------------------------------------------------------------------
# 3b. Loopback strategy — ext4-in-a-file mounted at /nix
# -------------------------------------------------------------------------
if [ "$STRATEGY" = loopback ]; then
  # Wipe any stale debris from a prior FAT-backed attempt.
  if [ -e "$NIX_DIR" ] && [ ! -L "$NIX_DIR" ]; then
    log "removing stale FAT-backed $NIX_DIR from previous attempt"
    rm -rf "$NIX_DIR"
  fi
  [ -L /nix ] && { log "removing stale /nix symlink"; rm /nix; }

  if [ ! -f "$NIX_IMG" ]; then
    log "creating ext4 image at $NIX_IMG (size=$NIX_IMG_SIZE)"
    command -v truncate >/dev/null 2>&1 || die "no 'truncate' — build image on host (size must be <4 GiB for FAT32)"
    truncate -s "$NIX_IMG_SIZE" "$NIX_IMG"
    if command -v mkfs.ext4 >/dev/null 2>&1; then
      mkfs.ext4 -F "$NIX_IMG"
    elif command -v mke2fs >/dev/null 2>&1; then
      mke2fs -t ext4 -F "$NIX_IMG"
    else
      rm -f "$NIX_IMG"; die "no mkfs.ext4 on device — build image on host"
    fi
  fi

  mkdir -p /nix
  if mountpoint -q /nix 2>/dev/null; then
    log "/nix already mounted — skipping"
  elif mount -o loop "$NIX_IMG" /nix 2>/dev/null; then
    log "loop-mounted $NIX_IMG -> /nix"
  else
    # Some busybox builds don't auto-allocate loop devices via -o loop.
    command -v losetup >/dev/null 2>&1 || die "mount -o loop failed and no losetup"
    LOOPDEV=$(losetup -f --show "$NIX_IMG") || die "losetup failed"
    mount "$LOOPDEV" /nix || die "mount $LOOPDEV /nix failed"
    log "loop-mounted via $LOOPDEV"
  fi
  mkdir -p /nix/store /nix/var/nix /nix/etc
fi

# -------------------------------------------------------------------------
# 4. Minimal nix.conf — lives inside /nix so it follows the storage backend
# -------------------------------------------------------------------------
cat > /nix/etc/nix.conf <<EOF
experimental-features = nix-command flakes
# Old Kobo/Kindle kernels lack CONFIG_USER_NS — sandboxing won't work.
sandbox = false
# Pull from your home cache (harmonia on the build host) first.
substituters = http://nixai.<tailnet>.ts.net:5000 https://cache.nixos.org
trusted-public-keys = nixai.<tailnet>.ts.net-1:<pubkey> cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
# Non-NixOS single-user setup — no nixbld group.
build-users-group =
# Disk-constrained device — auto-GC before we wedge.
min-free = 268435456       # auto-GC when free < 256 MB
max-free = 1073741824      # ...and reclaim up to 1 GB
keep-outputs = false       # don't pin build-time intermediates
keep-derivations = false
auto-optimise-store = true # hardlink identical files (~20-30% saved)
EOF

# -------------------------------------------------------------------------
# 5. Profile + activation symlinks
# -------------------------------------------------------------------------
mkdir -p /nix/var/nix/profiles/per-user/${USER:-root}
mkdir -p "$HOME/.config/nix"
ln -sfn /nix/etc/nix.conf "$HOME/.config/nix/nix.conf"

# -------------------------------------------------------------------------
# 6. Persistence reminder (loopback only)
# -------------------------------------------------------------------------
if [ "$STRATEGY" = loopback ]; then
  cat <<EOF
NOTE: the loopback mount of $NIX_IMG -> /nix does NOT survive reboot.
Re-run bootstrap.sh after every boot, or add a startup hook that runs:
    mount -o loop $NIX_IMG /nix
On Kobo, the clean way is a KFMon launcher script. /etc/init.d/rcS works
but gets wiped on firmware updates.
EOF
fi

# -------------------------------------------------------------------------
# 7. Print next steps for the operator
# -------------------------------------------------------------------------
cat <<EOF
next steps (run on your NixOS build host):
  nix build .#cyberdeckConfigurations.$DEVICE.activationPackage
  nix copy --to ssh://root@$(hostname || echo kobo.lan) ./result
  ssh root@$(hostname || echo kobo.lan) "/nix/store/\$(basename \$(readlink ./result))/activate"
EOF

A few things worth flagging in this script:

  • The arch + device detection in section 1 is the discovery-by-observation pattern made concrete. Don't ask the operator which device family they're on — read /mnt/onboard/.kobo/version (Kobo) or /etc/prettyversion.txt (Kindle) and decide.
  • The probe in section 2 doesn't enumerate filesystems. "FAT" / "exFAT" / "vfat" / "msdos" all behave identically from our perspective — try to make a symlink and watch what happens. One branch handles every present and future filesystem with that property.
  • The nix.conf in section 4 lives inside /nix/etc/ — that way it follows whichever storage backend the probe chose, instead of sitting on the busybox root and getting lost on the next bootstrap.
  • build-users-group = (empty) is the single-user-Nix incantation for a non-NixOS host with no nixbld group. Forgetting this gives you "no build users" errors at activation.
  • The mount fallback in section 3b (losetup -f --show then plain mount) catches busybox builds that don't auto-allocate loop devices via -o loop. Some do, some don't — try the easy path, fall through.

The Terminal UX: Do Not Write Your Own E-Ink Terminal

Writing a usable framebuffer terminal on e-ink is hard. Every refresh is slow, ghosts, and partial-vs-full refresh is governed by an explicit kernel ioctl (MXCFB_SEND_UPDATE on Kobo's i.MX SoCs). An LCD terminal emulator assumes "framebuffer == screen" and that redrawing is free. E-ink is the opposite: a naïve fbterm flickers, lags, and ghosts. KOReader spent ~5 years tuning all of this via NiLuJe's FBInk library, and KOReader's Terminal emulator plugin reuses that tuning.

The Three Realistic Paths

ApproachEffortUXVerdict
KOReader Terminal pluginnone — built-inreal PTY, on-screen keyboard, e-ink refresh tunedshipped in v0
SSH in from a phone or laptopminutes — dropbear is already therethe Kobo becomes a server; terminal lives elsewherealways available, doesn't satisfy the cyberdeck vision
Standalone framebuffer terminal via KFMonweekend project — fork or write something against FBInktrue "boot to shell", no KOReader dependencyphase 2

The answer for v0: piggyback on KOReader's terminal. Ship a launcher binary, point KOReader at it.

Recipe: cyberdeck-shell Launcher

(writeShellScriptBin "cyberdeck-shell" ''
  export PATH="$HOME/.nix-profile/bin:$PATH"
  if [ -f "$HOME/.nix-profile/etc/profile.d/hm-session-vars.sh" ]; then
    . "$HOME/.nix-profile/etc/profile.d/hm-session-vars.sh"
  fi
  exec ${tmux}/bin/tmux new-session -A -s cyberdeck ${fish}/bin/fish
'')

In KOReader: Tools → More Tools → Terminal Emulator → Shell command/root/.nix-profile/bin/cyberdeck-shell.

The tmux new-session -A -s cyberdeck does real ergonomic work: close the terminal, reopen it, sleep the device, wake it two hours later, and your cursor is exactly where you left it. That continuity is what makes the device feel like a coherent computer instead of a chat-with-the-shell window.

Composing the Flake

# flake.nix (excerpt)
outputs.cyberdeckConfigurations.kobo =
  home-manager.lib.homeManagerConfiguration {
    pkgs = import nixpkgs {
      system = "armv7l-linux";
      overlays = [ (import ./overlays/qemu-armv7l.nix) ];
    };
    modules = [
      ./nixos/cyberdeck/default.nix
      ./nixos/cyberdeck/profiles/kobo.nix
      { home.username = "root"; home.homeDirectory = "/root"; }
    ];
  };

cyberdeck/default.nix is the device-agnostic kit (fish + tmux + neovim + mosh + ssh + tailscale + ripgrep/fd/fzf + the cyberdeck-shell launcher). cyberdeck/profiles/kobo.nix is Kobo-specific overrides (nixRoot = "/mnt/onboard/.adds/nix";, userspace tailscale, etc).

Tailscale: Userspace Networking Is Not Optional

The Kobo runs a 2015-era 4.1.x kernel that does not expose /dev/net/tun. Tailscale's normal mode wants a TUN device; without one, tailscaled fails to start. Run it in userspace mode instead:

# nixos/cyberdeck/profiles/kobo.nix
cyberdeck.tailscale = {
  enable = true;
  userspaceNetworking = true; # kernel 4.1.x has no TUN exposed
};

Same constraint applies to most non-NixOS handhelds (Kindles, Androids, anything on a vendor-frozen old kernel). What you get and what you trade:

  • Inbound works trivially. ssh kobo.<tailnet>.ts.net from any other tailnet node Just Works. This is the cyberdeck portal use case.
  • Outbound from arbitrary apps needs the proxy. tailscaled --tun=userspace-networking exposes a SOCKS5 / HTTP CONNECT proxy on localhost:1055. Apps that respect ALL_PROXY=socks5://localhost:1055 Just Work — that covers ssh (via ProxyCommand), mosh, curl in modern versions, and most CLI tooling. Apps that don't need explicit wrapping.
  • Magic-DNS still works. kobo.tailnet-style names resolve via the tailscaled DNS proxy without TUN.

For the cyberdeck use case (terminal in → SSH out to other hosts), the default ssh client and mosh both honour ALL_PROXY cleanly, so the wrinkle is invisible in practice. tailscale_kual does the same pattern manually on Kindle via a KUAL extension; the cookbook version is the declarative form of the same thing.

Reference: Tailscale userspace networking docs, and the jailbroken-Kindle writeup for the canonical rationale.

Side-Lesson: Positive Imports Beat Negative Overrides

If a shared module pulls in something the cyberdeck profile can't use (e.g. neovim.nix had chromium in extraPackages for strudel.nvim), the wrong instinct is lib.mkForce from the cyberdeck side. It works but it's ugly — anyone reading neovim.nix later sees chromium and assumes it's used.

Better: move the strudel-only dependencies into the strudel-specific file.

# home/neovim.nix — lean core
extraPackages = with pkgs; [ tree-sitter ];

# home/neovim/strudel.nix — desktop bits next to the desktop plugin
extraPackages = with pkgs; [
  tree-sitter-strudel nodejs nodePackages.npm chromium
];

Workstations import both. nix-on-droid and cyberdeck import just neovim.nix. The smell of lib.mkForce over an imported module is almost always "this file should have been split."

Deploying — The Complete First-Time Sequence

From a clean Kobo to a working cyberdeck-shell:

# 1. Build the closure
nix build .#cyberdeckConfigurations.kobo.activationPackage

# 2. Pre-build the ext4 image (FAT32 ceiling: <4 GiB)
truncate -s 3500M /tmp/nix.img
mkfs.ext4 -F /tmp/nix.img

# 3. Ship the image and bootstrap script to the Kobo
scp /tmp/nix.img             root@kobo.lan:/mnt/onboard/.adds/nix.img
scp nixos/cyberdeck/bootstrap.sh root@kobo.lan:/mnt/onboard/.adds/

# 4. Mount /nix on the device
ssh root@kobo.lan sh /mnt/onboard/.adds/bootstrap.sh

# 5. First push: tar (nix-store isn't on the device yet)
tar -czf - -P -T <(nix-store -qR ./result) | \
  ssh root@kobo.lan 'cd / && tar -xzf -'

# 6. Activate
ssh root@kobo.lan "$(readlink -f ./result)/activate"

After that, the image file persists. Future updates skip steps 2–5 entirely:

nix build .#cyberdeckConfigurations.kobo.activationPackage
nix copy --to ssh://root@kobo.lan ./result
ssh root@kobo.lan ./result/activate

Disk Hygiene: Surviving Inside 3.5 GB

A 2.5 GB closure inside a 3.5 GB image leaves ~1 GB of headroom. That's only enough for one or two generations if they accumulate, so GC has to be active — not a thing you remember to run by hand.

nix.conf on the Device

min-free = 268435456       # auto-GC when free < 256 MB
max-free = 1073741824      # ...and reclaim up to 1 GB
auto-optimise-store = true # hardlinks identical files, ~20-30% saved

Post-Activation GC Hook

# nixos/cyberdeck/default.nix
home.activation.cyberdeckGC = lib.hm.dag.entryAfter [ "cyberdeckBanner" ] ''
  $DRY_RUN_CMD ${pkgs.nix}/bin/nix-collect-garbage -d || true
  $DRY_RUN_CMD ${pkgs.nix}/bin/nix-store --optimise || true
'';

-d deletes every generation except current. --optimise needs an explicit kick because tar and nix copy add paths without going through the nix-daemon hook that normally triggers optimisation.

Tradeoff: aggressive GC means nix-env --rollback has nothing to roll back to. For a Kobo where deploys come from elsewhere, that's the right call — if a generation breaks, you fix it on the build host and ship a replacement.

pkgs.nix itself in the closure (~80 MB) is what unlocks nix-collect-garbage for the hook and nix copy --to ssh:// for subsequent pushes. Worth the cost.

Maintenance: Pull from harmonia Instead of Pushing

The push model (nix copy --to ssh://kobo from the build host) requires the device online and reachable at deploy time. For a Kobo often asleep or behind a captive portal, the pull model fits better:

  1. Build on the NixOS host. The result lands in /nix/store.
  2. harmonia (already running on the build host) serves /nix/store over HTTP on the tailnet.
  3. The Kobo's nix.conf lists harmonia as primary substituter.
  4. On the Kobo: nix-store -r /nix/store/HASH-activation-script pulls just what's needed.
  5. Then <path>/activate.

The "what's the current path" problem reduces to publishing a small text file on the build host (e.g. http://nixai.tailnet/cyberdeck-latest) after every build, and the Kobo's cyberdeck-update becomes a curl | xargs one-liner.

Tradeoff: harmonia GC. If the build host prunes its store before the Kobo pulls, the device gets a 404. Mitigate by pinning the current cyberdeck generation as a profile on the build host.

Flake Patterns Worth Knowing

A grab-bag of small things that bit me or saved me. Each one is the kind of detail that's nowhere in the official docs and shows up in five different forum threads instead.

  • git add -N (intent-to-add) is the escape hatch for flake-tracked files. Flakes only see git-tracked files because they hash the tree state. git add -N path/ registers the path with git's index without staging the content. The file becomes "tracked enough" for Nix to see it without polluting your commit history while you iterate.
  • nix eval validates shape; nix build validates contents. Reading .config.cyberdeck.device forces only the module system, not the package derivations. Successful eval means imports resolved, types are right, conditional logic works. Build is what proves the closure actually links. Use nix eval as your fast feedback loop; reserve nix build for committing to the cross-compile cost.
  • boot.binfmt.emulatedSystems does not need a reboot. Common misconception. The binfmt_misc kernel module is already loaded on any standard Linux. NixOS's activation writes /etc/binfmt.d/*.conf and restarts systemd-binfmt.service. ls /proc/sys/fs/binfmt_misc/ | grep qemu-arm confirms the registration.
  • Assertions are eval-time fail-fast. Use assertions = [{ assertion = ...; message = "..."; }] to refuse impossible configs. The cross-compile module uses one to catch "host's own arch in cross.targets" — a no-op that always means the user is confused. Cheap to write, expensive to retrofit after you've shipped something subtle.
  • Turn raw NixOS option values into domain language the moment you reuse them across hosts. cross.targets = [ "armv7l-linux" ]; reads as "this host can build for armv7l." boot.binfmt.emulatedSystems = [ "armv7l-linux" ]; reads as a mechanism. The first form scales; the second is fine for one host and noise across many.
  • overrideAttrs on a wrapper does not propagate to inner derivations. If you override tmuxPlugins.fingers at the wrapper level, the inner Rust crate's meta.platforms still excludes armv7l. Conditional drop at the consumer site (lib.optionals (pkgs.stdenv.hostPlatform.system != "armv7l-linux")) is cleaner than fighting the override.
  • overridePythonAttrs strips .override from its result. If anything in the closure forces evaluation of pkgs.meson.override (e.g. python-packages.nix's toPythonModule wrapper), the whole eval explodes. Use overrideAttrs for stdenv attrs like doCheck even on Python packages, unless you genuinely need overridePythonAttrs.
  • Perl scope overrides need overrideScope, not .override. perlPackages.override { overrides = pkgs: { Foo = ...; }; } creates infinite recursion because Perl's passthruFun computes its scope as perlPackages // (overrides pkgs) — the pkgs arg IS the post-override scope. perl540Packages.overrideScope (pfinal: pprev: { ... }) gives you a proper layered fixed point.
  • Some packages are renamed in attr-space but keep their dashed display name. boehm-gc in build output is boehmgc in the attrset. source-highlight is sourceHighlight. No consistent rule. When in doubt, nix eval --raw .#pkgs.<name>.drvPath and trust the "did you mean...?" suggestion.

Quick Reference

Files and What They Own

PathOwns
flake.nixcyberdeckConfigurations.kobo output, wires cross-compile.nix into the build host's NixOS config
nixos/cross-compile.nixLayer 1: cross.targets API; binfmt + gccarch system-features in one place
overlays/qemu-armv7l.nixLayer 2: ~21 package overrides for qemu-user emulation gaps, with one-paragraph "Why:" comments
nixos/cyberdeck/default.nixDevice-agnostic kit: imports home/{fish,tmux,neovim,git}.nix; adds mosh + openssh + ripgrep + fd + fzf + bat + jq + curl + git + tealdeer + ...; ships cyberdeck-shell launcher; post-activation GC hook
nixos/cyberdeck/profiles/kobo.nixKobo-specific: nixRoot = "/mnt/onboard/.adds/nix", tailscale.userspaceNetworking = true
nixos/cyberdeck/profiles/kindle.nixPlaceholder for the Kindle target (Bluetooth-HID lockdown is a separate fight)
nixos/cyberdeck/bootstrap.shLayer 6: arch + device detection; symlink probe; ext4 loopback; nix.conf seeding; profile + activation symlinks; persistence reminder
home/neovim.nixLean neovim core — slim extraPackages, all plugins
home/neovim/strudel.nixDesktop-only heavy deps (chromium, nodejs, npm) next to the strudel.nvim plugin that needs them
home/neovim/python.nixOpt-in pynvim — import only if a plugin actually needs the Python ↔ neovim RPC bridge

When the Device Won't Take the Closure

SymptomLayerFix
Failed to find a machine for remote build! (..., [gccarch-armv7-a])1Add gccarch features to extra-system-features
Package builds fine, test suite fails2doCheck = false in overlay
cannot bootstrap GHC on this platformabove 2Find a non-Haskell substitute
your_linux_architecture_is_not_supported_by_X sentinelabove 2Strip the dep from the consumer's postPatch
EOVERFLOW from readdir() mid-buildabove 2-D_FILE_OFFSET_BITS=64 via NIX_CFLAGS_COMPILE
sh: nix-store: not found from nix copy5First push uses tar
tar: can't create symlink ... Operation not permitted6bLoop-mount ext4-in-a-file on FAT
scp: write remote ... Bad message at ~4 GB6dImage strictly under 4 GiB

References

Prior Art (the Giants This Stands On)

Tools and Projects the Build Sits On

Reference / Docs

Companion Posts

What's Not in This Cookbook

  • KFMon launcher entry. The bootstrap currently has to re-run after every reboot because loop mounts don't survive. KFMon-on-launch is the clean fix; I haven't wired it up yet.
  • pkgsCross.armv7l-hf-multiplatform migration. A true cross-compile (Layer 3) instead of qemu-emulated-native would be faster but more invasive — many packages have shaky pkgsCross support and would need their own overrides.
  • Pull-from-harmonia. Sketched above, not yet running.
  • Kindle profile. A placeholder for the same shape on a jailbroken Kindle. The Bluetooth-HID lockdown is a separate fight.

The flake isn't public — it carries personal infra alongside the cyberdeck bits — but if you're actually building one, email me@jquaintance.com and I'll send a snapshot of overlays/qemu-armv7l.nix, nixos/cross-compile.nix, and nixos/cyberdeck/.

Header photo by Amanz on Unsplash.

Content on this blog was created using human and AI-assisted workflows described here. Original ideas and editorial decisions by Justin Quaintance.