nixos netboot

builds x86_64 uefi nixos images in forgejo actions and pushes them to a nixos server running nginx and tftp

the client runs entirely from ram. it does not install anything or touch local disks unless you manually tell it to

how it works

1. dhcp tells the machine to get ipxe.efi from 10.0.0.9
2. ipxe gets an address and loads the image menu from 10.0.0.9
3. the menu remembers the last choice for that machine's mac address
4. the selected kernel and initrd are downloaded over http
5. nixos boots with a tmpfs root and an embedded squashfs nix store

the current images are:

• bigscreen - autologin plasma bigscreen media center
• snapclient - headless snapcast client

the menu waits 10 seconds and boots the last image selected by that machine. new mac addresses default to bigscreen. choices are stored on kalia in /var/lib/netboot-menu/choices.json, not in client firmware or on client disks

the initrd is pretty large, so clients need at least 4gb of ram. 8gb or more is recommended

layout

• nixos/netboot-client.nix - the nixos config that clients boot
• nixos/images - the bigscreen and snapclient configs
• nixos/netboot-server.nix - nginx, tftp, and the restricted deploy user
• hosts/netboot-server - config for kalia, the current netboot server
• ipxe/chain.ipxe - script embedded into ipxe
• scripts/deploy.sh - builds and uploads a release
• .forgejo/workflows/publish.yml - does the same thing in ci

client config

edit nixos/netboot-client.nix to change what gets booted

site-specific values are in site.nix. set snapserver there before using the snapclient image

you should probably add your ssh key before using it:

users.users.root.openssh.authorizedKeys.keys = [
  "ssh-ed25519 AAAA..."
];

packages and services can be added like any other nixos config. changes are lost when the client reboots because the writable filesystems are tmpfs

building

nix build .#netboot
nix build .#ipxe-efi

netboot contains both image directories, kernels, initrds, and generated boot.ipxe files

ipxe-efi contains the uefi ipxe binary with the generated chain script embedded in it

server

the current server is kalia at 10.0.0.9. its config is in hosts/netboot-server/configuration.nix

to rebuild it:

nixos-rebuild switch --flake .#netboot-server

the reusable module can also be imported into another nixos config:

{
  imports = [ inputs.nixos-netboot.nixosModules.netboot-server ];

  services.nixos-netboot = {
    enable = true;
    deployKeys = [
      "ssh-ed25519 AAAA... forgejo-netboot"
    ];
  };
}

this opens tcp 80 for http and udp 69 for tftp. releases are stored at /srv/netboot/releases/<git sha> and switched atomically so a client cannot get half of an upload

opnsense

this setup uses the existing opnsense dnsmasq dhcp server. kalia does not run dhcp

the dhcp boot config should be:

next server / tftp server: 10.0.0.9
boot filename:             ipxe.efi

the generated dnsmasq line looks like this:

dhcp-boot=tag:bce0,ipxe.efi,10.0.0.9,10.0.0.9

secure boot needs to be disabled unless you sign ipxe.efi

forgejo

the workflow runs on the docker runner label. it installs nix in the job container, so the container needs to run as root and have tar and ssh

repository variables:

NETBOOT_TARGET=netboot@kalia.network.orcachill.in

the lan address, gateway, interface, and future snapserver address live in site.nix. the deploy script reads the http base url from the flake, so it does not need to be duplicated in forgejo

repository secrets:

• NETBOOT_DEPLOY_KEY - the private forgejo deploy key
• NETBOOT_KNOWN_HOSTS - output from ssh-keyscan kalia.network.orcachill.in

the deploy key is forced to run the upload receiver and cannot open a normal shell

every push to main builds and publishes a new release. it can also be run manually from the actions page

manual deploy

export NETBOOT_TARGET=netboot@kalia.network.orcachill.in
./scripts/deploy.sh "$(git rev-parse HEAD)"

testing

the full path has been tested in a diskless proxmox vm with ovmf and 4gb of ram

if proxmox refuses to expose network boot, the built ipxe.efi can be put at EFI/BOOT/BOOTX64.EFI on a small fat image and attached as a usb disk. this still tests dhcp, http, and the nixos image, it just skips the firmware tftp step

useful checks:

curl 'http://10.0.0.9/menu.ipxe?mac=bc:24:11:7b:64:34'
ssh root@kalia.network.orcachill.in

shared static assets live in /srv/netboot/static on kalia. mount them from a linux machine on the lan with:

sudo mkdir -p /mnt/netboot-static
sudo mount -t nfs 10.0.0.9:/srv/netboot/static /mnt/netboot-static

wallpapers go in /mnt/netboot-static/wallpapers. clients discover image files there automatically, so adding or removing one does not require rebuilding

old releases are kept around for rollback. tftp and http are unauthenticated, so this is intended for a trusted lan