diff --git a/.gitignore b/.gitignore index dfc4fbd9..d4b58937 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,7 @@ srv.pid __pycache__ result* + +# Draw IO diagram backups +*.bkp +*.dtmp diff --git a/content/2023-11-20-Raspberry-Pi-Github-Actions.markdown b/content/2023-11-20-Raspberry-Pi-Github-Actions.markdown new file mode 100644 index 00000000..43c22a95 --- /dev/null +++ b/content/2023-11-20-Raspberry-Pi-Github-Actions.markdown @@ -0,0 +1,93 @@ +--- +title: Building Nix packages for the Raspberry Pi with GitHub Actions +--- + +Building Nix packages for the Raspberry Pi 3 or newer requires building for an +ARM 64 architecture, which Nix refers to as `aarch64-linux`. + +To build `aarch64-linux` binaries we can: + +1. Build natively on an `aarch64-linux` machine. +1. Cross compile for `aarch64-linux`. +1. Compile with an emulator. + +The first option is the simplest. For example, a Raspberry Pi with Nix +installed compiles the `aarch64-linux` binaries reasonably well. Typically, you +need to build only a few packages from source and the rest may be downloaded +from the [Nix public binary cache](https://cache.nixos.org). You can also use +a more powerful ARM 64 computer for building, if you own or rent one. + +I have little to say about the second option because I never managed to get it +working. I also suspect that cross compiled packages are not in the public +binary cache, so build times are longer than native builds. + +In the rest of the article I explore the third option: compiling with an +emulator, in particular with QEMU. + +# On NixOS + +On NixOS, it's trivial to use QEMU to build for different architectures. You +need to [adjust one parameter][EmulatedSystems] in your NixOS host +configuration: + +```nix +boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; +``` + +This snippet installs and configures QEMU and enables emulated builds for the +Raspberry Pi. For example, to build +[hello](https://www.gnu.org/software/hello/) from source, run: + +``` +nix build nixpkgs#legacyPackages.aarch64-linux.hello --no-substitute +``` + +For this demonstration I added the `--no-substitute` flag to disallow binary +substitutes. In other words, this flag forces Nix to build all packages from +source. + +If the compilation with the emulated tool chain works, Nix writes an ARM 64-bit +binary at `./result/bin/hello` + +# Using GitHub Actions + +Recently I discovered the [setup-qemu-action][QEMUAction] from Docker which +helps us to configure a hosted GitHub Action runner to build for the Raspberry +Pi: + +```yaml +jobs: + build: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: docker/setup-qemu-action@v3 # ⑴ + - uses: cachix/install-nix-action@v23 # ⑵ + with: + extra_nix_config: "extra-platforms = aarch64-linux" + -run: | # ⑶ + nix build nixpkgs#legacyPackages.aarch64-linux.hello --no-substitute +``` + +This GitHub Actions workflow starts an Ubuntu virtual machine and Installs Nix +on it for building an `aarch64-linux` package: + +1. Install the QEMU static binaries using a [GitHub Action from + Docker][QEMUAction]. +1. Install Nix using a [GitHub Action from Cachix][NixAction] and configure it + to allow building for `aarch64-linux`. +1. Build `hello` for `aarch64-linux`. + +# Summary + +Using two GitHub Actions from [Docker][QEMUAction] and [Cachix][NixAction] I +set up a workflow to build packages for the Raspberry Pi using freely available +Ubuntu runners from GitHub. + +In a [related article][HomelabDeployment] I explain how I use this technique to +build and deploy NixOS on Raspberry Pi. + +[EmulatedSystems]: https://nixos.org/manual/nixos/stable/options#opt-boot.binfmt.emulatedSystems +[QEMUAction]: https://github.com/marketplace/actions/docker-setup-qemu +[NixAction]: https://github.com/cachix/install-nix-action +[HomelabDeployment]: {filename}/2023-11-25-Homelab-deployment.markdown diff --git a/content/2023-11-25-Homelab-deployment.markdown b/content/2023-11-25-Homelab-deployment.markdown new file mode 100644 index 00000000..6298583e --- /dev/null +++ b/content/2023-11-25-Homelab-deployment.markdown @@ -0,0 +1,171 @@ +--- +title: Homelab deployment +--- + +I configured continuous deployment in [my homelab][Homelab]. This article +describes how I use Nix with GitHub Actions and Cachix Deploy to automatically +deploy NixOS machines on my home network. + +# Overview + +My home network comprises a wirelesss router, a few computers, temperature and +humidity sensors and remote controllable switches. I configure these devices +mainly using Nix, and when it's possible, I run NixOS on them. I use this +setup to experiment with [home automation and to learn about new +tools][Homelab]. The entire configuration of my home network is [available on +GitHub][HomelabRepo]. + +The configuration of the NixOS servers was always built from a declarative +specification stored in version control system. But the deployment of the new +configuration was manual, slow and often tedious. I ran `nixos-build` from the +command line, and sometimes I had to wait an hour to build and deploy the new +machine configuration. + +I wanted to reconfigure my servers automatically, triggered by committing to a +Git repository. + +To solve this problem I built a system based on Nix and freely available hosted +services: + +![Figure1]({static}/images/homelab-deployment.svg "Homelab deployment") + +The automatic deployments work as follows: + +1. The [Homelab repository][HomelabRepo] contains the NixOS host configurations. +1. A workflow in [GitHub Action][HomelabAction] _builds_ the NixOS system and + _stores_ it in a [hosted binary cache](https://www.cachix.org/). +1. An agent on the target machine _pulls_ the built binaries from the cache and + _activates_ the new deployment. + +# Host configuration + +The configuration of my servers is written in the Nix language. For example, +[host-nuc.nix][HomelabNuc] describes the hardware and software configuration of +my Intel NUC device: + +```nix +{ config, ... }: + +{ + imports = [ + ./hardware/nuc.nix + ./modules/cachix.nix + ./modules/common.nix + ./modules/consul/server.nix + ./modules/git.nix + ./modules/grafana + ./modules/loki.nix + ./modules/mqtt.nix + ./modules/prometheus.nix + ./modules/push-notifications.nix + ./modules/remote-builder + ./modules/traefik.nix + ./modules/vpn.nix + ]; + + system.stateVersion = "22.05"; +} +``` + +The configuration is split into modules. Glancing at the `imports` block you +can tell what is installed on this machine. The host `nuc` is my main home +server, it runs all my "production" services. + +For example, the `cachix.nix` module configures the [Cachix +Agent](https://docs.cachix.org/deploy/running-an-agent/) which is responsible +for reconfiguring the NixOS machine when a new build is available. The agent +runs on all machines whose configuration automatically managed. + +# Building with GitHub Actions + +The NixOS host specifications are built with a single `nix build` command: + +``` +nix build .#nixosConfigurations.nuc.config.system.build.toplevel +``` + +This builds the whole system for the `nuc` machine: the kernel, the installed +packages and their configuration. By default, `nix` downloads pre-built +packages from the [NixOS public binary cache](http://cache.nixos.org/), so a +typical execution of the build command takes only a few minutes. + +To run the build in GitHub Actions, the build job installs Nix and configures +the access to the Cachix binary cache where the deployment artifacts are +stored: + +```yaml +jobs: + build: + runs-on: ubuntu-latest + environment: + name: Homelab + url: "https://app.cachix.org/deploy/workspace/lab.thewagner.home/" # ⑴ + steps: + - uses: actions/checkout@v4 + - uses: docker/setup-qemu-action@v3 # ⑵ + - uses: cachix/install-nix-action@v23 + with: + extra_nix_config: "extra-platforms = aarch64-linux" # ⑶ + - uses: cachix/cachix-action@v12 # ⑷ + with: + name: wagdav + authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' + - run: nix build --print-build-logs .#cachix-deploy-spec # ⑸ + - run: | # ⑹ + cachix push wagdav ./result + cachix deploy activate --async ./result + env: + CACHIX_ACTIVATE_TOKEN: "${{ secrets.CACHIX_ACTIVATE_TOKEN }}" +``` + +The `build` job of the [workflow][HomelabAction]: + +1. Configures a [deployment target][GitHubEnvironment]. When a GitHub Actions + workflow deploys to an environment, the environment is displayed on the main + page of the repository. +1. Installs the QEMU static binaries for building packages for architectures + different than that of the build runner. +1. Configures Nix to use emulation to [build ARM 64 packages][QEMUBuilds]. +1. Configures the [Cachix hosted binary cache](https://cachix.org). +1. Builds the [deploy specification][DeploySpec] which is a set of the NixOS + systems to deploy. +1. Pushes the built binaries to the cache and sends an activation signal to the + Cachix Agent. + +The job uses two secrets: `CACHIX_AUTH_TOKEN` is the authentication token to +push to the binary cache and `CACHIX_ACTIVATE_TOKEN` is required to activate +the built NixOS configurations. + +# Deploying with Cachix Deploy + +To deploy the built NixOS configurations I use the generous free-tier of +[Cachix Deploy](https://docs.cachix.org/deploy/). Following their +documentation, I installed `cachix-agent` on the target hosts and configured a +few authentication keys. + +The agent process connects to the Cachix backend and waits for a deployment. +When a new deployment is available, the agent pulls the relevant binaries from +the binary cache and reconfigures the NixOS system it runs on. + +# Summary + +Since I configured [automatic deployment of this blog][BlogDeployment] I wanted +the same for my home infrastructure. I had my configuration repository and I +knew how to build the machine configurations with GitHub Actions. I was +missing the automatic deployment part until Cachix Deploy was announced. +Cachix has excellent documentation and the integration with my Homelab was +super simple. + +# Acknowledgement + +I'm grateful to [Cachix Deploy](https://www.cachix.org/) for offering a binary +cache and a deployment service. + +[BlogDeployment]: {filename}/2020-12-06-Blog-deployment-update.markdown +[DeploySpec]: https://docs.cachix.org/deploy/deploying-to-agents/#write-deploy-specification +[GitHubEnvironment]: https://docs.github.com/en/actions/deployment/about-deployments/deploying-with-github-actions#using-environments +[HomelabAction]: https://github.com/wagdav/homelab/blob/master/.github/workflows/build-and-deploy.yml +[HomelabNuc]: https://github.com/wagdav/homelab/blob/master/host-nuc.nix +[HomelabRepo]: https://github.com/wagdav/homelab +[Homelab]: {filename}/2020-05-31-Homelab.markdown +[QEMUBuilds]: {filename}/2023-11-20-Raspberry-Pi-Github-Actions.markdown diff --git a/content/images/homelab-deployment.svg b/content/images/homelab-deployment.svg new file mode 100644 index 00000000..d4a1d556 --- /dev/null +++ b/content/images/homelab-deployment.svg @@ -0,0 +1,4 @@ + + + +
Cachix
Cachix
GitHub Actions
GitHub Actions
NixOS
system
NixOS...
NixOS machine
NixOS machine
cachix-agent
cachix-agent
activate
activate
Homelab repository
Homelab repository
NixOS configuration
NixOS configurat...
pull
pull
Binary cache
Binary cache
build
build
store
storeText is not SVG - cannot display \ No newline at end of file