-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Building Nix packages for the Raspberry Pi in GitHub Actions * Homelab continous deployment
- Loading branch information
Showing
4 changed files
with
271 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -7,3 +7,7 @@ srv.pid | |
__pycache__ | ||
|
||
result* | ||
|
||
# Draw IO diagram backups | ||
*.bkp | ||
*.dtmp |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,93 @@ | ||
--- | ||
title: Building Nix packages for the Raspberry Pi with GitHub Actions | ||
--- | ||
|
||
Building Nix packages for the Raspberry Pi 3 or newer requires to build for an | ||
ARM 64 architecture, which Nix refers to as `aarch64-linux`. | ||
|
||
To build `aarch64-linux` binaries we can: | ||
|
||
1. Build natively on an `aarch64-linux` machine. | ||
1. Cross compile for `aarch64-linux`. | ||
1. Compile with an emulator. | ||
|
||
The first option is the simplest. For example, a Raspberry Pi with Nix | ||
installed compiles the `aarch64-linux` binaries reasonably well. Typically you | ||
need to build only a few packages from source and the rest may be downloaded | ||
from the [Nix public binary cache](https://cache.nixos.org). You can also use | ||
a more powerful ARM 64 computer for building, if you own or rent one. | ||
|
||
I have little to say about the second option because I never managed to get it | ||
working. I also suspect that cross compiled packages are not in the public | ||
binary cache, so build times are longer than native builds. | ||
|
||
In the rest of the article I explore the third option: compiling with an | ||
emulator, in particular with QEMU. | ||
|
||
# On NixOS | ||
|
||
On NixOS it's trivial to use QEMU to build for different architectures. You | ||
need to [adjust one parameter][EmulatedSystems] in your NixOS host | ||
configuration: | ||
|
||
```nix | ||
boot.binfmt.emulatedSystems = [ "aarch64-linux" ]; | ||
``` | ||
|
||
This snippet installs and configures QEMU and enables emulated builds for the | ||
Raspberry Pi. For example, to build | ||
[hello](https://www.gnu.org/software/hello/) from source, run: | ||
|
||
``` | ||
nix build nixpkgs#legacyPackages.aarch64-linux.hello --no-substitute | ||
``` | ||
|
||
For this demonstration I added the `--no-substitute` flag to disallow binary | ||
substitutes. In other words, this flag forces Nix to build all packages from | ||
source. | ||
|
||
If the compilation with the emulated tool chain works, Nix writes an ARM 64-bit | ||
binary at `./result/bin/hello` | ||
|
||
# Using GitHub Actions | ||
|
||
Recently I discovered the [setup-qemu-action][QEMUAction] from Docker which | ||
helps us configuring a hosted GitHub Action runner to build for the Raspberry | ||
Pi: | ||
|
||
```yaml | ||
jobs: | ||
build: | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- uses: docker/setup-qemu-action@v3 # ⑴ | ||
- uses: cachix/install-nix-action@v23 # ⑵ | ||
with: | ||
extra_nix_config: "extra-platforms = aarch64-linux" | ||
-run: | # ⑶ | ||
nix build nixpkgs#legacyPackages.aarch64-linux.hello --no-substitute | ||
``` | ||
This GitHub Actions workflow starts an Ubuntu virtual machine and Installs Nix | ||
on it for building an `aarch64-linux` package: | ||
|
||
1. Install the QEMU static binaries using a [GitHub Action from | ||
Docker][QEMUAction]. | ||
1. Install Nix using a [GitHub Action from Cachix][NixAction] and configure it | ||
to allow building for `aarch64-linux`. | ||
1. Build `hello` for `aarch64-linux`. | ||
|
||
# Summary | ||
|
||
Using two GitHub Actions from [Docker][QEMUAction] and [Cachix][NixAction] we | ||
set up a workflow to build a packages for the Raspberry Pi using freely | ||
available Ubuntu runners from GitHub. | ||
|
||
In a [related article][HomelabDeployment] I explain how I use this technique to | ||
build and deploy NixOS on Raspberry Pi. | ||
|
||
[EmulatedSystems]: https://nixos.org/manual/nixos/stable/options#opt-boot.binfmt.emulatedSystems | ||
[QEMUAction]: https://github.com/marketplace/actions/docker-setup-qemu | ||
[NixAction]: https://github.com/cachix/install-nix-action | ||
[HomelabDeployment]: {filename}/2023-11-25-Homelab-deployment.markdown |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,170 @@ | ||
--- | ||
title: Homelab deployment | ||
--- | ||
|
||
I configured continous deployment on [computers in my homelab][Homelab]. In | ||
this article I describe how I use Nix with GitHub Actions and Cachix Deploy to | ||
deploy computers on my home network. | ||
|
||
# Overview | ||
|
||
At home I have a wirelesss router, a few computers, temperature and humidity | ||
sensors and remote controllable switches. I configure these devices mainly | ||
using Nix, and where it's possible, I run NixOS on them. I use this setup to | ||
experiment with home automation and to learn about new tools. In 2020, I wrote | ||
an [article about the initial setup][Homelab] and all configuration data is | ||
[available on GitHub][HomelabRepo]. | ||
|
||
The configuration of the NixOS servers was always built from a declarative | ||
specification stored in version control system. But the deployment of the new | ||
configuration was manual, slow and often tedious. I ran `nixos-build` from the | ||
command line and sometimes I had to wait an hour to build and deploy the new | ||
machine configuration. | ||
|
||
I wanted to reconfigure my servers automatically, triggered by a commit in a | ||
Git repository. | ||
|
||
The rest of the article describes an automatic deployment system for my Homelab | ||
which looks like this: | ||
|
||
![Figure1]({static}/images/homelab-deployment.svg "Homelab deployment") | ||
|
||
A deployment comprises the following activities: | ||
|
||
1. The [Homelab repository][HomelabRepo] contains the NixOS host configurations. | ||
1. A workflow in [GitHub Action][HomelabAction] _builds_ the NixOS system and | ||
_stores_ it in a [hosted binary cache](https://www.cachix.org/). | ||
1. An agent on the target machine _pulls_ the built binaries from the binary | ||
cache and _activates_ the new deployment. | ||
|
||
Let's see the system's components in detail. | ||
|
||
# Host configuration | ||
|
||
The configuration of my servers is written in the Nix language. For example, | ||
the file [host-nuc.nix][HomelabNuc] describes the hardware and software | ||
configuration of my Intel NUC device: | ||
|
||
```nix | ||
{ config, ... }: | ||
{ | ||
imports = [ | ||
./hardware/nuc.nix | ||
./modules/cachix.nix | ||
./modules/common.nix | ||
./modules/consul/server.nix | ||
./modules/git.nix | ||
./modules/grafana | ||
./modules/loki.nix | ||
./modules/mqtt.nix | ||
./modules/prometheus.nix | ||
./modules/push-notifications.nix | ||
./modules/remote-builder | ||
./modules/traefik.nix | ||
./modules/vpn.nix | ||
]; | ||
system.stateVersion = "22.05"; | ||
} | ||
``` | ||
|
||
The configuration is split into modules. Glancing at the `imports` block you | ||
can tell what is installed on this machine. The host `nuc` is my main home | ||
server, it runs all my "production" services. | ||
|
||
For example, the `cachix.nix` module configures the [Cachix | ||
Agent](https://docs.cachix.org/deploy/running-an-agent/) which is responsible | ||
for reconfiguring the NixOS machine when a new build is available. The agent | ||
runs on all machines whose configuration automatically managed. | ||
|
||
# Building with GitHub Actions | ||
|
||
The NixOS host specifications are built with a single `nix build` command. For | ||
example, the command: | ||
|
||
``` | ||
nix build .#nixosConfigurations.nuc.config.system.build.toplevel | ||
``` | ||
|
||
builds the entire root file system of the `nuc` machine: the kernel, the installed | ||
packages and their configuration. By default, `nix` downloads pre-built packages | ||
from the [NixOS public binary cache](http://cache.nixos.org/), so a typical | ||
execution of the build command takes only a few minutes. | ||
|
||
To run the build in GitHub Actions, the build job installs Nix and configures | ||
the access to the Cachix binary cache where the deployment artifacts are | ||
stored: | ||
|
||
```yaml | ||
jobs: | ||
build: | ||
runs-on: ubuntu-latest | ||
environment: | ||
name: Homelab | ||
url: "https://app.cachix.org/deploy/workspace/lab.thewagner.home/" # ⑴ | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- uses: docker/setup-qemu-action@v3 # ⑵ | ||
- uses: cachix/install-nix-action@v23 | ||
with: | ||
extra_nix_config: "extra-platforms = aarch64-linux" # ⑶ | ||
- uses: cachix/cachix-action@v12 # ⑷ | ||
with: | ||
name: wagdav | ||
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}' | ||
- run: nix build --print-build-logs .#cachix-deploy-spec # ⑸ | ||
- run: | # ⑹ | ||
cachix push wagdav ./result | ||
cachix deploy activate --async ./result | ||
env: | ||
CACHIX_ACTIVATE_TOKEN: "${{ secrets.CACHIX_ACTIVATE_TOKEN }}" | ||
``` | ||
The [entire workflow][HomelabAction] is 40 lines, here I show only the `build` | ||
job which: | ||
|
||
1. Configures a [deployment target][GitHubEnvironment]. When a GitHub Actions | ||
workflow deploys to an environment, the environment is displayed on the main | ||
page of the repository. | ||
1. Installs the QEMU static binaries for emulating systems with architectures | ||
than that of the of the build runner. | ||
1. Configures Nix to use emulation for building. | ||
1. Configures the [Cachix hosted binary cache](https://cachix.org). | ||
1. Builds the NixOS systems. | ||
1. Pushes the built binaries to the cache and sends an activation signal to the | ||
Cachix Agent. | ||
|
||
The job uses two secrets: `CACHIX_AUTH_TOKEN` is the authentication token to | ||
push to the binary cache and `CACHIX_ACTIVATE_TOKEN` is required to activate | ||
the built NixOS configurations. | ||
|
||
# Deploying with Cachix Deploy | ||
|
||
To deploy built NixOS configurations I use the generous free-tier of [Cachix | ||
Deploy](https://docs.cachix.org/deploy/). The system needs the `cachix-agent` | ||
running on the target host, a few authentication keys and it just simply works. | ||
|
||
The agent process connects to the Cachix backend and waits for a deployment. | ||
When a new deployment is available, the agent pulls the relevant binaries from | ||
the binary cache and reconfigures the NixOS system it runs on. | ||
|
||
# Summary | ||
|
||
Since I configured [automatic deployment of this blog][BlogDeployment] I wanted | ||
the same for my home infrastructure. I had my configuration repository and I | ||
knew how to build the machine configurations with GitHub Actions. I was | ||
missing the automatic deployment part until Cachix Deploy was announced. Cachix | ||
has excellent documentation and I integration with my Homelab was super simple. | ||
|
||
# Acknowledgement | ||
|
||
I'm grateful to [Cachix Deploy](https://www.cachix.org/) for offering a binary | ||
cache and a deployment service. | ||
|
||
[BlogDeployment]: {filename}/2020-12-06-Blog-deployment-update.markdown | ||
[HomelabAction]: https://github.com/wagdav/homelab/blob/master/.github/workflows/build-and-deploy.yml | ||
[HomelabNuc]: https://github.com/wagdav/homelab/blob/master/host-nuc.nix | ||
[HomelabRepo]: https://github.com/wagdav/homelab | ||
[Homelab]: {filename}/2020-05-31-Homelab.markdown | ||
[GitHubEnvironment]: https://docs.github.com/en/actions/deployment/about-deployments/deploying-with-github-actions#using-environments |
Oops, something went wrong.