Skip to content

Commit

Permalink
Add two posts.
Browse files Browse the repository at this point in the history
* Building Nix packages for the Raspberry Pi in GitHub Actions
* Homelab continous deployment
  • Loading branch information
wagdav committed Nov 26, 2023
1 parent d306cbc commit 6ba2d40
Show file tree
Hide file tree
Showing 4 changed files with 272 additions and 0 deletions.
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,3 +7,7 @@ srv.pid
__pycache__

result*

# Draw IO diagram backups
*.bkp
*.dtmp
93 changes: 93 additions & 0 deletions content/2023-11-20-Raspberry-Pi-Github-Actions.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
---
title: Building Nix packages for the Raspberry Pi with GitHub Actions
---

Building Nix packages for the Raspberry Pi 3 or newer requires building for an
ARM 64 architecture, which Nix refers to as `aarch64-linux`.

To build `aarch64-linux` binaries we can:

1. Build natively on an `aarch64-linux` machine.
1. Cross compile for `aarch64-linux`.
1. Compile with an emulator.

The first option is the simplest. For example, a Raspberry Pi with Nix
installed compiles the `aarch64-linux` binaries reasonably well. Typically, you
need to build only a few packages from source and the rest may be downloaded
from the [Nix public binary cache](https://cache.nixos.org). You can also use
a more powerful ARM 64 computer for building, if you own or rent one.

I have little to say about the second option because I never managed to get it
working. I also suspect that cross compiled packages are not in the public
binary cache, so build times are longer than native builds.

In the rest of the article I explore the third option: compiling with an
emulator, in particular with QEMU.

# On NixOS

On NixOS, it's trivial to use QEMU to build for different architectures. You
need to [adjust one parameter][EmulatedSystems] in your NixOS host
configuration:

```nix
boot.binfmt.emulatedSystems = [ "aarch64-linux" ];
```

This snippet installs and configures QEMU and enables emulated builds for the
Raspberry Pi. For example, to build
[hello](https://www.gnu.org/software/hello/) from source, run:

```
nix build nixpkgs#legacyPackages.aarch64-linux.hello --no-substitute
```

For this demonstration I added the `--no-substitute` flag to disallow binary
substitutes. In other words, this flag forces Nix to build all packages from
source.

If the compilation with the emulated tool chain works, Nix writes an ARM 64-bit
binary at `./result/bin/hello`

# Using GitHub Actions

Recently I discovered the [setup-qemu-action][QEMUAction] from Docker which
helps us to configure a hosted GitHub Action runner to build for the Raspberry
Pi:

```yaml
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3 #
- uses: cachix/install-nix-action@v23 #
with:
extra_nix_config: "extra-platforms = aarch64-linux"
-run: | #
nix build nixpkgs#legacyPackages.aarch64-linux.hello --no-substitute
```
This GitHub Actions workflow starts an Ubuntu virtual machine and Installs Nix
on it for building an `aarch64-linux` package:

1. Install the QEMU static binaries using a [GitHub Action from
Docker][QEMUAction].
1. Install Nix using a [GitHub Action from Cachix][NixAction] and configure it
to allow building for `aarch64-linux`.
1. Build `hello` for `aarch64-linux`.

# Summary

Using two GitHub Actions from [Docker][QEMUAction] and [Cachix][NixAction] I
set up a workflow to build packages for the Raspberry Pi using freely available
Ubuntu runners from GitHub.

In a [related article][HomelabDeployment] I explain how I use this technique to
build and deploy NixOS on Raspberry Pi.

[EmulatedSystems]: https://nixos.org/manual/nixos/stable/options#opt-boot.binfmt.emulatedSystems
[QEMUAction]: https://github.com/marketplace/actions/docker-setup-qemu
[NixAction]: https://github.com/cachix/install-nix-action
[HomelabDeployment]: {filename}/2023-11-25-Homelab-deployment.markdown
171 changes: 171 additions & 0 deletions content/2023-11-25-Homelab-deployment.markdown
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
---
title: Homelab deployment
---

I configured continuous deployment in [my homelab][Homelab]. This article
describes how I use Nix with GitHub Actions and Cachix Deploy to automatically
deploy NixOS machines on my home network.

# Overview

My home network comprises a wirelesss router, a few computers, temperature and
humidity sensors and remote controllable switches. I configure these devices
mainly using Nix, and when it's possible, I run NixOS on them. I use this
setup to experiment with [home automation and to learn about new
tools][Homelab]. The entire configuration of my home network is [available on
GitHub][HomelabRepo].

The configuration of the NixOS servers was always built from a declarative
specification stored in version control system. But the deployment of the new
configuration was manual, slow and often tedious. I ran `nixos-build` from the
command line, and sometimes I had to wait an hour to build and deploy the new
machine configuration.

I wanted to reconfigure my servers automatically, triggered by committing to a
Git repository.

To solve this problem I built a system based on Nix and freely available hosted
services:

![Figure1]({static}/images/homelab-deployment.svg "Homelab deployment")

The automatic deployments work as follows:

1. The [Homelab repository][HomelabRepo] contains the NixOS host configurations.
1. A workflow in [GitHub Action][HomelabAction] _builds_ the NixOS system and
_stores_ it in a [hosted binary cache](https://www.cachix.org/).
1. An agent on the target machine _pulls_ the built binaries from the cache and
_activates_ the new deployment.

# Host configuration

The configuration of my servers is written in the Nix language. For example,
[host-nuc.nix][HomelabNuc] describes the hardware and software configuration of
my Intel NUC device:

```nix
{ config, ... }:
{
imports = [
./hardware/nuc.nix
./modules/cachix.nix
./modules/common.nix
./modules/consul/server.nix
./modules/git.nix
./modules/grafana
./modules/loki.nix
./modules/mqtt.nix
./modules/prometheus.nix
./modules/push-notifications.nix
./modules/remote-builder
./modules/traefik.nix
./modules/vpn.nix
];
system.stateVersion = "22.05";
}
```

The configuration is split into modules. Glancing at the `imports` block you
can tell what is installed on this machine. The host `nuc` is my main home
server, it runs all my "production" services.

For example, the `cachix.nix` module configures the [Cachix
Agent](https://docs.cachix.org/deploy/running-an-agent/) which is responsible
for reconfiguring the NixOS machine when a new build is available. The agent
runs on all machines whose configuration automatically managed.

# Building with GitHub Actions

The NixOS host specifications are built with a single `nix build` command:

```
nix build .#nixosConfigurations.nuc.config.system.build.toplevel
```

This builds the whole system for the `nuc` machine: the kernel, the installed
packages and their configuration. By default, `nix` downloads pre-built
packages from the [NixOS public binary cache](http://cache.nixos.org/), so a
typical execution of the build command takes only a few minutes.

To run the build in GitHub Actions, the build job installs Nix and configures
the access to the Cachix binary cache where the deployment artifacts are
stored:

```yaml
jobs:
build:
runs-on: ubuntu-latest
environment:
name: Homelab
url: "https://app.cachix.org/deploy/workspace/lab.thewagner.home/" #
steps:
- uses: actions/checkout@v4
- uses: docker/setup-qemu-action@v3 #
- uses: cachix/install-nix-action@v23
with:
extra_nix_config: "extra-platforms = aarch64-linux" #
- uses: cachix/cachix-action@v12 #
with:
name: wagdav
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'
- run: nix build --print-build-logs .#cachix-deploy-spec #
- run: | #
cachix push wagdav ./result
cachix deploy activate --async ./result
env:
CACHIX_ACTIVATE_TOKEN: "${{ secrets.CACHIX_ACTIVATE_TOKEN }}"
```
The `build` job of the [workflow][HomelabAction]:

1. Configures a [deployment target][GitHubEnvironment]. When a GitHub Actions
workflow deploys to an environment, the environment is displayed on the main
page of the repository.
1. Installs the QEMU static binaries for building packages for architectures
different than that of the build runner.
1. Configures Nix to use emulation to [build ARM 64 packages][QEMUBuilds].
1. Configures the [Cachix hosted binary cache](https://cachix.org).
1. Builds the [deploy specification][DeploySpec] which is a set of the NixOS
systems to deploy.
1. Pushes the built binaries to the cache and sends an activation signal to the
Cachix Agent.

The job uses two secrets: `CACHIX_AUTH_TOKEN` is the authentication token to
push to the binary cache and `CACHIX_ACTIVATE_TOKEN` is required to activate
the built NixOS configurations.

# Deploying with Cachix Deploy

To deploy the built NixOS configurations I use the generous free-tier of
[Cachix Deploy](https://docs.cachix.org/deploy/). Following their
documentation, I installed `cachix-agent` on the target hosts and configured a
few authentication keys.

The agent process connects to the Cachix backend and waits for a deployment.
When a new deployment is available, the agent pulls the relevant binaries from
the binary cache and reconfigures the NixOS system it runs on.

# Summary

Since I configured [automatic deployment of this blog][BlogDeployment] I wanted
the same for my home infrastructure. I had my configuration repository and I
knew how to build the machine configurations with GitHub Actions. I was
missing the automatic deployment part until Cachix Deploy was announced.
Cachix has excellent documentation and the integration with my Homelab was
super simple.

# Acknowledgement

I'm grateful to [Cachix Deploy](https://www.cachix.org/) for offering a binary
cache and a deployment service.

[BlogDeployment]: {filename}/2020-12-06-Blog-deployment-update.markdown
[DeploySpec]: https://docs.cachix.org/deploy/deploying-to-agents/#write-deploy-specification
[GitHubEnvironment]: https://docs.github.com/en/actions/deployment/about-deployments/deploying-with-github-actions#using-environments
[HomelabAction]: https://github.com/wagdav/homelab/blob/master/.github/workflows/build-and-deploy.yml
[HomelabNuc]: https://github.com/wagdav/homelab/blob/master/host-nuc.nix
[HomelabRepo]: https://github.com/wagdav/homelab
[Homelab]: {filename}/2020-05-31-Homelab.markdown
[QEMUBuilds]: {filename}/2023-11-20-Raspberry-Pi-Github-Actions.markdown
Loading

0 comments on commit 6ba2d40

Please sign in to comment.