ci: build binary artifacts for pushes/PRs

[cpu]

This branch adds a CI job for all pushes & PRs that builds binary artifacts for rustls-ffi for:

• Windows (x86_64 MSVC)
• Debian .deb (x86_64 GNU)
• Linux (x86_64 GNU)
• Apple (ARM64 and x86_64)

Binary artifacts are built in release mode using stable rust, with the default crypto provider (aws-lc-rs) and cert compression enabled.

Linux binaries are built on ubuntu-20.04 for greatest compatibility. Users will need glibc 2.31 or greater. For now I've opted not to include a statically linked musl build.

For FIPS, no cert-compression, the ring crypto provider, debug builds, or other customization you will need to build from source.

Users of the .pc files from the bare archive will need to override the prefix to wherever they extract the archive, or use cargo cinstall from src. Package config files in an archive aren't relocatable. The .deb for Debian/Ubuntu has the correct .pc prefix already.

The build library zips are automatically added to the CI run as artifacts. This has the advantage that we don't need any special write permissions and can manually attach binary artifacts to the releases we publish as appropriate. For now I'd like to avoid automating the release process. Artifact links associated with a branch are valid for ~90d.

You can see the artifacts produced for this PR here, under the annotations.

See an example of a simple C test application built using cmake and pkg-config from the distributed binary releases here: https://github.com/cpu/rustls-ffi-test It's used by testing CI stages that run after the packaging step for each OS/output.

Unpacking the archives gives structure like:

# Linux
.
├── include
│   └── rustls.h
├── lib
│  ├── librustls.a
│  ├── librustls.so
│  ├── librustls.so.0.15.0
│  └── pkgconfig
│      └── rustls.pc

# MacOS
.
├── include
│   └── rustls.h
├── lib
│   ├── librustls.0.15.0.dylib
│   ├── librustls.a
│   ├── librustls.dylib
│   └── pkgconfig
│       └── rustls.pc

# Windows
.
├── bin
│   ├── rustls.dll
│   └── rustls.pdb
├── include
│   └── rustls.h
├── lib
│   ├── pkgconfig
│   │   └── rustls.pc
│   ├── rustls.def
│   ├── rustls.dll.lib
│   └── rustls.lib

The .deb displays the following:

$ dpkg-deb --info librustls_0.15.0_amd64.deb
 new Debian package, version 2.0.
 size 8759588 bytes: control archive=420 bytes.
     211 bytes,     8 lines      control
      26 bytes,     3 lines   *  postinst             #!/bin/sh
 Package: librustls
 Version: 0.15.0
 Architecture: amd64
 Maintainer: Daniel McCarney <[email protected]>
 Description: FFI bindings for the Rustls TLS library
 Section: libs
 Depends: libc6
 Priority: optional
 
$ dpkg-deb --contents librustls_0.15.0_amd64.deb
drwxr-xr-x runner/docker     0 2024-12-22 20:09 ./
drwxr-xr-x runner/docker     0 2024-12-22 20:09 ./usr/
drwxr-xr-x runner/docker     0 2024-12-22 20:09 ./usr/include/
-rw-r--r-- runner/docker 122289 2024-12-22 20:09 ./usr/include/rustls.h
drwxr-xr-x runner/docker      0 2024-12-22 20:09 ./usr/lib/
drwxr-xr-x runner/docker      0 2024-12-22 20:09 ./usr/lib/x86_64-linux-gnu/
-rw-r--r-- runner/docker 38767436 2024-12-22 20:09 ./usr/lib/x86_64-linux-gnu/librustls.a
-rwxr-xr-x runner/docker  6082776 2024-12-22 20:09 ./usr/lib/x86_64-linux-gnu/librustls.so.0.15.0
drwxr-xr-x runner/docker        0 2024-12-22 20:09 ./usr/lib/x86_64-linux-gnu/pkgconfig/
-rw-r--r-- runner/docker      286 2024-12-22 20:09 ./usr/lib/x86_64-linux-gnu/pkgconfig/rustls.pc
lrwxrwxrwx runner/docker        0 2024-12-22 20:09 ./usr/lib/x86_64-linux-gnu/librustls.so -> librustls.so.0.15.0

Resolves #218

[cpu]

I haven't done extensive testing of the built artifacts from an external project yet, I'll try to that shortly.

This shook out some useful findings:

1. Avoid using xx-latest images in CI for this task... I lost a lot of time debugging some weird Glibc symbol resolution errors with the Linux artifacts before I realized my fork of the repo was using ubuntu-latest == 22.04 and this repo was using ubuntu-latest == 24.04. I knew that change was coming, but thought it would be universally applied .... 🤬

2. It's best to build with an older Linux image if we can, to keep maximum glibc compat. I'm using 20.04, which also requires building with CC=clang instead of gcc because aws-lc's build system flags the gcc as vulnerable to a nasty compiler bug.

3. For the MacOS dylib we need to patch the dylib ID to @rpath/librustls.dylib so that it can be located anywhere on the typical search path. The cargo cinstall dylib has its ID hardcoded to the location it spits it out.

In the end I was able to get a repo working with a simple C application that spits out rustls_version() on macos-latest (arm64), macos-13 (x86_64), windows-latest (x86-64) and ubuntu-latest (x86_64) using the pre-built artifacts & cmake's pkg-config integration. You can find that here: https://github.com/cpu/rustls-ffi-test The Linux binaries also work unmodified on my NixOS system.

One place for improvement: I had to lift some of the tests/CMakeList.txt for the Windows build. I think this should be handled automatically through the .pc file of our lib describing the required linker args, but it doesn't seem to be working right. I will chase this down another time.

rustls-ffi / Windows (aws-lc-rs, Debug) (pull_request) Failing after 4m

This seems to be unrelated but still needs to be debugged:

  research.cloudflare.com.ech.configs.bin
  client[2180]: using the platform verifier for certificate verification.
  client[2180]: using ECH config lists 'research.cloudflare.com.ech.configs.bin'
  client[2180]: no compatible/valid ECH configs found in 'research.cloudflare.com.ech.configs.bin'
  client[2180]: failed to configure ECH with any provided config files

[ctz]

• Avoid using xx-latest images in CI for this task... I lost a lot of time debugging some weird Glibc symbol resolution errors with the Linux artifacts before I realized my fork of the repo was using ubuntu-latest == 22.04 and this repo was using ubuntu-latest == 24.04. I knew that change was coming, but thought it would be universally applied .... 🤬

Oh! I think that has also been happening for rustls-libssl.

[cpu]

Oh! I think that has also been happening for rustls-libssl.

Looks like it 😓 Here's a fix PR: rustls/rustls-openssl-compat#45

[cpu]

cpu force-pushed the cpu-bin-artifacts branch

Tidied up my commit history & added testing to the workflow.

[jsha]

Looks good to me. Do you want to move the rustls-ffi-test repo into the rustls org at some point?

[cpu]

Thanks for taking a look 🙇

Do you want to move the rustls-ffi-test repo into the rustls org at some point?

Good question - I think with a bit of cmake work to allow finding an existing librustls instead of building it w/ cargo-c we could re-use the client/server tests that are already in-repo. I think that's a better route since it will actually test the functionality as well as building/linking.

I'll make a follow-up issue for that 📝 Edit: #519

[cpu]

cpu force-pushed the cpu-bin-artifacts branch

I removed the macos-13 runner. It added a long time to the build process because we had to build cargo-c from source (upstream only publishes binaries for arm64 mac).

Instead I realized we could use the arm64 runner/cargo-c bins to cross-compile the Mac x86_64 build. Much faster. We're now bottlenecked by Windows and I don't think can do much there.

[cpu]

@ctz (No pressure) would you like to review this or should I merge based on Jsha's +1?