Skip to content

Spack build cache for Github Actions

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT
Notifications You must be signed in to change notification settings

spack/github-actions-buildcache

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

70 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Spack buildcache for GitHub Actions

This repo provides a buildcache to speed up Spack in your GitHub Actions.

Currently it provides binaries from Spack develop for

which are compatible with

  • Ubuntu 20.04 and later
  • Debian 11 and later
  • RHEL 9 and later
  • Fedora 32 and later

To use it, add an environment spack.yaml to the root of your own repository

spack:
  view: my_view
  specs:
  - [email protected]

  config:
    install_tree:
      root: /opt/spack

  packages:
    all:
      require: 'target=x86_64_v3'

and Spack install it in a GitHub Action:

name: Build

on: push

jobs:
  example:
    runs-on: ubuntu-22.04
    steps:
    - name: Checkout
      uses: actions/checkout@v4

    - name: Setup Spack
      uses: spack/setup-spack@v2

    - name: Concretize
      run: spack -e . concretize

    - name: Install
      run: spack -e . install --no-check-signature

    - name: Run
        run: ./my_view/bin/python -c 'print("hello world")'

Caching your own binaries

If you want to cache your own binaries too, there are three steps to take:

  1. Use padding in the install root, and add an additional mirror to spack.yaml:

    spack:
      config:
        install_tree:
          root: /opt/spack
          padded_length: 128
      mirrors:
        local-buildcache: oci://ghcr.io/<username>/spack-buildcache
  2. Configure the permissions for GITHUB_TOKEN:

    jobs:
      example:
        runs-on: ubuntu-22.04
        permissions:
          packages: write
  3. Add an extra job step that pushes installed Spack packages to the local buildcache:

    jobs:
      example:
        steps:
        - name: Push packages and update index
          run: |
            spack -e . mirror set --push --oci-username ${{ github.actor }} --oci-password "${{ secrets.GITHUB_TOKEN }}" local-buildcache
            spack -e . buildcache push --base-image ubuntu:22.04 --unsigned --update-index local-buildcache
          if: ${{ !cancelled() }}

    NOTE: Make sure to add if: ${{ !cancelled() }}, so that binaries for successfully installed packages are available also when a dependent fails to build.

Caching your own binaries in private repos and buildcaches

When your local buildcache is stored in a private GitHub package, you need to specify the OCI credentials already before spack concretize. This is because Spack needs to fetch the buildcache index. Also, remember to remove the --push flag from spack mirror set, since fetching needs credentials too:

jobs:
  example-private:
    steps:
    - name: Login
      run: spack -e . mirror set --oci-username ${{ github.actor }} --oci-password "${{ secrets.GITHUB_TOKEN }}" local-buildcache

    - name: Concretize
      run: spack -e . concretize

    - name: Install
      run: spack -e . install --no-check-signature

    - name: Push packages and update index
      run: spack -e . buildcache push --base-image ubuntu:22.04 --unsigned --update-index local-buildcache

From a security perspective, notice that the GITHUB_TOKEN is exposed to every subsequent job step. (This is no different from docker login, which also likes to store credentials in the home directory.)

Contributing

If you want to make more packages available, contribute to spack.yaml.

Build strategy

Since compiling software in GitHub actions is relatively slow, this stack is built using concretizer:reuse:dependencies. That means that the latest versions of the packages listed in spack.yaml are built, but their dependencies are only updated when a package compatibility rule requires it. The stack is currently built on demand, not on a schdule.

License

This project is part of Spack. Spack is distributed under the terms of both the MIT license and the Apache License (Version 2.0). Users may choose either license, at their option.

All new contributions must be made under both the MIT and Apache-2.0 licenses.

See LICENSE-MIT, LICENSE-APACHE, COPYRIGHT, and NOTICE for details.

SPDX-License-Identifier: (Apache-2.0 OR MIT)

LLNL-CODE-811652

About

Spack build cache for Github Actions

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Packages