Updates for BGC build #2

aidanheerdegen · 2024-03-19T08:50:58Z

Add changes to convert from ACCESS-OM2 build type to ACCESS-OM2-BGC.

Requires changes to the README and the spack.yaml to use type=ACCESS-OM2-BGC which was added to the MOM5 spack package (see ACCESS-NRI/spack-packages#72).

CodeGat · 2024-03-19T22:09:25Z

Whoops, I'll need to update the CI in this repository since it was created before I got CI working.

CodeGat · 2024-03-19T22:41:48Z

Regarding https://github.com/ACCESS-NRI/ACCESS-OM2-BGC/pull/2/files#diff-e8582e74fa156f4e5729a850e52b24f2fde2d815c2c9c360f88c4cf90db851abL8 (spack.yaml Line 8 - [email protected]) - would we change the name of the package? I feel that it could clash with the environment names in prerelease/release if the versions are the same (for example, access-om2-2024_02_1).

CodeGat · 2024-03-19T23:55:50Z

So regarding the spack package for access-om2-bgc:

don't change the spack package name, but hardcode the name of the spack environment to include a -bgc (access-om2-bgc_2024_03_0). This would mean the environments wouldn't clash, but the packages outside the environment might, if the versions are the same. Needs more testing.
add an entry in spack-packages for BGC. This wouldn't clash with an environment or package name, but might be nonstandard - do we really want to do a new spack-package for every variant?

@harshula how are variants handled in spack, regarding the package name? If you do a spack find would there be something like access-om2+bgc for a bgc variant?

aidanheerdegen · 2024-03-20T03:37:35Z

don't change the spack package name, but hardcode the name of the spack environment to include a -bgc (access-om2-bgc_2024_03_0). This would mean the environments wouldn't clash, but the packages outside the environment might, if the versions are the same.

To be specific, this would require changes to the CD workflow to be able to pass in a spack environment name? As it currently defaults to the spack package in spack.yaml:

https://github.com/ACCESS-NRI/ACCESS-OM2/blob/main/.github/workflows/cd.yml#L28

add an entry in spack-packages for BGC. This wouldn't clash with an environment or package name, but might be nonstandard - do we really want to do a new spack-package for every variant?

No I don't think we do, but this doesn't have to be the last word. Unscrambling this egg wouldn't be too painful: leave deployed model versions as-is and new ones could follow a new approach.

This is part of why being able to rely on module load would be nice: it frees us a bit on the implementation detail as we wouldn't need to hard-code paths in model configs, which means we could update how we do it relatively simply and it could be backwards compatible.

@harshula how are variants handled in spack, regarding the package name? If you do a spack find would there be something like access-om2+bgc for a bgc variant?

I can answer: yes.

$ spack find -Lv mom5
-- linux-rocky8-x86_64 / [email protected] -----------------------
uznvr36p3km4qmjvdrxin7nhfyjubmd3 mom5@master~deterministic~optimisation_report build_system=makefile
oucqhr5r4yupkapwb6vsnqh2i6ow4vod mom5@master~deterministic~optimisation_report+repro build_system=makefile
4q2un3c6if5hynashormlklggv55bng5 mom5@master~deterministic~optimisation_report+restart_repro build_system=makefile type=ACCESS-OM
aollz5h3gjqjhb3pkgfkkqqifm75hals mom5@master~deterministic~optimisation_report build_system=makefile type=ACCESS-OM-BGC
4jfyunr427g3rk667hcyir7xrfytlqoo mom5@master~deterministic~optimisation_report build_system=makefile type=MOM_solo
==> 5 installed packages

The packages are handled fine, but as you point out environment names would clash. It is possible to store environments in non-default locations. Could we add a prefix (subdirectory) that is the repo name, then any environment created therein is guaranteed to be unique on a per-repository basis only.

CodeGat · 2024-03-20T21:37:20Z

This branch will need to be rebased onto main to get the CI updates

github-actions · 2024-03-20T22:35:12Z

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

!bump major for feature releases
!bump minor for bugfixes

aidanheerdegen · 2024-03-20T22:38:24Z

!bump major

This is a the first version of a new model so needs a new major (CalVer) version.

github-actions · 2024-03-20T22:38:35Z

❌ Failed to bump version or commit changes, see https://github.com/ACCESS-NRI/ACCESS-OM2-BGC/actions/runs/8366824447 ❌

aidanheerdegen · 2024-03-20T22:42:18Z

!bump major

github-actions · 2024-03-20T22:42:33Z

❌ Failed to bump version or commit changes, see https://github.com/ACCESS-NRI/ACCESS-OM2-BGC/actions/runs/8366852547 ❌

github-actions · 2024-03-20T23:04:19Z

The model version in the spack.yaml has not been updated.
Either update it manually, or comment the following to have it updated and committed automatically:

!bump major for feature releases
!bump minor for bugfixes

spack.yaml

CodeGat · 2024-03-22T04:29:49Z

So currently the CI is failing because it hasn't taken into account the possibility that the projections would be anything but {name}/{version}. For example, mom5 has the projection {name}-bgc/0000.00.00. This PR should fix that, and we can get a deployment on the way: #4

…e projections

…k-packages version

github-actions · 2024-03-22T06:02:52Z

This ACCESS-NRI/ACCESS-OM2-BGC model will be deployed with the following versions:

2024.03.0 as a Release (when merged).
2024.03.0-8 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

access-nri/spack-packages version 2024.03.22
access-nri/spack-config version 2024.02.1

If this is not what was expected, commit changes to config/versions.json.

…ystem openmpi directly)

github-actions · 2024-03-22T07:28:17Z

This ACCESS-NRI/ACCESS-OM2-BGC model will be deployed with the following versions:

2024.03.0 as a Release (when merged).
2024.03.0-9 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

access-nri/spack-packages version 2024.03.22
access-nri/spack-config version 2024.03.22

If this is not what was expected, commit changes to config/versions.json.

aidanheerdegen · 2024-03-22T09:18:42Z

Deployment is failing because the environment still specifies nci-openmpi which we've removed from spack-config

https://github.com/ACCESS-NRI/ACCESS-OM2-BGC/pull/2/files#diff-e8582e74fa156f4e5729a850e52b24f2fde2d815c2c9c360f88c4cf90db851abL24-L25

github-actions · 2024-03-22T09:22:22Z

This ACCESS-NRI/ACCESS-OM2-BGC model will be deployed with the following versions:

2024.03.0 as a Release (when merged).
2024.03.0-10 as a Prerelease (during this PR). This can be accessed on Gadi via spack at /g/data/vk83/prerelease/apps/spack/0.20/spack once deployed.

It will be deployed using:

access-nri/spack-packages version 2024.03.22
access-nri/spack-config version 2024.03.22

If this is not what was expected, commit changes to config/versions.json.

aidanheerdegen had a problem deploying to Gadi Prerelease March 19, 2024 08:51 — with GitHub Actions Failure

aidanheerdegen requested review from harshula and CodeGat March 19, 2024 22:07

aidanheerdegen force-pushed the initial-bgc branch from af942a8 to 00d5956 Compare March 20, 2024 22:35

CodeGat requested changes Mar 21, 2024

View reviewed changes

spack.yaml Outdated Show resolved Hide resolved

spack.yaml Outdated Show resolved Hide resolved

aidanheerdegen mentioned this pull request Mar 21, 2024

Separate SPD for ACCESS-OM2 biogeochemistry (BGC) build ACCESS-NRI/spack-packages#80

Closed

CodeGat self-assigned this Mar 21, 2024

aidanheerdegen and others added 8 commits March 22, 2024 16:58

Changed README to ACCESS-OM2-BGC model

e8be879

Change environment file to use BGC types and added bgc specific modul…

0f61d44

…e projections

More README updates adding bgc to model name

6579e17

Updared spack packages and config versions

1a30928

Manually updated model version

753bf16

Add missing quotes to requirements

dc1c843

spack.yaml: Removed explicit bgc for mom5, updated spec, updated spac…

a475e59

…k-packages version

spack.yaml: Updated module names to '-bgc' variant

5ff95e5

CodeGat force-pushed the initial-bgc branch from dd07778 to 5ff95e5 Compare March 22, 2024 06:01

CodeGat temporarily deployed to Gadi Prerelease March 22, 2024 06:02 — with GitHub Actions Inactive

versions.json: Updated version of spack-config to 2024.03.22 (using s…

01cf887

…ystem openmpi directly)

CodeGat had a problem deploying to Gadi Prerelease March 22, 2024 07:28 — with GitHub Actions Failure

Change to openmpi from nci-openmpi

029c5c6

aidanheerdegen temporarily deployed to Gadi Prerelease March 22, 2024 09:22 — with GitHub Actions Inactive

CodeGat approved these changes Mar 27, 2024

View reviewed changes

aidanheerdegen merged commit bf2cd3e into main Mar 27, 2024
10 checks passed

aidanheerdegen temporarily deployed to Gadi Prerelease March 27, 2024 05:32 — with GitHub Actions Inactive

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Updates for BGC build #2

Updates for BGC build #2

aidanheerdegen commented Mar 19, 2024 •

edited

Loading

CodeGat commented Mar 19, 2024

CodeGat commented Mar 19, 2024

CodeGat commented Mar 19, 2024 •

edited

Loading

aidanheerdegen commented Mar 20, 2024

CodeGat commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

aidanheerdegen commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

aidanheerdegen commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

CodeGat commented Mar 22, 2024

github-actions bot commented Mar 22, 2024

github-actions bot commented Mar 22, 2024

aidanheerdegen commented Mar 22, 2024

github-actions bot commented Mar 22, 2024

Updates for BGC build #2

Updates for BGC build #2

Conversation

aidanheerdegen commented Mar 19, 2024 • edited Loading

CodeGat commented Mar 19, 2024

CodeGat commented Mar 19, 2024

CodeGat commented Mar 19, 2024 • edited Loading

aidanheerdegen commented Mar 20, 2024

CodeGat commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

aidanheerdegen commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

aidanheerdegen commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

github-actions bot commented Mar 20, 2024

CodeGat commented Mar 22, 2024

github-actions bot commented Mar 22, 2024

github-actions bot commented Mar 22, 2024

aidanheerdegen commented Mar 22, 2024

github-actions bot commented Mar 22, 2024

aidanheerdegen commented Mar 19, 2024 •

edited

Loading

CodeGat commented Mar 19, 2024 •

edited

Loading