Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restructure order of user module load/use and setting executable paths #382

Closed
jo-basevi opened this issue Nov 9, 2023 · 1 comment
Closed
Assignees

Comments

@jo-basevi
Copy link
Collaborator

As brought up in this issue #379 regarding payu being able to find the path to spack built executables. A potential solution, as suggested by @micaeljtoliveira here, was to add the following in config.yaml:

exe: $SPACK_ACCESS_OM3_ROOT/bin/access-om3-MOM6-CICE6

modules:
  use:
      - /g/data/ik11/spack/0.20.1/modules/access-om3/0.3.0/linux-rocky8-cascadelake
  load:
      - access-om3

Where SPACK_ACCESS_OM3_ROOT is set by the access-om3 module.

However currently payu runs module load/use during payu run (Experiment.run()). The paths to the executable are set earlier in the experiment initialisation and if exe is not an absolute path, it prepends the path to the laboratory bin path. The executable paths are checked and added to the manifest during setup in the model driver.

The minimum to get the above solution working, would require moving use/load user modules to Experiment.setup before any Model.setup() is run. It will also require any environment variables to be expanded in the exec_path before the path is checked and added to the manifest. To bypass the absolute path check in initialisation, the exe path will need to start with a forward slash.

I think it could be worth restructuring the Model.exec_path initialisation and setup, so executable paths are set later in Model.setup (vs at payu Experiment initialisation). This means there is no need to ensure there's a starting forward slash in exe as the path will be fully expanded to an absolute path.

As spoke about in the #379, it may be worth adding a check that user module versions are specified. At minimum, it could just be emit a warning if it is using default version.

@jo-basevi jo-basevi self-assigned this May 1, 2024
@jo-basevi
Copy link
Collaborator Author

Closed by #439

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant