Bug when using conda-lock within environment deployment

[atteggiani]

There is currently this issue in the conda github repo related to a bug with conda-lock when a package in the environment yaml file contains a hash (#) in the package specification.

This happens to be the case for this repo's env.yml file in the mule installation through pip (note the #subdirectory=mule at the end of the line).

This, as reported in the conda GitHub repo issue above, causes the current deployment action to fail with the same error.
I also tested the deployment when commenting the "hashy" line in the env.yml file, and in that case conda-lock runs without problems, meaning the hash (#) is effectively the problem here.

Currently, mule needs to be installed through pip as specified in this repo's env.yml (containing a hash (#) within the pip package installation line).

Is there any way to solve this for the time being?
Do we have any alternatives to using conda-lock?

@CodeGat

[CodeGat]

Seems like this PR might be relevant: ACCESS-NRI/payu-condaenv#37 (comment) in which Jo and myself were talking about the #subdirectory=mule bit. @jo-basevi might be able to help as well.

[atteggiani]

In the PR you linked, only the env-dev.yml environment is deployed, so the deploy-dev.yml is run for the action.
But in the deploy-dev.yml file there is no conda-lock run in the Create Pack step:

https://github.com/ACCESS-NRI/payu-condaenv/blob/485b4d9dd02a46c070a603b837c9b7880f214cf2/.github/workflows/deploy-dev.yml#L88-L90

whereas in the deploy.yml there is:

https://github.com/ACCESS-NRI/payu-condaenv/blob/485b4d9dd02a46c070a603b837c9b7880f214cf2/.github/workflows/deploy.yml#L46-L50

The conda-lock run is what causes issues.
Whenever the env.yml file is going to be depoyed, it might trigger this bug also in the payu-condaenv case.

[jo-basevi]

Yeah we are going to run into issues with payu-condaenv too once pip install mule is added to env.yaml.. I think conda-lock is not essential for the deployment as it was a reference of what dependencies were installed. I don't have any great ideas on alternatives. Could use micromamba list to obtain the micromamba dependencies and a pip list to get list of pip dependencies and adding those to a file.

[jo-basevi]

As we are using conda-pack, the environment is already fixed and the environment can be reproduced by downloading the packed environment file so maybe using conda-lock file is not as necessary?

[atteggiani]

As we are using conda-pack, the environment is already fixed and the environment can be reproduced by downloading the packed environment file so maybe using conda-lock file is not as necessary?

The conda-lock is for "external usage" to be placed in the release files.
I commented it out for now so it won't be produced.
I would say it's not a must-have, but definitely a good thing to have if the issue gets solved!

[atteggiani]

Fixed by replacing mule installation through pip with mule from coecms conda channel!
Thank you @jo-basevi !

[CodeGat]

@atteggiani I think you meant @jo-basevi 😆

[atteggiani]

Oopssss 🤣