Software version reporting best practices

[glormph]

I have been trying to figure out how to best report software versions in a pipeline, somewhat dynamically, and have experimented a bit. This tends to get slightly convoluted, but I tried:

• a JSON file with containers, e.g. {"python": {"version": "3.10", "docker": "quay.io/....:3.10__...", "singularity": "https://...../python:3.10.2"}
• reading that in nextflow.config as allcontainers
• container engine defined there as engine = docker.enabled ? 'docker' : 'singularity' (with slightly more checks than that).
• after that process { withName do_stuff { container = allcontainers['python'][engine] } }

This will get my containers reported in the workflow.container object (unlike when defining the container in the process definition), albeit with the drawback of that all processes configured will be reported, so even those which aren't used. I'm not sure if that is expected behaviour, but in that case I'd like to be able to filter those out somehow.

Another way which I've seen in nf-core, is to for each process dedicatedly output the version to a channel, which implies one would have to write and parse the version in the script, which strikes me as somewhat of a contamination in the script (which is why I went to look for other methods). But it gets the job done, and it gets only the actually used software versions, so there's definitely something to say for that as well. My above experiment may be more convoluted.

Is there any other or better way to do this?

[jeffinrockey]

Very recently (May 24 I suppose) a new mechanism is in place which makes the version capture very straight forward.
Involves the newly introduced eval and topic. The below page has example.
https://nextflow.io/blog/2024/nextflow-2404-highlights.html#process-eval-outputs
(I've tried them and found this way very useful)