Conda is ok but singularity failed

[lindenb]

Description of the bug

Hi I'm trying to run a first nfcore workflow using fetchngs.

I managed to run it with conda : https://nfcore.slack.com/archives/CE6SDBX2A/p1674655464394899

But if fails when I switch to singularity

Command used and terminal output

echo "ERR9851248" > "${OUTDIR}/input.txt"


cat << EOF > "${OUTDIR}/sge.cfg"
process {
beforeScript = "hostname 1>&2"
executor="sge"
clusterOptions = "-S /bin/bash -q max-24h.q"
cache = "lenient"
penv = "smp"
}
EOF

OK with conda

nextflow -c /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/sge.cfg run nf-core/fetchngs -profile test,conda -work-dir /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/work --input /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/input.txt --outdir /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/output -resume

but singularity doesn't work:

nextflow -c /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/sge.cfg run nf-core/fetchngs -profile test,singularity -work-dir /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/work --input /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/input.txt --outdir /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/output -resume

output

Caused by:
  Process `NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO (ERR9851248)` terminated with an error exit status (1)

Command executed:

  echo ERR9851248 > id.txt
  sra_ids_to_runinfo.py \
      id.txt \
      ERR9851248.runinfo.tsv \
  
  
  cat <<-END_VERSIONS > versions.yml
  "NFCORE_FETCHNGS:SRA:SRA_IDS_TO_RUNINFO":
      python: $(python --version | sed 's/Python //g')
  END_VERSIONS

Command exit status:
  1

Command output:
  (empty)

Command error:
  WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMP as environment variable will not be supported in the future, use APPTAINERENV_TMP instead
  WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_TMPDIR as environment variable will not be supported in the future, use APPTAINERENV_TMPDIR instead
  WARNING: DEPRECATED USAGE: Forwarding SINGULARITYENV_NXF_DEBUG as environment variable will not be supported in the future, use APPTAINERENV_NXF_DEBUG instead
  [ERROR] We failed to reach a server.
  [ERROR] Reason: [Errno 110] Connection timed out

Work dir:
  /SCRATCH-BIRD/users/lindenbaum-p/work/NEXTFLOW/2023/20230125.nfcore/work/8b/f1972099f26ee351e766989e4fa599

HTTPS_PROXY and HTTP_PROXY are both set.

Relevant files

No response

System information

nf 22.10.4
Linux gkq4xd2.compute.bird2.prive 3.10.0-1127.19.1.el7.x86_64 #1 SMP Tue Aug 25 17:23:54 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
sge
singularity
linux

[charmoniumQ]

There may be a difference between setting HTTPS_PROXY for the host running Nextflow, but Nextflow starts several processes and containers without propagating environment variables (unless explicitly told), so perhaps the HTTPS_PROXY is not present inside the container.

Try going to the workdir indicated by the error and running .command.run. If you look at that script, you may be able to see where .command.run starts a Singularity container. See if you can hit any external network resource from inside that container.

[drpatelh]

Yep, that makes sense. You could try to export any network-type variables via the NF env scope to make them available within the container.

Did you find a solution to this @lindenb ? Could test independent of NF by running the command within the container and exporting the appropriate environment variables if you know what they are.

[lindenb]

Did you find a solution to this @lindenb ?

@drpatelh not for now. I haven't tried nf-core since my first post :-) My resistance to frustration is quite low 😆

[drpatelh]

No worries @lindenb ! Ok, I will close for now and if you muster the courage to try again then please feel free to re-open if the problem persists.

[lindenb]

FYI , one year later... setting HTTP_PROXY and HTTPS_PROXY in env did the job. Thanks !