Include input and output file sizes in the trace

[jorgee]

After the execution input and output sizes are stored in the task working directory using the du UNIX utility and storing in .command.filesizes.yaml file. For every input and output file/directory it stores the name and the size in bytes.

For instance:

outputs:
- name: multiqc_report.html
  size: 5285017
inputs:
- name: ggal_gut
  size: 34221
- name: fastqc_ggal_gut_logs
  size: 2102143
- name: config
  size: 26568

This information is also included in the TraceRecord and the trace file at the end of the execution

task_id hash    native_id       name    status  exit    submit  duration        realtime        %cpu    peak_rss        peak_vmem       rchar   wchar   inputs  outputs
2       f4/a70e1f       66582   RNASEQ:FASTQC (FASTQC on ggal_gut)      COMPLETED       0       2024-10-29 14:58:49.807 2.3s    2.2s    188.8%  179.2 MB        3.5 GB  12.4 MB 3.8 MB  [[name:ggal_gut_2.fq, size:691995], [name:ggal_gut_1.fq, size:691873]]  [[name:fastqc_ggal_gut_logs, size:2102143]]
1       6b/deab39       66576   RNASEQ:INDEX (ggal_1_48850000_49020000) COMPLETED       0       2024-10-29 14:58:49.796 5.6s    5.5s    3.1%    27.4 MB 88.2 MB 2.4 MB  1.3 MB  [[name:ggal_1_48850000_49020000.Ggal71.500bpflank.fa, size:173911]]     [[name:index, size:613419]]
3       3a/323ae9       67340   RNASEQ:QUANT (ggal_gut) COMPLETED       0       2024-10-29 14:58:55.431 906ms   808ms   18.4%   12.9 MB 51.2 MB 2 MB    21.8 KB [[name:ggal_gut_2.fq, size:691995], [name:index, size:613419], [name:ggal_gut_1.fq, size:691873]]       [[name:ggal_gut, size:34221]]
4       94/9fe1ca       67479   MULTIQC COMPLETED       0       2024-10-29 14:58:56.369 3s      2.9s    76.0%   99.4 MB 1.7 GB  38.7 MB 17.7 MB [[name:ggal_gut, size:34221], [name:fastqc_ggal_gut_logs, size:2102143], [name:config, size:26568]]     [[name:multiqc_report.html, size:5285017]]

[netlify]

✅ Deploy Preview for nextflow-docs-staging canceled.

Name	Link
🔨 Latest commit	4202754
🔍 Latest deploy log	https://app.netlify.com/sites/nextflow-docs-staging/deploys/672a005024069f00085b0255