Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

publishDir in process block in nextflow.config #3966

Closed
pdp10 opened this issue May 24, 2023 · 1 comment
Closed

publishDir in process block in nextflow.config #3966

pdp10 opened this issue May 24, 2023 · 1 comment

Comments

@pdp10
Copy link

pdp10 commented May 24, 2023

New feature

In the current pipeline I am working on, processes use the following publishDir directive:

process fastq2bam {
    publishDir "${params.output_path}/${task.process.replaceAll(":", "/")}", mode: 'copy', overwrite: true, pattern: '*.bam'

    input:
        tuple path(ch_fastq1), path(ch_fastq2)

    output:
        path "*.bam", emit: ch_bam

    shell:
    """
    touch "my.bam"
    """
}

Whilst I find the way published files are organised helpful for debugging as they replicate the pipeline structure, having to replicate the whole directive for each process over and over seems quite odd.

Usage scenario

Avoid having to replicate publishDir directive configuration for each process.

Suggest implementation

I came across with this post: #3107 (comment) .
It would be nice to code the publishDir directive in the process block in nextflow.config, e.g.:

process {
    // General publishDir setting
    publishDir = [
        path: { "${params.output_path}/${task.process.replaceAll(":", "/")}" },
        mode: "copy",
        overwrite: true
    ]
}

and then the actual process code to be:

process fastq2bam {
    publishDir pattern: '*.bam'     // the rest inherited from the config above

// as code above
}

p.s. specifying the patter is redundant in this case as it is the only output file, however it can be helpful in general.

@bentsherman
Copy link
Member

Closing in favor of #4186

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants