Specifying fields with Nextflow log command

[georgiesamaha]

Hi,

I'm using the nextflow log command to query specific trace fields for a workflow run, but running into an issue when I try to query %cpu and %mem. According to the current documentation:

The fields accepted by the -f options are the ones in the trace report, as well as: script, stdout, stderr, env. List available fields using the -l (-list-fields) option.

When I specify %cpu and/or %mem as they are specified in the trace field table, for example:

nextflow log modest_goldstine -f name,%cpu,%mem

I get a token recognition error:

unknown recognition error type: groovyjarjarantlr4.v4.runtime.LexerNoViableAltException
Failed to parse template script (your template may contain an error or be trying to use expressions not currently supported): org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/groovy/script/Script4DAF240CDD69B1DD0F546D98806ECADB: 1: token recognition error at: '%' @ line 1, column 26.
   __$$_out.print("""$name      $%cpu   $%mem""");
                            ^

1 error

While I understand this is an issue with parsing the %, I'm not familiar with Groovy syntax so am having trouble finding a workaround. Would you be able to clarify if its currently possible to apply these fields with the log command and how to do so?

Thanks :)

[bentsherman]

It looks like there are a few special cases as shown here:

nextflow/modules/nextflow/src/main/groovy/nextflow/cli/CmdLog.groovy

Lines 241 to 266 in eda0845

	@Override
	Object get(Object key) {
	if( delegate.containsKey(key) ) {
	return delegate.get(key)
	}
	if( key == 'stdout' ) {
	return fetch(getWorkDir().resolve(TaskRun.CMD_OUTFILE))
	}
	if( key == 'stderr' ) {
	return fetch(getWorkDir().resolve(TaskRun.CMD_ERRFILE))
	}
	if( key == 'log' ) {
	return fetch(getWorkDir().resolve(TaskRun.CMD_LOG))
	}
	if( key == 'pcpu' )
	return record.getFmtStr('%cpu')
	if( key == 'pmem' )
	return record.getFmtStr('%mem')
	return record.getFmtStr(normaliseKey(key))
	}

In your case you just need to use pcpu and pmem. I will make a note to document these on the CLI page.

[georgiesamaha]

thanks @bentsherman!

[YinYangKarly]

Hi, I don't know if this still be open. When I used pcpu and pmem, I get "-"in those fields which does not give information. How to extract cpu and memory usage from the nextflow log? (I did not include with-report in my nextflow command line).