change post wall-time

[SarahLu-NOAA]

Hello
My post failed due to wall-time exceeding 900 sec. The expt directory is: /glade/work/clu/ufs/expt_dirs/RRFS_conus_3km (at Derecho).

I plan to modify $DIR/ufs-srweather-app/parm/wflow/post.yaml to fix the wall-time issue. Just wonder whether there is an even better way to go around this. Thanks.

[gspetro-NOAA]

Hi @SarahLu-NOAA ,

You can set the walltime in the your config.yaml file in the rocoto: tasks: section. For example, to change the wall clock time from 15 to 20 minutes, you'd add everything from metatask_run_ens_post: down into your config.yaml Rocoto section under tasks:. If this is the only workflow change you're making, your rocoto: section would look like this:

rocoto:
  tasks:
    metatask_run_ens_post:
      metatask_run_post_mem#mem#_all_fhrs:
        task_run_post_mem#mem#_f#fhr#:
          walltime: 00:20:00

You can certainly adjust the walltime in parm as well, but I find it easier to do everything in config.yaml to avoid jumping between files and to keep changes isolated to one experiment. Of course, if this is a change you think you may need for several experiments, then it may make more sense to change it directly in the parm/wflow/post.yaml.

More generally, variables that are set in a parm/wflow yaml file can be changed by setting them in the rocoto: section of config.yaml. The structure of metatasks and tasks in the yaml is the structure you use to determine what metatasks and tasks to add under rocoto: tasks: (and in which order).

Best,
Gillian Petro

[SarahLu-NOAA]

Thanks, @gspetro-NOAA
Yes - the option to modify post's wall-time in config.yaml is a more elegant way. Thanks. I copy & paste the code into the config.yaml, and FV3LAM_wflow.xml shows the walltime for task "run_post_mem#mem#_f#fhr#" is now 20 min. /The jobs are in queues for hours, so I check the xml/.

My previous config.yaml has the following and the walltime for the post is 15 min (the default). Apparently my modification is incorrect.
metatask_run_ens_post:
run_fcst_mem#mem#:
walltime: 00:20:00

[gspetro-NOAA]

Hi @SarahLu-NOAA ,

I looked at the config.yaml file in /glade/work/clu/ufs/expt_dirs/RRFS_conus_3km, and it looks like after metatask_run_ens_post: you need to replace the run_fcst_mem#mem#: line with:

metatask_run_post_mem#mem#_all_fhrs:
  task_run_post_mem#mem#_f#fhr#:

So your whole rocoto section would look like this:

rocoto:
  tasks:
    taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml",
       "parm/wflow/plot.yaml"]|include
      }}'
    metatask_run_ensemble:
      task_run_fcst_mem#mem#:
        walltime: 02:00:00
    metatask_run_ens_post:
      metatask_run_post_mem#mem#_all_fhrs:
        task_run_post_mem#mem#_f#fhr#:
          walltime: 00:20:00

[SarahLu-NOAA]

Thanks @gspetro-NOAA
The config.yaml file has been modified accordingly. The xml shows 20 min walltime for post task and all post jobs are completed successfully. Cheers, Sarah