gridstat: how to put output results into one file per valid time?

[refszkentla]

The general question is how to reduce the number of output files. [The main reason for this is that we have a file number limit on our server]
Currently, when I run gridstat calculations with 1h intervals, the output that I get is one file per init-valid pair. It would be great however, to put e.g. results for all inits for a given valid into single file. All the information provided in the filename are also available in the file content, because all the tables are identical [in terms of column names]. Maybe it is somehow possible to set in the config files?

[jprestop]

Hi @refszkentla. There is currently no way to achieve this directly from Grid-Stat or the METplus Grid-Stat wrapper. That tool is intended to be run once per valid time, so you will end up with one output time for each init-valid pair. However, you can store as much or as little data in a .stat output file as you’d like.

So, if you are running Grid-Stat using METplus, you could add a call to STAT-Analysis after all of the lead times have been evaluated. Then you'd run a filter job to collect all the data for a single model initialization time and store it a single output file. The job would look something like this:

stat_analysis -job filter -lookin directory/with/hourly/grid_stat/output -job filter -fcst_init_beg YYYYMMDDHH -fcst_init_end YYYYMMDDHH -dump_row grid_stat_init_YYYYMMDDHH.stat

where “YYYYMMDD_HH” is the model initialization time. That time string can be specified in any of these formats: YYYYMMDD, YYYYMMDDHH, YYYYMMDD_HH, or YYYYMMDD_HHMMSS.

I hope that helps. Please let us know if you have any other questions.