TC_STAT produces CTS output even when told to produce CTC

[mollybsmith-noaa]

Hi,

This is a follow-up to a discussion answered by @JohnHalleyGotway the other day. I've been running TC_STAT to generate CTC scores, but it stops working as expected when I try to stratify the output by lead time.

For example, this works correctly and produced CTC tables:
TC_STAT_JOB_ARGS = -job rirw -line_type TCMPR -rirw_time 24 -rirw_exact false -rirw_thresh ge30 -basin {ENV[BASIN]} -by CYCLONE -out_line_type CTC -out_stat {OUTPUT_BASE}/{ENV[MODEL]}/{ENV[YYYYMMDDHH]}/{TC_STAT_OUTPUT_TEMPLATE}

But adding a ",LEAD" causes it to instead produce CTC and CTS output in the same file, neither of which is formatted to be loaded into a database:
TC_STAT_JOB_ARGS = -job rirw -line_type TCMPR -rirw_time 24 -rirw_exact false -rirw_thresh ge30 -basin {ENV[BASIN]} -by CYCLONE,LEAD -out_line_type CTC -out_stat {OUTPUT_BASE}/{ENV[MODEL]}/{ENV[YYYYMMDDHH]}/{TC_STAT_OUTPUT_TEMPLATE}

Here is an example of erroneous output:

JOB_LIST: -job rirw -basin AL -init_beg 20230827_000000 -init_end 20230903_000000 -line_type TCMPR -rirw_time 240000 -rirw_exact false -rirw_thresh >=30 -rirw_window 000000 000000 -by CYCLONE -out_line_type CTC -out_line_type CTS -out_alpha 0.05000 
COL_NAME: ATIME  BTIME  AEXACT BEXACT ATHRESH BTHRESH WINDOW_BEG WINDOW_END CYCLONE TOTAL FY_OY FY_ON FN_OY FN_ON EC_VALUE
RIRW_CTC: 240000 240000 false  false  >=30    >=30    000000     000000     08         23     0     0     3    20      0.5
RIRW_CTC: 240000 240000 false  false  >=30    >=30    000000     000000     10         27     0     0     3    24      0.5
RIRW_CTC: 240000 240000 false  false  >=30    >=30    000000     000000     11         17     0     0     0    17      0.5
RIRW_CTC: 240000 240000 false  false  >=30    >=30    000000     000000     92         10     0     0     0    10      0.5

COL_NAME: ATIME  BTIME  AEXACT BEXACT ATHRESH BTHRESH WINDOW_BEG WINDOW_END CYCLONE TOTAL   BASER  BASER_NCL BASER_NCU BASER_BCL BASER_BCU FMEAN  FMEAN_NCL FMEAN_NCU FMEAN_BCL FMEAN_BCU     ACC ACC_NCL ACC_NCU ACC_BCL ACC_BCU FBIAS FBIAS_BCL FBIAS_BCU PODY PODY_NCL PODY_NCU PODY_BCL PODY_BCU PODN PODN_NCL PODN_NCU PODN_BCL PODN_BCU POFD   POFD_NCL POFD_NCU POFD_BCL POFD_BCU FAR FAR_NCL FAR_NCU FAR_BCL FAR_BCU CSI CSI_NCL  CSI_NCU CSI_BCL CSI_BCU GSS GSS_BCL GSS_BCU HK   HK_NCL   HK_NCU HK_BCL HK_BCU HSS HSS_BCL HSS_BCU ODDS ODDS_NCL ODDS_NCU ODDS_BCL ODDS_BCU LODDS LODDS_NCL LODDS_NCU LODDS_BCL LODDS_BCU ORSS ORSS_NCL ORSS_NCU ORSS_BCL ORSS_BCU EDS EDS_NCL EDS_NCU EDS_BCL EDS_BCU SEDS SEDS_NCL SEDS_NCU SEDS_BCL SEDS_BCU EDI EDI_NCL EDI_NCU EDI_BCL EDI_BCU SEDI SEDI_NCL SEDI_NCU SEDI_BCL SEDI_BCU BAGSS BAGSS_BCL BAGSS_BCU  HSS_EC HSS_EC_BCL HSS_EC_BCU EC_VALUE
RIRW_CTS: 240000 240000 false  false  >=30    >=30    000000     000000     08         23 0.13043 0.045377     0.32127        NA        NA     0 0            0.14312        NA        NA 0.86957 0.67873 0.95462      NA      NA     0        NA        NA    0        0  0.14312       NA       NA    1  0.85688        1       NA       NA    0 0           0.14312       NA       NA  NA      NA      NA      NA      NA   0       0  0.14312      NA      NA   0      NA      NA  0 -0.29208  0.29208     NA     NA   0      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA        NA        NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA 0.73913         NA         NA      0.5
RIRW_CTS: 240000 240000 false  false  >=30    >=30    000000     000000     10         27 0.11111 0.03852      0.28058        NA        NA     0 0            0.12456        NA        NA 0.88889 0.71942 0.96148      NA      NA     0        NA        NA    0        0  0.12456       NA       NA    1  0.87544        1       NA       NA    0 0           0.12456       NA       NA  NA      NA      NA      NA      NA   0       0  0.12456      NA      NA   0      NA      NA  0 -0.2891   0.2891      NA     NA   0      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA        NA        NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA 0.77778         NA         NA      0.5
RIRW_CTS: 240000 240000 false  false  >=30    >=30    000000     000000     11         17 0       1.132e-17    0.18432        NA        NA     0 1.132e-17    0.18432        NA        NA 1       0.81568 1            NA      NA    NA        NA        NA   NA       NA NA             NA       NA    1  0.81568        1       NA       NA    0 1.132e-17   0.18432       NA       NA  NA      NA      NA      NA      NA  NA      NA NA            NA      NA  NA      NA      NA NA NA       NA           NA     NA  NA      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA        NA        NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA 1               NA         NA      0.5
RIRW_CTS: 240000 240000 false  false  >=30    >=30    000000     000000     92         10 0       2.0052e-17   0.27753        NA        NA     0 2.0052e-17   0.27753        NA        NA 1       0.72247 1            NA      NA    NA        NA        NA   NA       NA NA             NA       NA    1  0.72247        1       NA       NA    0 2.0052e-17  0.27753       NA       NA  NA      NA      NA      NA      NA  NA      NA NA            NA      NA  NA      NA      NA NA NA       NA           NA     NA  NA      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA        NA        NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA  NA      NA      NA      NA      NA   NA       NA       NA       NA       NA    NA        NA        NA 1               NA         NA      0.5

As you can see, it has spawned an -out_line_type CTS and I have no idea why.

[mollybsmith-noaa]

(To clarify, TC_STAT_OUTPUT_TEMPLATE is here set to tc_rirw_ctc_{ENV[BASIN]}.stat) which does end in .stat

[mollybsmith-noaa]

Just for the heck of it, I reordered the parameters being passed in, and this combination seems to produce the correct CTC output:
TC_STAT_JOB_ARGS = -job rirw -line_type TCMPR -rirw_time 24 -rirw_exact false -rirw_thresh ge30 -basin {ENV[BASIN]} -out_line_type CTC -out_stat {OUTPUT_BASE}/{ENV[MODEL]}/{ENV[YYYYMMDDHH]}/{TC_STAT_OUTPUT_TEMPLATE} -by LEAD,CYCLONE

I don't know if putting OUT_LINE_TYPE first did the trick or if it was putting LEAD before CYCLONE, but I'll do each in turn to find out.

[JohnHalleyGotway]

@mollybsmith-noaa, OK, I'm guessing this is actually an issue with the METplus StatAnalysis Python wrapper more so than the tc_stat MET tool itself. Specifically, my theory is that the embedded comma in -by LEAD,CYCLONE is causing a parsing issue. Can you please check to see if switching to using two -by options produces the desired result?

Please try changing FROM -by LEAD,CYCLONE TO -by LEAD -by CYCLONE.

If you find that does produce the expected result, we can loop @georgemccabe in to take a look at the suspected METplus parsing issue.

[mollybsmith-noaa]

@JohnHalleyGotway Will do! There definitely is an issue there, because if I do -by LEAD,CYCLONE it only stratifies by lead time, and if I do -by CYCLONE,LEAD it only stratifies by cyclone. Also, placing -out_line_type CTC -out_stat ... before the -by statement resulted in getting the correct CTC output. So that would make sense if it didn't parse anything after the comma.

[mollybsmith-noaa]

@JohnHalleyGotway Doing two -by statements does indeed fix everything. Good catch!

[georgemccabe]

@mollybsmith-noaa, you should be able to put quotation marks around the entire value for TC_STAT_JOB_ARGS to prevent it from splitting it into 2 sets of job args at the comma:

TC_STAT_JOB_ARGS = "-job rirw -line_type TCMPR -rirw_time 24 -rirw_exact false -rirw_thresh ge30 -basin {ENV[BASIN]} -out_line_type CTC -out_stat {OUTPUT_BASE}/{ENV[MODEL]}/{ENV[YYYYMMDDHH]}/{TC_STAT_OUTPUT_TEMPLATE} -by LEAD,CYCLONE"

[mollybsmith-noaa]

Ah, okay! I'm reprocessing everything with the dual -by clauses, but I'll try that afterwards. I think that should probably be clarified in the user's guide. I can try to do that when I get the chance.