ALLOW_MISSING_INPUTS Functionality in MET v12.0.0 and METplus v6.0.0 beta3 Testing

[ShannonShields-NOAA]

I tested MET v12.0.0-beta3 and METplus v6.0.0-beta3 with the subseasonal component of EVS. To test missing input files for grid-to-obs (which was easier to test but grid-to-grid would be similar), I set POINT_STAT_ALLOW_MISSING_INPUTS = True and POINT_STAT_INPUT_THRESH = 0.8 in my PB2NC and Point_Stat config files.
The threshold breakdown for each verification period is as follows:
Weekly 80% thresh: 12 runs must be successful out of 15 total runs
Days 6-10 80% thresh: 9 runs must be successful out of 11 total runs
Weeks 3-4 80% thresh: 23 runs must be successful out of 29 total runs

Removing 2 obs (prepbufr_nam) files yielded a successful run for all periods as this was in the 80% threshold...only WARNINGs were issued.
"03/22 22:36:28.797 metplus.f24a1c59 (command_builder.py:684) DEBUG: Could not find OBS_INPUT file /lfs/h2/emc/stmp/shannon.shields/evs_test/prod/tmp/jevs_subseasonal_cfs_grid2obs_stats.122357069.cbqs01/grid2obs_stats/data/prepbufr_nam/prepbufr.nam.2024031900 using template /lfs/h2/emc/stmp/shannon.shields/evs_test/prod/tmp/jevs_subseasonal_cfs_grid2obs_stats.122357069.cbqs01/grid2obs_stats/data/pr
epbufr_nam/prepbufr.nam.{valid?fmt=%Y%m%d%H}03/22 22:36:28.797 metplus.f24a1c59 (command_builder.py:469) WARNING: Could not find observation file
03/22 22:36:28.797 metplus.f24a1c59 (runtime_freq_wrapper.py:437) WARNING: A problem occurred trying to obtain input files"
Output:/lfs/h2/emc/vpppg/noscrub/shannon.shields/MET_beta3_testing/evs_cfs_subseasonal_grid2obs_stats_new_met_test_success.txt

However, removing 3 obs (prepbufr_nam) files yielded ERROR:
"03/22 23:32:08.628 metplus.f191d242 (command_builder.py:252) ERROR: (runtime_freq_wrapper.py:199) 72.72727272727273% of PointStat runs had all required inputs. Must have 80.0% to prevent error. 3 out of 11 runs had missing inputs."
Output: /lfs/h2/emc/vpppg/noscrub/shannon.shields/MET_beta3_testing/evs_cfs_subseasonal_grid2obs_stats_new_met_test_fail.txt
Weekly and Weeks 3-4 periods were fine, but now the Days 6-10 period had too many missing files for its threshold and produced an ERROR, causing the entire job to fail. So the weekly periods that were fine were not processed through (no final .stat file when there should have been one).

George suggested to add an option to loop through the times and gather the pass/fail numbers, then skip running with a warning if the threshold is not met, otherwise loop through the times again and actually run the METplus tools. I think this might work if the word "WARNING" was replaced with something like "NOTE" or maybe "ATTENTION" to comply with NCO standards for operational code.

[j-opatz]

Hi Shannon,

And thank you for bringing this effect to our attention. At the moment, George is out of the office and as he was the one who worked primarily on issue #2460 I'm going to defer to his direction on how to proceed with the fix.

Will it cause any major negative effects to wait until his return to the office (April 11th) to pursue a fix, or do we need to get this patched up prior to that?

[ShannonShields-NOAA]

Hi John, No worries, this can wait until George's return. This missing input functionality is not an immediate requirement; it is something we have been discussing in our telecon meetings that would be nice to have in the future for our EVS code. Thanks, Shannon

…

On Fri, Mar 29, 2024 at 12:09 PM j-opatz ***@***.***> wrote: Hi Shannon, And thank you for bringing this effect to our attention. At the moment, George is out of the office and as he was the one who worked primarily on issue #2460 <#2460> I'm going to defer to his direction on how to proceed with the fix. Will it cause any major negative effects to wait until his return to the office (April 11th) to pursue a fix, or do we need to get this patched up prior to that? — Reply to this email directly, view it on GitHub <#2528 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP4MEUBH2WGKTF5XW5IERZLY2WG43AVCNFSM6AAAAABFNBLLBGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4DSNJTGUYDO> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Shannon Shields SAIC at NOAA/NWS/NCEP/EMC 5830 University Research Ct. Rm 2180 College Park, MD 20740 ***@***.***

[j-opatz]

Shannon, I wanted to reach out again to communicate the status on our side. As you are probably already aware, we are waiting on funds to proceed with much of our work; as a result, we've had to put a temporary halt on most projects and issues that are not mission critical.

Given this context, we'll have to hold off on any further development on this topic until funding is secured and priorities have been reassessed. I hope you understand this does not mean this issue is unimportant, and the METplus team wants to return to this issue when we are able and find you a more permanent fix than the current script solution you're using.

[ShannonShields-NOAA]

I totally understand the situation and appreciate the update. We can continue discussion on this missing input functionality in the future. Thank you John, Shannon

…

On Tue, Apr 16, 2024 at 2:45 PM j-opatz ***@***.***> wrote: Shannon, I wanted to reach out again to communicate the status on our side. As you are probably already aware, we are waiting on funds to proceed with much of our work; as a result, we've had to put a temporary halt on most projects and issues that are not mission critical. Given this context, we'll have to hold off on any further development on this topic until funding is secured and priorities have been reassessed. I hope you understand this does not mean this issue is unimportant, and the METplus team wants to return to this issue when we are able and find you a more permanent fix than the current script solution you're using. — Reply to this email directly, view it on GitHub <#2528 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AP4MEUC2AE27NHNGQR7FR73Y5VWVRAVCNFSM6AAAAABFNBLLBGVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TCMZTHA3DE> . You are receiving this because you authored the thread.Message ID: ***@***.***>

-- Shannon Shields SAIC at NOAA/NWS/NCEP/EMC 5830 University Research Ct. Rm 2180 College Park, MD 20740 ***@***.***

[georgemccabe]

To follow up here, we discussed these details in the METplus/NOAA Telecon on 6/10/2024. The details of the enhancements are described in the GitHub issue comments.

I am going to close this discussion and lock it to prevent future posts. @ShannonShields-NOAA, please continue any communication about this in GitHub issue #2524 or create a new discussion if appropriate. Thanks!