Need Point2Grid metplus wrapper for EVSv2.0

[Ho-ChunHuang-NOAA]

@ShelleyMelchior-NOAA @AliciaBentley-NOAA I need to use Point2Grid app to map GOES ABI AOD to model grid for EVS2 AOD verification for AQM, RAP, HRRR, and GEFS-Chem, but I can not find the wrapper on WCOSS2

/apps/ops/prod/libs/intel/19.1.3.304/metplus/5.0.1/parm/met_config/Point2GridConfig_wrapped

Can it be added to the next beta version of MET v12.0.0 and METplus v6.0.0?

[jprestop]

Hi @Ho-ChunHuang-NOAA.

I'm wondering if this is these are the files you're looking for?
/apps/ops/prod/libs/intel/19.1.3.304/metplus/5.0.1/parm/use_cases/met_tool_wrapper/Point2Grid/Point2Grid.conf
/apps/ops/para/libs/intel/19.1.3.304/metplus/6.0.0-beta3/parm/use_cases/met_tool_wrapper/Point2Grid/Point2Grid.conf

Here you can find the documentation for the Point2Grid: Basic Use Case.

[Ho-ChunHuang-NOAA]

The wrapper should have something like ${METPLUS_...}, that I can define and overwrite the default for my user's case in python code. John or George should know more about it.

example coding ==================
fcst = {
${METPLUS_FCST_FILE_TYPE}
${METPLUS_FCST_FIELD}
}

obs = {
${METPLUS_OBS_FILE_TYPE}
${METPLUS_OBS_FIELD}
}

${METPLUS_OUTPUT_PREFIX}

${METPLUS_MET_CONFIG_OVERRIDES}

[Ho-ChunHuang-NOAA]

Despite I think there should be a CONFIG Python warpper for Point2Grid to interpret the point2grid setting, I have additional question to ask on "how to specify the selected QC flag for processed AOD".

I commented out the "POINT2GRID_CONFIG_FILE=" in Point2Grid_hourly_obsABI.conf.txt. The Point2Grid ran to finish but the result is not what I expected. It is not taking the >> export AOD_QC_FLAG="0" << defined in exevs_aqm_aqm_grid2grid_prep.sh.txt and used in Point2Grid_hourly_obsABI.conf.txt, i.e. POINT2GRID_QC_FLAGS = {ENV[AOD_QC_FLAG]}.

I intended the Point2Grid to be ran like
point2grid ${aod_file} ${mod_file} ${out_file} -field 'name="AOD"; level="(*,*)";' -method UW_MEAN -v 2 -qc 0

But it actually ran as shown in line 1999 of jevs_aqm_g2g_abi_prep_00.o140897259.txt (i.e., without -qc)
cmd: /apps/ops/prod/libs/intel/19.1.3.304/met/11.0.2/bin/point2grid /lfs/h2/emc/physics/noscrub/ho-chun.huang/GOES16_AOD/AOD/20240401/OR_ABI-L2-AODC-M6_G16_s20240922101170_e20240922103543_c20240922107079.nc /lfs/h2/emc/ptmp/jianping.huang/emc.para/com/aqm/v7.0/aqm.20240401/06/aqm.t06z.cmaq.f001.793.grib2 /lfs/h2/emc/stmp/ho-chun.huang/evs_test/prod/tmp/jevs_aqm_g2g_abi_prep_00.140897259.dbqs01/prepsave/abi_AOD_aqm_g16_20240401
_21.nc -field name="AOD"; level="(*,*)"; -method UW_MEAN -v 2

I checked the result and it is similar to use the -qc "0,1,2". I wonder whether "0,1,2" is the default?

Please let me know how to set the QC flag to overwrite the default..

Point2Grid_hourly_obsABI.conf.txt
exevs_aqm_aqm_grid2grid_prep.sh.txt
jevs_aqm_g2g_abi_prep_00.o140897259.txt

[jprestop]

Hi @Ho-ChunHuang-NOAA. I am hoping that @j-opatz can help provide further assistance to you. @j-opatz are you able to help while George is out?

[jprestop]

Hi @Ho-ChunHuang-NOAA. Our team talked about your questions in a meeting today. Regarding the call to point2grid, please try moving the -qc 0 before the -v 2 and see if that makes any difference. You can see an example of a call to point2grid from our testing infrastructure in our unit_point2grid.xml file.

Regarding how the QC flag, should be set, we'd recommend getting some advice on that from Partha Bhattacharjee (@perthsb) (tagged here) who has done more extensive work with AOD data.

[jprestop]

Thanks, @perthsb. @Ho-ChunHuang-NOAA Looking at the file used for our testing:

DQF:long_name = "ABI L2+ Aerosol Optical Depth at 550 nm data quality flags" ;

it seems this is the relevant information:

DQF:flag_values = 0b, 1b, 2b, 3b ;
DQF:flag_meanings = "high_quality_retrieval_qf medium_quality_retrieval_qf low_quality_retrieval_qf no_retrieval_qf" ;

Please let us know how the run goes, moving the -qc option before the -v 2.

[jprestop]

Hi @Ho-ChunHuang-NOAA. I created the following issue, Enhancement: Add support for a Point2Grid config wrapper #2540 and have assigned it to @georgemccabe. Please note that George is out, returning on April 11th.

[jprestop]

Hi @Ho-ChunHuang-NOAA. Since we have created an issue for this work, I'd like to go ahead and mark this as an answer and move any discussion over to comments on that issue. Any objections?

[Ho-ChunHuang-NOAA]

@jprestop I am good with your decision. So we will continue point2grid discussion on issue 2540, correct?

[jprestop]

Yes, exactly. George, will be back on April 11th and should be able to address any questions related to the Point2Grid wrapper at that time. I'll go ahead and marked this discussion as answered and we can have any additional discussion in Issue 2540.

[Ho-ChunHuang-NOAA]

Hi, @jprestop. Thank you. It is fine since my application is for EVSv2.0. Thanks. again.

[perthsb]

Thanks Julie. However, I have not done anything with point2grid function. Partha

…

On Thu, Apr 4, 2024 at 5:57 PM Julie Prestopnik ***@***.***> wrote: Hi @Ho-ChunHuang-NOAA <https://github.com/Ho-ChunHuang-NOAA>. Our team talked about your questions in a meeting today. Regarding the call to point2grid, please try moving the -qc 0 before the -v 2 and see if that makes any difference. You can see an example <https://github.com/dtcenter/MET/blob/a72e8c56745bc3b500ff2c088363060c70222e0c/internal/test_unit/xml/unit_point2grid.xml#L123> of a call to point2grid from our testing infrastructure in our unit_point2grid.xml file. Regarding how the QC flag, should be set, we'd recommend getting some advice on that from Partha Bhattacharjee ***@***.*** <https://github.com/perthsb>) (tagged here) who has done more extensive work with AOD data. — Reply to this email directly, view it on GitHub <#2539 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AVARVDS76XZ77TW4HF4JVR3Y3XEFTAVCNFSM6AAAAABFV2MWM6VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TAMJUHEYTE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Ho-ChunHuang-NOAA]

@jprestop Per request, I commented out the run_metplus.py and use the point2grid directly in in exevs_aqm_aqm_grid2grid_prep.sh.txt

## run_metplus.py ${conf_dir}/${config_file} ${config_common}

out_file=${RUNTIME_PREP_DIR}/${DATA_TYPE}AOD${MODELNAME}${satellite_name}${VDATE}_${VHOUR}.nc

point2grid ${filein_aod} ${filein_mdl_grid} ${out_file} -field 'name="AOD"; level="(*,*)";' -method UW_MEAN -v 2 -qc ${AOD_QC_FLAG}

and

point2grid ${filein_aod} ${filein_mdl_grid} ${out_file} -field 'name="AOD"; level="(*,*)";' -method UW_MEAN -qc ${AOD_QC_FLAG} -v 2

generated same and correct result, i.e., high quality AOD regridding.

I worked with John and George to create the capability of GOES AOD mapping. The qc_flag setting are

        for i in "${qc_flag[@]}"; do
           case ${i} in
              high) value="0"
                    ;;
              medium) value="0,1"
                      ;;
              low) value="0,1,2"
                   ;;
              med_only) value="1"
                  ;;
              low_only) value="2"
                  ;;
              *) echo "qc_flag=${i} is not defined"
                 exit;;
           esac