merge develop and resolved conflicts

.github/parm/use_case_groups.json

@@ -59,11 +59,14 @@
     "index_list": "0-1",
     "run": false
   },
+  {
+    "category": "fire",
+    "index_list": "0",
+    "run": false
+  },
   {
     "category": "land_surface",
     "index_list": "0",
-    "disabled": true,
-    "disabled_reason": "need to check reason",
     "run": false
   },
   {

.github/workflows/update_input_data.yml

@@ -13,7 +13,7 @@ jobs:
       - run: |
           branch_name=$(echo ${{ github.event.inputs.branch || github.ref }} | cut -d/ -f3)
           echo "branch_name=$branch_name" >> "$GITHUB_ENV"
-      - uses: dtcenter/metplus-action-data-update@v2
+      - uses: dtcenter/metplus-action-data-update@v3
         with:
           docker_name: ${{ secrets.DOCKER_USERNAME }}
           docker_pass: ${{ secrets.DOCKER_PASSWORD }}

docs/Contributors_Guide/add_use_case.rst

@@ -48,6 +48,7 @@ one of the following:
 * climate
 * clouds
 * data_assimilation
+* fire
 * extremes
 * land_surface
 * marine_and_cryosphere

docs/Users_Guide/getting_started.rst

@@ -377,11 +377,13 @@ the path to the METplus log file that was generated.
 * Review the :ref:`metplus_final.conf<metplus_final_conf>` file to see all
   of the settings that were used in the use case.
 
+Containers
+==========
 
 .. _metplus-docker:
 
 METplus in Docker
-=================
+-----------------
 
 METplus is available on DockerHub. The METplus Docker image includes all of
 the MET executables from the corresponding METplus Coordinated Release and
@@ -400,7 +402,7 @@ on DockerHub.
 .. _docker-sample-input:
 
 Sample Input Data
------------------
+^^^^^^^^^^^^^^^^^
 
 Sample input data for all of the use cases provided with the METplus wrappers
 are also available on DockerHub. These data are found in the
@@ -426,7 +428,7 @@ The input data will be available inside the container under
 /data/input/METplus_Data.
 
 Running METplus in Docker
--------------------------
+^^^^^^^^^^^^^^^^^^^^^^^^^
 
 The run_metplus.py script is in the user's path inside the container.
 The use case configuration files can be found under
@@ -451,3 +453,78 @@ the corresponding category can be run::
 Please note that use cases that have additional Python package dependencies
 may not run successfully unless those packages are installed inside the
 container or obtained elsewhere.
+
+
+.. _metplus-apptainer:
+
+METplus in Apptainer
+--------------------
+
+Apptainer (formerly Singularity) can also be used to run METplus.
+Images can be pulled from DockerHub using Apptainer commands.
+
+If running on the NCAR Casper cluster, be sure to first load the Apptainer module via::
+
+    module load apptainer
+
+Navigate to a working directory and pull an image from DockerHub, e.g.::
+
+    apptainer pull docker://dtcenter/metplus:5.1-latest
+
+This will create a *.sif* file in the current directory,
+e.g. **metplus_5.1-latest.sif**.
+
+See :ref:`metplus-docker` for information about available images on DockerHub.
+
+.. _apptainer-sample-input:
+
+Sample Input Data
+^^^^^^^^^^^^^^^^^
+
+The Docker data volumes that contain sample input data provided on DockerHub are
+not compatible with Apptainer. Therefore, sample input data must be obtained from from the METplus Data website.
+Navigate to the `METplus Data <https://dtcenter.ucar.edu/dfiles/code/METplus/METplus_Data>`_
+website. Next, navigate to the directory that corresponds to the vX.Y version that
+will be run, e.g. `v5.1 <https://dtcenter.ucar.edu/dfiles/code/METplus/METplus_Data/v5.1>`_
+The names of sample data tar files include the corresponding use case category.
+
+Download one or more of the sample data tar files and uncompress them into
+a local directory. The directory that contains the uncompressed sample data
+will need to be mounted to the container to make the data available inside it.
+
+The **APPTAINER_BIND** environment variable can be used to mount directories to
+the container. The local directory and the container directory will be separated
+by a colon (:).
+
+Mount the local directory (./input_data in this example) to the
+/data/input/METplus_Data directory inside the container::
+
+    export APPTAINER_BIND="./input_data:/data/input/METplus_Data"
+
+
+Running METplus in Apptainer
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The METplus wrappers inside the container come pre-configured so that
+:ref:`sys_conf_input_base` is set to /data/input/METplus_Data.
+Mounting the local data directory to this path will allow the use cases
+to be run without the need to modify the default configuration.
+
+The :ref:`sys_conf_output_base` value is set to /data/output by default.
+This directory can also be mounted to a local directory so the output from
+a use case run can be accessed locally. Multiple directory mounts can be specified
+by separating them by comma::
+
+    export APPTAINER_BIND="./input_data:/data/input/METplus_Data,./output_data:/data/output"
+
+In this example, the directory **./output_data** will be populated with the
+output from the use case.
+
+To run a use case, run the **apptainer exec** command with the **.sif**
+file that was created by the **apptainer pull** command and the call to the
+**run_metplus.py** script as arguments::
+
+    apptainer exec metplus_5.1-latest.sif /metplus/METplus/ush/run_metplus.py /metplus/METplus/parm/use_cases/met_tool_wrapper/GridStat/GridStat.conf
+
+Note that the full path to the run_metplus.py script and the configuration file for the use case you are running must be provided.
+

docs/Users_Guide/quicksearch.rst

@@ -76,14 +76,15 @@ Use Cases by Application:
    | `Air Quality and Composition <../search.html?q=AirQualityAndCompAppUseCase&check_keywords=yes&area=default>`_
    | `Climate <../search.html?q=ClimateAppUseCase&check_keywords=yes&area=default>`_
    | `Clouds <../search.html?q=CloudsAppUseCase&check_keywords=yes&area=default>`_
-   | `Short Range <../search.html?q=ShortRangeAppUseCase&check_keywords=yes&area=default>`_
    | `Data Assimilation  <../search.html?q=DataAssimilationAppUseCase&check_keywords=yes&area=default>`_
    | `Ensemble  <../search.html?q=EnsembleAppUseCase&check_keywords=yes&area=default>`_
+   | `Fire  <../search.html?q=FireAppUseCase&check_keywords=yes&area=default>`_
    | `Land Surface <../search.html?q=LandSurfaceAppUseCase&check_keywords=yes&area=default>`_
    | `Marine and Cryosphere <../search.html?q=MarineAndCryosphereAppUseCase&check_keywords=yes&area=default>`_
    | `Medium Range <../search.html?q=MediumRangeAppUseCase&check_keywords=yes&area=default>`_
    | `PBL <../search.html?q=PBLAppUseCase&check_keywords=yes&area=default>`_
    | `Precipitation <../search.html?q=PrecipitationAppUseCase&check_keywords=yes&area=default>`_
+   | `Short Range <../search.html?q=ShortRangeAppUseCase&check_keywords=yes&area=default>`_
    | `Space Weather <../search.html?q=SpaceWeatherAppUseCase&check_keywords=yes&area=default>`_
    | `Subseasonal to Seasonal <../search.html?q=S2SAppUseCase&check_keywords=yes&area=default>`_
    | `Subseasonal to Seasonal: Madden-Julian Oscillation <../search.html?q=S2SMJOAppUseCase&check_keywords=yes&area=default>`_
@@ -95,14 +96,15 @@ Use Cases by Application:
    | **Air Quality and Composition**: *AirQualityAndCompAppUseCase*
    | **Climate**: *ClimateAppUseCase*
    | **Clouds**: *CloudsAppUseCase*
-   | **Short Range**: *ShortRangeAppUseCase*
    | **Data Assimilation**: *DataAssimilationAppUseCase*
    | **Ensemble**: *EnsembleAppUseCase*
+   | **Fire**: *FireAppUseCase*
    | **LandSurface**: *LandSurfaceAppUseCase*
    | **Marine and Cryosphere**: *MarineAndCryosphereAppUseCase*
    | **Medium Range**: *MediumRangeAppUseCase*
    | **PBL**: *PBLAppUseCase*
    | **Precipitation**: *PrecipitationAppUseCase*
+   | **Short Range**: *ShortRangeAppUseCase*
    | **Space Weather**: *SpaceWeatherAppUseCase*
    | **Subseasonal to Seasonal**: *S2SAppUseCase*
    | **Subseasonal to Seasonal: Madden-Julian Oscillation**: *S2SMJOAppUseCase*

docs/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter.py

@@ -0,0 +1,218 @@
+"""
+GridStat: WRF and MMA Fire Perimeter
+====================================
+
+model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter.conf
+
+"""
+##############################################################################
+# .. contents::
+#   :depth: 1
+#   :local:
+#   :backlinks: none
+
+##############################################################################
+# Scientific Objective
+# --------------------
+#
+# This use case demonstrates the use of GridStat to evaluate the performance of the 
+# fire spread forecast from the WRF-Fire model for the 416 fire in Colorado in 2018. 
+# Using available fire perimeter observations and WRF-Fire forecasts, 
+# contingency statistics are produced to evaluate the forecast performance 
+# relative to the observed fire.
+
+##############################################################################
+# Version Added
+# -------------
+#
+# METplus version 6.0
+
+##############################################################################
+# Datasets
+# --------
+#
+#
+# **Forecast:** WRF Fire
+#
+# **Observations:** Multimission Aircraft (MMA)
+#
+# **Location:** All of the input data required for this use case can be
+# found in a sample data tarball. Each use case category will have
+# one or more sample data tarballs. It is only necessary to download
+# the tarball with the use case’s dataset and not the entire collection
+# of sample data. Click here to access the METplus releases page and download sample data
+# for the appropriate release: https://github.com/dtcenter/METplus/releases
+# This tarball should be unpacked into the directory that you will
+# set the value of INPUT_BASE. See :ref:`running-metplus` section for more information.
+#
+
+
+##############################################################################
+# METplus Components
+# ------------------
+#
+# This use case uses the UserScript wrapper to run a Python script to that
+# converts KML fire perimeter files to the poly line format that can be read by
+# MET. Then it runs GenVxMask to create gridded MET NetCDF files from the poly
+# files. Then it runs GridStat to process the WRF fire forecast files and the
+# observation mask files.
+#
+
+##############################################################################
+# METplus Workflow
+# ----------------
+#
+# **Beginning time (INIT_BEG):** 2018-06-01 at 16Z
+#
+# **End time (INIT_END):** 2018-06-01 at 16Z
+#
+# **Increment between beginning and end times (INIT_INCREMENT):** None
+#
+# **Sequence of forecast leads to process (LEAD_SEQ):** 4 hour, 23 hour, 32 hour
+#
+# This use case processes 3 forecast leads initialized at 16Z on June 1, 2018, running 3 times. 
+# First, the UserScript tool is called. This tool calls a Python script to convert the 
+# kml shapefiles to polylines. Then, GenVxMask is run to convert the shapefile 
+# produced by UserScript to a netCDF mask, providing the observation input for GridStat. 
+# Finally, GridStat is called to compare the WRF-Fire forecast of fire area to the 
+# observation mask created using the GenVxMask tool. The GridStat call uses Python Embedding 
+# in order to read the WRF-Fire subgrid into GridStat.
+
+##############################################################################
+# METplus Configuration
+# ---------------------
+#
+# METplus first loads the default configuration file,
+# then it loads any configuration files passed to METplus via the command line
+# e.g. parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter.conf
+#
+# .. highlight:: bash
+# .. literalinclude:: ../../../../parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter.conf
+#
+
+##############################################################################
+# MET Configuration
+# -----------------
+#
+# METplus sets environment variables based on user settings in the METplus
+# configuration file. See :ref:`How METplus controls MET config file settings<metplus-control-met>` for more details.
+#
+# **YOU SHOULD NOT SET ANY OF THESE ENVIRONMENT VARIABLES YOURSELF! THEY WILL BE OVERWRITTEN BY METPLUS WHEN IT CALLS THE MET TOOLS!**
+#
+# If there is a setting in the MET configuration file that is currently
+# not supported by METplus you’d like to control, please refer to:
+# :ref:`Overriding Unsupported MET config file settings<met-config-overrides>`
+#
+# .. dropdown:: GridStatConfig_wrapped
+#
+#    .. highlight:: bash
+#    .. literalinclude:: ../../../../parm/met_config/GridStatConfig_wrapped
+#
+
+##############################################################################
+# Python Embedding
+# ----------------
+#
+# This use case uses a Python embedding script to read the WRF-Fire forecast into GridStat. 
+# The script hard codes settings directly from WRF-Fire for the WRF-Fire netCDF variable name (FIRE_AREA) 
+# as well as the format of the WRF-Fire forecast file template. The input directory and valid time 
+# are read from the METplus script. Once the file(s) are found, the script sets attributes for 
+# initiation and valid times, variable name, level, and units in MET format and defines a grid 
+# based on the input WRF-Fire netCDF file's subgrid. The data from the FIRE_AREA variable and MET attributes 
+# are then printed to be read in by GridStat.
+#
+# .. dropdown:: parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter/read_wrfout_fire.py
+#
+#    .. highlight:: python
+#    .. literalinclude:: ../../../../parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter/read_wrfout_fire.py
+# 
+# For more information on the basic requirements to utilize Python Embedding in METplus, 
+# please refer to the MET User’s Guide section on `Python embedding <https://met.readthedocs.io/en/latest/Users_Guide/appendixF.html#appendix-f-python-embedding>`_
+
+##############################################################################
+# User Scripting
+# --------------
+#
+# This use case calls a Python script to read MMA fire perimeter .kml files and convert them 
+# into a poly line file that can be read by GenVxMask. The script hard codes the filename template 
+# for the .kml files and a valid time format. This valid time format is provided by the METplus configuration file. 
+# The script the sets up a variable for the previous valid time and an output file path. 
+# If a .kml file is not found for the current valid time, the script searches for a .kml file 
+# from the previous hours. Once a file is found, the Python script parses the input file to find 
+# the set of coordinates that define the fire perimeter. These coordinates are then written to 
+# a text file (.poly file) in the order longitude, latitude, elevation for each point. 
+# This .poly file is then ready to be read in by the GenVxMask tool.
+#
+# .. dropdown:: parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter/find_and_read_fire_perim_poly.py
+#
+#    .. highlight:: python
+#    .. literalinclude:: ../../../../parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter/find_and_read_fire_perim_poly.py
+#
+
+
+
+##############################################################################
+# Running METplus
+# ---------------
+#
+# Pass the use case configuration file to the run_metplus.py script
+# along with any user-specific system configuration files if desired::
+#
+#    run_metplus.py /path/to/METplus/parm/use_cases/model_applications/fire/GridStat_fcstWRF_obsMMA_fire_perimeter.conf /path/to/user_system.conf
+#
+# See :ref:`running-metplus` for more information.
+#
+
+##############################################################################
+# Expected Output
+# ---------------
+#
+# A successful run will output the following both to the screen and to the logfile::
+#
+#   INFO: METplus has successfully finished running.
+#
+# Refer to the value set for **OUTPUT_BASE** to find where the output data was generated.
+# There are three groups of outputs. First, three polyline files resulting from the UserScript tool
+#
+# * poly/fire_perim_20180601_20.poly
+# * poly/fire_perim_20180602_15.poly
+# * poly/fire_perim_20180603_00.poly
+#
+# Second, three netCDF files resulting from the GenVxMask tool containing fire perimeter observations
+#
+# * mask/fire_perim_20180601_20_mask.nc
+# * mask/fire_perim_20180602_15_mask.nc
+# * mask/fire_perim_20180603_00_mask.nc
+#
+# Finally, six files from the GridStat tool run with Python Embedding. The .stat files contain the 
+# CTC and CTS line types for the FIRE_AREA variable at the given lead time (two lines total per stat file). 
+# The netCDF files contain the following five fields: lat (latitude), lon (longitude), 
+# FCST_FIRE_AREA_Z0_FULL (fire spread area from the WRF-Fire forecast), OBS_FIRE_PERIM_all_all_FULL (fire spread area from the .kml observations), 
+# and DIFF_FIRE_AREA_Z0_FIRE_PERIM_all_all_FULL (both the forecast and observed fire spread areas, including overlaps and differences).
+#
+# * grid_stat/2018060120/grid_stat_040000L_20180601_200000V.stat
+# * grid_stat/2018060120/grid_stat_040000L_20180601_200000V_pairs.nc
+# * grid_stat/2018060215/grid_stat_230000L_20180602_150000V.stat
+# * grid_stat/2018060215/grid_stat_230000L_20180602_150000V_pairs.nc
+# * grid_stat/2018060300/grid_stat_320000L_20180603_000000V.stat
+# * grid_stat/2018060300/grid_stat_320000L_20180603_000000V_pairs.nc
+#
+
+##############################################################################
+# Keywords
+# --------
+#
+# .. note::
+#
+#   * UserScriptUseCase
+#   * GenVxMaskToolUseCase
+#   * GridStatToolUseCase
+#   * PythonEmbeddingFileUseCase
+#   * GRIB2FileUseCase
+#   * FireAppUseCase
+#
+#   Navigate to the :ref:`quick-search` page to discover other similar use cases.
+#
+#
+#
+# sphinx_gallery_thumbnail_path = '_static/fire-GridStat_fcstWRF_obsMMA_fire_perimeter.png'