From fc15ea51bcd26825bee442e7d006091d9f32d296 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cgspetro-NOAA=E2=80=9D?= Date: Wed, 11 Oct 2023 22:31:34 -0400 Subject: [PATCH 01/43] include info on NA_13km grid --- .../CustomizingTheWorkflow/LAMGrids.rst | 49 +++++++++++++------ 1 file changed, 33 insertions(+), 16 deletions(-) diff --git a/docs/UsersGuide/source/CustomizingTheWorkflow/LAMGrids.rst b/docs/UsersGuide/source/CustomizingTheWorkflow/LAMGrids.rst index db04ad5043..618f301eed 100644 --- a/docs/UsersGuide/source/CustomizingTheWorkflow/LAMGrids.rst +++ b/docs/UsersGuide/source/CustomizingTheWorkflow/LAMGrids.rst @@ -6,18 +6,19 @@ Limited Area Model (:term:`LAM`) Grids: Predefined and User-Generated Options In order to set up the workflow and generate an experiment with the SRW Application, the user must choose between various predefined :term:`FV3`-:term:`LAM` grids or generate a user-defined grid. At this time, full support is only provided to those using one of the four predefined -grids supported in the v2.1.0 release, but other predefined grids are available (see :numref:`Section %s ` for more detail). Preliminary information is also provided at the end of this chapter describing how users can leverage the SRW App workflow scripts to generate their own user-defined grid and/or adjust the number of vertical levels in the grid. Currently, this feature is not fully supported and is "use at your own risk." +grids supported in the v2.2.0 release, but other predefined grids are available (see :numref:`Section %s ` for more detail). Preliminary information is also provided at the end of this chapter describing how users can leverage the SRW App workflow scripts to generate their own user-defined grid and/or adjust the number of vertical levels in the grid. Currently, these features are not fully supported and are "use at your own risk." Predefined Grids ================= -The SRW App v2.1.0 release includes four predefined limited area model (:term:`LAM`) grids. To select a supported predefined grid, the ``PREDEF_GRID_NAME`` variable within the ``workflow:`` section of the ``config.yaml`` script must be set to one of the following four options: +The SRW App v2.2.0 release includes five predefined limited area model (:term:`LAM`) grids. To select a supported predefined grid, the ``PREDEF_GRID_NAME`` variable within the ``workflow:`` section of the ``config.yaml`` script must be set to one of the following five options: * ``RRFS_CONUS_3km`` * ``RRFS_CONUS_13km`` * ``RRFS_CONUS_25km`` * ``SUBCONUS_Ind_3km`` +* ``RRFS_NA_13km`` -These four options are provided for flexibility related to compute resources and supported physics options. Other predefined grids are listed :ref:`here `. The high-resolution 3-km :term:`CONUS` grid generally requires more compute power and works well with three of the five supported physics suites (see :numref:`Table %s `). Low-resolution grids (i.e., 13-km and 25-km domains) require less compute power and should generally be used with the other supported physics suites: ``FV3_GFS_v16`` and ``FV3_RAP``. +These five options are provided for flexibility related to compute resources and supported physics options. Other predefined grids are listed :ref:`here `. The high-resolution 3-km :term:`CONUS` grid generally requires more compute power and works well with three of the five supported physics suites (see :numref:`Table %s `). Low-resolution grids (i.e., 13-km and 25-km domains) require less compute power and should generally be used with the other supported physics suites: ``FV3_GFS_v16`` and ``FV3_RAP``. .. _GridPhysicsCombos: @@ -42,6 +43,10 @@ These four options are provided for flexibility related to compute resources and | | | | | FV3_RAP | +-------------------+------------------+ + | RRFS_NA_13km | FV3_RAP | + | | | + | | FV3_GFS_v16 | + +-------------------+------------------+ | RRFS_CONUS_25km | FV3_GFS_v16 | | | | | | FV3_RAP | @@ -49,7 +54,7 @@ These four options are provided for flexibility related to compute resources and In theory, it is possible to run any of the supported physics suites with any of the predefined grids, but the results will be more accurate and meaningful with appropriate grid/physics pairings. -The predefined :term:`CONUS` grids follow the naming convention (e.g., ``RRFS_CONUS_*km``) of the 3-km version of the continental United States (CONUS) grid being tested for the Rapid Refresh Forecast System (:term:`RRFS`). RRFS will be a convection-allowing, hourly-cycled, :term:`FV3`-:term:`LAM`-based ensemble planned for operational implementation in 2025. All four supported grids were created to fit completely within the High Resolution Rapid Refresh (`HRRR `_) domain to allow for use of HRRR data to initialize the SRW App. +The predefined :term:`CONUS` grids follow the naming convention (e.g., ``RRFS_CONUS_*km``) of the 3-km version of the continental United States (CONUS) grid being tested for the Rapid Refresh Forecast System (:term:`RRFS`). RRFS will be a convection-allowing, hourly-cycled, :term:`FV3`-:term:`LAM`-based ensemble planned for operational implementation in 2025. The four supported ``RRFS_*`` grids were created to fit completely within the High Resolution Rapid Refresh (`HRRR `__) domain to allow for use of HRRR data to initialize the SRW App. The ``RRFS_NA_13km`` grid covers all of North America and therefore requires different data to initialize. Predefined 3-km CONUS Grid ----------------------------- @@ -59,12 +64,12 @@ The 3-km CONUS domain is ideal for running the ``FV3_RRFS_v1beta`` physics suite .. _RRFS_CONUS_3km: .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/RRFS_CONUS_3km.sphr.native_wrtcmp.png - :alt: Map of the continental United States 3 kilometer domain. The computational grid boundaries appear in red and the write-component grid appears just inside the computational grid boundaries in blue. + :alt: Map of the continental United States 3 kilometer domain. The computational grid boundaries appear in red and the write component grid appears just inside the computational grid boundaries in blue. - *The boundary of the RRFS_CONUS_3km computational grid (red) and corresponding write-component grid (blue).* + *The boundary of the RRFS_CONUS_3km computational grid (red) and corresponding write component grid (blue).* -The boundary of the ``RRFS_CONUS_3km`` domain is shown in :numref:`Figure %s ` (in red), and the boundary of the :ref:`write-component grid ` sits just inside the computational domain (in blue). This extra grid is required because the post-processing utility (:term:`UPP`) is unable to process data on the native FV3 gnomonic grid (in red). Therefore, model data are interpolated to a Lambert conformal grid (the write component grid) in order for the :term:`UPP` to read in and correctly process the data. +The boundary of the ``RRFS_CONUS_3km`` domain is shown in :numref:`Figure %s ` (in red), and the boundary of the :ref:`write component grid ` sits just inside the computational domain (in blue). This extra grid is required because the post-processing utility (:term:`UPP`) is unable to process data on the native FV3 gnomonic grid (in red). Therefore, model data are interpolated to a Lambert conformal grid (the write component grid) in order for the :term:`UPP` to read in and correctly process the data. .. note:: While it is possible to initialize the FV3-LAM with coarser external model data when using the ``RRFS_CONUS_3km`` domain, it is generally advised to use external model data (such as HRRR or RAP data) that has a resolution similar to that of the native FV3-LAM (predefined) grid. @@ -76,33 +81,45 @@ Predefined SUBCONUS Grid Over Indianapolis .. _SUBCONUS_Ind_3km: .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/SUBCONUS_Ind_3km.png - :alt: Map of Indiana and portions of the surrounding states. The map shows the boundaries of the continental United States sub-grid centered over Indianapolis. The computational grid boundaries appear in red and the write-component grid appears just inside the computational grid boundaries in blue. + :alt: Map of Indiana and portions of the surrounding states. The map shows the boundaries of the continental United States sub-grid centered over Indianapolis. The computational grid boundaries appear in red and the write component grid appears just inside the computational grid boundaries in blue. - *The boundary of the SUBCONUS_Ind_3km computational grid (red) and corresponding write-component grid (blue).* + *The boundary of the SUBCONUS_Ind_3km computational grid (red) and corresponding write component grid (blue).* The ``SUBCONUS_Ind_3km`` grid covers only a small section of the :term:`CONUS` centered over Indianapolis. Like the ``RRFS_CONUS_3km`` grid, it is ideally paired with the ``FV3_RRFS_v1beta``, ``FV3_HRRR``, or ``FV3_WoFS`` physics suites, since these are all convection-allowing physics suites designed to work well on high-resolution grids. -Predefined 13-km Grid ------------------------- +Predefined 13-km CONUS Grid +----------------------------- .. _RRFS_CONUS_13km: .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/RRFS_CONUS_13km.sphr.native_wrtcmp.png - :alt: Map of the continental United States 13 kilometer domain. The computational grid boundaries appear in red and the write-component grid appears just inside the computational grid boundaries in blue. + :alt: Map of the continental United States 13 kilometer domain. The computational grid boundaries appear in red and the write component grid appears just inside the computational grid boundaries in blue. - *The boundary of the RRFS_CONUS_13km computational grid (red) and corresponding write-component grid (blue).* + *The boundary of the RRFS_CONUS_13km computational grid (red) and corresponding write component grid (blue).* The ``RRFS_CONUS_13km`` grid (:numref:`Fig. %s `) covers the full :term:`CONUS`. This grid is meant to be run with the ``FV3_GFS_v16`` or ``FV3_RAP`` physics suites. These suites use convective :term:`parameterizations`, whereas the other supported suites do not. Convective parameterizations are necessary for low-resolution grids because convection occurs on scales smaller than 25-km and 13-km. +Predefined 13-km North American Grid +-------------------------------------- + +.. _RRFS_NA_13km: + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/RRFS_NA_13km.sphr.native_wrtcmp.png + :alt: Map of the North American 13 kilometer domain. The computational grid boundaries appear in red and the write component grid appears just inside the computational grid boundaries in blue. + + *The boundary of the RRFS_NA_13km computational grid (red) and corresponding write component grid (blue).* + +The ``RRFS_NA_13km`` grid (:numref:`Fig. %s `) covers all of North America. This grid was designed to run with the ``FV3_RAP`` physics suite but can also be run with the ``FV3_GFS_v16`` suite. These suites use convective :term:`parameterizations`, whereas the other supported suites do not. Convective parameterizations are necessary for low-resolution grids because convection occurs on scales smaller than 25-km and 13-km. + Predefined 25-km Grid ------------------------ .. _RRFS_CONUS_25km: .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/RRFS_CONUS_25km.sphr.native_wrtcmp.png - :alt: Map of the continental United States 25 kilometer domain. The computational grid boundaries appear in red and the write-component grid appears just inside the computational grid boundaries in blue. + :alt: Map of the continental United States 25 kilometer domain. The computational grid boundaries appear in red and the write component grid appears just inside the computational grid boundaries in blue. - *The boundary of the RRFS_CONUS_25km computational grid (red) and corresponding write-component grid (blue).* + *The boundary of the RRFS_CONUS_25km computational grid (red) and corresponding write component grid (blue).* The final predefined :term:`CONUS` grid (:numref:`Fig. %s `) uses a 25-km resolution and is meant mostly for quick testing to ensure functionality prior to using a higher-resolution domain. @@ -183,7 +200,7 @@ The following is an example of a code stanza for "NEW_GRID" to be added to ``pre LAYOUT_Y: 2 BLOCKSIZE: 40 - # Parameters for the write-component (aka "quilting") grid. + # Parameters for the write component (aka "quilting") grid. QUILTING: WRTCMP_write_groups: 1 From 61d55cfbc902c8a73c1340063efd4bd969051044 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cgspetro-NOAA=E2=80=9D?= Date: Wed, 11 Oct 2023 23:02:33 -0400 Subject: [PATCH 02/43] automate substitutions for name of workflow env --- .../source/BuildingRunningTesting/AQM.rst | 4 ++-- .../ContainerQuickstart.rst | 2 +- .../BuildingRunningTesting/Quickstart.rst | 2 +- .../source/BuildingRunningTesting/RunSRW.rst | 20 +++++++++---------- .../BuildingRunningTesting/Tutorial.rst | 2 +- docs/UsersGuide/source/Reference/FAQ.rst | 4 ++-- docs/UsersGuide/source/conf.py | 7 +++++++ 7 files changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/UsersGuide/source/BuildingRunningTesting/AQM.rst b/docs/UsersGuide/source/BuildingRunningTesting/AQM.rst index 4c83bcee2b..409524327d 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/AQM.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/AQM.rst @@ -68,7 +68,7 @@ If the SRW-AQM builds correctly, users should see the standard executables liste * - nexus - Runs the NOAA Emission and eXchange Unified System (:ref:`NEXUS `) emissions processing system -Load the ``workflow_tools`` Environment +Load the |wflow_env| Environment -------------------------------------------- Load the python environment for the workflow: @@ -90,7 +90,7 @@ If the console outputs a message, the user should run the commands specified in Please do the following to activate conda: > conda activate workflow_tools -then the user should run ``conda activate workflow_tools``. Otherwise, the user can continue with configuring the workflow. +then the user should run |activate|. Otherwise, the user can continue with configuring the workflow. .. _AQMConfig: diff --git a/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst b/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst index bcc17e73ce..3c4fc7de3f 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/ContainerQuickstart.rst @@ -273,7 +273,7 @@ The ``wflow_`` modulefile will then output instructions to activate th Please do the following to activate conda: > conda activate workflow_tools -then the user should run ``conda activate workflow_tools``. This will activate the ``workflow_tools`` conda environment. The command(s) will vary from system to system, but the user should see ``(workflow_tools)`` in front of the Terminal prompt at this point. +then the user should run |activate|. This will activate the |wflow_env| conda environment. The command(s) will vary from system to system, but the user should see |prompt| in front of the Terminal prompt at this point. .. _SetUpConfigFileC: diff --git a/docs/UsersGuide/source/BuildingRunningTesting/Quickstart.rst b/docs/UsersGuide/source/BuildingRunningTesting/Quickstart.rst index df5a61b1ef..9aa119a9eb 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/Quickstart.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/Quickstart.rst @@ -62,7 +62,7 @@ For a detailed explanation of how to build and run the SRW App on any supported Please do the following to activate conda: > conda activate workflow_tools - then the user should run ``conda activate workflow_tools`` to activate the workflow environment. + then the user should run |activate| to activate the workflow environment. #. Configure the experiment: diff --git a/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst b/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst index c7ea16d6b0..8d8609b8b5 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/RunSRW.rst @@ -111,7 +111,7 @@ The first two steps depend on the platform being used and are described here for Load the Conda/Python Environment ------------------------------------ -The SRW App workflow is often referred to as the *regional workflow* because it runs experiments on a regional scale (unlike the *global workflow* used in other applications). The SRW App workflow requires installation of Python3 using conda; it also requires additional packages built in a separate conda evironment named ``workflow_tools``. On Level 1 systems, a ``workflow_tools`` environment already exists, and users merely need to load the environment. On Level 2-4 systems, users must create and then load the environment. The process for each is described in detail below. +The SRW App workflow is often referred to as the *regional workflow* because it runs experiments on a regional scale (unlike the *global workflow* used in other applications). The SRW App workflow requires installation of Python3 using conda; it also requires additional packages built in a separate conda evironment named |wflow_env|. On Level 1 systems, a |wflow_env| environment already exists, and users merely need to load the environment. On Level 2-4 systems, users must create and then load the environment. The process for each is described in detail below. .. _Load-WF-L1: @@ -122,7 +122,7 @@ Loading the Workflow Environment on Level 1 Systems Users on a Level 2-4 system should skip to the :ref:`next section ` for instructions. -The ``workflow_tools`` conda/Python environment has already been set up on Level 1 platforms and can be activated in the following way: +The |wflow_env| conda/Python environment has already been set up on Level 1 platforms and can be activated in the following way: .. code-block:: console @@ -142,7 +142,7 @@ The ``wflow_`` modulefile will then output instructions to activate th Please do the following to activate conda: > conda activate workflow_tools -then the user should run ``conda activate workflow_tools``. This activates the ``workflow_tools`` conda environment, and the user typically sees ``(workflow_tools)`` in front of the Terminal prompt at this point. +then the user should run |activate|. This activates the |wflow_env| conda environment, and the user typically sees |prompt| in front of the Terminal prompt at this point. After loading the workflow environment, users may continue to :numref:`Section %s ` for instructions on setting the experiment configuration parameters. @@ -181,10 +181,10 @@ MacOS requires the installation of a few additional packages and, possibly, an u .. _LinuxMacVEnv: -Creating the ``workflow_tools`` Environment on Linux and Mac OS +Creating the |wflow_env| Environment on Linux and Mac OS """"""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" -On generic Mac and Linux systems, users need to create a conda ``workflow_tools`` environment. The environment can be stored in a local path, which could be a default location or a user-specified location (e.g., ``$HOME/condaenv/venvs/`` directory). (To determine the default location, use the ``conda info`` command, and look for the ``envs directories`` list.) The following is a brief recipe for creating a virtual conda environment on non-Level 1 platforms. It uses the aarch64 (64-bit ARM) Miniforge for Linux and installs into $HOME/conda. Adjust as necessary for your target system. +On generic Mac and Linux systems, users need to create a conda |wflow_env| environment. The environment can be stored in a local path, which could be a default location or a user-specified location (e.g., ``$HOME/condaenv/venvs/`` directory). (To determine the default location, use the ``conda info`` command, and look for the ``envs directories`` list.) The following is a brief recipe for creating a virtual conda environment on non-Level 1 platforms. It uses the aarch64 (64-bit ARM) Miniforge for Linux and installs into $HOME/conda. Adjust as necessary for your target system. .. code-block:: console @@ -211,12 +211,12 @@ See the `workflow-tools repository `` File `````````````````````````````````````` -Users can copy one of the provided ``wflow_`` files from the ``modulefiles`` directory and use it as a template to create a ``wflow_`` file that functions on their system. The ``wflow_macos`` and ``wflow_linux`` template modulefiles are provided as a starting point, but any ``wflow_`` file could be used. Users must modify the files to provide paths for python, miniconda modules, module loads, conda initialization, and the user's ``workflow_tools`` conda environment. +Users can copy one of the provided ``wflow_`` files from the ``modulefiles`` directory and use it as a template to create a ``wflow_`` file that functions on their system. The ``wflow_macos`` and ``wflow_linux`` template modulefiles are provided as a starting point, but any ``wflow_`` file could be used. Users must modify the files to provide paths for python, miniconda modules, module loads, conda initialization, and the user's |wflow_env| conda environment. Load the Workflow Environment ``````````````````````````````` -After creating a ``workflow_tools`` environment and making modifications to a ``wflow_`` file, users can run the commands below to activate the workflow environment: +After creating a |wflow_env| environment and making modifications to a ``wflow_`` file, users can run the commands below to activate the workflow environment: .. code-block:: console @@ -236,10 +236,10 @@ The ``wflow_`` modulefile will then output the following instructions: Please do the following to activate conda: > conda activate workflow_tools -After running ``conda activate workflow_tools``, the user will typically see ``(workflow_tools)`` in front of the Terminal prompt. This indicates that the workflow environment has been loaded successfully. +After running |activate|, the user will typically see |prompt| in front of the Terminal prompt. This indicates that the workflow environment has been loaded successfully. .. note:: - ``conda`` needs to be initialized before running ``conda activate workflow_tools`` command. Depending on the user's system and login setup, this may be accomplished in a variety of ways. Conda initialization usually involves the following command: ``source /etc/profile.d/conda.sh``, where ```` is the base conda installation directory. + ``conda`` needs to be initialized before running |activate| command. Depending on the user's system and login setup, this may be accomplished in a variety of ways. Conda initialization usually involves the following command: ``source /etc/profile.d/conda.sh``, where ```` is the base conda installation directory. .. _ExptConfig: @@ -416,7 +416,7 @@ A correct ``config.yaml`` file will output a ``SUCCESS`` message. A ``config.yam .. hint:: - * The ``workflow_tools`` environment must be loaded for the ``config_utils.py`` script to validate the ``config.yaml`` file. + * The |wflow_env| environment must be loaded for the ``config_utils.py`` script to validate the ``config.yaml`` file. * Valid values for configuration variables should be consistent with those in the ``ush/valid_param_vals.yaml`` script. diff --git a/docs/UsersGuide/source/BuildingRunningTesting/Tutorial.rst b/docs/UsersGuide/source/BuildingRunningTesting/Tutorial.rst index c27544f54e..ef06840a4b 100644 --- a/docs/UsersGuide/source/BuildingRunningTesting/Tutorial.rst +++ b/docs/UsersGuide/source/BuildingRunningTesting/Tutorial.rst @@ -65,7 +65,7 @@ To load the workflow environment, source the lmod-setup file. Then load the work where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values). -After loading the workflow, users should follow the instructions printed to the console. Usually, the instructions will tell the user to run ``conda activate workflow_tools``. For example, a user on Hera with permissions on the ``nems`` project may issue the following commands to load the workflow (replacing ``User.Name`` with their actual username): +After loading the workflow, users should follow the instructions printed to the console. Usually, the instructions will tell the user to run |activate|. For example, a user on Hera with permissions on the ``nems`` project may issue the following commands to load the workflow (replacing ``User.Name`` with their actual username): .. code-block:: console diff --git a/docs/UsersGuide/source/Reference/FAQ.rst b/docs/UsersGuide/source/Reference/FAQ.rst index 237980789c..f178545a18 100644 --- a/docs/UsersGuide/source/Reference/FAQ.rst +++ b/docs/UsersGuide/source/Reference/FAQ.rst @@ -226,7 +226,7 @@ In addition to the options above, many standard terminal commands can be run to How can I run a new experiment? ================================== -To run a new experiment at a later time, users need to rerun the commands in :numref:`Section %s ` that reactivate the *workflow_tools* environment: +To run a new experiment at a later time, users need to rerun the commands in :numref:`Section %s ` that reactivate the |wflow_env| environment: .. code-block:: console @@ -234,7 +234,7 @@ To run a new experiment at a later time, users need to rerun the commands in :nu module use /path/to/modulefiles module load wflow_ -Follow any instructions output by the console (e.g., ``conda activate workflow_tools``). +Follow any instructions output by the console (e.g., |activate|). Then, users can configure a new experiment by updating the environment variables in ``config.yaml`` to reflect the desired experiment configuration. Detailed instructions can be viewed in :numref:`Section %s `. Parameters and valid values are listed in :numref:`Section %s `. After adjusting the configuration file, generate the new experiment by running ``./generate_FV3LAM_wflow.py``. Check progress by navigating to the ``$EXPTDIR`` and running ``rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10``. diff --git a/docs/UsersGuide/source/conf.py b/docs/UsersGuide/source/conf.py index a6023d339d..4e1059f6d0 100644 --- a/docs/UsersGuide/source/conf.py +++ b/docs/UsersGuide/source/conf.py @@ -85,6 +85,13 @@ # The name of the Pygments (syntax highlighting) style to use. pygments_style = 'sphinx' +# Documentation-wide substitutions + +rst_prolog = """ +.. |wflow_env| replace:: ``workflow_tools`` +.. |activate| replace:: ``conda activate workflow_tools`` +.. |prompt| replace:: ``(workflow_tools)`` +""" # -- Options for HTML output ------------------------------------------------- From 91a0c0b3ab621fb6ff2fd7ba5ce431fb1eb63258 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?=E2=80=9Cgspetro-NOAA=E2=80=9D?= Date: Wed, 11 Oct 2023 23:03:58 -0400 Subject: [PATCH 03/43] updates to intro --- docs/UsersGuide/source/BackgroundInfo/Introduction.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/UsersGuide/source/BackgroundInfo/Introduction.rst b/docs/UsersGuide/source/BackgroundInfo/Introduction.rst index f4aa344fa7..b59917e816 100644 --- a/docs/UsersGuide/source/BackgroundInfo/Introduction.rst +++ b/docs/UsersGuide/source/BackgroundInfo/Introduction.rst @@ -11,10 +11,10 @@ The UFS includes `multiple applications ` for a detailed summary of updates that came in ahead of the v2.1.0 release. - * Support for the :term:`UPP` inline post option (see :ref:`here `) + * Introduced option to change vertical coordinate file (`PR #813 `__) and :ref:`instructions on how to use this feature `. * Documentation updates to reflect the changes above The SRW App v2.1.0 citation is as follows and should be used when presenting results based on research conducted with the App: From 3b8f2467746ef68d76093d352323659f8364fdea Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Wed, 31 Jul 2024 11:54:15 -0400 Subject: [PATCH 04/43] Tutorial information for Halloween storm --- .../BuildingRunningTesting/Tutorial.rst | 231 ++++++++++++++++++ 1 file changed, 231 insertions(+) diff --git a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst index 445dee1b8f..93636ef864 100644 --- a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst @@ -581,6 +581,237 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t *Halloween Storm 2019* +Data +------- + +On :srw-wiki:`Level 1 ` systems, users can find data for the Halloween Storm in the usual input model data locations (see :numref:`Section %s ` for a list). The data can also be downloaded from the `UFS SRW Halloween Storm Case Study `__. + +Load the workflow +--------------------- + +To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment. From the ``ufs-srweather-app`` directory, run: + +.. code-block:: console + + source etc/lmod-setup.sh # OR: source etc/lmod-setup.csh when running in a csh/tcsh shell + module use modulefiles + module load wflow_ + +where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values). + +After loading the workflow, users should follow the instructions printed to the console. Usually, the instructions will tell the user to run |activate|. For example, a user on Hera with permissions on the ``nems`` project may issue the following commands to load the workflow (replacing ``User.Name`` with their actual username): + +.. code-block:: console + + source /scratch1/NCEPDEV/nems/User.Name/ufs-srweather-app/etc/lmod-setup.sh hera + module use /scratch1/NCEPDEV/nems/User.Name/ufs-srweather-app/modulefiles + module load wflow_hera + conda activate srw_app + +Configuration +------------------------- + +Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: + +.. code-block:: console + + cd /path/to/ufs-srweather-app/ush + cp config.community.yaml config.yaml + +Users can save the location of the ``ush`` directory in an environment variable (``$USH``). This makes it easier to navigate between directories later. For example: + +.. code-block:: console + + export USH=/path/to/ufs-srweather-app/ush + +Users should substitute ``/path/to/ufs-srweather-app/ush`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $USH``, and it will take them to the ``ush`` directory. The variable will need to be reset for each login session. + +Experiment 1: Control +^^^^^^^^^^^^^^^^^^^^^^^^ + +Edit the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below. + +.. Hint:: + + To open the configuration file in the command line, users may run the command: + + .. code-block:: console + + vi config.yaml + + To modify the file, hit the ``i`` key and then make any changes required. To close and save, hit the ``esc`` key and type ``:wq`` to write the changes to the file and exit/quit the file. Users may opt to use their preferred code editor instead. + +Start in the ``user:`` section and change the ``MACHINE`` and ``ACCOUNT`` variables. For example, when running on a personal MacOS device, users might set: + +.. code-block:: console + + user: + RUN_ENVIR: community + MACHINE: macos + ACCOUNT: none + +For a detailed description of these variables, see :numref:`Section %s `. + +Users do not need to change the ``platform:`` section of the configuration file for this tutorial. The default parameters in the ``platform:`` section pertain to METplus verification, which is not addressed here. For more information on verification, see :numref:`Section %s `. + +In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PREDEF_GRID_NAME``. + +.. code-block:: console + + workflow: + USE_CRON_TO_RELAUNCH: false + EXPT_SUBDIR: control + CCPP_PHYS_SUITE: FV3_GFS_v16 + PREDEF_GRID_NAME: RRFS_CONUS_13km + DATE_FIRST_CYCL: '2019102812' + DATE_LAST_CYCL: '2019102812' + FCST_LEN_HRS: 6 + PREEXISTING_DIR_METHOD: rename + VERBOSE: true + COMPILER: intel + +.. _CronNote: + +.. note:: + + Users may also want to set ``USE_CRON_TO_RELAUNCH: true`` and add ``CRON_RELAUNCH_INTVL_MNTS: 3``. This will automate submission of workflow tasks when running the experiment. However, not all systems have :term:`cron`. + +``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``control`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different forecasts --- one that uses the FV3_GFS_v16 physics suite and one that uses the FV3_RRFS_v1beta physics suite --- "gfsv16_physics_fcst" could be a good alternative directory name. + +``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. Using the RRFS_CONUS_13km grid allows for fewer computational restraints compared to the 25km grid. For more information on this grid, see :numref:`Section %s `. + +For a detailed description of other ``workflow:`` variables, see :numref:`Section %s `. + +To turn on the plotting for the experiment, the YAML configuration file +should be included in the ``rocoto:tasks:taskgroups:`` section, like this: + +.. code-block:: console + + rocoto: + tasks: + task_get_extrn_ics: + walltime: 06:00:00 + task_get_extrn_lbcs: + walltime: 06:00:00 + metatask_run_ensemble: + task_make_lbcs_mem#mem#: + walltime: 06:00:00 + task_run_fcst_mem#mem#: + walltime: 06:00:00 + taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' + + +For more information on how to turn on/off tasks in the workflow, please +see :numref:`Section %s `. + +In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_SOURCE_BASEDIR_ICS``. Users will need to adjust the file path to reflect the location of data on their system (see :numref:`Section %s ` for locations on :srw-wiki:`Level 1 ` systems). + +.. code-block:: console + + task_get_extrn_ics: + EXTRN_MDL_NAME_ICS: UFS-CASE-STUDY + FV3GFS_FILE_FMT_ICS: nemsio + USE_USER_STAGED_EXTRN_FILES: true + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/UFS-CASE-STUDY/nemsio/${yyyymmddhh} + +For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. + +Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_SOURCE_BASEDIR_LBCS``. Users will need to adjust the file path to reflect the location of data on their system (see :numref:`Section %s ` for locations on Level 1 systems). + +.. code-block:: console + + task_get_extrn_lbcs: + EXTRN_MDL_NAME_LBCS: FV3GFS + LBC_SPEC_INTVL_HRS: 6 + FV3GFS_FILE_FMT_LBCS: grib2 + USE_USER_STAGED_EXTRN_FILES: true + EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/FV3GFS/grib2/${yyyymmddhh} + +For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. + +Users do not need to modify the ``task_run_fcst:`` section for this tutorial. + + +Lastly, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6`` and ``PLOT_DOMAINS: ["regional"]``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 12`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. + +.. code-block:: console + + task_plot_allvars: + COMOUT_REF: "" + PLOT_FCST_INC: 6 + PLOT_DOMAINS: ["regional"] + +``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 18z on June 15, 2019 (the 0th forecast hour) through the 12th forecast hour (June 16, 2019 at 06z). + +``PLOT_DOMAINS:`` The plotting scripts are designed to generate plots over the entire CONUS by default, but by setting this variable to ["regional"], the experiment will generate plots for the smaller SUBCONUS_Ind_3km regional domain instead. + +After configuring the forecast, users can generate the forecast by running: + +.. code-block:: console + + ./generate_FV3LAM_wflow.py + +To see experiment progress, users should navigate to their experiment directory. Then, use the ``rocotorun`` command to launch new workflow tasks and ``rocotostat`` to check on experiment progress. + +.. code-block:: console + + cd /path/to/expt_dirs/control + rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 + rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 + +Users will need to rerun the ``rocotorun`` and ``rocotostat`` commands above regularly and repeatedly to continue submitting workflow tasks and receiving progress updates. + +.. note:: + + When using cron to automate the workflow submission (as described :ref:`above `), users can omit the ``rocotorun`` command and simply use ``rocotostat`` to check on progress periodically. + +Users can save the location of the ``control`` directory in an environment variable (``$CONTROL``). This makes it easier to navigate between directories later. For example: + +.. code-block:: console + + export CONTROL=/path/to/expt_dirs/control + +Users should substitute ``/path/to/expt_dirs/control`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $CONTROL``, and it will take them to the ``control`` experiment directory. The variable will need to be reset for each login session. + +Experiment 2: Test +^^^^^^^^^^^^^^^^^^^^^^ + +Once the control case is running, users can return to the ``config.yaml`` file (in ``$USH``) and adjust the parameters for a new forecast. Most of the variables will remain the same. However, users will need to adjust ``EXPT_SUBDIR`` and ``CCPP_PHYS_SUITE`` in the ``workflow:`` section as follows: + +.. code-block:: console + + workflow: + EXPT_SUBDIR: test_expt + CCPP_PHYS_SUITE: FV3_RRFS_v1beta + +``EXPT_SUBDIR:`` This name must be different than the ``EXPT_SUBDIR`` name used in the previous forecast experiment. Otherwise, the first forecast experiment will be renamed, and the new experiment will take its place (see :numref:`Section %s ` for details). To avoid this issue, this tutorial uses ``test_expt`` as the second experiment's name, but the user may select a different name if desired. + +``CCPP_PHYS_SUITE:`` The FV3_RRFS_v1beta physics suite was specifically created for convection-allowing scales and is the precursor to the operational physics suite that will be used in the Rapid Refresh Forecast System (:term:`RRFS`). + +.. hint:: + + Later, users may want to conduct additional experiments using the FV3_HRRR and FV3_WoFS_v0 physics suites. Like FV3_RRFS_v1beta, these physics suites were designed for use with high-resolution grids for storm-scale predictions. + +.. COMMENT: Maybe also FV3_RAP? + +Next, users will need to modify the data parameters in ``task_get_extrn_ics:`` and ``task_get_extrn_lbcs:`` to use HRRR and RAP data rather than FV3GFS data. Users will need to change the following lines in each section: + +.. code-block:: console + + task_get_extrn_ics: + EXTRN_MDL_NAME_ICS: HRRR + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/HRRR/${yyyymmddhh} + task_get_extrn_lbcs: + EXTRN_MDL_NAME_LBCS: RAP + EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} + EXTRN_MDL_LBCS_OFFSET_HRS: '-0' + +HRRR and RAP data are better than FV3GFS data for use with the FV3_RRFS_v1beta physics scheme because these datasets use the same physics :term:`parameterizations` that are in the FV3_RRFS_v1beta suite. They focus on small-scale weather phenomena involved in storm development, so forecasts tend to be more accurate when HRRR/RAP data are paired with FV3_RRFS_v1beta and a high-resolution (e.g., 3-km) grid. Using HRRR/RAP data with FV3_RRFS_v1beta also limits the "spin-up adjustment" that takes place when initializing with model data coming from different physics. + +``EXTRN_MDL_LBCS_OFFSET_HRS:`` This variable allows users to use lateral boundary conditions (:term:`LBCs`) from a previous forecast run that was started earlier than the start time of the forecast being configured in this experiment. This variable is set to 0 by default except when using RAP data; with RAP data, the default value is 3, so the forecast will look for LBCs from a forecast started 3 hours earlier (i.e., at 2019061515 --- 15z --- instead of 2019061518). To avoid this, users must set ``EXTRN_MDL_LBCS_OFFSET_HRS`` explicitly. + +Under ``rocoto:tasks:``, add a section to increase the maximum wall time for the postprocessing tasks. The walltime is the maximum length of time a task is allowed to run. On some systems, the default of 15 minutes may be enough, but on others (e.g., NOAA Cloud), the post-processing time exceeds 15 minutes, so the tasks fail. + Tutorial Content ------------------- From 88baa05fc1ba5726898092be13be10f7f48a0266 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Wed, 31 Jul 2024 11:58:55 -0400 Subject: [PATCH 05/43] Changed CCCP_PHYS_SUITE info --- doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-) diff --git a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst index 93636ef864..9d2456a2ba 100644 --- a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst @@ -782,12 +782,10 @@ Once the control case is running, users can return to the ``config.yaml`` file ( workflow: EXPT_SUBDIR: test_expt - CCPP_PHYS_SUITE: FV3_RRFS_v1beta + CCPP_PHYS_SUITE: FV3_GFS_v16 ``EXPT_SUBDIR:`` This name must be different than the ``EXPT_SUBDIR`` name used in the previous forecast experiment. Otherwise, the first forecast experiment will be renamed, and the new experiment will take its place (see :numref:`Section %s ` for details). To avoid this issue, this tutorial uses ``test_expt`` as the second experiment's name, but the user may select a different name if desired. -``CCPP_PHYS_SUITE:`` The FV3_RRFS_v1beta physics suite was specifically created for convection-allowing scales and is the precursor to the operational physics suite that will be used in the Rapid Refresh Forecast System (:term:`RRFS`). - .. hint:: Later, users may want to conduct additional experiments using the FV3_HRRR and FV3_WoFS_v0 physics suites. Like FV3_RRFS_v1beta, these physics suites were designed for use with high-resolution grids for storm-scale predictions. From 6cf899280dca31ff145cd10930763d1e35f8bdc8 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Wed, 31 Jul 2024 13:54:34 -0400 Subject: [PATCH 06/43] Adding in Halloween storm Tutorial --- .../source/BuildingRunningTesting/Tutorial.rst | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst index 9d2456a2ba..ec684b05e7 100644 --- a/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/source/BuildingRunningTesting/Tutorial.rst @@ -608,7 +608,7 @@ After loading the workflow, users should follow the instructions printed to the module load wflow_hera conda activate srw_app -Configuration +Configurationcd ------------------------- Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: @@ -616,7 +616,7 @@ Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") .. code-block:: console cd /path/to/ufs-srweather-app/ush - cp config.community.yaml config.yaml + Users can save the location of the ``ush`` directory in an environment variable (``$USH``). This makes it easier to navigate between directories later. For example: @@ -694,11 +694,11 @@ should be included in the ``rocoto:tasks:taskgroups:`` section, like this: task_get_extrn_lbcs: walltime: 06:00:00 metatask_run_ensemble: - task_make_lbcs_mem#mem#: - walltime: 06:00:00 - task_run_fcst_mem#mem#: - walltime: 06:00:00 - taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' + task_make_lbcs_mem#mem#: + walltime: 06:00:00 + task_run_fcst_mem#mem#: + walltime: 06:00:00 + taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' For more information on how to turn on/off tasks in the workflow, please @@ -712,7 +712,7 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and EXTRN_MDL_NAME_ICS: UFS-CASE-STUDY FV3GFS_FILE_FMT_ICS: nemsio USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/UFS-CASE-STUDY/nemsio/${yyyymmddhh} + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. From 9ffb2e9ba6808eabd02757cb7c7dbcd5f90c6117 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Wed, 31 Jul 2024 14:03:49 -0400 Subject: [PATCH 07/43] Adding in Halloween Tutorial --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index ec684b05e7..30503fb857 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -574,7 +574,7 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t **Weather Phenomena:** Flooding and high winds - * `Storm Prediction Center (SPC) Storm Report for 20191031 `__ + * `Storm Prediction Center (SPC) Storm Report for 20191031 .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/Tutorial/HalloweenStorm.jpg :alt: Radar animation of the Halloween Storm that swept across the Eastern United States in 2019. @@ -810,6 +810,7 @@ HRRR and RAP data are better than FV3GFS data for use with the FV3_RRFS_v1beta p Under ``rocoto:tasks:``, add a section to increase the maximum wall time for the postprocessing tasks. The walltime is the maximum length of time a task is allowed to run. On some systems, the default of 15 minutes may be enough, but on others (e.g., NOAA Cloud), the post-processing time exceeds 15 minutes, so the tasks fail. + Tutorial Content ------------------- From 3edd708b7fdaa73879b3f03918672aaed232fffc Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 15 Aug 2024 10:59:37 -0400 Subject: [PATCH 08/43] Changes to config --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 30503fb857..3b4d2b0885 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -608,7 +608,7 @@ After loading the workflow, users should follow the instructions printed to the module load wflow_hera conda activate srw_app -Configurationcd +Configurations ------------------------- Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: @@ -711,8 +711,7 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and task_get_extrn_ics: EXTRN_MDL_NAME_ICS: UFS-CASE-STUDY FV3GFS_FILE_FMT_ICS: nemsio - USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data + USE_USER_STAGED_EXTRN_FILES: false For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. @@ -721,11 +720,10 @@ Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_ .. code-block:: console task_get_extrn_lbcs: - EXTRN_MDL_NAME_LBCS: FV3GFS - LBC_SPEC_INTVL_HRS: 6 - FV3GFS_FILE_FMT_LBCS: grib2 - USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/FV3GFS/grib2/${yyyymmddhh} + EXTRN_MDL_NAME_LBCS: UFS-CASE-STUDY + LBC_SPEC_INTVL_HRS: 3 + FV3GFS_FILE_FMT_LBCS: nemsio + USE_USER_STAGED_EXTRN_FILES: false For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. From a95b9a2228e364d26498cf5d50ecf1719b4d267c Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Wed, 28 Aug 2024 09:17:22 -0400 Subject: [PATCH 09/43] Added section about wget --- .../BuildingRunningTesting/Tutorial.rst | 50 ++++++++++++++++--- 1 file changed, 43 insertions(+), 7 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 3b4d2b0885..97138953ee 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -665,7 +665,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR PREDEF_GRID_NAME: RRFS_CONUS_13km DATE_FIRST_CYCL: '2019102812' DATE_LAST_CYCL: '2019102812' - FCST_LEN_HRS: 6 + FCST_LEN_HRS: 12 PREEXISTING_DIR_METHOD: rename VERBOSE: true COMPILER: intel @@ -771,10 +771,43 @@ Users can save the location of the ``control`` directory in an environment varia Users should substitute ``/path/to/expt_dirs/control`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $CONTROL``, and it will take them to the ``control`` experiment directory. The variable will need to be reset for each login session. -Experiment 2: Test -^^^^^^^^^^^^^^^^^^^^^^ +Experiment 2: Changing the forecast input +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Once the control case is running, users can return to the ``config.yaml`` file (in ``$USH``) and adjust the parameters for a new forecast. Most of the variables will remain the same. However, users will need to adjust ``EXPT_SUBDIR`` and ``CCPP_PHYS_SUITE`` in the ``workflow:`` section as follows: +Users who need to download the ``halloween_rap.tgz`` file using any of the following methods will follow the instructions below: + + #. Download directly from the S3 bucket using a browser. The data is available at https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz. + + #. Download from a terminal using the AWS command line interface (CLI), if installed: + + .. code-block:: console + + aws s3 cp https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz halloween_rap.tgz + + #. Download from a terminal using ``wget``: + + .. code-block:: console + + wget https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz + +After downloading ``halloween_rap.tgz`` using one of the three methods above, untar the downloaded compressed archive file: + +.. code-block:: console + + tar xvfz halloween_rap.tgz + +Save the path to this file in and ``HALLOWEENDATA`` environment variable: + +.. code-block:: console + + cd Indy-Severe-Weather + export HALLOWEENDATA=$PWD + +.. note:: + + Users can untar the fix files and Natural Earth files by substituting those file names in the commands above. + +Once the control case is running, users can return to the ``config.yaml`` file (in ``$USH``) and adjust the parameters for a new forecast. Most of the variables will remain the same. However, users will need to adjust ``EXPT_SUBDIR`` in the ``workflow:`` section as follows: .. code-block:: console @@ -790,15 +823,18 @@ Once the control case is running, users can return to the ``config.yaml`` file ( .. COMMENT: Maybe also FV3_RAP? -Next, users will need to modify the data parameters in ``task_get_extrn_ics:`` and ``task_get_extrn_lbcs:`` to use HRRR and RAP data rather than FV3GFS data. Users will need to change the following lines in each section: +Next, users will need to modify the data parameters in ``task_get_extrn_ics:`` and ``task_get_extrn_lbcs:`` to use RAP data. Users will need to change the following lines in each section: .. code-block:: console task_get_extrn_ics: - EXTRN_MDL_NAME_ICS: HRRR - EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/HRRR/${yyyymmddhh} + EXTRN_MDL_NAME_ICS: RAP + USE_USER_STAGED_EXTRN_FILES: true + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} task_get_extrn_lbcs: EXTRN_MDL_NAME_LBCS: RAP + LBC_SPEC_INTVL_HRS: 3 + USE_USER_STAGED_EXTRN_FILES: true EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} EXTRN_MDL_LBCS_OFFSET_HRS: '-0' From d538daedb84611d97cc37defe2c0a5904ee2b567 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 12 Sep 2024 10:57:56 -0400 Subject: [PATCH 10/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 97138953ee..8751ff62d7 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -574,7 +574,7 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t **Weather Phenomena:** Flooding and high winds - * `Storm Prediction Center (SPC) Storm Report for 20191031 + * `Storm Prediction Center (SPC) Storm Report for 20191031 `_ .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/Tutorial/HalloweenStorm.jpg :alt: Radar animation of the Halloween Storm that swept across the Eastern United States in 2019. From ac8f8676a2dd4ece463b6d6d1a250668557b4c66 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 12 Sep 2024 10:58:17 -0400 Subject: [PATCH 11/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 8751ff62d7..ca35ee0795 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -584,7 +584,12 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t Data ------- -On :srw-wiki:`Level 1 ` systems, users can find data for the Halloween Storm in the usual input model data locations (see :numref:`Section %s ` for a list). The data can also be downloaded from the `UFS SRW Halloween Storm Case Study `__. +On :srw-wiki:`Level 1 ` systems, users can find data for the Halloween Storm in the usual input model data locations (see :numref:`Section %s ` for a list). The RAP data can also be downloaded from the `SRW App data bucket `_ using ``wget``: + +.. code-block:: console + + wget https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz + tar -xzf halloween_rap.tgz Load the workflow --------------------- From 336ab82de98d922ac3bd415436460608e6a2b305 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 12 Sep 2024 10:58:25 -0400 Subject: [PATCH 12/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index ca35ee0795..67b834cb08 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -598,7 +598,7 @@ To load the workflow environment, source the lmod-setup file. Then load the work .. code-block:: console - source etc/lmod-setup.sh # OR: source etc/lmod-setup.csh when running in a csh/tcsh shell + source etc/lmod-setup.sh module use modulefiles module load wflow_ From 754f95d54132f0d782750b359180623ce9e8d7f0 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 12 Sep 2024 10:58:46 -0400 Subject: [PATCH 13/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 67b834cb08..3e34a11e48 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -613,7 +613,7 @@ After loading the workflow, users should follow the instructions printed to the module load wflow_hera conda activate srw_app -Configurations +Configuration ------------------------- Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: From ebb2cb51f4e9fdcd0f274c6e947d9b53751eaeff Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 12 Sep 2024 10:59:22 -0400 Subject: [PATCH 14/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 3e34a11e48..3440d76fc5 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -683,7 +683,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR ``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``control`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different forecasts --- one that uses the FV3_GFS_v16 physics suite and one that uses the FV3_RRFS_v1beta physics suite --- "gfsv16_physics_fcst" could be a good alternative directory name. -``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. Using the RRFS_CONUS_13km grid allows for fewer computational restraints compared to the 25km grid. For more information on this grid, see :numref:`Section %s `. +``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. Using the RRFS_CONUS_13km grid provides a higher resolution forecast, more detailed forecast; however, it is more computationally expensive compared to the 25km grid. For more information on this grid, see :numref:`Section %s `. For a detailed description of other ``workflow:`` variables, see :numref:`Section %s `. From 15240870f5d5dbd123a0c6eb93e12fc7748e41a1 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 12 Sep 2024 12:12:19 -0400 Subject: [PATCH 15/43] Updating config on halloween tutorial --- .../BuildingRunningTesting/Tutorial.rst | 107 +++--------------- 1 file changed, 17 insertions(+), 90 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index f2c450506b..f5196dc564 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -631,7 +631,7 @@ Users can save the location of the ``ush`` directory in an environment variable Users should substitute ``/path/to/ufs-srweather-app/ush`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $USH``, and it will take them to the ``ush`` directory. The variable will need to be reset for each login session. -Experiment 1: Control +Experiment 1: Rap Data ^^^^^^^^^^^^^^^^^^^^^^^^ Edit the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below. @@ -665,12 +665,12 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR workflow: USE_CRON_TO_RELAUNCH: false - EXPT_SUBDIR: control + EXPT_SUBDIR: halloween CCPP_PHYS_SUITE: FV3_GFS_v16 PREDEF_GRID_NAME: RRFS_CONUS_13km - DATE_FIRST_CYCL: '2019102812' - DATE_LAST_CYCL: '2019102812' - FCST_LEN_HRS: 12 + DATE_FIRST_CYCL: '2019103012' + DATE_LAST_CYCL: '2019103012' + FCST_LEN_HRS: 6 PREEXISTING_DIR_METHOD: rename VERBOSE: true COMPILER: intel @@ -681,7 +681,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR Users may also want to set ``USE_CRON_TO_RELAUNCH: true`` and add ``CRON_RELAUNCH_INTVL_MNTS: 3``. This will automate submission of workflow tasks when running the experiment. However, not all systems have :term:`cron`. -``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``control`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different forecasts --- one that uses the FV3_GFS_v16 physics suite and one that uses the FV3_RRFS_v1beta physics suite --- "gfsv16_physics_fcst" could be a good alternative directory name. +``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``halloween`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different types of forecast data input --- halloween_rap could be a good alternative name. ``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. Using the RRFS_CONUS_13km grid provides a higher resolution forecast, more detailed forecast; however, it is more computationally expensive compared to the 25km grid. For more information on this grid, see :numref:`Section %s `. @@ -703,7 +703,7 @@ should be included in the ``rocoto:tasks:taskgroups:`` section, like this: walltime: 06:00:00 task_run_fcst_mem#mem#: walltime: 06:00:00 - taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' + taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' For more information on how to turn on/off tasks in the workflow, please @@ -714,9 +714,9 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and .. code-block:: console task_get_extrn_ics: - EXTRN_MDL_NAME_ICS: UFS-CASE-STUDY - FV3GFS_FILE_FMT_ICS: nemsio - USE_USER_STAGED_EXTRN_FILES: false + EXTRN_MDL_NAME_ICS: RAP + USE_USER_STAGED_EXTRN_FILES: True + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. @@ -725,28 +725,25 @@ Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_ .. code-block:: console task_get_extrn_lbcs: - EXTRN_MDL_NAME_LBCS: UFS-CASE-STUDY + EXTRN_MDL_NAME_LBCS: RAP LBC_SPEC_INTVL_HRS: 3 - FV3GFS_FILE_FMT_LBCS: nemsio - USE_USER_STAGED_EXTRN_FILES: false + USE_USER_STAGED_EXTRN_FILES: true + EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. Users do not need to modify the ``task_run_fcst:`` section for this tutorial. -Lastly, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6`` and ``PLOT_DOMAINS: ["regional"]``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 12`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. +Lastly, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 12`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. .. code-block:: console task_plot_allvars: COMOUT_REF: "" PLOT_FCST_INC: 6 - PLOT_DOMAINS: ["regional"] -``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 18z on June 15, 2019 (the 0th forecast hour) through the 12th forecast hour (June 16, 2019 at 06z). - -``PLOT_DOMAINS:`` The plotting scripts are designed to generate plots over the entire CONUS by default, but by setting this variable to ["regional"], the experiment will generate plots for the smaller SUBCONUS_Ind_3km regional domain instead. +``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 18z on June 15, 2019 (the 0th forecast hour) through the 12th forecast hour (June 16, 2019 at 06z). After configuring the forecast, users can generate the forecast by running: @@ -768,87 +765,17 @@ Users will need to rerun the ``rocotorun`` and ``rocotostat`` commands above reg When using cron to automate the workflow submission (as described :ref:`above `), users can omit the ``rocotorun`` command and simply use ``rocotostat`` to check on progress periodically. -Users can save the location of the ``control`` directory in an environment variable (``$CONTROL``). This makes it easier to navigate between directories later. For example: +Users can save the location of the ``Halloween`` directory in an environment variable (``$CONTROL``). This makes it easier to navigate between directories later. For example: .. code-block:: console export CONTROL=/path/to/expt_dirs/control -Users should substitute ``/path/to/expt_dirs/control`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $CONTROL``, and it will take them to the ``control`` experiment directory. The variable will need to be reset for each login session. +Users should substitute ``/path/to/expt_dirs/Halloween`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $Halloween``, and it will take them to the ``halloween`` experiment directory. The variable will need to be reset for each login session. Experiment 2: Changing the forecast input ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Users who need to download the ``halloween_rap.tgz`` file using any of the following methods will follow the instructions below: - - #. Download directly from the S3 bucket using a browser. The data is available at https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz. - - #. Download from a terminal using the AWS command line interface (CLI), if installed: - - .. code-block:: console - - aws s3 cp https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz halloween_rap.tgz - - #. Download from a terminal using ``wget``: - - .. code-block:: console - - wget https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz - -After downloading ``halloween_rap.tgz`` using one of the three methods above, untar the downloaded compressed archive file: - -.. code-block:: console - - tar xvfz halloween_rap.tgz - -Save the path to this file in and ``HALLOWEENDATA`` environment variable: - -.. code-block:: console - - cd Indy-Severe-Weather - export HALLOWEENDATA=$PWD - -.. note:: - - Users can untar the fix files and Natural Earth files by substituting those file names in the commands above. - -Once the control case is running, users can return to the ``config.yaml`` file (in ``$USH``) and adjust the parameters for a new forecast. Most of the variables will remain the same. However, users will need to adjust ``EXPT_SUBDIR`` in the ``workflow:`` section as follows: - -.. code-block:: console - - workflow: - EXPT_SUBDIR: test_expt - CCPP_PHYS_SUITE: FV3_GFS_v16 - -``EXPT_SUBDIR:`` This name must be different than the ``EXPT_SUBDIR`` name used in the previous forecast experiment. Otherwise, the first forecast experiment will be renamed, and the new experiment will take its place (see :numref:`Section %s ` for details). To avoid this issue, this tutorial uses ``test_expt`` as the second experiment's name, but the user may select a different name if desired. - -.. hint:: - - Later, users may want to conduct additional experiments using the FV3_HRRR and FV3_WoFS_v0 physics suites. Like FV3_RRFS_v1beta, these physics suites were designed for use with high-resolution grids for storm-scale predictions. - -.. COMMENT: Maybe also FV3_RAP? - -Next, users will need to modify the data parameters in ``task_get_extrn_ics:`` and ``task_get_extrn_lbcs:`` to use RAP data. Users will need to change the following lines in each section: - -.. code-block:: console - - task_get_extrn_ics: - EXTRN_MDL_NAME_ICS: RAP - USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} - task_get_extrn_lbcs: - EXTRN_MDL_NAME_LBCS: RAP - LBC_SPEC_INTVL_HRS: 3 - USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} - EXTRN_MDL_LBCS_OFFSET_HRS: '-0' - -HRRR and RAP data are better than FV3GFS data for use with the FV3_RRFS_v1beta physics scheme because these datasets use the same physics :term:`parameterizations` that are in the FV3_RRFS_v1beta suite. They focus on small-scale weather phenomena involved in storm development, so forecasts tend to be more accurate when HRRR/RAP data are paired with FV3_RRFS_v1beta and a high-resolution (e.g., 3-km) grid. Using HRRR/RAP data with FV3_RRFS_v1beta also limits the "spin-up adjustment" that takes place when initializing with model data coming from different physics. - -``EXTRN_MDL_LBCS_OFFSET_HRS:`` This variable allows users to use lateral boundary conditions (:term:`LBCs`) from a previous forecast run that was started earlier than the start time of the forecast being configured in this experiment. This variable is set to 0 by default except when using RAP data; with RAP data, the default value is 3, so the forecast will look for LBCs from a forecast started 3 hours earlier (i.e., at 2019061515 --- 15z --- instead of 2019061518). To avoid this, users must set ``EXTRN_MDL_LBCS_OFFSET_HRS`` explicitly. - -Under ``rocoto:tasks:``, add a section to increase the maximum wall time for the postprocessing tasks. The walltime is the maximum length of time a task is allowed to run. On some systems, the default of 15 minutes may be enough, but on others (e.g., NOAA Cloud), the post-processing time exceeds 15 minutes, so the tasks fail. - Tutorial Content ------------------- From ab31032848f7f55da82cea85f136a26140a70390 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Sun, 22 Sep 2024 23:00:40 -0400 Subject: [PATCH 16/43] Adding HRRR forecast input option --- .../BuildingRunningTesting/Tutorial.rst | 58 +++++++++++++++++-- 1 file changed, 53 insertions(+), 5 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index f5196dc564..c91494c100 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -665,12 +665,12 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR workflow: USE_CRON_TO_RELAUNCH: false - EXPT_SUBDIR: halloween - CCPP_PHYS_SUITE: FV3_GFS_v16 + EXPT_SUBDIR: halloweenRAP + CCPP_PHYS_SUITE: FV3_RAP PREDEF_GRID_NAME: RRFS_CONUS_13km DATE_FIRST_CYCL: '2019103012' DATE_LAST_CYCL: '2019103012' - FCST_LEN_HRS: 6 + FCST_LEN_HRS: 36 PREEXISTING_DIR_METHOD: rename VERBOSE: true COMPILER: intel @@ -716,7 +716,7 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and task_get_extrn_ics: EXTRN_MDL_NAME_ICS: RAP USE_USER_STAGED_EXTRN_FILES: True - EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/for_ICS For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. @@ -728,7 +728,7 @@ Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_ EXTRN_MDL_NAME_LBCS: RAP LBC_SPEC_INTVL_HRS: 3 USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/${yyyymmddhh} + EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/for_LBCS For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. @@ -775,7 +775,55 @@ Users should substitute ``/path/to/expt_dirs/Halloween`` with the actual path on Experiment 2: Changing the forecast input ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +In this experiment we will be changing the forecast input to use HRRR data. This will include edits to the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below + +In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PREDEF_GRID_NAME``. + +.. code-block:: console + + workflow: + USE_CRON_TO_RELAUNCH: false + EXPT_SUBDIR: halloweenHRRR + CCPP_PHYS_SUITE: FV3_RAP + PREDEF_GRID_NAME: RRFS_CONUScompact_13km + DATE_FIRST_CYCL: '2019103012' + DATE_LAST_CYCL: '2019103012' + FCST_LEN_HRS: 36 + PREEXISTING_DIR_METHOD: rename + VERBOSE: true + COMPILER: intel + +.. note:: + + Since HRRR is a high-resolution model than RAP, we will need to utilize the ``RRFS_CONUScompact_13km`` grid in order for the experiment to run successfully. + +In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_FILES_ICS``. +.. code-block:: console + + task_get_extrn_ics: + EXTRN_MDL_NAME_ICS: HRRR + USE_USER_STAGED_EXTRN_FILES: false + EXTRN_MDL_FILES_ICS: + - '{yy}{jjj}{hh}00{fcst_hr:02d}00' + +For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. + +Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_FILES_LBCS``. Users will need to adjust the file path to reflect the location of data on their system (see :numref:`Section %s ` for locations on Level 1 systems). + +.. code-block:: console + + task_get_extrn_lbcs: + EXTRN_MDL_NAME_LBCS: HRRR + LBC_SPEC_INTVL_HRS: 3 + USE_USER_STAGED_EXTRN_FILES: false + EXTRN_MDL_FILES_LBCS: + - '{yy}{jjj}{hh}00{fcst_hr:02d}00' + + +For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. + +Users do not need to modify the ``task_run_fcst:`` section for this tutorial. Tutorial Content ------------------- From 301bfcdc91ebe4f8f66313a1e5226ceda4de884b Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Tue, 24 Sep 2024 13:44:40 -0400 Subject: [PATCH 17/43] Minor changes and adding plotting explanationss --- .../BuildingRunningTesting/Tutorial.rst | 49 ++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index c91494c100..3fd912c0ad 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -809,7 +809,7 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. -Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_FILES_LBCS``. Users will need to adjust the file path to reflect the location of data on their system (see :numref:`Section %s ` for locations on Level 1 systems). +Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_FILES_LBCS``. .. code-block:: console @@ -825,6 +825,53 @@ For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numre Users do not need to modify the ``task_run_fcst:`` section for this tutorial. +How to Analyze Results +----------------------- +Navigate to ``halloweenHRRR/2019103012/postprd`` or ``halloweenRAP/2019203012/postprd``. This directory contains the post-processed data generated by the :term:`UPP` from the ``halloween`` forecast. After the ``plot_allvars`` task completes, this directory will contain ``.png`` images for several forecast variables. + +Copy ``.png`` Files onto Local System +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Users who are working on the cloud or on an HPC cluster may want to copy the ``.png`` files onto their local system to view in their preferred image viewer. Detailed instructions are available in the :ref:`Introduction to SSH & Data Transfer `. + +In summary, users can run the ``scp`` command in a new terminal/command prompt window to securely copy files from a remote system to their local system if an SSH tunnel is already established between the local system and the remote system. Users can adjust one of the following commands for their system: + +.. code-block:: console + + scp username@your-IP-address:/path/to/source_file_or_directory /path/to/destination_file_or_directory + # OR + scp -P 12345 username@localhost:/path/to/source_file_or_directory /path/to/destination_file_or_directory + +Users would need to modify ``username``, ``your-IP-address``, ``-P 12345``, and the file paths to reflect their systems' information. See the :ref:`Introduction to SSH & Data Transfer ` for example commands. + + +Experiment 3: Examining Forecast Plots at Peak Intensity +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +In this experiment will be looking at plots from HRRR and RAP input files while the Halloween Storm is at or approaching peak intensity. + +Analysis +^^^^^^^^^^^ + +.. _fcst4_250wind: + +250mb Wind +`````````` +Any good forecast starts with analyzing a 250mb wind chart. Using a 250mb wind plot, users will be able to point out key features such as jet placement, jet maxes, troughs, ridges and more. Analyzing this also allows us to see where the potential for the strongest severe weather is. + +In the 250mb Wind plots, the ``halloweenHRRR`` and ``halloweenRAP`` plots are nearly identical at forecast hour f036. This shows great model agreement. Analyzing this chart we can see multiple ingredients signaling a significant severe weather event over eastern CONUS. The first thing to notice is the placement of the jet streak along with troughing approaching the eastern US. Also notice an extreme 150KT jet max over southern Ohio further fueling severe weather. The last thing to notice is the divegence aloft present over Eastern Conus, seeing divergence present all the way up to 250mb indicates a strong system. + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/250wind_rap_conus_f036.png + :align: center + :width: 75% + + *Rap plot for 250mb wind* + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/250wind_hrrr_conus_f036.png + :align: center + :width: 75% + + *HRRR plot for 250mb wind* + Tutorial Content ------------------- From 36ac4c134deb31adfcb42cddbbbc176897bead66 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Wed, 25 Sep 2024 13:27:21 -0400 Subject: [PATCH 18/43] Finishing up plotting explanations --- .../BuildingRunningTesting/Tutorial.rst | 47 +++++++++++++++++-- 1 file changed, 42 insertions(+), 5 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 3fd912c0ad..24c6f9bb83 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -849,16 +849,13 @@ Experiment 3: Examining Forecast Plots at Peak Intensity ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In this experiment will be looking at plots from HRRR and RAP input files while the Halloween Storm is at or approaching peak intensity. -Analysis -^^^^^^^^^^^ - .. _fcst4_250wind: 250mb Wind `````````` -Any good forecast starts with analyzing a 250mb wind chart. Using a 250mb wind plot, users will be able to point out key features such as jet placement, jet maxes, troughs, ridges and more. Analyzing this also allows us to see where the potential for the strongest severe weather is. +An effective weather forecast begins with analyzing a 250mb wind chart. By using this wind plot, forecasters can identify key features such as jet stream placement, jet maxima, troughs, ridges, and more. This analysis also helps pinpoint areas with the potential for the strongest severe weather. -In the 250mb Wind plots, the ``halloweenHRRR`` and ``halloweenRAP`` plots are nearly identical at forecast hour f036. This shows great model agreement. Analyzing this chart we can see multiple ingredients signaling a significant severe weather event over eastern CONUS. The first thing to notice is the placement of the jet streak along with troughing approaching the eastern US. Also notice an extreme 150KT jet max over southern Ohio further fueling severe weather. The last thing to notice is the divegence aloft present over Eastern Conus, seeing divergence present all the way up to 250mb indicates a strong system. +In the 250mb wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` plots are nearly identical at forecast hour f036. This shows great model agreement. Analyzing this chart we can see multiple ingredients signaling a significant severe weather event over eastern CONUS. The first thing to notice is the placement of the jet streak along with troughing approaching the eastern US. Also notice an extreme 150KT jet max over southern Ohio further fueling severe weather. The last thing to notice is the divegence aloft present over Eastern Conus, seeing divergence present all the way up to 250mb indicates a strong system. .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/250wind_rap_conus_f036.png :align: center @@ -872,6 +869,46 @@ In the 250mb Wind plots, the ``halloweenHRRR`` and ``halloweenRAP`` plots are ne *HRRR plot for 250mb wind* +.. _fcst4_10mwind: + +10m Wind +`````````` +The 10m wind plots allows forecasters to pick up on patterns closer to the surface. It shows things such as convergence and pressure areas. + +In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once again very similar, which makes sense given that the 250mb wind plots are also so similar. We can see a few key feautres on this chart. The most important is the area of convergence taking place over the east coast which is driving the line of severe storms. + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/10mwind_rap_conus_f036.png + :align: center + :width: 75% + + *Rap plot for 10m winds* + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/10mwind_hrrr_conus_f036.png + :align: center + :width: 75% + + *HRRR plot for 10m winds* + +.. _fcst4_refc: + +Composite Reflectivity +`````````` +Reflectivity images visually represent the weather based on the energy (measured in decibels [dBZ]) reflected back from radar. Composite reflectivity generates an image based on reflectivity scans at multiple elevation angles, or "tilts", of the antenna. See https://www.noaa.gov/jetstream/reflectivity for a more detailed explanation of composite reflectivity. + +In the composite reflectivity plots below, the ``halloweenHRRR`` and ``halloweenrap`` are still very similar as expected. Utlizing the reflectivity plots is the last piece of the puzzle. Based on things we had identified using the previous plots we already had a good idea of where the storms should be. Composite reflecitvity is just another tool that allows us to see where these models are placing storms. We can see that the strongest storms appear to be in the NC/VA vicinity which is represented by a higher dBZ. + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/refc_rap_conus_f036.png + :align: center + :width: 75% + + *Rap plot for Composite Reflectivity* + +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/refc_hrrr_conus_f036.png + :align: center + :width: 75% + + *HRRR plot for Composite Reflectivity* + Tutorial Content ------------------- From fe70d0eaa61ca5ac797cac9571f014cbeac1a70e Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Thu, 26 Sep 2024 11:00:35 -0400 Subject: [PATCH 19/43] Minor formatting corrections --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 24c6f9bb83..8c2be813a0 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -715,7 +715,7 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and task_get_extrn_ics: EXTRN_MDL_NAME_ICS: RAP - USE_USER_STAGED_EXTRN_FILES: True + USE_USER_STAGED_EXTRN_FILES: true EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/for_ICS For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. @@ -775,7 +775,7 @@ Users should substitute ``/path/to/expt_dirs/Halloween`` with the actual path on Experiment 2: Changing the forecast input ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In this experiment we will be changing the forecast input to use HRRR data. This will include edits to the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below +In this experiment we will be changing the forecast input to use ``HRRR`` data. This will include edits to the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PREDEF_GRID_NAME``. @@ -895,7 +895,7 @@ Composite Reflectivity `````````` Reflectivity images visually represent the weather based on the energy (measured in decibels [dBZ]) reflected back from radar. Composite reflectivity generates an image based on reflectivity scans at multiple elevation angles, or "tilts", of the antenna. See https://www.noaa.gov/jetstream/reflectivity for a more detailed explanation of composite reflectivity. -In the composite reflectivity plots below, the ``halloweenHRRR`` and ``halloweenrap`` are still very similar as expected. Utlizing the reflectivity plots is the last piece of the puzzle. Based on things we had identified using the previous plots we already had a good idea of where the storms should be. Composite reflecitvity is just another tool that allows us to see where these models are placing storms. We can see that the strongest storms appear to be in the NC/VA vicinity which is represented by a higher dBZ. +In the composite reflectivity plots below, the ``halloweenHRRR`` and ``halloweenRAP`` models remain quite similar, as expected. Utilizing the reflectivity plots provides the final piece of the puzzle. From the previous analyses, we already had a good understanding of where the storms were likely to occur. Composite reflectivity serves as an additional tool, allowing us to visualize where the models predict storm placement. In this case, the strongest storms are indicated by higher dBZ values and appear to be concentrated in the NC/VA region. .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/refc_rap_conus_f036.png :align: center From a5401c79b84fa142a5979081a3338145e336ec39 Mon Sep 17 00:00:00 2001 From: Joshua Kublnick Date: Thu, 26 Sep 2024 11:23:13 -0400 Subject: [PATCH 20/43] More minor changes to the text --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 8c2be813a0..e48d658a67 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -681,7 +681,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR Users may also want to set ``USE_CRON_TO_RELAUNCH: true`` and add ``CRON_RELAUNCH_INTVL_MNTS: 3``. This will automate submission of workflow tasks when running the experiment. However, not all systems have :term:`cron`. -``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``halloween`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different types of forecast data input --- halloween_rap could be a good alternative name. +``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "halloweenRAP" to "halloweenHRRR" to "a;skdfj". However, the best names will indicate useful information about the experiment. Since this tutorial helps users to compare the output from two different types of forecast data input --- halloweenRAP and halloweenHRRR could be good names. ``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. Using the RRFS_CONUS_13km grid provides a higher resolution forecast, more detailed forecast; however, it is more computationally expensive compared to the 25km grid. For more information on this grid, see :numref:`Section %s `. From acee59cd54815d036f5e076be15cac5442663bcc Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Thu, 26 Sep 2024 17:04:23 -0400 Subject: [PATCH 21/43] add prelim env load snippet to be reused throughout docs --- doc/doc-snippets/load-env.rst | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 doc/doc-snippets/load-env.rst diff --git a/doc/doc-snippets/load-env.rst b/doc/doc-snippets/load-env.rst new file mode 100644 index 0000000000..1afbbe498b --- /dev/null +++ b/doc/doc-snippets/load-env.rst @@ -0,0 +1,7 @@ +.. code-block:: console + + module use /path/to/ufs-srweather-app/modulefiles + module load wflow_ + conda activate srw_app + + From 6abdc762599c735ebeb7bc76a914327a32fd5d6d Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 12:29:31 -0400 Subject: [PATCH 22/43] factor out module use/load cmds; use include dirctive to add --- .../BuildingRunningTesting/Quickstart.rst | 10 ++----- .../BuildingRunningTesting/RunSRW.rst | 11 ++------ .../BuildingRunningTesting/Tutorial.rst | 28 +++++++------------ .../BuildingRunningTesting/VXCases.rst | 7 +---- doc/UsersGuide/Reference/FAQ.rst | 7 +---- doc/doc-snippets/cron-note.rst | 3 ++ doc/doc-snippets/load-env.rst | 6 ++-- 7 files changed, 24 insertions(+), 48 deletions(-) create mode 100644 doc/doc-snippets/cron-note.rst diff --git a/doc/UsersGuide/BuildingRunningTesting/Quickstart.rst b/doc/UsersGuide/BuildingRunningTesting/Quickstart.rst index 36ea8eb7fb..1cc83744d0 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Quickstart.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Quickstart.rst @@ -50,13 +50,9 @@ For a detailed explanation of how to build and run the SRW App on any supported #. Load the python environment for the workflow. Users on Level 2-4 systems will need to use one of the existing ``wflow_`` modulefiles (e.g., ``wflow_macos``) and adapt it to their system. Then, run: - .. code-block:: console - - source /path/to/ufs-srweather-app/etc/lmod-setup.sh - module use /path/to/ufs-srweather-app/modulefiles - module load wflow_ - - where ```` refers to a valid machine name (see :numref:`Section %s `). After loading the workflow, users should follow the instructions printed to the console. For example, if the output says: + .. include:: ../../doc-snippets/load-env.rst + + After loading the workflow, users should follow the instructions printed to the console. For example, if the output says: .. code-block:: console diff --git a/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst index 831b2a6345..fe4ac5d117 100644 --- a/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst +++ b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst @@ -134,13 +134,8 @@ Loading the Workflow Environment The |wflow_env| conda/Python environment can be activated in the following way: -.. code-block:: console - - source /path/to/ufs-srweather-app/etc/lmod-setup.sh - module use /path/to/ufs-srweather-app/modulefiles - module load wflow_ - -where ```` refers to a valid machine name (see :numref:`Section %s ` for ``MACHINE`` options). In a csh shell environment, users should replace ``lmod-setup.sh`` with ``lmod-setup.csh``. +.. include:: ../../doc-snippets/load-env.rst +In a csh shell environment, users should replace ``lmod-setup.sh`` with ``lmod-setup.csh``. .. note:: If users source the lmod-setup file on a system that doesn't need it, it will not cause any problems (it will simply do a ``module purge``). @@ -155,7 +150,7 @@ The ``wflow_`` modulefile will then output instructions to activate th then the user should run |activate|. This activates the |wflow_env| conda environment, and the user typically sees |prompt| in front of the Terminal prompt at this point. .. note:: - If users do not use the wflow module to load conda, ``conda`` will need to be initialized before running ``conda activate srw_app`` command. Depending on the user's system and login setup, this may be accomplished in a variety of ways. Conda initialization usually involves the following command: ``source /etc/profile.d/conda.sh``, where ```` is the base conda installation directory and by default will be the full path to ``ufs-srweather-app/conda``. + If users do not use the ``wflow_`` module to load conda, ``conda`` will need to be initialized before running ``conda activate srw_app`` command. Depending on the user's system and login setup, this may be accomplished in a variety of ways. Conda initialization usually involves the following command: ``source /etc/profile.d/conda.sh``, where ```` is the base conda installation directory and by default will be the full path to ``ufs-srweather-app/conda``. After loading the workflow environment, users may continue to :numref:`Section %s ` for instructions on setting the experiment configuration parameters. diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index e48d658a67..e44550bcc4 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -11,7 +11,7 @@ Users can run through the entire set of tutorials or jump to the one that intere #. :ref:`Severe Weather Over Indianapolis `: Change physics suites and compare graphics plots. #. :ref:`Cold Air Damming `: Coming soon! #. :ref:`Southern Plains Winter Weather Event `: Coming soon! - #. :ref:`Halloween Storm `: Coming soon! + #. :ref:`Halloween Storm `: Change :term:`IC/LBC ` sources and compare results. #. :ref:`Hurricane Barry `: Coming soon! Each section provides a summary of the weather event and instructions for configuring an experiment. @@ -57,13 +57,7 @@ Load the Workflow To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment. From the ``ufs-srweather-app`` directory, run: -.. code-block:: console - - source etc/lmod-setup.sh # OR: source etc/lmod-setup.csh when running in a csh/tcsh shell - module use modulefiles - module load wflow_ - -where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values). +.. include:: ../../doc-snippets/load-env.rst After loading the workflow, users should follow the instructions printed to the console. Usually, the instructions will tell the user to run |activate|. For example, a user on Hera with permissions on the ``nems`` project may issue the following commands to load the workflow (replacing ``User.Name`` with their actual username): @@ -555,18 +549,20 @@ A polar vortex brought arctic air to much of the U.S. and Mexico. A series of co *Southern Plains Winter Weather Event Over Oklahoma City* -.. COMMENT: Upload a png to the SRW wiki and change the hyperlink to point to that. - Tutorial Content ------------------- -Coming Soon! +Coming Soon! .. _fcst4: Sample Forecast #4: Halloween Storm ======================================= +**Objective:** + * Compare forecast outputs for similar experiments that use different :term:`IC/LBC ` sources. + * Coming soon: Option to use verification tools to assess forecast quality. + Weather Summary -------------------- @@ -675,11 +671,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR VERBOSE: true COMPILER: intel -.. _CronNote: - -.. note:: - - Users may also want to set ``USE_CRON_TO_RELAUNCH: true`` and add ``CRON_RELAUNCH_INTVL_MNTS: 3``. This will automate submission of workflow tasks when running the experiment. However, not all systems have :term:`cron`. +.. include:: ../../doc-snippets/cron-note.rst ``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "halloweenRAP" to "halloweenHRRR" to "a;skdfj". However, the best names will indicate useful information about the experiment. Since this tutorial helps users to compare the output from two different types of forecast data input --- halloweenRAP and halloweenHRRR could be good names. @@ -846,7 +838,7 @@ Users would need to modify ``username``, ``your-IP-address``, ``-P 12345``, and Experiment 3: Examining Forecast Plots at Peak Intensity -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ In this experiment will be looking at plots from HRRR and RAP input files while the Halloween Storm is at or approaching peak intensity. .. _fcst4_250wind: @@ -892,7 +884,7 @@ In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once .. _fcst4_refc: Composite Reflectivity -`````````` +```````````````````````` Reflectivity images visually represent the weather based on the energy (measured in decibels [dBZ]) reflected back from radar. Composite reflectivity generates an image based on reflectivity scans at multiple elevation angles, or "tilts", of the antenna. See https://www.noaa.gov/jetstream/reflectivity for a more detailed explanation of composite reflectivity. In the composite reflectivity plots below, the ``halloweenHRRR`` and ``halloweenRAP`` models remain quite similar, as expected. Utilizing the reflectivity plots provides the final piece of the puzzle. From the previous analyses, we already had a good understanding of where the storms were likely to occur. Composite reflectivity serves as an additional tool, allowing us to visualize where the models predict storm placement. In this case, the strongest storms are indicated by higher dBZ values and appear to be concentrated in the NC/VA region. diff --git a/doc/UsersGuide/BuildingRunningTesting/VXCases.rst b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst index 2bf6f775d0..595adb1cab 100644 --- a/doc/UsersGuide/BuildingRunningTesting/VXCases.rst +++ b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst @@ -83,12 +83,7 @@ Load the Workflow First, navigate to the ``ufs-srweather-app/ush`` directory. Then, load the workflow environment: -.. code-block:: console - - source /path/to/etc/lmod-setup.sh - module use /path/to/ufs-srweather-app/modulefiles - module load wflow_ - +.. include:: ../../doc-snippets/load-env.rst Users running a csh/tcsh shell would run ``source /path/to/etc/lmod-setup.csh `` in place of the first command above. After loading the workflow, users should follow the instructions printed to the console. Usually, the instructions will tell the user to run |activate|. diff --git a/doc/UsersGuide/Reference/FAQ.rst b/doc/UsersGuide/Reference/FAQ.rst index e8c3df0dec..b1c7bcce1f 100644 --- a/doc/UsersGuide/Reference/FAQ.rst +++ b/doc/UsersGuide/Reference/FAQ.rst @@ -281,12 +281,7 @@ How can I run a new experiment? To run a new experiment at a later time, users need to rerun the commands in :numref:`Section %s ` that reactivate the |wflow_env| environment: -.. code-block:: console - - source /path/to/etc/lmod-setup.sh/or/lmod-setup.csh - module use /path/to/modulefiles - module load wflow_ - +.. include:: ../../doc-snippets/load-env.rst Follow any instructions output by the console (e.g., |activate|). Then, users can configure a new experiment by updating the experiment parameters in ``config.yaml`` to reflect the desired experiment configuration. Detailed instructions can be viewed in :numref:`Section %s `. Parameters and valid values are listed in :numref:`Section %s `. After adjusting the configuration file, generate the new experiment by running ``./generate_FV3LAM_wflow.py``. Check progress by navigating to the ``$EXPTDIR`` and running ``rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10``. diff --git a/doc/doc-snippets/cron-note.rst b/doc/doc-snippets/cron-note.rst new file mode 100644 index 0000000000..99192d0a1e --- /dev/null +++ b/doc/doc-snippets/cron-note.rst @@ -0,0 +1,3 @@ +.. note:: + + Users may also want to set ``USE_CRON_TO_RELAUNCH: true`` and add ``CRON_RELAUNCH_INTVL_MNTS: 3``. This will automate submission of workflow tasks when running the experiment. However, not all systems have :term:`cron`. \ No newline at end of file diff --git a/doc/doc-snippets/load-env.rst b/doc/doc-snippets/load-env.rst index 1afbbe498b..ce537e7824 100644 --- a/doc/doc-snippets/load-env.rst +++ b/doc/doc-snippets/load-env.rst @@ -1,7 +1,7 @@ .. code-block:: console + source /path/to/ufs-srweather-app/etc/lmod-setup.sh module use /path/to/ufs-srweather-app/modulefiles - module load wflow_ - conda activate srw_app - + module load wflow_ +where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values). \ No newline at end of file From 1d1c47a4afa56b7296f3778236e71f3a6c4814ff Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 13:25:44 -0400 Subject: [PATCH 23/43] add code snippets to tutorial --- .../BuildingRunningTesting/Tutorial.rst | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index e44550bcc4..cc995c873a 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -55,7 +55,7 @@ On :srw-wiki:`Level 1 ` systems, users can fi Load the Workflow -------------------- -To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment. From the ``ufs-srweather-app`` directory, run: +To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment by running: .. include:: ../../doc-snippets/load-env.rst @@ -132,9 +132,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR .. _CronNote: -.. note:: - - Users may also want to set ``USE_CRON_TO_RELAUNCH: true`` and add ``CRON_RELAUNCH_INTVL_MNTS: 3``. This will automate submission of workflow tasks when running the experiment. However, not all systems have :term:`cron`. +.. include:: ../../doc-snippets/cron-note.rst ``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``control`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different forecasts --- one that uses the FV3_GFS_v16 physics suite and one that uses the FV3_RRFS_v1beta physics suite --- "gfsv16_physics_fcst" could be a good alternative directory name. @@ -590,15 +588,9 @@ On :srw-wiki:`Level 1 ` systems, users can fi Load the workflow --------------------- -To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment. From the ``ufs-srweather-app`` directory, run: +To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment by running: -.. code-block:: console - - source etc/lmod-setup.sh - module use modulefiles - module load wflow_ - -where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values). +.. include:: ../../doc-snippets/load-env.rst After loading the workflow, users should follow the instructions printed to the console. Usually, the instructions will tell the user to run |activate|. For example, a user on Hera with permissions on the ``nems`` project may issue the following commands to load the workflow (replacing ``User.Name`` with their actual username): From 38fd16fe3e3773eb1b9044187cdde0db7a09ba71 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 13:33:44 -0400 Subject: [PATCH 24/43] update snippet & VX ch --- doc/UsersGuide/BuildingRunningTesting/VXCases.rst | 2 +- doc/doc-snippets/load-env.rst | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/VXCases.rst b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst index 595adb1cab..81be70024c 100644 --- a/doc/UsersGuide/BuildingRunningTesting/VXCases.rst +++ b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst @@ -81,7 +81,7 @@ Save the path to this file in and ``INDYDATA`` environment variable: Load the Workflow ^^^^^^^^^^^^^^^^^^^^ -First, navigate to the ``ufs-srweather-app/ush`` directory. Then, load the workflow environment: +To load the workflow environment, run: .. include:: ../../doc-snippets/load-env.rst Users running a csh/tcsh shell would run ``source /path/to/etc/lmod-setup.csh `` in place of the first command above. diff --git a/doc/doc-snippets/load-env.rst b/doc/doc-snippets/load-env.rst index ce537e7824..e73d8234fc 100644 --- a/doc/doc-snippets/load-env.rst +++ b/doc/doc-snippets/load-env.rst @@ -4,4 +4,4 @@ module use /path/to/ufs-srweather-app/modulefiles module load wflow_ -where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values). \ No newline at end of file +where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values), and ``path/to/`` is replaced by the actual path to the ``ufs-srweather-app``. \ No newline at end of file From 1b7cffceb7ab88573e3b381ea2bc4865106c6865 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 14:57:34 -0400 Subject: [PATCH 25/43] fix typo --- doc/doc-snippets/load-env.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/doc-snippets/load-env.rst b/doc/doc-snippets/load-env.rst index e73d8234fc..85afec6199 100644 --- a/doc/doc-snippets/load-env.rst +++ b/doc/doc-snippets/load-env.rst @@ -4,4 +4,4 @@ module use /path/to/ufs-srweather-app/modulefiles module load wflow_ -where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values), and ``path/to/`` is replaced by the actual path to the ``ufs-srweather-app``. \ No newline at end of file +where ```` is a valid, lowercased machine name (see ``MACHINE`` in :numref:`Section %s ` for valid values), and ``/path/to/`` is replaced by the actual path to the ``ufs-srweather-app``. \ No newline at end of file From f41e41e400edb6f16d292c18156f7e8bc16ad426 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 16:03:36 -0400 Subject: [PATCH 26/43] replace PNG w/GIF --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index cc995c873a..f996de848b 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -570,7 +570,7 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t * `Storm Prediction Center (SPC) Storm Report for 20191031 `_ -.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/Tutorial/HalloweenStorm.jpg +.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/Tutorial/HalloweenStorm.gif :alt: Radar animation of the Halloween Storm that swept across the Eastern United States in 2019. *Halloween Storm 2019* From 48cd3e60c4ea9ddad41e1cb0679afb7d5d3e57e7 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 19:13:19 -0400 Subject: [PATCH 27/43] factor out conf intro section for reuse --- .../BuildingRunningTesting/Tutorial.rst | 32 +++---------------- doc/doc-snippets/expt-conf-intro.rst | 14 ++++++++ 2 files changed, 18 insertions(+), 28 deletions(-) create mode 100644 doc/doc-snippets/expt-conf-intro.rst diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index f996de848b..0b5a2f304a 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -71,20 +71,7 @@ After loading the workflow, users should follow the instructions printed to the Configuration ------------------------- -Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: - -.. code-block:: console - - cd /path/to/ufs-srweather-app/ush - cp config.community.yaml config.yaml - -Users can save the location of the ``ush`` directory in an environment variable (``$USH``). This makes it easier to navigate between directories later. For example: - -.. code-block:: console - - export USH=/path/to/ufs-srweather-app/ush - -Users should substitute ``/path/to/ufs-srweather-app/ush`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $USH``, and it will take them to the ``ush`` directory. The variable will need to be reset for each login session. +.. include:: ../../doc-snippets/expt-conf-intro.rst Experiment 1: Control ^^^^^^^^^^^^^^^^^^^^^^^^ @@ -580,6 +567,8 @@ Data On :srw-wiki:`Level 1 ` systems, users can find data for the Halloween Storm in the usual input model data locations (see :numref:`Section %s ` for a list). The RAP data can also be downloaded from the `SRW App data bucket `_ using ``wget``: +.. COMMENT: Revisit data - it's getting pulled from a bucket! + .. code-block:: console wget https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz @@ -604,20 +593,7 @@ After loading the workflow, users should follow the instructions printed to the Configuration ------------------------- -Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: - -.. code-block:: console - - cd /path/to/ufs-srweather-app/ush - - -Users can save the location of the ``ush`` directory in an environment variable (``$USH``). This makes it easier to navigate between directories later. For example: - -.. code-block:: console - - export USH=/path/to/ufs-srweather-app/ush - -Users should substitute ``/path/to/ufs-srweather-app/ush`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $USH``, and it will take them to the ``ush`` directory. The variable will need to be reset for each login session. +.. include:: ../../doc-snippets/expt-conf-intro.rst Experiment 1: Rap Data ^^^^^^^^^^^^^^^^^^^^^^^^ diff --git a/doc/doc-snippets/expt-conf-intro.rst b/doc/doc-snippets/expt-conf-intro.rst new file mode 100644 index 0000000000..d23fc546cb --- /dev/null +++ b/doc/doc-snippets/expt-conf-intro.rst @@ -0,0 +1,14 @@ +Navigate to the ``ufs-srweather-app/ush`` directory. The default (or "control") configuration for this experiment is based on the ``config.community.yaml`` file in that directory. Users can copy this file into ``config.yaml`` if they have not already done so: + +.. code-block:: console + + cd /path/to/ufs-srweather-app/ush + cp config.community.yaml config.yaml + +Users can save the location of the ``ush`` directory in an environment variable (``$USH``). This makes it easier to navigate between directories later. For example: + +.. code-block:: console + + export USH=/path/to/ufs-srweather-app/ush + +Users should substitute ``/path/to/ufs-srweather-app/ush`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $USH``, and it will take them to the ``ush`` directory. The variable will need to be reset for each login session. \ No newline at end of file From 6fd03491f05f791b11fe3956be41730b1a1ccac9 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 19:20:03 -0400 Subject: [PATCH 28/43] factor out file editing hint --- .../BuildingRunningTesting/Tutorial.rst | 22 +++---------------- doc/doc-snippets/file-edit-hint.rst | 9 ++++++++ 2 files changed, 12 insertions(+), 19 deletions(-) create mode 100644 doc/doc-snippets/file-edit-hint.rst diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 0b5a2f304a..aab5176630 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -78,15 +78,7 @@ Experiment 1: Control Edit the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below. -.. Hint:: - - To open the configuration file in the command line, users may run the command: - - .. code-block:: console - - vi config.yaml - - To modify the file, hit the ``i`` key and then make any changes required. To close and save, hit the ``esc`` key and type ``:wq`` to write the changes to the file and exit/quit the file. Users may opt to use their preferred code editor instead. +.. include:: ../../doc-snippets/file-edit-hint.rst Start in the ``user:`` section and change the ``MACHINE`` and ``ACCOUNT`` variables. For example, when running on a personal MacOS device, users might set: @@ -595,20 +587,12 @@ Configuration .. include:: ../../doc-snippets/expt-conf-intro.rst -Experiment 1: Rap Data +Experiment 1: RAP Data ^^^^^^^^^^^^^^^^^^^^^^^^ Edit the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below. -.. Hint:: - - To open the configuration file in the command line, users may run the command: - - .. code-block:: console - - vi config.yaml - - To modify the file, hit the ``i`` key and then make any changes required. To close and save, hit the ``esc`` key and type ``:wq`` to write the changes to the file and exit/quit the file. Users may opt to use their preferred code editor instead. +.. include:: ../../doc-snippets/file-edit-hint.rst Start in the ``user:`` section and change the ``MACHINE`` and ``ACCOUNT`` variables. For example, when running on a personal MacOS device, users might set: diff --git a/doc/doc-snippets/file-edit-hint.rst b/doc/doc-snippets/file-edit-hint.rst new file mode 100644 index 0000000000..b7a9b99b6a --- /dev/null +++ b/doc/doc-snippets/file-edit-hint.rst @@ -0,0 +1,9 @@ +.. Hint:: + + To open the configuration file in the command line, users may run the command: + + .. code-block:: console + + vi config.yaml + + To modify the file, hit the ``i`` key and then make any changes required. To close and save, hit the ``esc`` key and type ``:wq`` to write the changes to the file and exit/quit the file. Users may opt to use their preferred code editor instead. \ No newline at end of file From 72225c7339093c88bb894b50e78ded24f32c4360 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Fri, 27 Sep 2024 21:04:13 -0400 Subject: [PATCH 29/43] update data & plotting info --- .../BuildingRunningTesting/Tutorial.rst | 73 ++++++++++--------- 1 file changed, 40 insertions(+), 33 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index aab5176630..89e5b62e70 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -557,14 +557,15 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t Data ------- -On :srw-wiki:`Level 1 ` systems, users can find data for the Halloween Storm in the usual input model data locations (see :numref:`Section %s ` for a list). The RAP data can also be downloaded from the `SRW App data bucket `_ using ``wget``: +Data for the Halloween Storm is publicly in S3 data buckets. The Rapid Refresh (`RAP `_) data can be downloaded from the `SRW App data bucket `_ using ``wget``: .. COMMENT: Revisit data - it's getting pulled from a bucket! .. code-block:: console wget https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz - tar -xzf halloween_rap.tgz + +This will untar the ``halloween_rap.tgz`` data into a directory named ``RAP``. Load the workflow --------------------- @@ -605,9 +606,9 @@ Start in the ``user:`` section and change the ``MACHINE`` and ``ACCOUNT`` variab For a detailed description of these variables, see :numref:`Section %s `. -Users do not need to change the ``platform:`` section of the configuration file for this tutorial. The default parameters in the ``platform:`` section pertain to METplus verification, which is not addressed here. For more information on verification, see :numref:`Section %s `. +Users do not need to change the ``platform:`` section of the configuration file for this tutorial. -In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PREDEF_GRID_NAME``. +In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR``, ``CCPP_PHYS_SUITE``, ``PREDEF_GRID_NAME``, ``DATE_FIRST_CYCL``, ``DATE_LAST_CYCL``, and ``FCST_LEN_HRS``. .. code-block:: console @@ -625,46 +626,28 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR .. include:: ../../doc-snippets/cron-note.rst -``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "halloweenRAP" to "halloweenHRRR" to "a;skdfj". However, the best names will indicate useful information about the experiment. Since this tutorial helps users to compare the output from two different types of forecast data input --- halloweenRAP and halloweenHRRR could be good names. - -``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. Using the RRFS_CONUS_13km grid provides a higher resolution forecast, more detailed forecast; however, it is more computationally expensive compared to the 25km grid. For more information on this grid, see :numref:`Section %s `. - -For a detailed description of other ``workflow:`` variables, see :numref:`Section %s `. - -To turn on the plotting for the experiment, the YAML configuration file -should be included in the ``rocoto:tasks:taskgroups:`` section, like this: +``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "halloweenRAP" to "HalloweenStorm1" to "a;skdfj". However, the best names will indicate useful information about the experiment. Since this tutorial helps users to compare the output from RAP and HRRR forecast input data, this tutorial will use ``halloweenRAP`` for the Halloween Storm experiment that uses RAP forecast data. -.. code-block:: console +``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. This 13-km resolution is used in the NOAA operational Rapid Refresh (`RAP `_) model and is the resolution envisioned for the initial operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`). For more information on this grid, see :numref:`Section %s `. - rocoto: - tasks: - task_get_extrn_ics: - walltime: 06:00:00 - task_get_extrn_lbcs: - walltime: 06:00:00 - metatask_run_ensemble: - task_make_lbcs_mem#mem#: - walltime: 06:00:00 - task_run_fcst_mem#mem#: - walltime: 06:00:00 - taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' +``CCPP_PHYS_SUITE:`` The FV3_RAP physics suite contains the evolving :term:`parameterizations` used operationally in the NOAA Rapid Refresh (`RAP `_) model; the suite is also a prime candidate under consideration for initial RRFS implementation and has been well-tested at the 13-km resolution. It is therefore an appropriate physics choice when using the RRFS_CONUS_13km grid. +``DATE_FIRST_CYCL``, ``DATE_LAST_CYCL``, and ``FCST_LEN_HRS`` set parameters related to the date and duration of the forecast. Because this is a one-cycle experiment that does not use cycling or :term:`data assimilation`, the date of the first :term:`cycle` and last cycle are the same. -For more information on how to turn on/off tasks in the workflow, please -see :numref:`Section %s `. +For a detailed description of other ``workflow:`` variables, see :numref:`Section %s `. -In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_SOURCE_BASEDIR_ICS``. Users will need to adjust the file path to reflect the location of data on their system (see :numref:`Section %s ` for locations on :srw-wiki:`Level 1 ` systems). +In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_SOURCE_BASEDIR_ICS``. Users will need to adjust the file path to point to the location of the data on their system. .. code-block:: console task_get_extrn_ics: EXTRN_MDL_NAME_ICS: RAP USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/for_ICS + EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/RAP/for_ICS For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. -Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_SOURCE_BASEDIR_LBCS``. Users will need to adjust the file path to reflect the location of data on their system (see :numref:`Section %s ` for locations on Level 1 systems). +Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_SOURCE_BASEDIR_LBCS``. Users will need to adjust the file path to point to the location of the data on their system. .. code-block:: console @@ -672,14 +655,38 @@ Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_ EXTRN_MDL_NAME_LBCS: RAP LBC_SPEC_INTVL_HRS: 3 USE_USER_STAGED_EXTRN_FILES: true - EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/UFS_SRW_App/develop/input_model_data/RAP/for_LBCS + EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/RAP/for_LBCS For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. Users do not need to modify the ``task_run_fcst:`` section for this tutorial. +.. COMMENT: Do we need to set QUILTING to true? + +In the ``rocoto:tasks:`` section, increase the walltime for the data-related tasks and metatasks. Then include the YAML configuration file containing the plotting task in the ``rocoto:tasks:taskgroups:`` section, like this: + +.. code-block:: console + + rocoto: + tasks: + task_get_extrn_ics: + walltime: 06:00:00 + task_get_extrn_lbcs: + walltime: 06:00:00 + metatask_run_ensemble: + task_make_lbcs_mem#mem#: + walltime: 06:00:00 + task_run_fcst_mem#mem#: + walltime: 06:00:00 + taskgroups: '{{ ["parm/wflow/prep.yaml", "parm/wflow/coldstart.yaml", "parm/wflow/post.yaml", "parm/wflow/plot.yaml"]|include }}' + +.. note:: + + Tasks are run once each. A :ref:`Rocoto ` metatask expands into one or more similar tasks by replacing the values between ``#`` symbols with the values under the ``var:`` key. See the `Rocoto documentation `_ for more information. + +For more information on how to turn on/off tasks in the workflow, please see :numref:`Section %s `. -Lastly, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 12`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. +Then, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 36`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. .. code-block:: console @@ -687,7 +694,7 @@ Lastly, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users m COMOUT_REF: "" PLOT_FCST_INC: 6 -``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 18z on June 15, 2019 (the 0th forecast hour) through the 12th forecast hour (June 16, 2019 at 06z). +``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 12z on October 30, 2019 (the 0th forecast hour) through the 36th forecast hour (November 2, 2019 at 12z). After configuring the forecast, users can generate the forecast by running: From 9d06f5591a715ad26b2b192a829dd9fbc0008097 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 10:10:33 -0400 Subject: [PATCH 30/43] refactor scp instructions --- .../BuildingRunningTesting/Tutorial.rst | 64 +++++-------------- doc/doc-snippets/scp-files.rst | 11 ++++ 2 files changed, 26 insertions(+), 49 deletions(-) create mode 100644 doc/doc-snippets/scp-files.rst diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 89e5b62e70..edbdfc2e5f 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -302,17 +302,7 @@ Navigate to ``test_expt/2019061518/postprd``. This directory contains the post-p Copy ``.png`` Files onto Local System ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Users who are working on the cloud or on an HPC cluster may want to copy the ``.png`` files onto their local system to view in their preferred image viewer. Detailed instructions are available in the :ref:`Introduction to SSH & Data Transfer `. - -In summary, users can run the ``scp`` command in a new terminal/command prompt window to securely copy files from a remote system to their local system if an SSH tunnel is already established between the local system and the remote system. Users can adjust one of the following commands for their system: - -.. code-block:: console - - scp username@your-IP-address:/path/to/source_file_or_directory /path/to/destination_file_or_directory - # OR - scp -P 12345 username@localhost:/path/to/source_file_or_directory /path/to/destination_file_or_directory - -Users would need to modify ``username``, ``your-IP-address``, ``-P 12345``, and the file paths to reflect their systems' information. See the :ref:`Introduction to SSH & Data Transfer ` for example commands. +.. include:: ../../doc-snippets/scp-files.rst .. _ComparePlots: @@ -682,11 +672,11 @@ In the ``rocoto:tasks:`` section, increase the walltime for the data-related tas .. note:: - Tasks are run once each. A :ref:`Rocoto ` metatask expands into one or more similar tasks by replacing the values between ``#`` symbols with the values under the ``var:`` key. See the `Rocoto documentation `_ for more information. + Rocoto tasks are run once each. A :ref:`Rocoto ` metatask expands into one or more similar tasks by replacing the values between ``#`` symbols with the values under the ``var:`` key. See the `Rocoto documentation `_ for more information. For more information on how to turn on/off tasks in the workflow, please see :numref:`Section %s `. -Then, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 36`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. +In the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users may also want to add ``PLOT_FCST_START: 0`` and ``PLOT_FCST_END: 36`` explicitly, but these can be omitted since the default values are the same as the forecast start and end time respectively. .. code-block:: console @@ -694,7 +684,7 @@ Then, in the ``task_plot_allvars:`` section, add ``PLOT_FCST_INC: 6``. Users may COMOUT_REF: "" PLOT_FCST_INC: 6 -``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 12z on October 30, 2019 (the 0th forecast hour) through the 36th forecast hour (November 2, 2019 at 12z). +``PLOT_FCST_INC:`` This variable indicates the forecast hour increment for the plotting task. By setting the value to ``6``, the task will generate a ``.png`` file for every 6th forecast hour starting from 12z on October 30, 2019 (the 0th forecast hour) through the 36th forecast hour (November 1, 2019 at 0z). After configuring the forecast, users can generate the forecast by running: @@ -706,7 +696,7 @@ To see experiment progress, users should navigate to their experiment directory. .. code-block:: console - cd /path/to/expt_dirs/control + cd /path/to/expt_dirs/halloweenRAP rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 @@ -716,39 +706,31 @@ Users will need to rerun the ``rocotorun`` and ``rocotostat`` commands above reg When using cron to automate the workflow submission (as described :ref:`above `), users can omit the ``rocotorun`` command and simply use ``rocotostat`` to check on progress periodically. -Users can save the location of the ``Halloween`` directory in an environment variable (``$CONTROL``). This makes it easier to navigate between directories later. For example: +Users can save the location of the ``halloweenRAP`` directory in an environment variable (e.g., ``$HRAP``). This makes it easier to navigate between directories later. For example: .. code-block:: console - export CONTROL=/path/to/expt_dirs/control + export CONTROL=/path/to/expt_dirs/HRAP -Users should substitute ``/path/to/expt_dirs/Halloween`` with the actual path on their system. As long as a user remains logged into their system, they can run ``cd $Halloween``, and it will take them to the ``halloween`` experiment directory. The variable will need to be reset for each login session. +Users should substitute ``/path/to/expt_dirs/HRAP`` with the actual path to the experiment directory on their system. As long as a user remains logged into their system, they can run ``cd $HRAP``, and it will take them to the ``halloweenRAP`` experiment directory. The variable will need to be reset for each login session. -Experiment 2: Changing the forecast input +Experiment 2: Changing the Forecast Input ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In this experiment we will be changing the forecast input to use ``HRRR`` data. This will include edits to the configuration file (``config.yaml``) to include the variables and values in the sample configuration excerpts below +Once the ``halloweenRAP`` case is running, users can return to the ``config.yaml`` file (in ``$USH``) and adjust the parameters for a new forecast. In this forecast, users will change the forecast input to use ``HRRR`` data and alter a few associated parameters. -In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PREDEF_GRID_NAME``. +In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PREDEF_GRID_NAME``. Other parameters should remain the same. .. code-block:: console workflow: - USE_CRON_TO_RELAUNCH: false EXPT_SUBDIR: halloweenHRRR - CCPP_PHYS_SUITE: FV3_RAP PREDEF_GRID_NAME: RRFS_CONUScompact_13km - DATE_FIRST_CYCL: '2019103012' - DATE_LAST_CYCL: '2019103012' - FCST_LEN_HRS: 36 - PREEXISTING_DIR_METHOD: rename - VERBOSE: true - COMPILER: intel .. note:: - Since HRRR is a high-resolution model than RAP, we will need to utilize the ``RRFS_CONUScompact_13km`` grid in order for the experiment to run successfully. + Relative to the original CONUS domain, the "compact" CONUS domains are slightly smaller. The original CONUS domains were a bit too large to run with :term:`LBCs` from HRRR, so the "compact" domains were created to be just small enough to work with HRRR data. -In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_FILES_ICS``. +In the ``task_get_extrn_ics:`` section, update the values for ``EXTRN_MDL_NAME_ICS`` and ``USE_USER_STAGED_EXTRN_FILES`` and add ``EXTRN_MDL_FILES_ICS``. Users may choose to comment out or remove ``EXTRN_MDL_SOURCE_BASEDIR_ICS``, but this is not necessary. .. code-block:: console @@ -760,7 +742,7 @@ In the ``task_get_extrn_ics:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and For a detailed description of the ``task_get_extrn_ics:`` variables, see :numref:`Section %s `. -Similarly, in the ``task_get_extrn_lbcs:`` section, add ``USE_USER_STAGED_EXTRN_FILES`` and ``EXTRN_MDL_FILES_LBCS``. +Update the same values in the ``task_get_extrn_lbcs:`` section: .. code-block:: console @@ -783,18 +765,7 @@ Navigate to ``halloweenHRRR/2019103012/postprd`` or ``halloweenRAP/2019203012/po Copy ``.png`` Files onto Local System ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Users who are working on the cloud or on an HPC cluster may want to copy the ``.png`` files onto their local system to view in their preferred image viewer. Detailed instructions are available in the :ref:`Introduction to SSH & Data Transfer `. - -In summary, users can run the ``scp`` command in a new terminal/command prompt window to securely copy files from a remote system to their local system if an SSH tunnel is already established between the local system and the remote system. Users can adjust one of the following commands for their system: - -.. code-block:: console - - scp username@your-IP-address:/path/to/source_file_or_directory /path/to/destination_file_or_directory - # OR - scp -P 12345 username@localhost:/path/to/source_file_or_directory /path/to/destination_file_or_directory - -Users would need to modify ``username``, ``your-IP-address``, ``-P 12345``, and the file paths to reflect their systems' information. See the :ref:`Introduction to SSH & Data Transfer ` for example commands. - +.. include:: ../../doc-snippets/scp-files.rst Experiment 3: Examining Forecast Plots at Peak Intensity ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -860,11 +831,6 @@ In the composite reflectivity plots below, the ``halloweenHRRR`` and ``halloween *HRRR plot for Composite Reflectivity* -Tutorial Content -------------------- - -Coming Soon! - .. _fcst5: Sample Forecast #5: Hurricane Barry diff --git a/doc/doc-snippets/scp-files.rst b/doc/doc-snippets/scp-files.rst new file mode 100644 index 0000000000..bc29780e9c --- /dev/null +++ b/doc/doc-snippets/scp-files.rst @@ -0,0 +1,11 @@ +Users who are working on the cloud or on an HPC cluster may want to copy the ``.png`` files onto their local system to view in their preferred image viewer. Detailed instructions are available in the :ref:`Introduction to SSH & Data Transfer `. + +In summary, users can run the ``scp`` command in a new terminal/command prompt window to securely copy files from a remote system to their local system if an SSH tunnel is already established between the local system and the remote system. Users can adjust one of the following commands for their system: + +.. code-block:: console + + scp username@your-IP-address:/path/to/source_file_or_directory /path/to/destination_file_or_directory + # OR + scp -P 12345 username@localhost:/path/to/source_file_or_directory /path/to/destination_file_or_directory + +Users would need to modify ``username``, ``your-IP-address``, ``-P 12345``, and the file paths to reflect their systems' information. See the :ref:`Introduction to SSH & Data Transfer ` for example commands. \ No newline at end of file From 52029c072ce7f22bc37cb8e62d220b320333aec5 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 10:22:32 -0400 Subject: [PATCH 31/43] misc typos --- .../BuildingRunningTesting/Tutorial.rst | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index edbdfc2e5f..4aeee99be8 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -760,16 +760,16 @@ Users do not need to modify the ``task_run_fcst:`` section for this tutorial. How to Analyze Results ----------------------- -Navigate to ``halloweenHRRR/2019103012/postprd`` or ``halloweenRAP/2019203012/postprd``. This directory contains the post-processed data generated by the :term:`UPP` from the ``halloween`` forecast. After the ``plot_allvars`` task completes, this directory will contain ``.png`` images for several forecast variables. +Navigate to ``halloweenHRRR/2019103012/postprd`` and/or ``halloweenRAP/2019203012/postprd``. These directories contain the post-processed data generated by the :term:`UPP` from the Halloween Storm forecasts. After the ``plot_allvars`` task completes, this directory will contain ``.png`` images for several forecast variables. Copy ``.png`` Files onto Local System ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. include:: ../../doc-snippets/scp-files.rst -Experiment 3: Examining Forecast Plots at Peak Intensity +Examining Forecast Plots at Peak Intensity ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -In this experiment will be looking at plots from HRRR and RAP input files while the Halloween Storm is at or approaching peak intensity. +This experiment will be looking at plots from HRRR and RAP input files while the Halloween Storm is at or approaching peak intensity. .. _fcst4_250wind: @@ -777,25 +777,25 @@ In this experiment will be looking at plots from HRRR and RAP input files while `````````` An effective weather forecast begins with analyzing a 250mb wind chart. By using this wind plot, forecasters can identify key features such as jet stream placement, jet maxima, troughs, ridges, and more. This analysis also helps pinpoint areas with the potential for the strongest severe weather. -In the 250mb wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` plots are nearly identical at forecast hour f036. This shows great model agreement. Analyzing this chart we can see multiple ingredients signaling a significant severe weather event over eastern CONUS. The first thing to notice is the placement of the jet streak along with troughing approaching the eastern US. Also notice an extreme 150KT jet max over southern Ohio further fueling severe weather. The last thing to notice is the divegence aloft present over Eastern Conus, seeing divergence present all the way up to 250mb indicates a strong system. +In the 250mb wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` plots are nearly identical at forecast hour f036. This shows great model agreement. Analyzing this chart we can see multiple ingredients signaling a significant severe weather event over the eastern CONUS. The first thing to notice is the placement of the jet streak along with troughing approaching the eastern US. Also notice an extreme 150KT jet max over southern Ohio further fueling severe weather. The last thing to notice is the divergence aloft present over the eastern CONUS; seeing divergence present all the way up to 250mb indicates a strong system. .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/250wind_rap_conus_f036.png :align: center :width: 75% - *Rap plot for 250mb wind* + *RAP Plot for 250mb Wind* .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/250wind_hrrr_conus_f036.png :align: center :width: 75% - *HRRR plot for 250mb wind* + *HRRR Plot for 250mb Wind* .. _fcst4_10mwind: 10m Wind `````````` -The 10m wind plots allows forecasters to pick up on patterns closer to the surface. It shows things such as convergence and pressure areas. +The 10m wind plots allows forecasters to pick up on patterns closer to the surface. It shows features such as convergence and pressure areas. In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once again very similar, which makes sense given that the 250mb wind plots are also so similar. We can see a few key feautres on this chart. The most important is the area of convergence taking place over the east coast which is driving the line of severe storms. @@ -803,13 +803,13 @@ In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once :align: center :width: 75% - *Rap plot for 10m winds* + *Rap Plot for 10m Winds* .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/10mwind_hrrr_conus_f036.png :align: center :width: 75% - *HRRR plot for 10m winds* + *HRRR Plot for 10m Winds* .. _fcst4_refc: @@ -823,13 +823,13 @@ In the composite reflectivity plots below, the ``halloweenHRRR`` and ``halloween :align: center :width: 75% - *Rap plot for Composite Reflectivity* + *RAP Plot for Composite Reflectivity* .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/refc_hrrr_conus_f036.png :align: center :width: 75% - *HRRR plot for Composite Reflectivity* + *HRRR Plot for Composite Reflectivity* .. _fcst5: From 9786a916a20af1066c5fdb89052422fb28a803d5 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 13:08:18 -0400 Subject: [PATCH 32/43] update links --- doc/ContribGuide/contributing.rst | 2 +- doc/UsersGuide/BackgroundInfo/Components.rst | 4 ++-- .../BackgroundInfo/Introduction.rst | 4 ++-- .../BuildingRunningTesting/BuildSRW.rst | 2 +- .../BuildingRunningTesting/RunSRW.rst | 2 +- .../BuildingRunningTesting/Tutorial.rst | 9 +++++---- .../BuildingRunningTesting/VXCases.rst | 2 +- doc/UsersGuide/Reference/Glossary.rst | 20 +++++++++---------- doc/UsersGuide/SSHIntro.rst | 2 +- doc/conf.py | 6 ++++-- 10 files changed, 28 insertions(+), 25 deletions(-) diff --git a/doc/ContribGuide/contributing.rst b/doc/ContribGuide/contributing.rst index eb995efb41..0f7231e265 100644 --- a/doc/ContribGuide/contributing.rst +++ b/doc/ContribGuide/contributing.rst @@ -11,7 +11,7 @@ Fork and PR Overview Contributions to the ``ufs-srweather-app`` project are made via a :github-docs:`Fork` and :github-docs:`Pull Request (PR)` model. GitHub provides a thorough description of this contribution model in their `Contributing to a project` :github-docs:`Quickstart`, but the steps, with respect to ``ufs-srweather-app`` contributions, can be summarized as: -#. :github-docs:`Create an issue ` to document proposed changes. +#. :github-docs:`Create an issue ` to document proposed changes. #. :github-docs:`Fork` the :srw-repo:`ufs-srweather-app repository<>` into your personal GitHub account. #. :github-docs:`Clone` your fork onto your development system. #. :github-docs:`Create a branch` in your clone for your changes. All development should take place on a branch, *not* on ``develop``. diff --git a/doc/UsersGuide/BackgroundInfo/Components.rst b/doc/UsersGuide/BackgroundInfo/Components.rst index 559576725d..c70f6cefb0 100644 --- a/doc/UsersGuide/BackgroundInfo/Components.rst +++ b/doc/UsersGuide/BackgroundInfo/Components.rst @@ -38,12 +38,12 @@ Supported model resolutions in this release include 3-, 13-, and 25-km predefine Model Physics --------------- -The Common Community Physics Package (CCPP), described `here `__, supports interoperable atmospheric physics and land surface model options. Atmospheric physics are a set of numerical methods describing small-scale processes such as clouds, turbulence, radiation, and their interactions. The most recent SRW App release (|latestr|) included five supported physics suites: FV3_RRFS_v1beta, FV3_GFS_v16, FV3_WoFS_v0, FV3_HRRR, and FV3_RAP. The FV3_RRFS_v1beta physics suite is being tested for use in the future operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`) planned for 2023-2024, and the FV3_GFS_v16 is an updated version of the physics suite used in the operational Global Forecast System (GFS) v16. A detailed list of CCPP updates since the SRW App v2.1.0 release is available :ref:`here `. A full scientific description of CCPP parameterizations and suites can be found in the `CCPP Scientific Documentation `__, and CCPP technical aspects are described in the :doc:`CCPP Technical Documentation `. The model namelist has many settings beyond the physics options that can optimize various aspects of the model for use with each of the supported suites. Additional information on Stochastic Physics options is available :doc:`here `. +The Common Community Physics Package (CCPP), described `here `__, supports interoperable atmospheric physics and land surface model options. Atmospheric physics are a set of numerical methods describing small-scale processes such as clouds, turbulence, radiation, and their interactions. The most recent SRW App release (|latestr|) included five supported physics suites: FV3_RRFS_v1beta, FV3_GFS_v16, FV3_WoFS_v0, FV3_HRRR, and FV3_RAP. The FV3_RRFS_v1beta physics suite is being tested for use in the future operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`) planned for 2023-2024, and the FV3_GFS_v16 is an updated version of the physics suite used in the operational Global Forecast System (GFS) v16. A detailed list of CCPP updates since the SRW App v2.1.0 release is available :ref:`here `. A full scientific description of CCPP parameterizations and suites can be found in the `CCPP Scientific Documentation `_, and CCPP technical aspects are described in the :doc:`CCPP Technical Documentation `. The model namelist has many settings beyond the physics options that can optimize various aspects of the model for use with each of the supported suites. Additional information on Stochastic Physics options is available :doc:`here `. .. note:: SPP is currently only available for specific physics schemes used in the RAP/HRRR physics suite. Users need to be aware of which physics suite definition file (:term:`SDF`) is chosen when turning this option on. Among the supported physics suites, the full set of parameterizations can only be used with the ``FV3_HRRR`` option for ``CCPP_PHYS_SUITE``. -Additionally, a CCPP single-column model (`CCPP-SCM `__) option has also been developed as a child repository. Users can refer to the `CCPP Single Column Model User and Technical Guide `__ for more details. This CCPP-SCM user guide contains a Quick Start Guide with instructions for obtaining the code, compiling, and running test cases, which include five standard test cases and two additional FV3 replay cases (refer to section 5.2 in the CCPP-SCM user guide for more details). Moreover, the CCPP-SCM supports a precompiled version in a docker container, allowing it to be easily executed on NOAA's cloud computing platforms without any issues (see section 2.5 in the CCPP-SCM user guide for more details). +Additionally, a CCPP single-column model (`CCPP-SCM `_) option has also been developed as a child repository. Users can refer to the `CCPP Single Column Model User and Technical Guide `_ for more details. This CCPP-SCM user guide contains a Quick Start Guide with instructions for obtaining the code, compiling, and running test cases, which include five standard test cases and two additional FV3 replay cases (refer to section 5.2 in the CCPP-SCM user guide for more details). Moreover, the CCPP-SCM supports a precompiled version in a docker container, allowing it to be easily executed on NOAA's cloud computing platforms without any issues (see section 2.5 in the CCPP-SCM user guide for more details). The SRW App supports the use of both :term:`GRIB2` and :term:`NEMSIO` input data. The UFS Weather Model ingests initial and lateral boundary condition files produced by :term:`chgres_cube` and outputs files in netCDF format on a specific projection (e.g., Lambert Conformal) in the horizontal direction and model levels in the vertical direction. diff --git a/doc/UsersGuide/BackgroundInfo/Introduction.rst b/doc/UsersGuide/BackgroundInfo/Introduction.rst index f1a384e025..4b0978ef23 100644 --- a/doc/UsersGuide/BackgroundInfo/Introduction.rst +++ b/doc/UsersGuide/BackgroundInfo/Introduction.rst @@ -4,9 +4,9 @@ Introduction ============== -The Unified Forecast System (:term:`UFS`) is a community-based, coupled, comprehensive Earth modeling system. NOAA's operational model suite for numerical weather prediction (:term:`NWP`) is quickly transitioning to the UFS from a number of different modeling systems. The UFS enables research, development, and contribution opportunities within the broader :term:`Weather Enterprise` (including government, industry, and academia). For more information about the UFS, visit the `UFS Portal `__. +The Unified Forecast System (:term:`UFS`) is a community-based, coupled, comprehensive Earth modeling system. NOAA's operational model suite for numerical weather prediction (:term:`NWP`) is quickly transitioning to the UFS from a number of different modeling systems. The UFS enables research, development, and contribution opportunities within the broader :term:`Weather Enterprise` (including government, industry, and academia). For more information about the UFS, visit the `UFS Portal `_. -The UFS includes `multiple applications `__ that support different forecast durations and spatial domains. This documentation describes the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The most recent SRW Application includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end. These components are documented within this User's Guide and supported through the `GitHub Discussions `__ forum. The SRW App also includes support for a verification package (METplus) for both deterministic and ensemble simulations and support for four stochastically perturbed physics schemes. +The UFS includes `multiple applications `_ that support different forecast durations and spatial domains. This documentation describes the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The most recent SRW Application includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end. These components are documented within this User's Guide and supported through the `GitHub Discussions `_ forum. The SRW App also includes support for a verification package (METplus) for both deterministic and ensemble simulations and support for four stochastically perturbed physics schemes. Since the last release, developers have added a variety of features: diff --git a/doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst b/doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst index 3076a5f6eb..73334828da 100644 --- a/doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst +++ b/doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst @@ -35,7 +35,7 @@ Install the Prerequisite Software Stack Users on any sufficiently up-to-date machine with a UNIX-based operating system should be able to install the prerequisite software stack and run the SRW Application. However, a list of prerequisites is available in :numref:`Section %s ` for reference. Users should install or update their system as required before attempting to install the software stack. -Currently, installation of the prerequisite software stack is supported via spack-stack on most systems. :term:`Spack-stack` is a :term:`repository` that provides a Spack-based system to build the software stack required for `UFS `__ applications such as the SRW App. Spack-stack is the software stack validated by the UFS Weather Model (:term:`WM`), and the SRW App has likewise shifted to spack-stack for most Level 1 systems. +Currently, installation of the prerequisite software stack is supported via spack-stack on most systems. :term:`Spack-stack` is a :term:`repository` that provides a Spack-based system to build the software stack required for `UFS `_ applications such as the SRW App. Spack-stack is the software stack validated by the UFS Weather Model (:term:`WM`), and the SRW App has likewise shifted to spack-stack for most Level 1 systems. .. hint:: Skip the spack-stack installation if working on a :srw-wiki:`Level 1 system ` (e.g., Hera, Jet, Derecho, NOAA Cloud), and :ref:`continue to the next section `. diff --git a/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst index fe4ac5d117..0239c74b01 100644 --- a/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst +++ b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst @@ -685,7 +685,7 @@ More information about configuring the ``rocoto:`` section can be found in :numr If users have access to NOAA :term:`HPSS` but have not pre-staged the data, the default ``verify_pre.yaml`` taskgroup will activate the tasks, and the workflow will attempt to download the appropriate data from NOAA HPSS. In this case, the ``*_OBS_DIR`` paths must be set to the location where users want the downloaded data to reside. -Users who do not have access to NOAA HPSS and do not have the data on their system will need to download :term:`CCPA`, :term:`MRMS`, and :term:`NDAS` data manually from collections of publicly available data, such as the ones listed `here `__. +Users who do not have access to NOAA HPSS and do not have the data on their system will need to download :term:`CCPA`, :term:`MRMS`, and :term:`NDAS` data manually from collections of publicly available data, such as the ones listed `here `_. Users who have already staged the observation data needed for METplus (i.e., the :term:`CCPA`, :term:`MRMS`, and :term:`NDAS` data) on their system should set the path to this data in ``config.yaml``. diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 4aeee99be8..d7e64d54e8 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -41,7 +41,7 @@ A surface boundary associated with a vorticity maximum over the northern Great P Data ------- -On :srw-wiki:`Level 1 ` systems, users can find data for the Indianapolis Severe Weather Forecast in the usual input model data locations (see :numref:`Section %s ` for a list). The data can also be downloaded from the `UFS SRW Application Data Bucket `__. +On :srw-wiki:`Level 1 ` systems, users can find data for the Indianapolis Severe Weather Forecast in the usual input model data locations (see :numref:`Section %s ` for a list). The data can also be downloaded from the `UFS SRW Application Data Bucket `_. * FV3GFS data for the first forecast (``control``) is located at: @@ -547,16 +547,17 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t Data ------- -Data for the Halloween Storm is publicly in S3 data buckets. The Rapid Refresh (`RAP `_) data can be downloaded from the `SRW App data bucket `_ using ``wget``: - -.. COMMENT: Revisit data - it's getting pulled from a bucket! +Data for the Halloween Storm is publicly available in S3 data buckets. The Rapid Refresh (`RAP `_) data can be downloaded from the `SRW App data bucket `_ using ``wget``: .. code-block:: console wget https://noaa-ufs-srw-pds.s3.amazonaws.com/develop-20240618/halloween_rap.tgz + tar -xzf halloween_rap.tgz This will untar the ``halloween_rap.tgz`` data into a directory named ``RAP``. +The SRW App can pull HRRR data directly from the `HRRR data bucket `_. Users do not need to download the data separately. + Load the workflow --------------------- diff --git a/doc/UsersGuide/BuildingRunningTesting/VXCases.rst b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst index 81be70024c..b36afcefd4 100644 --- a/doc/UsersGuide/BuildingRunningTesting/VXCases.rst +++ b/doc/UsersGuide/BuildingRunningTesting/VXCases.rst @@ -262,7 +262,7 @@ Point STAT Files The Point-Stat files contain continuous variables like temperature, pressure, and wind speed. A description of the Point-Stat file can be found :ref:`here ` in the MET documentation. -The Point-Stat files contain a potentially overwhelming amount of information. Therefore, it is recommended that users focus on the CNT MET test, which contains the `RMSE `__ and `MBIAS `__ statistics. The MET tests are defined in column 24 'LINE_TYPE' of the ``.stat`` file. Look for 'CNT' in this column. Then find column 66-68 for MBIAS and 78-80 for RMSE statistics. A full description of this file can be found :ref:`here `. +The Point-Stat files contain a potentially overwhelming amount of information. Therefore, it is recommended that users focus on the CNT MET test, which contains the `RMSE `_ and `MBIAS `_ statistics. The MET tests are defined in column 24 'LINE_TYPE' of the ``.stat`` file. Look for 'CNT' in this column. Then find column 66-68 for MBIAS and 78-80 for RMSE statistics. A full description of this file can be found :ref:`here `. To narrow down the variable field even further, users can focus on these weather variables: diff --git a/doc/UsersGuide/Reference/Glossary.rst b/doc/UsersGuide/Reference/Glossary.rst index 2612d4fbe8..99463adea3 100644 --- a/doc/UsersGuide/Reference/Glossary.rst +++ b/doc/UsersGuide/Reference/Glossary.rst @@ -10,7 +10,7 @@ Glossary To transport substances in the atmostphere by :term:`advection`. advection - According to the American Meteorological Society (AMS) `definition `__, advection is "The process of transport of an atmospheric property solely by the mass motion (velocity field) of the atmosphere." In common parlance, advection is movement of atmospheric substances that are carried around by the wind. + According to the American Meteorological Society (AMS) definition, `advection `_ is "The process of transport of an atmospheric property solely by the mass motion (velocity field) of the atmosphere." In common parlance, advection is movement of atmospheric substances that are carried around by the wind. AQM The `Air Quality Model `__ (AQM) is a UFS Application that dynamically couples the Community Multiscale Air Quality (:term:`CMAQ`) model with the UFS Weather Model through the :term:`NUOPC` Layer to simulate temporal and spatial variations of atmospheric compositions (e.g., ozone and aerosol compositions). The CMAQ, treated as a column chemistry model, updates concentrations of chemical species (e.g., ozone and aerosol compositions) at each integration time step. The transport terms (e.g., :term:`advection` and diffusion) of all chemical species are handled by the UFS Weather Model as :term:`tracers`. @@ -22,7 +22,7 @@ Glossary Climatology-Calibrated Precipitation Analysis (CCPA) data. This data is required for METplus precipitation verification tasks within the SRW App. The most recent 8 days worth of data are publicly available and can be accessed `here `__. CCPP - The `Common Community Physics Package `__ is a forecast-model agnostic, vetted collection of code containing atmospheric physical parameterizations and suites of parameterizations for use in Numerical Weather Prediction (NWP) along with a framework that connects the physics to the host forecast model. + The `Common Community Physics Package `_ is a forecast-model agnostic, vetted collection of code containing atmospheric physical parameterizations and suites of parameterizations for use in Numerical Weather Prediction (NWP) along with a framework that connects the physics to the host forecast model. chgres_cube The preprocessing software used to create initial and boundary condition files to @@ -78,10 +78,10 @@ Glossary The radar-indicated top of an area of precipitation. Specifically, it contains the height of the 18 dBZ reflectivity value. EMC - The `Environmental Modeling Center `__. + The `Environmental Modeling Center `_. EPIC - The `Earth Prediction Innovation Center `__ seeks to accelerate scientific research and modeling contributions through continuous and sustained community engagement in order to produce the most accurate and reliable operational modeling system in the world. + The `Earth Prediction Innovation Center `_ seeks to accelerate scientific research and modeling contributions through continuous and sustained community engagement in order to produce the most accurate and reliable operational modeling system in the world. ESG Extended Schmidt Gnomonic (ESG) grid. The ESG grid uses the map projection developed by Jim Purser of NOAA :term:`EMC` (:cite:t:`Purser_2020`). @@ -118,7 +118,7 @@ Glossary High-Performance Computing. HPC-Stack - The `HPC-Stack `__ is a repository that provides a unified, shell script-based build system for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `__ and the `Joint Effort for Data assimilation Integration (JEDI) `__ framework. View the HPC-Stack documentation :doc:`here `. + The `HPC-Stack `__ is a repository that provides a unified, shell script-based build system for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `_ and the `Joint Effort for Data assimilation Integration (JEDI) `_ framework. View the HPC-Stack documentation :doc:`here `. HPSS High Performance Storage System (HPSS). @@ -227,25 +227,25 @@ Glossary A central location in which files (e.g., data, code, documentation) are stored and managed. RRFS - The `Rapid Refresh Forecast System `__ (RRFS) is NOAA's next-generation convection-allowing, rapidly-updated, ensemble-based data assimilation and forecasting system currently scheduled for operational implementation in 2024. It is designed to run forecasts on a 3-km :term:`CONUS` domain, see also `NOAA Rapid Refresh Forecast System (RRFS) `__. Experimental data is currently available from the `AWS S3 NOAA-RRFS `__ bucket for deterministic forecasts out to 60 hours at 00, 06, 12, and 18 UTC. Additionally, hourly forecasts out to 18 hours may be available for more recent RRFS model runs; the user needs to verify that data exists for needed dates. + The `Rapid Refresh Forecast System `_ (RRFS) is NOAA's next-generation convection-allowing, rapidly-updated, ensemble-based data assimilation and forecasting system currently scheduled for operational implementation in 2024. It is designed to run forecasts on a 3-km :term:`CONUS` domain, see also `NOAA Rapid Refresh Forecast System (RRFS) `__. Experimental data is currently available from the `AWS S3 NOAA-RRFS `__ bucket for deterministic forecasts out to 60 hours at 00, 06, 12, and 18 UTC. Additionally, hourly forecasts out to 18 hours may be available for more recent RRFS model runs; the user needs to verify that data exists for needed dates. SDF Suite Definition File. An external file containing information about the construction of a physics suite. It describes the schemes that are called, in which order they are called, whether they are subcycled, and whether they are assembled into groups to be called together. Spack - `Spack `__ is a package management tool designed to support multiple versions and configurations of software on a wide variety of platforms and environments. It was designed for large supercomputing centers, where many users and application teams share common installations of software on clusters with exotic architectures. + `Spack `_ is a package management tool designed to support multiple versions and configurations of software on a wide variety of platforms and environments. It was designed for large supercomputing centers, where many users and application teams share common installations of software on clusters with exotic architectures. spack-stack - The `spack-stack `__ is a collaborative effort between the NOAA Environmental Modeling Center (EMC), the UCAR Joint Center for Satellite Data Assimilation (JCSDA), and the Earth Prediction Innovation Center (EPIC). *spack-stack* is a repository that provides a :term:`Spack`-based method for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `__ and the `Joint Effort for Data assimilation Integration (JEDI) `__ framework. *spack-stack* uses the Spack package manager along with custom Spack configuration files and Python scripts to simplify installation of the libraries required to run various applications. The *spack-stack* can be installed on a range of platforms and comes pre-configured for many systems. Users can install the necessary packages for a particular application and later add the missing packages for another application without having to rebuild the entire stack. To get started, check out the documentation :doc:`here `. + The `spack-stack `_ is a collaborative effort between the NOAA Environmental Modeling Center (EMC), the UCAR Joint Center for Satellite Data Assimilation (JCSDA), and the Earth Prediction Innovation Center (EPIC). *spack-stack* is a repository that provides a :term:`Spack`-based method for building the software stack required for numerical weather prediction (NWP) tools such as the `Unified Forecast System (UFS) `_ and the `Joint Effort for Data assimilation Integration (JEDI) `_ framework. *spack-stack* uses the Spack package manager along with custom Spack configuration files and Python scripts to simplify installation of the libraries required to run various applications. The *spack-stack* can be installed on a range of platforms and comes pre-configured for many systems. Users can install the necessary packages for a particular application and later add the missing packages for another application without having to rebuild the entire stack. To get started, check out the documentation :doc:`here `. tracer - According to the American Meteorological Society (AMS) `definition `__, a tracer is "Any substance in the atmosphere that can be used to track the history [i.e., movement] of an air mass." Tracers are carried around by the motion of the atmosphere (i.e., by :term:`advection`). These substances are usually gases (e.g., water vapor, CO2), but they can also be non-gaseous (e.g., rain drops in microphysics parameterizations). In weather models, temperature (or potential temperature), absolute humidity, and radioactivity are also usually treated as tracers. According to AMS, "The main requirement for a tracer is that its lifetime be substantially longer than the transport process under study." + According to the American Meteorological Society (AMS) definition, a `tracer `_ is "Any substance in the atmosphere that can be used to track the history [i.e., movement] of an air mass." Tracers are carried around by the motion of the atmosphere (i.e., by :term:`advection`). These substances are usually gases (e.g., water vapor, CO2), but they can also be non-gaseous (e.g., rain drops in microphysics parameterizations). In weather models, temperature (or potential temperature), absolute humidity, and radioactivity are also usually treated as tracers. According to AMS, "The main requirement for a tracer is that its lifetime be substantially longer than the transport process under study." UFS The Unified Forecast System is a community-based, coupled, comprehensive Earth modeling system consisting of several applications (apps). These apps span regional to global - domains and sub-hourly to seasonal time scales. The UFS is designed to support the :term:`Weather Enterprise` and to be the source system for NOAA's operational numerical weather prediction applications. For more information, visit https://ufscommunity.org/. + domains and sub-hourly to seasonal time scales. The UFS is designed to support the :term:`Weather Enterprise` and to be the source system for NOAA's operational numerical weather prediction applications. For more information, visit https://ufs.epic.noaa.gov/. UFS_UTILS A collection of code used by multiple :term:`UFS` apps (e.g., the UFS Short-Range Weather App, diff --git a/doc/UsersGuide/SSHIntro.rst b/doc/UsersGuide/SSHIntro.rst index 7292a37e97..25837b9cc0 100644 --- a/doc/UsersGuide/SSHIntro.rst +++ b/doc/UsersGuide/SSHIntro.rst @@ -139,7 +139,7 @@ Download the Data from a Remote System to a Local System .. note:: - Users should transfer data to or from non-:srw-wiki:`Level 1 ` platforms using the recommended approach for that platform. This section outlines some basic guidance, but users may need to supplement with research of their own. On Level 1 systems, users may find it helpful to refer to the `RDHPCS CommonDocs Wiki `__. + Users should transfer data to or from non-:srw-wiki:`Level 1 ` platforms using the recommended approach for that platform. This section outlines some basic guidance, but users may need to supplement with research of their own. On Level 1 systems, users may find it helpful to refer to the `RDHPCS Data Transfer Documentation `_. To download data using ``scp``, users can typically adjust one of the following commands for use on their system: diff --git a/doc/conf.py b/doc/conf.py index 6b0f461ba8..e2a0e92013 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -99,14 +99,16 @@ # Ignore working links that cause a linkcheck 403 error. linkcheck_ignore = [r'https://www\.intel\.com/content/www/us/en/docs/cpp\-compiler/developer\-guide\-reference/2021\-10/thread\-affinity\-interface\.html', r'https://www\.intel\.com/content/www/us/en/developer/tools/oneapi/hpc\-toolkit\-download\.html', - #r'https://glossary.ametsoc.org/.*', + r'https://glossary.ametsoc.org/.*', ] # Ignore anchor tags for SRW App data bucket. Shows Not Found even when they exist. linkcheck_anchors_ignore = [r"current_srw_release_data/", r"input_model_data/.*", r"fix.*", - r"sample_cases/.*", + r"experiment-user-cases/.*", + r"rrfs_a/*", + r"develop-20240618/*", ] linkcheck_allowed_redirects = {r"https://github\.com/ufs-community/ufs-srweather-app/wiki/.*": From 698e0d71ef1075fef3030feafb143e122330e215 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 13:09:12 -0400 Subject: [PATCH 33/43] minor fix --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 -- 1 file changed, 2 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index d7e64d54e8..d3db5e5f77 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -222,8 +222,6 @@ Once the control case is running, users can return to the ``config.yaml`` file ( Later, users may want to conduct additional experiments using the FV3_HRRR and FV3_WoFS_v0 physics suites. Like FV3_RRFS_v1beta, these physics suites were designed for use with high-resolution grids for storm-scale predictions. -.. COMMENT: Maybe also FV3_RAP? - Next, users will need to modify the data parameters in ``task_get_extrn_ics:`` and ``task_get_extrn_lbcs:`` to use HRRR and RAP data rather than FV3GFS data. Users will need to change the following lines in each section: .. code-block:: console From 82fa33d8d2b22e6afe55b0fd0f014af360e2a720 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 14:59:30 -0400 Subject: [PATCH 34/43] minor fixes --- doc/UsersGuide/BackgroundInfo/Components.rst | 10 +++++----- doc/UsersGuide/BuildingRunningTesting/RunSRW.rst | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/doc/UsersGuide/BackgroundInfo/Components.rst b/doc/UsersGuide/BackgroundInfo/Components.rst index c70f6cefb0..6df12dbd23 100644 --- a/doc/UsersGuide/BackgroundInfo/Components.rst +++ b/doc/UsersGuide/BackgroundInfo/Components.rst @@ -38,7 +38,7 @@ Supported model resolutions in this release include 3-, 13-, and 25-km predefine Model Physics --------------- -The Common Community Physics Package (CCPP), described `here `__, supports interoperable atmospheric physics and land surface model options. Atmospheric physics are a set of numerical methods describing small-scale processes such as clouds, turbulence, radiation, and their interactions. The most recent SRW App release (|latestr|) included five supported physics suites: FV3_RRFS_v1beta, FV3_GFS_v16, FV3_WoFS_v0, FV3_HRRR, and FV3_RAP. The FV3_RRFS_v1beta physics suite is being tested for use in the future operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`) planned for 2023-2024, and the FV3_GFS_v16 is an updated version of the physics suite used in the operational Global Forecast System (GFS) v16. A detailed list of CCPP updates since the SRW App v2.1.0 release is available :ref:`here `. A full scientific description of CCPP parameterizations and suites can be found in the `CCPP Scientific Documentation `_, and CCPP technical aspects are described in the :doc:`CCPP Technical Documentation `. The model namelist has many settings beyond the physics options that can optimize various aspects of the model for use with each of the supported suites. Additional information on Stochastic Physics options is available :doc:`here `. +The Common Community Physics Package (CCPP), described `here `_, supports interoperable atmospheric physics and land surface model options. Atmospheric physics are a set of numerical methods describing small-scale processes such as clouds, turbulence, radiation, and their interactions. The most recent SRW App release (|latestr|) included five supported physics suites: FV3_RRFS_v1beta, FV3_GFS_v16, FV3_WoFS_v0, FV3_HRRR, and FV3_RAP. The FV3_RRFS_v1beta physics suite is being tested for use in the future operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`) planned for 2023-2024, and the FV3_GFS_v16 is an updated version of the physics suite used in the operational Global Forecast System (GFS) v16. A detailed list of CCPP updates since the SRW App v2.1.0 release is available :ref:`here `. A full scientific description of CCPP parameterizations and suites can be found in the `CCPP Scientific Documentation `_, and CCPP technical aspects are described in the :doc:`CCPP Technical Documentation `. The model namelist has many settings beyond the physics options that can optimize various aspects of the model for use with each of the supported suites. Additional information on Stochastic Physics options is available :doc:`here `. .. note:: SPP is currently only available for specific physics schemes used in the RAP/HRRR physics suite. Users need to be aware of which physics suite definition file (:term:`SDF`) is chosen when turning this option on. Among the supported physics suites, the full set of parameterizations can only be used with the ``FV3_HRRR`` option for ``CCPP_PHYS_SUITE``. @@ -57,17 +57,17 @@ The Unified Post Processor (:term:`UPP`) processes raw output from a variety of METplus Verification Suite ============================= -The Model Evaluation Tools (MET) package is a set of statistical verification tools developed by the `Developmental Testbed Center `__ (DTC) for use by the :term:`NWP` community to help them assess and evaluate the performance of numerical weather predictions. MET is the core component of the enhanced `METplus `__ verification framework; the suite also includes the associated database and display systems called METviewer and METexpress. +The Model Evaluation Tools (MET) package is a set of statistical verification tools developed by the `Developmental Testbed Center `_ (DTC) for use by the :term:`NWP` community to help them assess and evaluate the performance of numerical weather predictions. MET is the core component of the enhanced `METplus `_ verification framework; the suite also includes the associated database and display systems called METviewer and METexpress. -The METplus verification framework has been integrated into the SRW App to facilitate forecast evaluation. METplus is a verification framework that spans a wide range of temporal scales (warn-on-forecast to climate) and spatial scales (storm to global). It is supported by the `Developmental Testbed Center (DTC) `__. +The METplus verification framework has been integrated into the SRW App to facilitate forecast evaluation. METplus is a verification framework that spans a wide range of temporal scales (warn-on-forecast to climate) and spatial scales (storm to global). It is supported by the `Developmental Testbed Center (DTC) `_. METplus comes preinstalled with :term:`spack-stack` but can also be installed on other systems individually or as part of :term:`HPC-Stack` installation. Users on systems without a previous installation of METplus can follow the :ref:`MET Installation Guide ` and :ref:`METplus Installation Guide ` for individual installation. Currently, METplus *installation* is only supported as part of spack-stack installation; users attempting to install METplus individually or as part of HPC-Stack will need to direct assistance requests to the METplus team. However, METplus *use* is supported on any system with a functioning METplus installation. -The core components of the METplus framework include the statistical driver (MET), the associated database and display systems known as METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use cases. MET is a set of verification tools developed for use by the :term:`NWP` community. It matches up gridded forecast fields with either gridded analyses or point observations and applies configurable methods to compute statistics and diagnostics. Extensive documentation is available in the :doc:`METplus User's Guide ` and :doc:`MET User's Guide `. Documentation for all other components of the framework can be found at the *Documentation* link for each component on the METplus `downloads `__ page. +The core components of the METplus framework include the statistical driver (MET), the associated database and display systems known as METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use cases. MET is a set of verification tools developed for use by the :term:`NWP` community. It matches up gridded forecast fields with either gridded analyses or point observations and applies configurable methods to compute statistics and diagnostics. Extensive documentation is available in the :doc:`METplus User's Guide ` and :doc:`MET User's Guide `. Documentation for all other components of the framework can be found at the *Documentation* link for each component on the METplus `downloads `_ page. Among other techniques, MET provides the capability to compute standard verification scores for comparing deterministic gridded model data to point-based and gridded observations. It also provides ensemble and probabilistic verification methods for comparing gridded model data to point-based or gridded observations. Verification tasks to accomplish these comparisons are defined in the SRW App in :numref:`Table %s `. Currently, the SRW App supports the use of :term:`NDAS` observation files (which include conventional point-based surface and upper-air data) `in prepBUFR format `__ for point-based verification. It also supports gridded Climatology-Calibrated Precipitation Analysis (:term:`CCPA`) data for accumulated precipitation evaluation and Multi-Radar/Multi-Sensor (:term:`MRMS`) gridded analysis data for composite reflectivity and :term:`echo top` verification. -METplus is being actively developed by :term:`NCAR`/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (`ESRL `__), and NOAA/Environmental Modeling Center (:term:`EMC`), and it is open to community contributions. More details about METplus can be found on the `METplus website `__. +METplus is being actively developed by :term:`NCAR`/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (`ESRL `__), and NOAA/Environmental Modeling Center (:term:`EMC`), and it is open to community contributions. More details about METplus can be found on the `METplus website `_. Air Quality Modeling (AQM) Utilities ======================================= diff --git a/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst index 0239c74b01..f3c89679a9 100644 --- a/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst +++ b/doc/UsersGuide/BuildingRunningTesting/RunSRW.rst @@ -685,7 +685,7 @@ More information about configuring the ``rocoto:`` section can be found in :numr If users have access to NOAA :term:`HPSS` but have not pre-staged the data, the default ``verify_pre.yaml`` taskgroup will activate the tasks, and the workflow will attempt to download the appropriate data from NOAA HPSS. In this case, the ``*_OBS_DIR`` paths must be set to the location where users want the downloaded data to reside. -Users who do not have access to NOAA HPSS and do not have the data on their system will need to download :term:`CCPA`, :term:`MRMS`, and :term:`NDAS` data manually from collections of publicly available data, such as the ones listed `here `_. +Users who do not have access to NOAA HPSS and do not have the data on their system will need to download :term:`CCPA`, :term:`MRMS`, and :term:`NDAS` data manually from collections of publicly available data. Users who have already staged the observation data needed for METplus (i.e., the :term:`CCPA`, :term:`MRMS`, and :term:`NDAS` data) on their system should set the path to this data in ``config.yaml``. From b325985603782879b25113e5ab0d18b6ae3d8104 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 16:46:26 -0400 Subject: [PATCH 35/43] minor fixes --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index d3db5e5f77..d50c82d347 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -545,7 +545,7 @@ A line of severe storms brought strong winds, flash flooding, and tornadoes to t Data ------- -Data for the Halloween Storm is publicly available in S3 data buckets. The Rapid Refresh (`RAP `_) data can be downloaded from the `SRW App data bucket `_ using ``wget``: +Data for the Halloween Storm is publicly available in S3 data buckets. The Rapid Refresh (`RAP `_) data can be downloaded from the `SRW App data bucket `_ using ``wget``. Make sure to issue the command from the folder where you want to place the data. .. code-block:: console @@ -656,7 +656,7 @@ In the ``rocoto:tasks:`` section, increase the walltime for the data-related tas .. code-block:: console - rocoto: + rocoto: tasks: task_get_extrn_ics: walltime: 06:00:00 @@ -709,7 +709,7 @@ Users can save the location of the ``halloweenRAP`` directory in an environment .. code-block:: console - export CONTROL=/path/to/expt_dirs/HRAP + export HRAP=/path/to/expt_dirs/halloweenRAP Users should substitute ``/path/to/expt_dirs/HRAP`` with the actual path to the experiment directory on their system. As long as a user remains logged into their system, they can run ``cd $HRAP``, and it will take them to the ``halloweenRAP`` experiment directory. The variable will need to be reset for each login session. From b8ca925cd88c1f6a2e56ca41a1446d3e38730e38 Mon Sep 17 00:00:00 2001 From: gspetro-NOAA Date: Mon, 30 Sep 2024 17:13:28 -0400 Subject: [PATCH 36/43] add info on generating 2nd expt & saving env var --- .../BuildingRunningTesting/Tutorial.rst | 33 +++++++++++++++++-- 1 file changed, 31 insertions(+), 2 deletions(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index d50c82d347..1484fadb74 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -711,7 +711,7 @@ Users can save the location of the ``halloweenRAP`` directory in an environment export HRAP=/path/to/expt_dirs/halloweenRAP -Users should substitute ``/path/to/expt_dirs/HRAP`` with the actual path to the experiment directory on their system. As long as a user remains logged into their system, they can run ``cd $HRAP``, and it will take them to the ``halloweenRAP`` experiment directory. The variable will need to be reset for each login session. +Users should substitute ``/path/to/expt_dirs/halloweenRAP`` with the actual path to the experiment directory on their system. As long as a user remains logged into their system, they can run ``cd $HRAP``, and it will take them to the ``halloweenRAP`` experiment directory. The variable will need to be reset for each login session. Experiment 2: Changing the Forecast Input ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -755,7 +755,36 @@ Update the same values in the ``task_get_extrn_lbcs:`` section: For a detailed description of the ``task_get_extrn_lbcs:`` variables, see :numref:`Section %s `. -Users do not need to modify the ``task_run_fcst:`` section for this tutorial. +After configuring the forecast, users can generate the second forecast by running: + +.. code-block:: console + + ./generate_FV3LAM_wflow.py + +To see experiment progress, users should navigate to their experiment directory. As in the first forecast, the following commands allow users to launch new workflow tasks and check on experiment progress. + +.. code-block:: console + + cd /path/to/expt_dirs/test_expt + rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 + rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 + +.. note:: + + When using cron to automate the workflow submission (as described :ref:`above `), users can omit the ``rocotorun`` command and simply use ``rocotostat`` to check on progress periodically. + +.. note:: + + If users have not automated their workflow using cron, they will need to ensure that they continue issuing ``rocotorun`` commands to launch all of the tasks in each experiment. While switching between experiment directories to run ``rocotorun`` and ``rocotostat`` commands in both directories is possible, it may be easier to finish the ``halloweenRAP`` experiment's tasks before starting on ``halloweenHRRR``. + +As with the ``halloweenRAP`` experiment, users can save the location of the ``halloweenHRRR`` directory in an environment variable (e.g., ``$HHRRR``). This makes it easier to navigate between directories later. For example: + +.. code-block:: console + + export HHRRR=/path/to/expt_dirs/halloweenHRRR + +Users should substitute ``/path/to/expt_dirs/halloweenHRRR`` with the actual path on their system. + How to Analyze Results ----------------------- From 8e7dddd0c9e344358be7ecf16b4abeda7432b33a Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 10 Oct 2024 10:08:44 -0400 Subject: [PATCH 37/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Michael Lueken <63728921+MichaelLueken@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 1484fadb74..fb5d1f1a40 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -825,7 +825,7 @@ In the 250mb wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` plots `````````` The 10m wind plots allows forecasters to pick up on patterns closer to the surface. It shows features such as convergence and pressure areas. -In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once again very similar, which makes sense given that the 250mb wind plots are also so similar. We can see a few key feautres on this chart. The most important is the area of convergence taking place over the east coast which is driving the line of severe storms. +In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once again very similar, which makes sense given that the 250mb wind plots are also so similar. We can see a few key features on this chart. The most important is the area of convergence taking place over the east coast which is driving the line of severe storms. .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/10mwind_rap_conus_f036.png :align: center From ea464c3e90a701c6251b0f59af6807a68ee6d968 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 10 Oct 2024 10:09:17 -0400 Subject: [PATCH 38/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Michael Lueken <63728921+MichaelLueken@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index fb5d1f1a40..177b402167 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -831,7 +831,7 @@ In the 10m wind plots below, the ``halloweenHRRR`` and ``halloweenRAP`` are once :align: center :width: 75% - *Rap Plot for 10m Winds* + *RAP Plot for 10m Winds* .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/fcst4_plots/10mwind_hrrr_conus_f036.png :align: center From edbe55eb8147b0d1b83783db799e46ca5b8a4abd Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 10 Oct 2024 10:09:25 -0400 Subject: [PATCH 39/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 177b402167..17ec8acc18 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -55,7 +55,7 @@ On :srw-wiki:`Level 1 ` systems, users can fi Load the Workflow -------------------- -To load the workflow environment, source the lmod-setup file. Then load the workflow conda environment by running: +To load the workflow environment, source the lmod-setup file and load the workflow conda environment by running: .. include:: ../../doc-snippets/load-env.rst From d0e53c3cacb6fb4db053eecf7d92aa9e5f54a965 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 10 Oct 2024 10:31:16 -0400 Subject: [PATCH 40/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 17ec8acc18..aec3f8444c 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -113,7 +113,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR`` and ``PR .. include:: ../../doc-snippets/cron-note.rst -``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "a;skdfj". However, the best names will indicate useful information about the experiment. This tutorial uses ``control`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different forecasts --- one that uses the FV3_GFS_v16 physics suite and one that uses the FV3_RRFS_v1beta physics suite --- "gfsv16_physics_fcst" could be a good alternative directory name. +``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "gfsv16_physics_fcst" to "forecast1" to "askdfj" (but note that whitespace and some punctuation characters are not allowed). However, the best names will indicate useful information about the experiment. This tutorial uses ``control`` to establish a baseline, or "control", forecast. Since this tutorial helps users to compare the output from two different forecasts --- one that uses the FV3_GFS_v16 physics suite and one that uses the FV3_RRFS_v1beta physics suite --- "gfsv16_physics_fcst" could be a good alternative directory name. ``PREDEF_GRID_NAME:`` This experiment uses the SUBCONUS_Ind_3km grid, rather than the default RRFS_CONUS_25km grid. The SUBCONUS_Ind_3km grid is a high-resolution grid (with grid cell size of approximately 3km) that covers a small area of the U.S. centered over Indianapolis, IN. For more information on this grid, see :numref:`Section %s `. From e7c4ff2fdb12b202b7792598f460edfba6f6f04a Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 10 Oct 2024 10:31:25 -0400 Subject: [PATCH 41/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index aec3f8444c..f5c0cbb2e4 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -615,7 +615,7 @@ In the ``workflow:`` section of ``config.yaml``, update ``EXPT_SUBDIR``, ``CCPP_ .. include:: ../../doc-snippets/cron-note.rst -``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "halloweenRAP" to "HalloweenStorm1" to "a;skdfj". However, the best names will indicate useful information about the experiment. Since this tutorial helps users to compare the output from RAP and HRRR forecast input data, this tutorial will use ``halloweenRAP`` for the Halloween Storm experiment that uses RAP forecast data. +``EXPT_SUBDIR:`` This variable can be changed to any name the user wants from "halloweenRAP" to "HalloweenStorm1" to "askdfj" (but note that whitespace and some punctuation characters are not allowed). However, the best names will indicate useful information about the experiment. Since this tutorial helps users to compare the output from RAP and HRRR forecast input data, this tutorial will use ``halloweenRAP`` for the Halloween Storm experiment that uses RAP forecast data. ``PREDEF_GRID_NAME:`` This experiment uses the RRFS_CONUS_13km, rather than the default RRFS_CONUS_25km grid. This 13-km resolution is used in the NOAA operational Rapid Refresh (`RAP `_) model and is the resolution envisioned for the initial operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`). For more information on this grid, see :numref:`Section %s `. From b270da98f82390e803cefc64e90290fb821099f9 Mon Sep 17 00:00:00 2001 From: jdkublnick <47824899+jdkublnick@users.noreply.github.com> Date: Thu, 10 Oct 2024 15:25:34 -0400 Subject: [PATCH 42/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index f5c0cbb2e4..872c23eacb 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -559,7 +559,7 @@ The SRW App can pull HRRR data directly from the `HRRR data bucket Date: Thu, 10 Oct 2024 15:25:44 -0400 Subject: [PATCH 43/43] Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst Co-authored-by: Gillian Petro <96886803+gspetro-NOAA@users.noreply.github.com> --- doc/UsersGuide/BuildingRunningTesting/Tutorial.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst index 872c23eacb..2b7f169711 100644 --- a/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst +++ b/doc/UsersGuide/BuildingRunningTesting/Tutorial.rst @@ -765,7 +765,7 @@ To see experiment progress, users should navigate to their experiment directory. .. code-block:: console - cd /path/to/expt_dirs/test_expt + cd /path/to/expt_dirs/halloweenHRRR rocotorun -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10