Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[develop]: Adding in the tutorial for the Halloween Storm #1124

Merged
merged 63 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from 61 commits
Commits
Show all changes
63 commits
Select commit Hold shift + click to select a range
fc15ea5
include info on NA_13km grid
gspetro-NOAA Oct 12, 2023
61d55cf
automate substitutions for name of workflow env
gspetro-NOAA Oct 12, 2023
91a0c0b
updates to intro
gspetro-NOAA Oct 12, 2023
40291c4
Merge branch 'develop' of github.com:gspetro-NOAA/ufs-srweather-app i…
gspetro-NOAA Oct 13, 2023
bc09e91
Merge branch 'develop' of github.com:gspetro-NOAA/ufs-srweather-app i…
gspetro-NOAA Oct 27, 2023
3c4e161
update develop branch
gspetro-NOAA Jan 30, 2024
a0d1e67
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Feb 13, 2024
6851ed6
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Feb 16, 2024
de99fa7
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Feb 23, 2024
85cbead
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA May 22, 2024
6d55f0c
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Jun 5, 2024
5accded
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Jul 12, 2024
3b8f246
Tutorial information for Halloween storm
jdkublnick Jul 31, 2024
88baa05
Changed CCCP_PHYS_SUITE info
jdkublnick Jul 31, 2024
6cf8992
Adding in Halloween storm Tutorial
jdkublnick Jul 31, 2024
e7d7413
Merge branch 'text/us-216' of https://github.com/jdkublnick/ufs-srwea…
jdkublnick Jul 31, 2024
9ffb2e9
Adding in Halloween Tutorial
jdkublnick Jul 31, 2024
86a463a
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Aug 9, 2024
3edd708
Changes to config
jdkublnick Aug 15, 2024
2528bba
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Aug 27, 2024
a95b9a2
Added section about wget
jdkublnick Aug 28, 2024
7dc7fed
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Sep 9, 2024
d538dae
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Sep 12, 2024
ac8f867
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Sep 12, 2024
336ab82
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Sep 12, 2024
754f95d
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Sep 12, 2024
ebb2cb5
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Sep 12, 2024
c2f31bf
Merge branch 'ufs-community:develop' into text/us-216
jdkublnick Sep 12, 2024
1524087
Updating config on halloween tutorial
jdkublnick Sep 12, 2024
d554562
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Sep 17, 2024
ab31032
Adding HRRR forecast input option
jdkublnick Sep 23, 2024
301bfcd
Minor changes and adding plotting explanationss
Sep 24, 2024
36ac4c1
Finishing up plotting explanations
Sep 25, 2024
fe70d0e
Minor formatting corrections
Sep 26, 2024
a5401c7
More minor changes to the text
Sep 26, 2024
6956802
Merge branch 'ufs-community:develop' into text/us-216
jdkublnick Sep 26, 2024
a73af0a
Merge branch 'ufs-community:develop' into develop
gspetro-NOAA Sep 26, 2024
acee59c
add prelim env load snippet to be reused throughout docs
gspetro-NOAA Sep 26, 2024
cb0dcd6
Merge branch 'text/us-216' of https://github.com/jdkublnick/ufs-srwea…
gspetro-NOAA Sep 27, 2024
6abdc76
factor out module use/load cmds; use include dirctive to add
gspetro-NOAA Sep 27, 2024
1d1c47a
add code snippets to tutorial
gspetro-NOAA Sep 27, 2024
38fd16f
update snippet & VX ch
gspetro-NOAA Sep 27, 2024
1b7cffc
fix typo
gspetro-NOAA Sep 27, 2024
f41e41e
replace PNG w/GIF
gspetro-NOAA Sep 27, 2024
48cd3e6
factor out conf intro section for reuse
gspetro-NOAA Sep 27, 2024
6fd0349
factor out file editing hint
gspetro-NOAA Sep 27, 2024
72225c7
update data & plotting info
gspetro-NOAA Sep 28, 2024
9d06f55
refactor scp instructions
gspetro-NOAA Sep 30, 2024
52029c0
misc typos
gspetro-NOAA Sep 30, 2024
9786a91
update links
gspetro-NOAA Sep 30, 2024
698e0d7
minor fix
gspetro-NOAA Sep 30, 2024
82fa33d
minor fixes
gspetro-NOAA Sep 30, 2024
b325985
minor fixes
gspetro-NOAA Sep 30, 2024
b8ca925
add info on generating 2nd expt & saving env var
gspetro-NOAA Sep 30, 2024
52a09ad
Merge pull request #1 from gspetro-NOAA/text/doc-snippets
jdkublnick Oct 1, 2024
8e7dddd
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
ea464c3
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
edbe55e
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
d0e53c3
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
e7c4ff2
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
5aa3582
Merge branch 'develop' into text/us-216
jdkublnick Oct 10, 2024
b270da9
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
57c2a49
Update doc/UsersGuide/BuildingRunningTesting/Tutorial.rst
jdkublnick Oct 10, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/ContribGuide/contributing.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Fork and PR Overview

Contributions to the ``ufs-srweather-app`` project are made via a :github-docs:`Fork<pull-requests/collaborating-with-pull-requests/working-with-forks/about-forks>` and :github-docs:`Pull Request (PR)<pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/about-pull-requests>` model. GitHub provides a thorough description of this contribution model in their `Contributing to a project` :github-docs:`Quickstart<get-started/exploring-projects-on-github/contributing-to-a-project>`, but the steps, with respect to ``ufs-srweather-app`` contributions, can be summarized as:

#. :github-docs:`Create an issue <issues/tracking-your-work-with-issues/creating-an-issue>` to document proposed changes.
#. :github-docs:`Create an issue <issues/tracking-your-work-with-issues/using-issues/creating-an-issue>` to document proposed changes.
#. :github-docs:`Fork<get-started/exploring-projects-on-github/contributing-to-a-project#forking-a-repository>` the :srw-repo:`ufs-srweather-app repository<>` into your personal GitHub account.
#. :github-docs:`Clone<get-started/exploring-projects-on-github/contributing-to-a-project>` your fork onto your development system.
#. :github-docs:`Create a branch<get-started/exploring-projects-on-github/contributing-to-a-project#creating-a-branch-to-work-on>` in your clone for your changes. All development should take place on a branch, *not* on ``develop``.
Expand Down
12 changes: 6 additions & 6 deletions doc/UsersGuide/BackgroundInfo/Components.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,12 +38,12 @@ Supported model resolutions in this release include 3-, 13-, and 25-km predefine
Model Physics
---------------

The Common Community Physics Package (CCPP), described `here <https://dtcenter.org/community-code/common-community-physics-package-ccpp>`__, supports interoperable atmospheric physics and land surface model options. Atmospheric physics are a set of numerical methods describing small-scale processes such as clouds, turbulence, radiation, and their interactions. The most recent SRW App release (|latestr|) included five supported physics suites: FV3_RRFS_v1beta, FV3_GFS_v16, FV3_WoFS_v0, FV3_HRRR, and FV3_RAP. The FV3_RRFS_v1beta physics suite is being tested for use in the future operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`) planned for 2023-2024, and the FV3_GFS_v16 is an updated version of the physics suite used in the operational Global Forecast System (GFS) v16. A detailed list of CCPP updates since the SRW App v2.1.0 release is available :ref:`here <CCPPUpdates>`. A full scientific description of CCPP parameterizations and suites can be found in the `CCPP Scientific Documentation <https://dtcenter.ucar.edu/GMTB/UFS_SRW_App_v2.2.0/sci_doc/index.html>`__, and CCPP technical aspects are described in the :doc:`CCPP Technical Documentation <ccpp-techdoc:index>`. The model namelist has many settings beyond the physics options that can optimize various aspects of the model for use with each of the supported suites. Additional information on Stochastic Physics options is available :doc:`here <stochphys:index>`.
The Common Community Physics Package (CCPP), described `here <https://dtcenter.org/community-code/common-community-physics-package-ccpp>`_, supports interoperable atmospheric physics and land surface model options. Atmospheric physics are a set of numerical methods describing small-scale processes such as clouds, turbulence, radiation, and their interactions. The most recent SRW App release (|latestr|) included five supported physics suites: FV3_RRFS_v1beta, FV3_GFS_v16, FV3_WoFS_v0, FV3_HRRR, and FV3_RAP. The FV3_RRFS_v1beta physics suite is being tested for use in the future operational implementation of the Rapid Refresh Forecast System (:term:`RRFS`) planned for 2023-2024, and the FV3_GFS_v16 is an updated version of the physics suite used in the operational Global Forecast System (GFS) v16. A detailed list of CCPP updates since the SRW App v2.1.0 release is available :ref:`here <CCPPUpdates>`. A full scientific description of CCPP parameterizations and suites can be found in the `CCPP Scientific Documentation <https://dtcenter.ucar.edu/GMTB/UFS_SRW_App_v2.2.0/sci_doc/index.html>`_, and CCPP technical aspects are described in the :doc:`CCPP Technical Documentation <ccpp-techdoc:index>`. The model namelist has many settings beyond the physics options that can optimize various aspects of the model for use with each of the supported suites. Additional information on Stochastic Physics options is available :doc:`here <stochphys:index>`.

.. note::
SPP is currently only available for specific physics schemes used in the RAP/HRRR physics suite. Users need to be aware of which physics suite definition file (:term:`SDF`) is chosen when turning this option on. Among the supported physics suites, the full set of parameterizations can only be used with the ``FV3_HRRR`` option for ``CCPP_PHYS_SUITE``.

Additionally, a CCPP single-column model (`CCPP-SCM <https://github.com/NCAR/ccpp-scm>`__) option has also been developed as a child repository. Users can refer to the `CCPP Single Column Model User and Technical Guide <https://github.com/NCAR/ccpp-scm/blob/main/scm/doc/TechGuide/main.pdf>`__ for more details. This CCPP-SCM user guide contains a Quick Start Guide with instructions for obtaining the code, compiling, and running test cases, which include five standard test cases and two additional FV3 replay cases (refer to section 5.2 in the CCPP-SCM user guide for more details). Moreover, the CCPP-SCM supports a precompiled version in a docker container, allowing it to be easily executed on NOAA's cloud computing platforms without any issues (see section 2.5 in the CCPP-SCM user guide for more details).
Additionally, a CCPP single-column model (`CCPP-SCM <https://github.com/NCAR/ccpp-scm>`_) option has also been developed as a child repository. Users can refer to the `CCPP Single Column Model User and Technical Guide <https://ccpp-scm.readthedocs.io/en/latest/>`_ for more details. This CCPP-SCM user guide contains a Quick Start Guide with instructions for obtaining the code, compiling, and running test cases, which include five standard test cases and two additional FV3 replay cases (refer to section 5.2 in the CCPP-SCM user guide for more details). Moreover, the CCPP-SCM supports a precompiled version in a docker container, allowing it to be easily executed on NOAA's cloud computing platforms without any issues (see section 2.5 in the CCPP-SCM user guide for more details).

The SRW App supports the use of both :term:`GRIB2` and :term:`NEMSIO` input data. The UFS Weather Model ingests initial and lateral boundary condition files produced by :term:`chgres_cube` and outputs files in netCDF format on a specific projection (e.g., Lambert Conformal) in the horizontal direction and model levels in the vertical direction.

Expand All @@ -57,17 +57,17 @@ The Unified Post Processor (:term:`UPP`) processes raw output from a variety of
METplus Verification Suite
=============================

The Model Evaluation Tools (MET) package is a set of statistical verification tools developed by the `Developmental Testbed Center <https://dtcenter.org/>`__ (DTC) for use by the :term:`NWP` community to help them assess and evaluate the performance of numerical weather predictions. MET is the core component of the enhanced `METplus <https://dtcenter.org/community-code/metplus>`__ verification framework; the suite also includes the associated database and display systems called METviewer and METexpress.
The Model Evaluation Tools (MET) package is a set of statistical verification tools developed by the `Developmental Testbed Center <https://dtcenter.org/>`_ (DTC) for use by the :term:`NWP` community to help them assess and evaluate the performance of numerical weather predictions. MET is the core component of the enhanced `METplus <https://dtcenter.org/community-code/metplus>`_ verification framework; the suite also includes the associated database and display systems called METviewer and METexpress.

The METplus verification framework has been integrated into the SRW App to facilitate forecast evaluation. METplus is a verification framework that spans a wide range of temporal scales (warn-on-forecast to climate) and spatial scales (storm to global). It is supported by the `Developmental Testbed Center (DTC) <https://dtcenter.org/>`__.
The METplus verification framework has been integrated into the SRW App to facilitate forecast evaluation. METplus is a verification framework that spans a wide range of temporal scales (warn-on-forecast to climate) and spatial scales (storm to global). It is supported by the `Developmental Testbed Center (DTC) <https://dtcenter.org/>`_.

METplus comes preinstalled with :term:`spack-stack` but can also be installed on other systems individually or as part of :term:`HPC-Stack` installation. Users on systems without a previous installation of METplus can follow the :ref:`MET Installation Guide <met:installation>` and :ref:`METplus Installation Guide <metplus:install>` for individual installation. Currently, METplus *installation* is only supported as part of spack-stack installation; users attempting to install METplus individually or as part of HPC-Stack will need to direct assistance requests to the METplus team. However, METplus *use* is supported on any system with a functioning METplus installation.

The core components of the METplus framework include the statistical driver (MET), the associated database and display systems known as METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use cases. MET is a set of verification tools developed for use by the :term:`NWP` community. It matches up gridded forecast fields with either gridded analyses or point observations and applies configurable methods to compute statistics and diagnostics. Extensive documentation is available in the :doc:`METplus User's Guide <metplus:index>` and :doc:`MET User's Guide <met:index>`. Documentation for all other components of the framework can be found at the *Documentation* link for each component on the METplus `downloads <https://dtcenter.org/community-code/metplus/download>`__ page.
The core components of the METplus framework include the statistical driver (MET), the associated database and display systems known as METviewer and METexpress, and a suite of Python wrappers to provide low-level automation and examples, also called use cases. MET is a set of verification tools developed for use by the :term:`NWP` community. It matches up gridded forecast fields with either gridded analyses or point observations and applies configurable methods to compute statistics and diagnostics. Extensive documentation is available in the :doc:`METplus User's Guide <metplus:index>` and :doc:`MET User's Guide <met:index>`. Documentation for all other components of the framework can be found at the *Documentation* link for each component on the METplus `downloads <https://dtcenter.org/community-code/metplus/download>`_ page.

Among other techniques, MET provides the capability to compute standard verification scores for comparing deterministic gridded model data to point-based and gridded observations. It also provides ensemble and probabilistic verification methods for comparing gridded model data to point-based or gridded observations. Verification tasks to accomplish these comparisons are defined in the SRW App in :numref:`Table %s <VXWorkflowTasksTable>`. Currently, the SRW App supports the use of :term:`NDAS` observation files (which include conventional point-based surface and upper-air data) `in prepBUFR format <https://nomads.ncep.noaa.gov/pub/data/nccf/com/nam/prod/>`__ for point-based verification. It also supports gridded Climatology-Calibrated Precipitation Analysis (:term:`CCPA`) data for accumulated precipitation evaluation and Multi-Radar/Multi-Sensor (:term:`MRMS`) gridded analysis data for composite reflectivity and :term:`echo top` verification.

METplus is being actively developed by :term:`NCAR`/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (`ESRL <https://www.esrl.noaa.gov/>`__), and NOAA/Environmental Modeling Center (:term:`EMC`), and it is open to community contributions. More details about METplus can be found on the `METplus website <https://dtcenter.org/community-code/metplus>`__.
METplus is being actively developed by :term:`NCAR`/Research Applications Laboratory (RAL), NOAA/Earth Systems Research Laboratories (`ESRL <https://www.esrl.noaa.gov/>`__), and NOAA/Environmental Modeling Center (:term:`EMC`), and it is open to community contributions. More details about METplus can be found on the `METplus website <https://dtcenter.org/community-code/metplus>`_.

Air Quality Modeling (AQM) Utilities
=======================================
Expand Down
4 changes: 2 additions & 2 deletions doc/UsersGuide/BackgroundInfo/Introduction.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
Introduction
==============

The Unified Forecast System (:term:`UFS`) is a community-based, coupled, comprehensive Earth modeling system. NOAA's operational model suite for numerical weather prediction (:term:`NWP`) is quickly transitioning to the UFS from a number of different modeling systems. The UFS enables research, development, and contribution opportunities within the broader :term:`Weather Enterprise` (including government, industry, and academia). For more information about the UFS, visit the `UFS Portal <https://ufscommunity.org/>`__.
The Unified Forecast System (:term:`UFS`) is a community-based, coupled, comprehensive Earth modeling system. NOAA's operational model suite for numerical weather prediction (:term:`NWP`) is quickly transitioning to the UFS from a number of different modeling systems. The UFS enables research, development, and contribution opportunities within the broader :term:`Weather Enterprise` (including government, industry, and academia). For more information about the UFS, visit the `UFS Portal <https://ufs.epic.noaa.gov/>`_.

The UFS includes `multiple applications <https://ufscommunity.org/science/aboutapps/>`__ that support different forecast durations and spatial domains. This documentation describes the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The most recent SRW Application includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end. These components are documented within this User's Guide and supported through the `GitHub Discussions <https://github.com/ufs-community/ufs-srweather-app/discussions/categories/q-a>`__ forum. The SRW App also includes support for a verification package (METplus) for both deterministic and ensemble simulations and support for four stochastically perturbed physics schemes.
The UFS includes `multiple applications <https://ufs.epic.noaa.gov/applications/>`_ that support different forecast durations and spatial domains. This documentation describes the UFS Short-Range Weather (SRW) Application, which targets predictions of atmospheric behavior on a limited spatial domain and on time scales from minutes to several days. The most recent SRW Application includes a prognostic atmospheric model, pre- and post-processing, and a community workflow for running the system end-to-end. These components are documented within this User's Guide and supported through the `GitHub Discussions <https://github.com/ufs-community/ufs-srweather-app/discussions/categories/q-a>`_ forum. The SRW App also includes support for a verification package (METplus) for both deterministic and ensemble simulations and support for four stochastically perturbed physics schemes.

Since the last release, developers have added a variety of features:

Expand Down
2 changes: 1 addition & 1 deletion doc/UsersGuide/BuildingRunningTesting/BuildSRW.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ Install the Prerequisite Software Stack

Users on any sufficiently up-to-date machine with a UNIX-based operating system should be able to install the prerequisite software stack and run the SRW Application. However, a list of prerequisites is available in :numref:`Section %s <software-prereqs>` for reference. Users should install or update their system as required before attempting to install the software stack.

Currently, installation of the prerequisite software stack is supported via spack-stack on most systems. :term:`Spack-stack` is a :term:`repository` that provides a Spack-based system to build the software stack required for `UFS <https://ufscommunity.org/>`__ applications such as the SRW App. Spack-stack is the software stack validated by the UFS Weather Model (:term:`WM`), and the SRW App has likewise shifted to spack-stack for most Level 1 systems.
Currently, installation of the prerequisite software stack is supported via spack-stack on most systems. :term:`Spack-stack` is a :term:`repository` that provides a Spack-based system to build the software stack required for `UFS <https://ufs.epic.noaa.gov/>`_ applications such as the SRW App. Spack-stack is the software stack validated by the UFS Weather Model (:term:`WM`), and the SRW App has likewise shifted to spack-stack for most Level 1 systems.

.. hint::
Skip the spack-stack installation if working on a :srw-wiki:`Level 1 system <Supported-Platforms-and-Compilers>` (e.g., Hera, Jet, Derecho, NOAA Cloud), and :ref:`continue to the next section <DownloadSRWApp>`.
Expand Down
10 changes: 3 additions & 7 deletions doc/UsersGuide/BuildingRunningTesting/Quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,9 @@ For a detailed explanation of how to build and run the SRW App on any supported

#. Load the python environment for the workflow. Users on Level 2-4 systems will need to use one of the existing ``wflow_<platform>`` modulefiles (e.g., ``wflow_macos``) and adapt it to their system. Then, run:

.. code-block:: console
source /path/to/ufs-srweather-app/etc/lmod-setup.sh <platform>
module use /path/to/ufs-srweather-app/modulefiles
module load wflow_<platform>
where ``<platform>`` refers to a valid machine name (see :numref:`Section %s <user>`). After loading the workflow, users should follow the instructions printed to the console. For example, if the output says:
.. include:: ../../doc-snippets/load-env.rst

After loading the workflow, users should follow the instructions printed to the console. For example, if the output says:

.. code-block:: console
Expand Down
Loading
Loading