Skip to content

Commit

Permalink
Updates to RTD documentation (#2418)
Browse files Browse the repository at this point in the history
Updates to the RTD documentation include:

- Textual updates
- Better definition of GDA subfolder structure
- Added note about GDAur having been discontinued
- Adjust copyright and author information
- Fix Git version table and update its contents
- Add "Table of Contents" header before table of contents on front page
- Add AWS ICs path to init page
- Add link to UFS_UTILS gdas_init RTD documentation
- Add note about bash to `gw_setup.sh` section and added a warning block

Refs #2395
  • Loading branch information
KateFriedman-NOAA authored Mar 20, 2024
1 parent afe874e commit 4d1bf52
Show file tree
Hide file tree
Showing 6 changed files with 39 additions and 26 deletions.
18 changes: 10 additions & 8 deletions docs/source/components.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ Components included as submodules:
* **GSI Monitor** (https://github.com/NOAA-EMC/GSI-Monitor): These tools monitor the GSI package's data assimilation, detecting and reporting missing data sources, low observation counts, and high penalty values
* **GDAS** (https://github.com/NOAA-EMC/GDASApp): Jedi based Data Assimilation system. This system is currently being developed for marine Data Assimilation and in time will replace GSI for atmospheric data assimilation as well
* **UFS UTILS** (https://github.com/ufs-community/UFS_UTILS): Utility codes needed for UFS-weather-model
* **wxflow** Collection of python utilities for weather workflows (https://github.com/NOAA-EMC/wxflow)
* **wxflow** (https://github.com/NOAA-EMC/wxflow): Collection of python utilities for weather workflows
* **Verif global** (https://github.com/NOAA-EMC/EMC_verif-global): Verification package to evaluate GFS parallels. It uses MET and METplus. At this moment the verification package is limited to providing atmospheric metrics only

.. note::
Expand Down Expand Up @@ -57,19 +57,20 @@ Data

Observation data, also known as dump data, is prepared in production and then archived in a global dump archive (GDA) for use by users when running cycled experiments. The GDA (identified as ``$DMPDIR`` in the workflow) is available on supported platforms and the workflow system knows where to find the data.

* Hera: /scratch1/NCEPDEV/global/glopara/dump
* Orion/Hercules: /work/noaa/rstprod/dump
* Jet: /mnt/lfs4/HFIP/hfv3gfs/glopara/dump
* WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/dump
* S4: /data/prod/glopara/dump
* Hera: ``/scratch1/NCEPDEV/global/glopara/dump``
* Orion/Hercules: ``/work/noaa/rstprod/dump``
* Jet: ``/mnt/lfs4/HFIP/hfv3gfs/glopara/dump``
* WCOSS2: ``/lfs/h2/emc/global/noscrub/emc.global/dump``
* S4: ``/data/prod/glopara/dump``

-----------------------------
Global Dump Archive Structure
-----------------------------

The global dump archive (GDA) mimics the structure of its production source: ``DMPDIR/CDUMP.PDY/[CC/atmos/]FILES``
The global dump archive (GDA) mimics the structure of its production source:

The ``CDUMP`` is either gdas, gfs, or rtofs. All three contain production output for each day (``PDY``). The gdas and gfs folders are further broken into cycle (``CC``) and component (``atmos``).
* GDAS/GFS: ``DMPDIR/gdas[gfs].PDY/CC/atmos/FILES``
* RTOFS: ``DMPDIR/rtofs.PDY/FILES``

The GDA also contains special versions of some datasets and experimental data that is being evaluated ahead of implementation into production. The following subfolder suffixes exist:

Expand All @@ -81,6 +82,7 @@ The GDA also contains special versions of some datasets and experimental data th
+--------+------------------------------------------------------------------------------------------------------+
| ur | Un-restricted versions of restricted files in production. Produced and archived on a 48hrs delay. |
| | Some restricted datasets are unrestricted. Data amounts: restricted > un-restricted > non-restricted |
| | Limited availability. Discontinued producing mid-2023. |
+--------+------------------------------------------------------------------------------------------------------+
| x | Experimental global datasets being evaluated for production. Dates and types vary depending on |
| | upcoming global upgrades. |
Expand Down
7 changes: 4 additions & 3 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,14 @@
import os
import sys
sys.path.insert(0, os.path.abspath('.'))

from datetime import datetime

# -- Project information -----------------------------------------------------

project = 'Global-workflow'
copyright = '2023, Kate Friedman, Walter Kolczynski, Rahul Mahajan, Lin Gan, Arun Chawla'
author = 'Kate Friedman, Walter Kolczynski, Rahul Mahajan, Lin Gan, Arun Chawla'
year = datetime.now().year
copyright = f"2015-{year} NOAA/NWS/NCEP/EMC"
author = 'Kate Friedman, Walter Kolczynski, Rahul Mahajan, Lin Gan, and numerous collaborators and contributors'

# The full version, including alpha/beta/rc tags
release = '0.1'
Expand Down
22 changes: 11 additions & 11 deletions docs/source/hpc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -72,23 +72,23 @@ Version

It is advised to use Git v2+ when available. At the time of writing this documentation the default Git clients on the different machines were as noted in the table below. It is recommended that you check the default modules before loading recommended ones:

+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| Machine | Default | Recommended |
+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| Hera | v2.18.0 | default |
+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| Hercules | v2.31.1 | default |
+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| Orion | v1.8.3.1 | **module load git/2.28.0** |
+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| Jet | v2.18.0 | default |
+---------+----------+---------------------------------------+
| WCOSS2 | v2.26.2 | default or **module load git/2.29.0** |
+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| WCOSS2 | v2.35.3 | default |
+----------+----------+---------------------------------------+
| S4 | v1.8.3.1 | **module load git/2.30.0** |
+---------+----------+---------------------------------------+
| AWS PW | v1.8.3.1 | default
+---------+----------+---------------------------------------+
+----------+----------+---------------------------------------+
| AWS PW | v1.8.3.1 | default |
+----------+----------+---------------------------------------+

^^^^^^^^^^^^^
Output format
Expand Down
4 changes: 4 additions & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,10 @@ General updates: NOAA employees and affiliates can join the gfs-announce distrib

GitHub updates: Users should adjust their "Watch" settings for this repo so they receive notifications as they'd like to. Find the "Watch" or "Unwatch" button towards the top right of the `authoritative global-workflow repository page <https://github.com/NOAA-EMC/global-workflow>`_ and click it to adjust how you watch the repo.

=================
Table of Contents
=================

.. toctree::
:numbered:
:maxdepth: 3
Expand Down
8 changes: 5 additions & 3 deletions docs/source/init.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,7 @@ Cold-start atmosphere-only cycled C96 deterministic C48 enkf (80 members) ICs ar
Hera: /scratch1/NCEPDEV/global/glopara/data/ICSDIR/C96C48
Orion/Hercules: /work/noaa/global/glopara/data/ICSDIR/C96C48
WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR/C96C48
AWS: https://noaa-nws-global-pds.s3.amazonaws.com/index.html#data/ICSDIR/C96C48

Start date = 2021122018

Expand Down Expand Up @@ -111,6 +112,7 @@ Warm-start cycled w/ coupled (S2S) model C48 atmosphere C48 enkf (80 members) 5
Orion/Hercules: /work/noaa/global/glopara/data/ICSDIR/C48C48mx500
WCOSS2: /lfs/h2/emc/global/noscrub/emc.global/data/ICSDIR/C48C48mx500
Jet: /lfs4/HFIP/hfv3gfs/glopara/data/ICSDIR/C48C48mx500
AWS: https://noaa-nws-global-pds.s3.amazonaws.com/index.html#data/ICSDIR/C48C48mx500

Start date = 2021032312

Expand Down Expand Up @@ -246,7 +248,7 @@ Automated Generation
Cycled mode
-----------

Not yet supported.
Not yet supported. See the UFS_UTILS documentation on the gdas_init utility to generate your own ICs for cycled or forecast-only mode: https://noaa-emcufs-utils.readthedocs.io/en/latest/ufs_utils.html#gdas-init

.. _forecastonly-coupled:

Expand Down Expand Up @@ -319,14 +321,14 @@ Manual Generation

The following information is for users needing to generate cold-start initial conditions for a cycled experiment that will run at a different resolution or layer amount than the operational GFS (C768C384L127).

The ``chgres_cube`` code is available from the `UFS_UTILS repository <https://github.com/ufs-community/UFS_UTILS>`_ on GitHub and can be used to convert GFS ICs to a different resolution or number of layers. Users should see the documentation to generation initial conditions in the UFS_UTILS repository. The ``chgres_cube`` code/scripts currently support the following GFS inputs:
The ``chgres_cube`` code is available from the `UFS_UTILS repository <https://github.com/ufs-community/UFS_UTILS>`_ on GitHub and can be used to convert GFS ICs to a different resolution or number of layers. Users should see the `documentation to generation initial conditions in the UFS_UTILS repository <https://noaa-emcufs-utils.readthedocs.io/en/latest/ufs_utils.html#gdas-init>`_. The ``chgres_cube`` code/scripts currently support the following GFS inputs:

* pre-GFSv14
* GFSv14
* GFSv15
* GFSv16

See instructions in UFS_UTILS to clone, build and generate initial conditions.
See instructions in UFS_UTILS to clone, build and generate initial conditions: https://noaa-emcufs-utils.readthedocs.io/en/latest/ufs_utils.html#gdas-init

.. _warmstarts-prod:

Expand Down
6 changes: 5 additions & 1 deletion docs/source/setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,13 @@ Experiment Setup

::

# Note: this will wipe your existing lmod environment
source workflow/gw_setup.sh

.. warning::
Sourcing gw_setup.sh will wipe your existing lmod environment

.. note::
Bash shell is required to source gw_setup.sh

^^^^^^^^^^^^^^^^^^^^^^^^
Forecast-only experiment
Expand Down

0 comments on commit 4d1bf52

Please sign in to comment.