Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[develop] Add AQM to SRW user's guide #846

Merged
merged 10 commits into from
Jun 28, 2023
Merged

[develop] Add AQM to SRW user's guide #846

Changes from 9 commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
29a572a
add aqm to users guide
chan-hoo Jun 26, 2023
0b6a056
fix typo
chan-hoo Jun 26, 2023
98b9f83
fix typo and remove dead link
chan-hoo Jun 27, 2023
33ff779
update conda env command
chan-hoo Jun 27, 2023
f28598e
missing update
chan-hoo Jun 27, 2023
b84f943
add line between table and caption
chan-hoo Jun 27, 2023
f7c9835
change to workflow_tools
chan-hoo Jun 27, 2023
592d86a
remove unnecessary note
chan-hoo Jun 28, 2023
2611bbc
move png to wiki
chan-hoo Jun 28, 2023
598fd59
adjust aligning
chan-hoo Jun 28, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
175 changes: 131 additions & 44 deletions docs/UsersGuide/source/AQM.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _AQM:

=====================================
Air Quality Modeling (Online-CMAQ)
Air Quality Modeling (SRW-AQM)
=====================================

The standard SRW App distribution uses the uncoupled version of the UFS Weather Model (atmosphere-only). However, users have the option to use a coupled version of the SRW App that includes the standard distribution (atmospheric model) plus the Air Quality Model (AQM).
Expand All @@ -16,7 +16,7 @@ The AQM is a UFS Application that dynamically couples the Community Multiscale A

These instructions should work smoothly on Hera and WCOSS2, but users on other systems may need to make additional adjustments.

Quick Start Guide (AQM/Online-CMAQ)
Quick Start Guide (SRW-AQM)
=====================================

Download the Code
Expand All @@ -29,44 +29,10 @@ Clone the ``develop`` branch of the authoritative SRW App repository:
git clone -b develop https://github.com/ufs-community/ufs-srweather-app
cd ufs-srweather-app

Note that the latest hash of the ``develop`` branch might not be tested with the sample scripts of for AQM. To check out the stable (verified) version for AQM/online-CMAQ, users can check out hash ``#ff6f103``:

.. code-block:: console

git checkout ff6f103

This hash will then check out the following hashes (as of 03/08/2023) of the external components, which are specified in ``ufs-srweather-app/Externals.cfg``:

.. _ExternalsAQM:

.. table:: Externals for Online-CMAQ

+--------------------+--------------+
| Component | Hash |
+====================+==============+
| UFS_UTILS | ca9bed8 |
+--------------------+--------------+
| ufs-weather-model | e051e0e |
+--------------------+--------------+
| UPP | 2b2c84a |
+--------------------+--------------+
| NEXUS | 3842818 |
+--------------------+--------------+
| AQM-utils | e078c70 |
+--------------------+--------------+

Users may replace the hashes above with different ones if they prefer. For example, users can comment out the hash line and uncomment the branch line with a new ``repo_url`` address to use a different branch for development. In the example below, the ``ufs-weather-model`` repository URL has been changed to check out code from a user's personal fork rather than from the authoritative UFS repository.

.. code-block:: console

repo_url = https://github.com/chan-hoo/ufs-weather-model
branch = feature/for_example
#hash = ff6f103

Checkout Externals
---------------------

MichaelLueken marked this conversation as resolved.
Show resolved Hide resolved
Along with the components detailed in :numref:`Chapter %s <Components>`, the AQM version of the SRW App pulls in the externals listed in :numref:`Table %s <ExternalsAQM>`. Users must run the ``checkout_externals`` script to collect (or "check out") the individual components of the SRW App (AQM version) from their respective GitHub repositories.
Users must run the ``checkout_externals`` script to collect (or "check out") the individual components of the SRW App (AQM version) from their respective GitHub repositories.

.. code-block:: console

Expand All @@ -85,7 +51,7 @@ where ``<machine>`` is ``hera``, or ``wcoss2``. The ``-a`` argument indicates th

Building the SRW App with AQM on other machines, including other `Level 1 <https://github.com/ufs-community/ufs-srweather-app/wiki/Supported-Platforms-and-Compilers>`__ platforms, is not currently guaranteed to work, and users may have to make adjustments to the modulefiles for their system.

Load the ``regional_workflow`` Environment
Load the ``workflow_tools`` Environment
--------------------------------------------

Load the python environment for the workflow:
Expand All @@ -105,9 +71,9 @@ If the console outputs a message, the user should run the commands specified in
.. code-block:: console
MichaelLueken marked this conversation as resolved.
Show resolved Hide resolved

Please do the following to activate conda:
> conda activate regional_workflow
> conda activate workflow_tools

then the user should run ``conda activate regional_workflow``. Otherwise, the user can continue with configuring the workflow.
then the user should run ``conda activate workflow_tools``. Otherwise, the user can continue with configuring the workflow.

.. _AQMConfig:

Expand All @@ -123,10 +89,6 @@ Users will need to configure their experiment by setting parameters in the ``con

Users may prefer to copy the ``config.aqm.nco.realtime.yaml`` for a default "nco" mode experiment instead.

.. note::

Additional sample configuration files can be found in the ``online-cmaq`` branch of Chan-Hoo Jeon's (NOAA/NCEP/EMC) ``ufs-srweather-app`` repository fork on `GitHub <https://github.com/chan-hoo/ufs-srweather-app/tree/online-cmaq>`__.

Users will need to change the ``MACHINE`` and ``ACCOUNT`` variables in ``config.yaml`` to match their system. They may also wish to adjust other experiment settings. For more information on each task and variable, see :numref:`Chapter %s <ConfigWorkflow>`.

Users may also wish to change :term:`cron`-related parameters in ``config.yaml``. In the ``config.aqm.community.yaml`` file, which was copied into ``config.yaml``, cron is used for automatic submission and resubmission of the workflow:
Expand Down Expand Up @@ -170,3 +132,128 @@ Users may check experiment status from the experiment directory with either of t
# Check the experiment status and relaunch the workflow (for manual jobs)
./launch_FV3LAM_wflow.sh; tail -n 40 log.launch_FV3LAM_wflow


WE2E Test for AQM
=======================

Build the app for AQM:

.. code-block:: console

./devbuild.sh -p=hera -a=ATMAQ


Add the WE2E test for AQM to the list file:

.. code-block:: console

echo "custom_ESGgrid" > my_tests.txt
echo "aqm_grid_AQM_NA13km_suite_GFS_v16" >> my_tests.txt


Run the WE2E test:

.. code-block:: console

$ ./run_WE2E_tests.py -t my_tests.txt -m hera -a gsd-fv3 -q



Additional Tasks for AQM
===============================

Structure of SRW-AQM
-------------------------

The flowchart of the non-DA (data assimilation) SRW-AQM (Air Quality Modeling) is illustrated in :numref:`Figure %s <FlowProcAQM>`. Compared to the non-coupled (ATM stand-alone) FV3-LAM, SRW-AQM has additional tasks for pre- and post-processing. For pre-processing, multiple emission data such as NEXUS, fire, and point-source emission are retrieved or created for air quality modeling. Moreover, the chemical initial conditions (ICs) are extracted from the restart files of the previous cycle and added to the existing IC files. The chemical lateral boundary conditions (LBCs) and the GEFS aerosol data are also adeded to the existing LBC files. For post-processing, air quality forecast products for O3 and PM2.5 are generated and the bias-correction technique is applied to improve the accuracy of the results.

.. _FlowProcAQM:

.. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/SRW-AQM_workflow.png
:alt: Flowchart of the SRW-AQM tasks.
MichaelLueken marked this conversation as resolved.
Show resolved Hide resolved

*Workflow structure of SRW-AQM (non-DA)*



Pre-processing Tasks of SRW-AQM
------------------------------------

The pre-processing tasks for air quality modeling (AQM) are shown in :numref:`Table %s <TasksPrepAQM>`.

.. _TasksPrepAQM:

.. table:: Tasks for pre-processing of AQM

+-----------------------+--------------------------------------------------------------------+
| **Task name** | **Description** |
+=======================+====================================================================+
| nexus_gfs_sfc | This task retrieves the GFS surface files from the previous cycle |
| | in NRT (Near-Real-Time) or current cycle in retrospective cases. |
| | The surface radiation, soil moisture and temperature fields are |
| | needed for the MEGAN biogenics emissions within nexus_emission. |
+-----------------------+--------------------------------------------------------------------+
| nexus_emission | This task prepares the run directory with gridded emission inputs, |
| | run nexus to create model ready emission for the given simulation |
| | day, and post processes nexus output to make it more readable. The |
| | task will also split the task into multiple jobs set by the user. |
+-----------------------+--------------------------------------------------------------------+
| nexus_post_split | This task combines the nexus_emission outputs into a single job. |
+-----------------------+--------------------------------------------------------------------+
| fire_emission | This tasks is used to convert both satellite-retrieved gas and |
| | aerosol species emissions (RAVE) from mass (kg) to emission rates |
| | (kg/m2/s) and create 3-day hourly model-ready fire emission input |
| | files. |
+-----------------------+--------------------------------------------------------------------+
| point_source | This task aggregates the anthropogenic point source sectors of the |
| | National Emission Inventory(NEI) into a ready-to-input point-source|
| | emission file based on the weekday/weekend/holiday patterns of each|
| | sector and date/time of the simulation. |
+-----------------------+--------------------------------------------------------------------+
| aqm_ics | This task creates a chemical initial condition file by using the |
| | previous cycle restart files. |
+-----------------------+--------------------------------------------------------------------+
| aqm_lbcs | This task adds the chemical lateral boundary condition (LBC) upon |
| | the meteorological lateral boundary condition to form the full-set |
| | ready-to-input LBC for the simulation. It includes two sub-tasks: |
| | the gaseous species LBC and dynamic aerosol LBC. The former adds |
| | static gaseous LBC using monthly mean global data. The latter is |
| | the parallel job, which extracts the GEFS-Aerosol Model's output |
| | along the regional domain, and performs the species conversion |
| | from GOCART aerosols to CMAQ aerosols. |
+-----------------------+--------------------------------------------------------------------+


Post-processing Tasks of SRW-AQM
------------------------------------

The post-processing tasks for air quality modeling (AQM) are shown in :numref:`Table %s <TasksPostAQM>`. Since the module required to run these tasks is available only on WCOSS2, these tasks should not be defined in the configuration file ``config.yaml`` on other platforms.

.. _TasksPostAQM:

.. table:: Tasks for post-processing of AQM

+-----------------------+--------------------------------------------------------------------+
| **Task name** | **Description** |
+=======================+====================================================================+
| pre_post_stat | This task creates surface (i.e., model 1st level) meteorological |
| | and chemical files to support air quality product generation and |
| | generate training data to support bias correction tasks. |
+-----------------------+--------------------------------------------------------------------+
| post_stat_o3 | This task generates air quality forecast products including hourly |
| | -average and statistical products for O3 (e.g., daily 8-hour |
| | average maximum O3). |
+-----------------------+--------------------------------------------------------------------+
| post_stat_pm25 | This task generates air quality forecast products including hourly |
| | -average and statistical products for PM2.5 (e.g., 24-hour average |
| | PM2.5). |
+-----------------------+--------------------------------------------------------------------+
| bias_correction_o3 | This task applies a bias-correction technique (e.g., analog |
| | ensemble) to improve model raw forecast for O3 and generates the |
| | bias-corrected O3 products. |
+-----------------------+--------------------------------------------------------------------+
| bias_correction_pm25 | This task applies a bias-correction technique (e.g., analog |
| | ensemble) to improve model raw forecast for PM2.5 and generates the|
| | bias-corrected PM2.5 products. |
+-----------------------+--------------------------------------------------------------------+