V8.2.2 UGWP updates -- added non-stationary GWD and updated orographic GWD scheme #60
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The Noah-MP LSM sourcecode is copied from the Noah-MP github repository and corresponds to release-v5.0.1 (at hash 12a0b51aa52f6f5189525009ff249dfcbc95d29a), i.e.: commit 12a0b51aa52f6f5189525009ff249dfcbc95d29a Merge: 9fd80cb 843a742 Author: Cenlin_He <[email protected]> Date: Thu Jan 4 20:21:19 2024 -0700 Merge pull request MPAS-Dev#108 from NCAR/develop
…physics_noahmp,
modified sourcecode to compile the original Noah-MP sourcecode inside the MPAS
framework:
-> in ./physics_noahmp/utility, modified Machine.F90 to become dependent to
mpas_kind_types.F.
-> in ./physics_noahmp/utility, ./physics_noahmp/src, and ./physics_noahmp/drivers/hrldas,
modified Makefile to compile Noah-MP inside the MPAS framework.
-> in src/core_atmosphere/physics/physics_noahmp/drivers/hrldas/NoahmpDriverMainMod.F90,
removed the WRF modules module_ra_gfdleta and module_data_gocart_dust.
-> modified Makefile to compile all physics_noahmp subdirectories.
…p/parameters/NoahmpTable.TBL as we do for all the files in ./physics/physics_wrf. NoahmpTable.TBL is need to run the Noah-MP land surface scheme.
-> in /physics_noahmp/drivers, renamed the directory hrldas to mpas. -> in Makefile, changed all instance of hrldas to mpas.
…ed all instances of the J dimension and all references to WRF-related dimensions.
… NoahmpIOVarFinalizeMod.F90 to deallocate all arrays first allocated in NoahmpIOVarInitMod.F90. Modified Makefile accordingly.
…ic variables soilcomp, soilcl1, soilcl2, soilcl3, and soilcl4 needed to run the Noah-MP land surface scheme.
…he Noah-MP
land surface scheme:
-> added the Noah-MP Registry file ./physics/Registry_noahmp.xml to Registry.xml.
That file includes all the extra arrays to run Noah-MP.
-> in ./physics/mpas_atmphys_control.F, added the option sf_noahmp as an alternate
to sf_noah.
-> in ./physics/mpas_atmphys_manager.F, added the outputs month and day as outputs
to mpas_get_time. month and day are needed in ./physics_noahmp.
-> in ./physics/mpas_atmphys_packages, added the package sf_noahmp_in.
…DriverMainMod.F90, added print statements for NoahmpIO%itimestep and NoahmpIO%xice_threshold.
…InitMainMod.F90, remove the initialization of NoahmpIO%areaxy.
…IOVarFinalizeMod.F90, removed the sourcecode that initializes NoahmpIO variables since NoahmpIOVarFinalizeMod.F90 is only called to deallocate variables at the end of forecasts.
-> in Registry.xml, added Noahmp variables to the input and restart streams to run
the Noah-MP land surface scheme.
-> added ./physics/Registry_noahmp.xml which contains all the Noah-MP variables
needed to run the Noah-MP land surface scheme.
…Dev#1176) This merge enables the scalar transport routines in MPAS-Atmosphere to run on GPUs through the addition of OpenACC directives. Specifically, the atm_advance_scalars_work and atm_advance_scalars_mono_work routines have been ported via OpenACC. Additionally, timing information for OpenACC data transfers in the scalar transport routines is captured and reported in the log file by two new timers: atm_advance_scalars [ACC_data_xfer] atm_advance_scalars_mono [ACC_data_xfer] To minimize data movement, the MPAS-Atmosphere dycore has also been modified to copy all invariant fields needed by the scalar transport routines onto the device before the first dynamics timestep and to delete those invariant fields after the last dynamics timestep. * atmosphere/acc_scalar_transport: Add OpenACC directives throughout atm_advance_scalars_mono_work Add OpenACC directives throughout atm_advance_scalars_work Copy invariant fields used by scalar transport via OpenACC data directives
Right now this just contains a stub routine, but this gets some of the "plumbing work" done.
Guard the call to mpas_test_openacc routine with ifdef MPAS_OPENACC so the routines are only run if the test_core is built with OpenACC enabled.
This test carries out some calculations on the CPU and the GPU based on patterns in the dynamics of core_atmosphere. If the difference between the CPU results and the GPU results is 0, the test succeeds.
This merge adds a new module to the test core that tests OpenACC functionality and any OpenACC-specific framework routines that are added to MPAS. The only test added in this merge is based on patterns from the MPAS-A dycore. If the test core is compiled without OPENACC=true, the OpenACC tests are skipped. * framework/openacc_test: Remove OpenACC stub test, fix a comment Add representative array routines to OpenACC tests Add new variables to test_core Registry for OpenACC tests Add mpas_test_openacc to build and test_core_run Add module to exercise OpenACC to the test core
…_vers from 8.0 to 8.2 to access the latest version of the MPAS-Data repository. When mpas_vers="8.2", the file CCN_ACTIVATE_DATA needed to run the aerosol-aware Thompson cloud microphysics scheme is downloaded to /physics/physics_wrf/files.
…mmented extra print statements at the beginning and end of each subroutine to clarify the logfile.
…osols.F:
-> in subroutine calc_aerosol_rrtmg_sw, moved the declaration of dimensions ims,ime,...
above the declaration of variables p,t3d,dz8w,qv3d.
-> this allows module_ra_rrtmg_sw_aerosols.F to be compiled with gcc/12.2.0.
-> commented extra print statements related to the definition of the AOPs.
…ected typos at lines 697 and 700: replaced "\" with "&".
… package mp_thompson_aers_in. mp_thompson_aers_in refers to the aerosol-aware version of the Thompson cloud microphysics scheme, NOT the scale-aware version.
…t, tools include the subdirectory manage_externals. manage_externals is part of the Earth System Model Computational Infrastructure (ESMCI) and is a management utility used to check out groups of externals from revision control based on an externals description file (Externals.cfg). the master repository for manage_externals is https://github.com/ESMCI/manage_externals. we checked out manage_externals at the hash: commit f545b7ba8341e1da79dd0550c6a800f4972c8531 Merge: 876b344 2c4174b Author: Jim Edwards <[email protected]> Date: Tue Jan 2 06:19:40 2024 -0700 Merge pull request MPAS-Dev#210 from jedwards4b/defer_svn_access_error defer svn access to github error to actual checkout point, this will …
-> Modified Makefile to compile the physics_mmm directory accessed from the
github repository https://github.com/NCAR/MMM-physics.
-> Modified .gitignore to ignore the files LICENSE and README.md in physics_mmm.
…s used to access the github repository https://github.com/NCAR/MMM-physics and download and compiled the shared physics directory ./physics/physics_mmm.
This merge introduces the use of the manage_externals tool to access the GitHub repository NCAR/MMM-physics.git and to download the directory physics_mmm. NCAR/MMM-physics.git contains physics parameterizations shared by MPAS, WRF, and CM1. Modules follow CCPP coding standards. Added CCPP compliance for all modules contained in MMM-physics produce bit-for-bit reproducilbity of the physics_mmm develop branch with -O3 optimization, except for mp_wsm6.F90 with both the Intel and GCC compilers. Bit for bit reproducibility of the physics_mmm develop branch is achieved with no optimization (-g) and -O1 optimization.
…named "QNWFA_QNIFA_Monthly_Sigma.dat" to "QNWFA_QNIFA_SIGMA_MONTHLY.dat".
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_convection.F, added comments at the
top of module.
* * In ./src/core_atmosphere/physics/physics_wrf, removed the variables f_qv and f_qc
in module_ra_cam.F (subroutine camrad). Those two variables are not used in the
parameterization.
* In ./src/core_atmosphere/physics/mpas_atmphys_driver_radiation_lw(sw).F, removed
f_qv and f_qc in the calls to camrad.
* * In ./src/core_atmosphere/physics/mpas_atmphys_vars.F, removed the variable f_qv
which is not needed in any of the physics parameterizations.
* * In ./src/core_atmosphere/physics:
-> in mpas_atmphys_vars.F, removed the "parameter" definition for f_qc,f_qr,f_qi,
f_qs,f_qg,f_nc,and f_ni. these variables are now simply defined as logicals.
also renamed f_qnc to f_nc, and f_qni to f_ni.
-> in mpas_atmphys_init.F, added subroutine init_physics_flags to initialize f_qc,
f_qr,f_qi,f_qs,f_nc,and f_ni.
-> in mpas_atmphys_driver_pbl.F, changed the call in mynn_bl_driver after renaming
f_qnc to f_nc, and f_qni to f_ni.
* Allow MPAS to use either of 'mpi_f08' or 'mpi' modules
By defining the MPAS_USE_MPI_F08 macro at compile time, MPAS can now use the
more modern 'mpi_f08' module for MPI. If the MPAS_USE_MPI_F08 macro is not
defined, MPAS uses the 'mpi' module as prior to this commit.
With the use of the 'mpi_f08' module, certain MPI types are no longer integers
in Fortran, but are derived types; e.g., MPI_Comm, MPI_Request, MPI_Datatype,
MPI_Info.
In some instances, an integer-typed MPI type is still needed for
interoperability, and the MPI standard permits this to be done through the
'mpi_val' member of all MPI derived types, e.g., MPI_Comm % mpi_val.
* Add logic in top-level Makefile to detect mpi_f08 module availability
By trying to compile a test program that uses the 'mpi_f08' module, the
top-level Makefile can now detect whether the MPI implementation provides an
'mpi_f08' module and, if so, adds a definition of the MPAS_USE_MPI_F08 macro to
the CPPFLAGS used in the build.
* * In ./src/core_atmosphere/physics:
-> renamed module mpas_atmphys_driver_lsm_shared.F to mpas_atmphys_lsm_shared.F.
-> in mpas_atmphys_driver_lsm.F, now use mpas_atmphys_lsm_shared to call subroutine
correct_tsk_over_seaice. removed subroutine correct_tsk_over_seaice at the bottom
of the module.
-> corrected Makefile accordingly.
* Generalize mpas_sphere_angle function to work for non-unit-radius spheres
The methods previously employed in the mpas_sphere_angle function for computing
the side lengths of a spherical triangle were only applicable to unit-radius
spheres. This commit generalizes the computation of these side lengths so that
the mpas_sphere_angle function can be used to compute spherical angles on
spheres of arbitrary radius.
* Merge mpas_dmpar_exch_halo_adj_field2d_real() from the
JCSDA-internal/MPAS-Model repository.
Note the number of halo layers impacts the number of cells which will be updated
by this routine:
The first halo layer will update the owned 'edge' cells, where 'edge' cells are
adjacent to ghost cells.
The second halo layer will update owned cells which are adjacent to the 'edge' cells.
The third halo layer will update owned cells which are adjacent to the cells
updated by the seconds halo layer, etc.
* Add unit tests to verify the halo adjoint exchange.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_lsm.F, now weigh the local variables
needed in the calculation of t2m, th2m, and q2 prior to updating all LSM variables (see
subroutine lsm_to_MPAS).
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_lsm.F, moved the call to sfcdiags
from subroutine driver_lsm to subroutine lsm_to_MPAS. This allows t2m, th2m, and q2 to
be correctly computed over seaice points.
* * In ./src/core_atmosphere/physics, added the file mpas_atmphys_driver_seaice.F:
-> mpas_atmphys_driver_seaice.F contains all the sourcecode related to surface
processes over seaice points.
-> in mpas_atmphys_driver_lsm.F, removed all the sourcecode related to surface
processes over seaice points and moved the sourcecode to the seaice driver.
-> changed Makefile accordingly.
* Initialize mvd_r and mvd_c before first use in Thompson microphysics scheme
This commit initializes two local variables, mvd_r and mvd_c, in the mp_thompson
routine to avoid the potential use of uninitialized memory.
Note: Identical changes to those included in this commit have been introduced
into newer versions of the Thompson microphysics scheme, and so the changes here
can be considered a back-port of fixes.
* Remove LIBS from openacc_test in top-level Makefile
Since these OpenACC compilation tests don't depend on NetCDF, GPTL, or
other libraries, problems can be prevented by removing the linking
commands that come from the LIBS variable. For example, the openacc_test
rule will fail and the build will abort if a library the compiler
expects to link against hasn't built yet.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_cloudiness.F, corrected the units of
the variable dx_p from meters to kilometers (dx_p is used in the Thompson parameterization
of the cloud fraction).
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver.F, added OpenMP directives around
the loop that calls the seaice driver.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_seaice.F, updated the Copyright
at the top of the module.
* Add '-std=f2008' to FFLAGS_DEBUG for the 'gnu' build target
For the 'gnu' target, FFLAGS_OPT contained '-std=f2008' to check for Fortran
2008 conformance, but the debug build flags in FFLAGS_DEBUG lacked this option.
This commit simply adds '-std=f2008' to FFLAGS_DEBUG so that both optimized and
debug builds check that code conforms to the Fortran 2008 standard.
* * In ./src/core_atmosphere/physics, updated the MYNN surface layer scheme using the
sourcecode from WRF version 4.6.
-> in ./physics_mmm, added the CCPP-compliant MYNN surface layer scheme (sf_mynn.F) and
mynn_shared.F that contains functions also shared with the MYNN PBL scheme. Modified
Makefile accordingly.
-> in ./physics_wrf, added the file sf_mynn_pre.F to initialize variables needed in the
surface layer scheme. Modified module_sf_mynn.F which called the MYNN surface layer
scheme. Modified Makefile accordingly.
-> Modified mpas_atmphys_driver_sfclayer.F and mpas_atmphys_vars.F to accomodate updates
to the MYNN surface layer scheme.
* * In ./src/core_atmosphere/physics, very minor corrections to Makefile,
mpas_atmphys_driver_convection.F, and mpas_atmphys_driver_gwdo.F.
* * In ./src/core_atmosphere/physics/physics_mmm, very minor corrections to mp_radar.F
and sf_sfclayrev.F.
* * In ./src/core_atmosphere/physics, updated the MYNN PBL with the sourcecode available
in the WRF version 4.6.
-> in ./physics_mmm, added bl_mynn.F and bl_mynn_subroutines.F modules which are CCPP-
compliant versions of the MYNN PBL scheme. Modified Makefile accordingly.
-> in ./physics_wrf, added bl_mynn_pre.F and bl_mynn_post.F, and modified module_bl_mynn.F
that calls the MYNN PBL scheme. Modified Makefile accordingly.
-> in ./physics:
in mpas_atmphys_vars.F, added (and deleted) variables need to run the updated scheme.
in mpas_atmphys_init.F, initialized the added f_* flags needed to run the updated scheme.
in mpas_atmphys_driver_pbl.F, modified and updated the call to run the updated scheme.
in mpas_atmphys_todynamics.F, added rqsblten to the calculation to tend_scalars(index_qs,.).
-> in ./../Registry.xml, added new variables needed to run the updated MYNN PBL scheme.
* * In ./src/core_atmosphere/physics/physics_mmm/bl_mynn_subroutines.F, removed extra blank spaces
introduced when updating the sourcecode.
* * In ./src/core_atmosphere/physics, made minor corrections to ./physics_mmm/Makefile
and ./physics_wrf/module_cu_ntiedtke.F.
* * In ./src/core_atmosphere/physics/physics_mmm/bl_mynn_subroutines.F, commented out
the line:
6537 ! wm2 = wm2 + wm3**twothirds
in subroutine topdown_cloudrad. The variable wm2 is not initialized and not used
anywhere in the subroutine.
* * In ./src/core_atmosphere/physics/physics_mmm/bl_mynn.F, removed the lines:
!JOE-add GRIMS parameters & variables
real(kind=kind_phys),parameter:: d1 = 0.02, d2 = 0.05, d3 = 0.001
real(kind=kind_phys),parameter:: h1 = 0.33333335, h2 = 0.6666667
real(kind=kind_phys):: govrth,sflux,bfx0,wstar3,wm2,wm3,delb
!JOE-end GRIMS
in subroutine bl_mynn_run. The removed GRIMS parameters and variables are
not needed in bl_mynn.F.
* * In ./src/core_atmosphere/physics/physics_mmm/bl_mynn_subroutines.F, removed the extra
definition of parameters cphm_st, cphm_unst, cphh_st, and cphh_unst.
* * In ./src/core_atmosphere/physics:
-> in ./physics_mmm/bl_mynn.F, added the initialization of the optional variable dqke.
-> in ./physics_wrf/module_bl_mynn.F, made the variables dqke, qwt, qshear, qbuoy, and
qdiss optional variables.
-> in mpas_atmphys_driver_pbl.F, rewrite outputs to logfiles.
* * In ./src/core_atmosphere/Registry.xml, changed the option config_mynn_topdown
from 1 to 0.
* * In ./src/core_atmosphere/physics:
-> in ./physics_wrf/module_bl_mynn.F, added the local logical mynn_tkebudget_l which
is initialized as a function of the input integer variable bl_mynn_tkebudget.
mynn_tkebudget_l == true (false) when bl_mynn_tkebudget == 1 (0). this change is
needed to for CCPP-compliance of the MYNN files in ./physics_mmm.
-> in ./physics_mmm, modified bl_mynn.F and bl_mynn_subroutines.F accordingly.
* * In ./src/core_atmosphere/physics:
-> in physics_wrf/module_bl_mynn.F, added the local logicals mynn_mixclouds_l,
mynn_mixscalars_l, and mynn_mixqt_l which are initialized as functions of the
input integer variables bl_mynn_mixclouds, bl_mynn_mixscalars, and bl_mynn_mixqt.
the local logicals are set to true (false) when their respective input integer
variables are equal to 1 (0). this change is needed for CCPP-compliance of the MYNN
files in ./physics_mmm.
-> in physics_mmm, modified bl_mynn.F and bl_mynn_subroutines.F accordingly.
* * In ./src/core_atmosphere/physics:
-> in physics_wrf/module_bl_mynn.F, added the local logicals mynn_edmf_l and
mynn_edmf_dd_l which are initialized as functions of the input integer variables
bl_mynn_edmf and bl_mynn_edmf_dd. the logicals are set to true (false) when when
their respective input integer variables are equal to 1 (0). this change is needed
for CCPP-compliance of the MYNN files in ./physics_mmm.
-> in physics_mmm, modified bl_mynn.F and bl_mynn_subroutines.F accordingly.
* * In ./src/core_atmosphere/physics:
-> in physics_wrf/module_bl_mynn.F, added the local logicals mynn_edmf_mom_l and
mynn_edmf_tke_l which are initialized as functions of the input integer variables
bl_mynn_edmf_mom and bl_mynn_edmf_tke. the logicals are set to true (false) when
their respective input integer variables are equal to 1 (0). this change is needed
for CCPP-compliance of the MYNN files in ./physics_mmm.
-> in physics_mmm, modified bl_mynn.F and bl_mynn_subroutines.F accordingly.
* * In ./src/core_atmosphere/physics:
-> in physics_wrf/module_bl_mynn.F, added the local logicals mynn_output_l, mynn_dheatopt_l,
mynn_scaleaware_l, and mynn_topdown_l which are initialized as functions of the input integer
variables bl_mynn_output, bl_mynn_dheat_opt, bl_mynn_scaleaware_l, and bl_mynn_topdown. the
logicals are set to true (false) when when their respective input integer variables are equal
to 1 (0). this change is needed for CCPP-compliance of the MYNN files in ./physics_mmm.
-> in physics_mmm, modified bl_mynn.F and bl_mynn_subroutines.F accordingly.
* * In ./src/core_atmosphere/physics/physics_mm/bl_mynn.F, removed extra commented
out lines.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_pbl.F, commented out extra
print statements before and after the call to subroutine mynn_bl_driver.
* * In ./src/core_atmosphere/physics/physics_mmm/bl_mynn_subroutines.F, changed all
instances of DOUBLE PRECISION with kind_phys8 where kind_phys8 points to R8KIND.
* Update version number to 8.1.0
* Move call to halo exchange adjoint tests in test core
This commit moves the call to the routine test_halo_adj_exch_fields, which tests
the halo exchange adjoint code, up to the test_core_halo_exch_test routine;
additionally, the log file now contains lines to identify the testing of the
halo exchange adjoint code as well as the result of that testing, e.g.,
- Performing halo exchange adjoint tests
-- Return code: 0
* Adjust whitespace in comment block for mpas_dmpar_exch_halo_adj_field2d_real
* Update comments for halo exch adjoint tests in test_core_halo_exch module
This commit updates some of the comments in code related to halo exchange
adjoint testing in the test_core_halo_exch module to better align with
conventional MPAS terminology.
* * In ./src/core_atmosphere/physics/physics_mmm/bl_ysu.F, added the initialization
of local variable we.
* Fix memory leaks due to duplicate `allocate` calls on pointers
The `mpas_pool_create_pool` subroutine already allocates the pointer in its arguments.
Therefore, calling `allocate` before `mpas_pool_create_pool` actually causes memory leaks.
* * Made a couple of corrections in subroutines advance_scalars and atm_advance_scalars_mono_work
so that they can be called with state arrays different than "scalars":
-> in subroutine advance_scalars, modified the line:
call mpas_pool_get_dimension(state, 'num_scalars', num_scalars)
to
call mpas_pool_get_dimension(state, 'num_'//trim(field_name), num_scalars)
to expand the definition of num_scalars to state arrays different than "scalars".
-> in subroutine atm_advance_scalars_mono_work, modified the line:
call exchange_halo_group(block % domain, 'dynamics:scalars_old')
to
call exchange_halo_group(block % domain, 'dynamics:'//trim(field_name)//'_old')
so that halo exchange can be applied to state arrays different than "scalars".
* Add routine for reporting compile- and run-time settings to the log file
The new mpas_framework_report_settings routine writes information about compile-
time options and run-time settings to the log file.
A call to mpas_framework_report_settings has been added to the test core's
test_setup_log routine, so that information like the following is written to the
log file for the test core:
Output from 'git describe --dirty': v8.1.0-dirty
Compile-time options:
Build target: intel
OpenMP support: no
OpenACC support: no
Default real precision: single
Compiler flags: optimize
I/O layer: SMIOL
Run-time settings:
MPI task count: 32
* Use mpas_framework_report_settings in atm_setup_log for atmosphere core
This commit replaces code in the atmosphere core's atm_setup_log routine for
writing compile-time options and run-time settings with a single call to the
framework routine mpas_framework_report_settings.
* Use mpas_framework_report_settings in init_atm_setup_log for init_atmosphere core
This commit replaces code in the init_atmosphere core's init_atm_setup_log
routine for writing compile-time options and run-time settings with a single
call to the framework routine mpas_framework_report_settings.
* Include OpenACC device info in mpas_framework_report_settings output
If MPAS is compiled with OPENACC=true, the mpas_framework_report_settings
routine now calls a new private routine, report_acc_devices, in mpas_framework
to report information about OpenACC devices to the log. The report_acc_devices
routine uses the OpenACC runtime library to inquire about the number of
available devices, the device number for the calling MPI task, and info about
the device type and driver. As an example from Derecho with the NVHPC 24.3
compilers, the following information is reported in the log file:
OpenACC configuration:
Number of visible devices: 1
Device # for this MPI task: 0
Device vendor: NVIDIA
Device name: NVIDIA A100-SXM4-40GB
Device driver version: 12020
If MPAS is not compiled with OPENACC=true, then the report_acc_devices routine,
and the call to it from mpas_framework_report_settings, are pre-processed out.
* Add "-check all" to LDFLAGS_DEBUG for 'intel' build target
Regarding the '-check all' option, the Intel 2024.1 manual states:
Caution
Files compiled with option check all should also be
linked with this same option, or the link step may fail.
* Move 21 additional fields into the "invariant" stream in the atmosphere core
This commit moves 21 additional time-invariant fields from the "input" and
"restart" streams into the "invariant" stream for the atmosphere core. These
additional fields are time-invariant terrestrial fields that are used by various
physics parameterization schemes, and as they were in the "input" and "restart"
streams, they are conditionally included in the "invariant" stream with the
DO_PHYSICS preprocessing macro.
Specifically, the 21 fields now included in the "invariant" stream are:
isltyp
ivgtyp
mminlu
isice_lu
iswater_lu
landmask
shdmin
shdmax
snoalb
albedo12m
greenfrac
var2d
con
oa1
oa2
oa3
oa4
ol1
ol2
ol3
ol4
* Add stream_function and velocity_potential fields to the diag struct.
* Add da_state input/output stream.
This stream is only used by MPAS-JEDI.
This stream is associated with the "jedi_da" package, and so it will only be active if config_jedi_da = true.
* * In ./src/core_atmosphere/physics/physics_mmm, changes all the file extension .F to .F90.
Modified Makefile accordingly.
* * In ./src/core_atmosphere/physics/physics_mmm, updated bl_gwdo.F90 with additional
CCPP compliance requirements.
* * In ./src/core_atmosphere/physics/physics_mmm, updated bl_ysu.F90 with additional
CCPP compliance requirements.
Modified ./src/core_atmosphere/physics/physics_wrf/module_bl_ysu.F to accomodate
changes in ./physics_mmm/bl_ysu.F90.
* * In ./src/core_atmosphere/physics, updated ./physics_mmm/cu_ntiedtke.F90 and
./physics_wrf/module_cu_ntiedtke.F.
* * In ./src/core_atmosphere/physics/physics_wrf, finished to update module_cu_ntiedtke.F.
Added modules cu_ntiedtke_pre.F and cu_ntiedtke_post.F to comply with CCPP requirements,
and to initialize and finalize variables used in ./physics_mmm/cu_ntiedtke.F90. Modified
Makefile accordingly.
* * In ./src/core_atmosphere/physics:
-> in physics_mmm/sf_sfclayrev.F90, updated the revised surface layer scheme.
-> in physics_wrf, updated module_sf_sfclayrev.F and added the new module
sf_sfclayrev_pre.F. Modified aMakefile accordingly.
-> corrected the call to to sfclayrev.
* * In ./src/core_atmosphere/physics/physics_mmm, updated bl_mynn.F90 with CCPP-compliance
requirements for declarations of arrays with intent "in", "inout", and "out".
* * In ./src/core_atmosphere/physics/physics_mmm/cu_ntiedtke.F90, added doxygen comments
in subroutine cu_ntiedtke_run.
* * In ./src/core_atmosphere/physics/physics_mmm, updated mp_wsm6.F90 with CCPP-compliance
requirements for declarations of arrays with intent "in", "inout", and "out".
* * In ./src/core_atmosphere/physics:,
-> renamed module ccpp_kinds.F to ccpp_kind_types.F.
-> in ./physics_wrf, changed "use ccpp_kinds" to "use ccpp_kind_types" in modules
bl_mynn_post.F, bl_mynn_pre.F, cu_ntiedtke_post.F, sf_mynn_pre.F, sf_sfclayrev_pre.
-> in ./physics_mmm, changed "use ccpp_kinds" to "use ccpp_kind_types" in modules
bl_gwdo.F90, cu_ntiedtke.F90, mp_radar.F90, mp_wsm6.F90, sf_sfclayrev.F90, and
sf_mynn.F90.
-> in ./physics_mmm, changed "use mpas_kind_types" to "use ccpp_kind_types" in
modules bl_mynn.F90, bl_mynn_subroutines.F90, and mynn_shared.F90.
-> in ./physics_mmm, also updated mp_wsm6_effectRad.F90, with CCPP-compliance
requirements for declarations of arrays with intent "in", "inout", and "out".
-> Modified Makefile accordingly.
* * As an intermediary step to moving ./physics_mmm to its own repository and then removing
./physics_mmm/Makefile which is currently specific to MPAS, we first renamed Makefile
to Makefile.mpas. We modified ./physics/Makefile accordingly.
* Remove Fortran only flag from LDFLAGS_DEBUG for NV targets
The -Mchkptr flag is only valid for Fortran compilation. If included in
a C build with NVHPC compilers (like the check with openacc.c does), the
build fails.
* Add optional CMake build for atmosphere and init_atmosphere core of MPAS-Model
Introduces an optional CMake build for the atmosphere and init_atmosphere core
of the MPAS-Model, designed to improve support for MPAS-JEDI which employs
ecbuild/CMake. The intention is to enhance compatibility and facilitate
integration with MPAS-JEDI, while retaining the default build method using GNU
make. The CMake build addition is based on contributions from the NSF NCAR
Mesoscale and Microscale Meteorology (MMM) Laboratory and the Joint Center for
Satellite Data Assimilation (JCSDA).
The CMake build does not support SMIOL and requires PIO2. SMIOL support for the
CMake build is planned for future updates. Outside of the PIO2 requirement, the
only additional dependency for the CMake build is CMake itself.
To build with CMake, make sure CMake, netcdf-c, netcdf-fortran, parallel-netcdf,
and parallelio are installed and discoverable. If using a module package
environment, load the modules for the above libraries. Create a build directory
and enter it: mkdir <build_dir>; cd <build_dir>. Configure the build with
CMake by running cmake <mpas_src_dir> <cmake_flags>, where mpas_src_dir is
the MPAS source directory and cmake_flags is a string of flags used to set
various build options.
CMAKE_BUILD_TYPE: Specifies the build type. Possible values are Release,
Debug, and RelWithDebInfo. Defaults to Release.
MPAS_DOUBLE_PRECISION: Controls if MPAS is built with single or double
precision. Possible values are ON and OFF. Defaults to ON.
MPAS_CORES: Controls which MPAS Cores are built. Possible values are atmosphere,
init_atmosphere atmosphere, and init_atmosphere.
Next, run make -j<number_of_jobs> where number_of_jobs is the number of jobs
GNU Make will execute in parallel.
* Copy invariant fields used by scalar transport via OpenACC data directives
This commit adds code and OpenACC directives in the MPAS_atm_dynamics_init
routine to transfer invariant fields used by scalar transport code to the
device, and it adds code and directives in the MPAS_atm_dynamics_finalize
routine to delete these invariant fields.
The new code and directives are surrounded by tests on the MPAS_OPENACC
preprocessing macro, and so the changes in this commit have no effect unless
MPAS-A is compiled with OPENACC=true.
Since the MPAS_atm_dynamics_init routine is called once during model startup and
the MPAS_atm_dynamics_finalize once during model shutdown, the invariant fields
will remain resident on the device across all dynamics time steps.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_pbl.F, commented print
statements in subroutine init_pbl.
* * In ./src/core_init_atmosphere:
-> added the module mpas_init_atm_thompson_aerosols.F to interpolate climatological
monthly-mean Gocart-based water-friendly and ice-friendly aerosols for use with
the aerosol-aware Thompson cloud microphysics scheme.
-> in Registry.xml, added climatological, and first-guess water- and ice-friendly
aerosols for use with the aerosol-aware Thompson cloud microphysics scheme.
-> in mpas_init_atm_cases.F, added call to initialize first-guess water- and ice-
friendly aerosols.
-> modified Makefile accordingly.
* * In ./src/core_atmosphere/physics/physics_wrf, updated module_mp_thompson.F to WRF-v4.1.4.
* * In ./src/core_atmosphere/physics, started the implementation of the aerosol-aware
option of the Thompson cloud microphysics parameterization:
-> in mpas_atmphys_control.F, added the option mp_thompson_aerosols in subroutine
physics_namelist_check to trigger the aerosol-aware option of the microphysics
scheme.
-> in mpas_atmphys_packages.F, added the option mp_thompson_aerosols to activate
the package mp_thompson_in in subroutine atmphys_setup_packages.
-> in mpas_atmphys_manager.F, added the option mp_thompson_aerosols when initializing
the microphysics timestep dt_microp, and to activate the computation of effective
radii in subroutine physics_run_init. also edited a few print statements.
* * In ./src/core_atmosphere/Registry.xml, ./src/core_atmosphere/dynamics/mpas_atm_time_integration.F,
and ./src/core_atmosphere/physics, finished the implementation of the Thompson cloud microphysics
parameterization:
-> in Registry.xml, added the prognostic variables and associated tendencies to run the aerosol-
aware option, i.e. nifa, nwfa, and nc. also added surface emissions of nwfa (nwfa2d) and nifa
(nifa2d).
-> in mpas_atm_time_integration.F, added tend_physics to the argument list of the call to subroutine
driver_microphysics to compute cloud microphysics tendencies.
-> in all other modules, modified sourcecode to run the aerosol-aware option of the thompson cloud
microphysics scheme when we use the option config_microp_scheme = "mp_thompson_aerosols".
* * In ./src/core_atmosphere/physics, added the option "mp_thompson_aerosols" to compute
the radiative effective radii of cloud liquid water, cloud ice, and snow in modules
mpas_atmphys_driver_radiation_lw.F and mpas_atmphys_driver_radiation_sw.F.
* * In ./src/core_atmosphere/physics, cleaned-up mpas_atmphys_todynamics.F.
* * In ./src/core_atmosphere/physics, added the PBL and free-tropospheric mixing of
the cloud liquid water number concentration, and water-friendly and ice-friendly
Thompson_aerosols when the option mp_thompson_aerosols is set to true.
* * In ./src/core_atmosphere/physics, added the RRTMG shortwave aerosol optical properties of
the Thompson water-friendly and ice-friendly aerosols.
-> in ./physics_wrf, added module_mp_thompson_aerosols.F and module_ra_rrtmg_sw_aerosols.F
to compute the RRTMG aerosol optical properties (AOPs)of the Thompson aerosols. added the
aerosol effects in module_ra_rrtmg_sw.F.
-> in mpas_atmphys_vars.F, added local variables of the AOPs of the Thompson aerosols.
-> in mpas_atmphys_driver_radiation_sw.F, added calls to subroutine gt_aod and subroutine
calc_aerosol_rrtmg_sw to compute the AOPs of the Thompson aerosols.
* * In ./src/core_atmosphere/physics/mpas_atmphys_finalize, added the option
mp_thompson_aerosols in subroutine atmphys_finalize. Ths option is used
to deallocate all arrays used in the "MP_THOMPSON" tables.
* * In src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, corrected the
initialization of the array sorted_arr in the vertical interpolation of nifa
and nwfa in subroutine init_vinterp_gocart.
* * In ./src/core_atmosphere/physics/mpas_atmphys_init_microphysics.F, added the
computation of the surface emission of water friendly aerosols (nwfa2d).
* Add OpenACC directives throughout atm_advance_scalars_work
This commit adds OpenACC directives throughout the atm_advance_scalars_work
routine.
Also as part of this commit, macros have been defined at the top of
mpas_atm_time_integration.F to activate timers for OpenACC data transfers when
MPAS-A is compiled with OPENACC=true. For the data transfers in the
atm_advance_scalars_work routine, the new timer
atm_advance_scalars [ACC_data_xfer]
will be written to the timer summary in log files.
With the NVHPC 24.3 compilers and flags provided in the 'nvhpc' target, results
for a regional test case are bitwise identical when the original code and code
in this commit are both compiled with OPENACC=true.
* Add OpenACC directives throughout atm_advance_scalars_mono_work
This commit adds OpenACC directives throughout the atm_advance_scalars_mono_work
routine. At present, halo exchanges in the monotonic scalar transport routine
require fields to be transferred to the host before the halo exchange, and then
back to the device afterward.
For the data transfers in the atm_advance_scalars_mono_work routine, the new
timer
atm_advance_scalars_mono [ACC_data_xfer]
will be written to the timer summary in log files.
With the NVHPC 24.3 compilers and flags provided in the 'nvhpc' target, results
for a regional test case are bitwise identical when the original code and code
in this commit are both compiled with OPENACC=true.
* * Initial implementation of the Noah-MP land surface scheme (LSM):
The Noah-MP LSM sourcecode is copied from the Noah-MP github repository and corresponds
to release-v5.0.1 (at hash 12a0b51aa52f6f5189525009ff249dfcbc95d29a), i.e.:
commit 12a0b51aa52f6f5189525009ff249dfcbc95d29a
Merge: 9fd80cb 843a742
Author: Cenlin_He <[email protected]>
Date: Thu Jan 4 20:21:19 2024 -0700
Merge pull request #108 from NCAR/develop
* * In ./src/core_atmosphere/physics and ./src/core_atmosphere/physics/physics_noahmp,
modified sourcecode to compile the original Noah-MP sourcecode inside the MPAS
framework:
-> in ./physics_noahmp/utility, modified Machine.F90 to become dependent to
mpas_kind_types.F.
-> in ./physics_noahmp/utility, ./physics_noahmp/src, and ./physics_noahmp/drivers/hrldas,
modified Makefile to compile Noah-MP inside the MPAS framework.
-> in src/core_atmosphere/physics/physics_noahmp/drivers/hrldas/NoahmpDriverMainMod.F90,
removed the WRF modules module_ra_gfdleta and module_data_gocart_dust.
-> modified Makefile to compile all physics_noahmp subdirectories.
* * In ./src/core_atmosphere/Makefile, now link ./physics/physics_noahmp/parameters/NoahmpTable.TBL
as we do for all the files in ./physics/physics_wrf. NoahmpTable.TBL is need to run the Noah-MP
land surface scheme.
* * In ./src/core_atmosphere/physics:
-> in /physics_noahmp/drivers, renamed the directory hrldas to mpas.
-> in Makefile, changed all instance of hrldas to mpas.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas, removed all instances of the J dimension
and all references to WRF-related dimensions.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas, added NoahmpIOVarFinalizeMod.F90
to deallocate all arrays first allocated in NoahmpIOVarInitMod.F90. Modified Makefile accordingly.
* * In ./src/core_init_atmosphere, added the initialization of the static variables
soilcomp, soilcl1, soilcl2, soilcl3, and soilcl4 needed to run the Noah-MP land
surface scheme.
* * In ./src/core_atmosphere, starting the first modifications to run the Noah-MP
land surface scheme:
-> added the Noah-MP Registry file ./physics/Registry_noahmp.xml to Registry.xml.
That file includes all the extra arrays to run Noah-MP.
-> in ./physics/mpas_atmphys_control.F, added the option sf_noahmp as an alternate
to sf_noah.
-> in ./physics/mpas_atmphys_manager.F, added the outputs month and day as outputs
to mpas_get_time. month and day are needed in ./physics_noahmp.
-> in ./physics/mpas_atmphys_packages, added the package sf_noahmp_in.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas/NoahmpDriverMainMod.F90,
added print statements for NoahmpIO%itimestep and NoahmpIO%xice_threshold.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas/NoahmpInitMainMod.F90,
remove the initialization of NoahmpIO%areaxy.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas/NoahmpIOVarFinalizeMod.F90,
removed the sourcecode that initializes NoahmpIO variables since NoahmpIOVarFinalizeMod.F90
is only called to deallocate variables at the end of forecasts.
* * In ./src/core_atmosphere:
-> in Registry.xml, added Noahmp variables to the input and restart streams to run
the Noah-MP land surface scheme.
-> added ./physics/Registry_noahmp.xml which contains all the Noah-MP variables
needed to run the Noah-MP land surface scheme.
* Add module to exercise OpenACC to the test core
Right now this just contains a stub routine, but this gets some of the
"plumbing work" done.
* Add mpas_test_openacc to build and test_core_run
Guard the call to mpas_test_openacc routine with ifdef MPAS_OPENACC so
the routines are only run if the test_core is built with OpenACC
enabled.
* Add new variables to test_core Registry for OpenACC tests
* Add representative array routines to OpenACC tests
This test carries out some calculations on the CPU and the GPU based on
patterns in the dynamics of core_atmosphere. If the difference between
the CPU results and the GPU results is 0, the test succeeds.
* Remove OpenACC stub test, fix a comment
* * In src/core_atmosphere/physics/checkout_data_files.sh, updated mpas_vers from 8.0 to
8.2 to access the latest version of the MPAS-Data repository. When mpas_vers="8.2",
the file CCN_ACTIVATE_DATA needed to run the aerosol-aware Thompson cloud microphysics
scheme is downloaded to /physics/physics_wrf/files.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, commented extra
print statements at the beginning and end of each subroutine to clarify the logfile.
* * In ./src/core_atmosphere/physics/physics_wrf/module_ra_rrtmg_sw_aerosols.F:
-> in subroutine calc_aerosol_rrtmg_sw, moved the declaration of dimensions ims,ime,...
above the declaration of variables p,t3d,dz8w,qv3d.
-> this allows module_ra_rrtmg_sw_aerosols.F to be compiled with gcc/12.2.0.
-> commented extra print statements related to the definition of the AOPs.
* * In src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, corrected typos
at lines 697 and 700: replaced "\" with "&".
* * In ./src/core_atmosphere/Registry.xml, corrected the description of package
mp_thompson_aers_in. mp_thompson_aers_in refers to the aerosol-aware version
of the Thompson cloud microphysics scheme, NOT the scale-aware version.
* * In ./src/core_atmosphere, added the subdirectory "tools". At present, tools include the
subdirectory manage_externals.
manage_externals is part of the Earth System Model Computational Infrastructure (ESMCI)
and is a management utility used to check out groups of externals from revision control
based on an externals description file (Externals.cfg).
the master repository for manage_externals is https://github.com/ESMCI/manage_externals.
we checked out manage_externals at the hash:
commit f545b7ba8341e1da79dd0550c6a800f4972c8531
Merge: 876b344 2c4174b
Author: Jim Edwards <[email protected]>
Date: Tue Jan 2 06:19:40 2024 -0700
Merge pull request #210 from jedwards4b/defer_svn_access_error
defer svn access to github error to actual checkout point, this will …
* * In ./src/core_atmosphere/physics,removed the directory physics_mmm.
* * In ./src/core_atmosphere/physics:
-> Modified Makefile to compile the physics_mmm directory accessed from the
github repository https://github.com/NCAR/MMM-physics.
-> Modified .gitignore to ignore the files LICENSE and README.md in physics_mmm.
* * In ./src/core_atmosphere, added the file Externals.cfg. This file is used
to access the github repository https://github.com/NCAR/MMM-physics and
download and compiled the shared physics directory ./physics/physics_mmm.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, renamed
"QNWFA_QNIFA_Monthly_Sigma.dat" to "QNWFA_QNIFA_SIGMA_MONTHLY.dat".
* * In ./src/core_init_atmosphere, do not include the Thompson water- and ice-friendly
aerosols in the initial conditions and lateral boundary conditions files when the
when the GOCART input "QNWFA_QNIFA_SIGMA_MONTHLY.dat" is not present in the run
directory.
-> In Registry.xml, added package mp_thompson_aers_in.
-> In mpas_init_atm_core_interface.F, added the initialization of mp_thompson_aers_in.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, added subroutine
init_atm_thompson_aerosols_lbc to compute the lateral boundary conditions for the
water-friendly and ice-friendly aerosols in the aerosol-aware Thompson cloud
microphysics.
The lateral boundary conditions for the water-friendly and ice-friendly aerosols
are computing using the monthly mean aerosol climatology interpolated to the MPAS
mesh and available in the regional initial conditions.
* When running the 'parse' tool, provide Registry file as command-line argument
The 'parse' tool for generating code based on a Registry file can either read
the Registry from stdin or from a file whose name is supplied as a command-line
argument. This tool assumes that if its command-line argument count is 2, then
the argument is a Registry filename (the argument count is 1 if no command-line
arguments are given).
Since future development will add processing for an arbitrary number of
command-line arguments to the parse tool, deciding whether to read the Registry
from stdin based on the argument count is not a viable strategy, so with this
commit, each core's Makefile now passes a filename as the only argument to the
parse tool, and the parse tool itself has been modified to require a
command-line argument.
* * In ./src/core_init_atmosphere, continued the implementation of the lateral boundary
conditions for nc, nifa, and nwfa.
-> in Registry.xml, added the fields nifa_gocart_clim, nwfa_gocart_clim, and
pwif_gocart_clim to the output stream so that they can be read later from
the initial conditions file when building the lateral boundary conditions.
-> in Registry.xml, added the fields nofa_gocart_clim, nwfa_gocart_clim, and
pwif_gocart_clim to the input stream so that they can be read using the
initial conditions file when building the lateral boundary conditions.
-> in mpas_init_atm_core_interface.F, revised the initialization of package
mp_thompson_aerosols.
-> in mpas_init_atm_cases.F, add call to subroutine init_atm_thompson_aerosol_lbc
to build the lateral boundary conditions lbc_nc, lbc_nifa, and lbc_nwfa after
call to subroutine init_atm_case_lbc.
* * In ./src/core_atmosphere/physics:
-> added mpas_atmphys_lsm_noahmpinit.F to initialize Noah-MP variables within
the MPAS framework.
-> added mpas_atmphys_driver_lsm_noahmp.F to run the Noah-MP land surface scheme
within the MPAS framework.
* * In ./src/core_atmosphere/physics:
-> in mpas_atmphys_vars.F, added the structure mpas_noahmp of type NoahmIO_type
that contains all the Noah-MP variables and arrays needed for Noah-MP to
communicate with the MPAS physics.
-> in mpas_atmphys_init.F, added call to init_lsm_noahmp to initialize Noah-MP.
-> in mpas_atmphys_driver.F, added call to driver_lsm_noahmp to run Noah-MP.
-> Modified Makefile accordingly.
-> in ./../mpas_atm_core.F, modified call to physics_init to include the pools
diag_physics_noahmp and output_noahmp.
* * In ./src/core_atmosphere/physics/Registry_noahmp.xml, added the argument missing_value
to the definition of Noah-MP variables. missing_value is set to -9999 for integer and
-9999.0 to real variables, as defined in Machine.F90 used in Noah-MP.
* * In ./src/core_atmosphere/physics/mpas_atmphys_lsm_noahmpinit.F, removed variables needed in
the crop model. The crop model is currently not available and related variables are not
used.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_lsm_noahmp.F, removed variables needed
in the ground water parameterization.The ground water model is currently not available and
related variables are not used.
* * In src/core_atmosphere/physics/physics_noahmp/drivers/mpas:
-> in NoahmpInitMod.F90, removed the initialization of xland, xice, and
xice_threshold, and move all three variables to ./physics.
-> in NoahmpDriverMainMod.F90, rearranged print statements.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_lsm_noahmp.F, added the
initialization of xland to subroutine lsm_noahmp_fromMPAS.
* * In ./src/core_atmosphere/physics/Registry_noahmp.xml, corrected the description
of a few Noah-MP variables.
* * In ./src/core_atmosphere/physics, deallocate all Noah-MP arrays needed to run
the Noah-MP land surface scheme.
* * In Makefile, removed the option "-fp-model precise" for the ifort option.
* * In ./src/core_atmosphere/Registry.xml:
-> moved the variables soilcomp, soilcl1, soilcl2, and soilcl3 from the input stream
to the invariant stream.
-> added the variable soilcl4 to the input stream.
* * In ./src/core_atmosphere/physics:
-> in ./physics_noahmp/drivers/mpas/NoahmpDriverMainMod.F90, commented extra
print statements.
-> in mpas_atmphys_driver_lsm_noahmp.F, commented extra print statements.
* * In src/core_atmosphere/Registry.xml, removed the static fields needed to run Noah-MP
from the restart stream since they are available in the "invariant" stream and the
invariant stream is included in the "restart" stream.
* * In ./src/core_atmosphere/physics/mpas_atmphys_lsm_noahmpinit.F, commented extra
print statements originally written during the initialization of Noah-MP.
* * In ./src/core_atmosphere/physics/mpas_atmphys_lsm_noahmpinit.F, removed extra
print statements before and after call to subroutine NoahmpInitMain.
* Change URL for MMM-physics external to use https rather than git
So that the checkout of the MMM-physics repository will work even for those
who don't have SSH keys set up on GitHub, this commit changes the URL for
the MMM-physics external to use https rather than git.
* Add new function 'parse_macros' to registry tool utility code
The parse_macros function takes an array of strings representing C
pre-processing macro definitions (as would be used on the command-line when
invoking cpp or a C/C++ compiler) and parses the macro name and macro value,
passing those as two arguments to a user-provided callback function.
At present, the parse_macros function is not used.
* Generate core_variables.inc and domain_variables.inc using run-time macros
Previously, the 'parse' tool used macro definitions at the time it was compiled
to set the content that it writes to the core_variables.inc and
domain_variables.inc files. This has the drawback that the parse tool needs to
be recompiled any time a different core is being compiled.
With this commit, the parse tool now takes additional command-line arguments in
the form of C pre-processor macro definitions to set the values of variables
that are written to core_variables.inc and domain_variables.inc at runtime.
The following macros must now be defined with command-line arguments of the form
-D<macro>=<value> when running the parse tool:
MPAS_EXE_NAME
MPAS_GIT_VERSION
MPAS_BUILD_TARGET
MPAS_NAMELIST_SUFFIX
For example, when compiling the atmosphere core, the parse tool might be invoked
as
parse Registry.xml -DMPAS_EXE_NAME=atmosphere_model \
-DMPAS_GIT_VERSION=v8.1.0 \
-DBUILD_TARGET=gnu \
-DNAMELIST_SUFFIX=atmosphere
Accordingly, this commit also modifies the main Makefile for each core so that
the content of $(CPPFLAGS) is passed as command-line arguments to the parse
tool.
Other content written to the core_variables.inc file by the parse tool is still
based on the contents of the Registry.xml file that is supplied as the first
command-line argument.
* Implement new Makefile logic for sharing framework builds across cores
This commit updates the top-level Makefile (and also the Makefile in src/) to
enable reuse of existing builds of either the shared infrastructure or a core
when all build options in the current build are compatible with those used to
previously compile the infrastructure or core.
Previously, after compiling one core but before compiling another, an
intermediate clean step was required (or the AUTOCLEAN=true option could be
specified, which automatically executed the clean step). However, the
infrastructure code in src/framework, src/operators, and src/external is
core-independent and can be reused by multiple MPAS cores, provided build
options are the same.
Now when compiling, build options are saved to the files .build_opts.$(CORE)
and .build_opts.framework. If either of these files already exist, the build
options saved in them are compared with the current build options to determine
whether the core or the shared infrastructure needs to be recompiled.
The AUTOCLEAN=true option now cleans the core, shared infrastructure, or both as
needed.
Also in this commit, old logic in the top-level Makefile for determining whether
cleaning was necessary has been removed and the .gitignore file has been updated
to ignore the new .build_opts.* files.
* * In ./src/core_atmosphere/dynamics/mpas_atm_time_integration.F, apply subroutine
scalars_driving to nc, nifa, nwfa for regional forecasts using the cloud
microphysics option mp_thompson_aerosols.
* * In ./src/core_atmosphere, modified sourcecode so that the option mp_thompson_aerosols
works in both the "convection permitting" and "mesoscale_reference" suites:
-> in Registry.xml, removed package bl_mynn_in from the definition of nifa and nwfa,
and tend_nifa and tend_nwfa.
-> in mpas_atmphys_interface.F, expand the initialization of nifa_p and nwfa_p outside
of simply using bl_mynn. this change allows the computation of the aerosol optical
properties in mpas_atmphys_driver_radiation_sw.F with the mesoscale_reference suite.
* * In ./src/core_atmosphere/dynamics/mpas_atm_time_integration.F, corrected calls to
subroutine scalars_driving for scalars nc, nifa, and nwfa when config_apply_lbcs
is true in subroutine atm_srk3.
* * In ./src/core_atmosphere/physics/mpas_atmphys_control.F, increased the format A10 to
format A20 when checking the name of each physics parameterization.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F:
-> in subroutine init_hinterp_gocart, replaced the calls to subroutine
read_gocart_init with calls to subroutine read_met_init.
-> removed subroutine read_gocart_init.
* Parallelize the remapping of Noah-MP static fields in init_atmosphere core
Following the strategy employed for the parallel remapping of other static
fields in the init_atmosphere core, this commit reworks the code for remapping
soilcomp, soilcl1, soilcl2, soilcl3, and soilcl4 so that no special graph
partition files are needed in order to get bit-wise identical values for these
fields when using any number of MPI tasks.
The soilcomp field is handled as a multi-layer, continuous field, while
soilcl{1,2,3,4} are handled as categorical fields (although they are declared
in the Registry as "real" fields).
* * In ./src/core_init_atmosphere, removed the variables nifa_gocart_clim, nwfa_gocart_clim,
pwif_gocart_clim in the input and output streams when computing the lateral boundary
conditions when mp_thompson_aerosols is set to true (note that we get the exact same
boundary conditions as earlier):
-> in Registry.xml, removed the variables nifa_gocart_clim, nwfa_gocart_clim, and
pwif_gocart_clim in the input and output streams.
-> in mpas_init_atm_cases.F, added the timeStart timestamp in the call to subroutine
init_atm_thompson_aerosols_lbc.
-> in mpas_init_atm_thompson_aerosols.F, added a call to subroutine init_hinterp_gocart
to read the binary file QNWFA_QNIFA_SIGMA_MONTHLY.dat and interpolate the variables
nifa_gocart_clim, nwfa_gocart_clim, and pwif_gocart_clim to the MPAS mesh when the
timestamp is the same as timestart.
* * In ./src/core_atmosphere/physics:
-> in mpas_atmphys_driver_microphysics.F, commented extra print statements in subroutine
init_microphysics.
-> in mpas_atmphys_init_microphysics.F, commented extra print statements in subroutine
init_thompson_aerosols_forMPAS.
* * In ./src/core_atmosphere/physics/mpas_atmphys_control.F, replaced the MM5 surface layer
scheme option (sf_monin_obukhov) with the revised MM5 surface layer scheme option
(sf_monin_obukhov_rev) in the definition of the mesoscale_reference suite.
* Work around Noah-MP compilation issues with GNU -std=f2008 flag
At present, the Noah-MP code fails to compile with the GNU compilers using the
-std=f2008 flag, which is included in the FFLAGS set for the 'gnu' build
target:
RunoffSurfaceDynamicVicMod.F90:205:4:
151 | goto 1003
| 2
......
205 | 1003 if ( DepthYTmp <= 0.0 ) DepthYTmp = 0.0
| 1
Error: Legacy Extension: Label at (1) is not in the same block as the GOTO statement at (2)
RunoffSurfaceDynamicVicMod.F90:205:4:
175 | goto 1003
| 2
......
205 | 1003 if ( DepthYTmp <= 0.0 ) DepthYTmp = 0.0
| 1
Error: Legacy Extension: Label at (1) is not in the same block as the GOTO statement at (2)
As a work-around for this issue, this commit adds logic to the Noah-MP Makefile
to remove '-std=f2008' if it is present in FFLAGS. The modified FFLAGS are only
used when compiling Noah-MP, and so Fortran 2008 compliance is still enforced
for other code when building with the 'gnu' target.
* Add ONLY clause to USE statements for the 'mpi_f08' module
Besides being a generally good practice, the use of only those entities from the
'mpi_f08' module that are actually needed eliminates link-time issues with debug
builds using the Intel oneAPI compilers (with the Cray MPICH library).
* Fix issue in destroying halo groups when there are multiple domain instances
When there exist multiple domain instances, and atm_destroy_halo_groups and
mpas_deallocate_domain are called in a particular order for the domains, the
config_halo_exch_method module variable in the mpas_atm_halos module may be
invalid, leading to an error when destroying halo groups. This results in the
error message
ERROR: Failed to destroy halo exchange groups.
in the MPAS-Atmosphere log file.
Consider a situation in which we have two domain instances, domain_0 and
domain_1. If the atm_build_halo_groups routine and the atm_destroy_halo_groups
plus mpas_deallocate_domain routines are called in "last-in, first-out" order,
i.e.,
call atm_build_halo_groups(domain_0, ierr)
call atm_build_halo_groups(domain_1, ierr)
...
call atm_destroy_halo_groups(domain_1, ierr)
call mpas_deallocate_domain(domain_1)
call atm_destroy_halo_groups(domain_0, ierr)
call mpas_deallocate_domain(domain_0)
the module variable config_halo_exch_method would be first set to point to
memory owned by domain_0 and then re-set to point to memory owned by domain_1 in
the calls to atm_build_halo_groups. Upon calling atm_destroy_halo_groups for
domain_1 followed by a call to mpas_deallocate_domain, the
config_halo_exch_method pointer would become invalid, leading to an error when
calling mpas_destroy_halo_groups for domain_0.
Since the domain instance is passed as an argument to atm_destroy_halo_groups
routine, rather than using a module variable set when halo exchange groups were
built, we can simply retrieve config_halo_exch_method from the domain argument,
thereby avoiding the original problem.
* Improve check on deallocation of hash table in mpas_geotile_mgr_finalize
In some cases (typically with the Intel oneAPI compilers), parallel remapping
of static fields in the init_atmosphere core will fail with the message
ERROR: Problem deallocating the geotile hash table
for some MPI ranks. There is apparently a problem in deallocating the 'hash'
member of mpas_geotile_mgr_type instances in mpas_geotile_mgr_finalize.
This commit improves the checks on the deallocation of 'mgr % hash' in the
mpas_geotile_mgr_finalize routine, making them more stringent. With the
modifications to the deallocation checks, the deallocation errors no longer
occur, suggesting that they were entirely spurious.
* Update version number to 8.2.0
* Add module search paths to support compilation of Noah-MP code with nvfortran
The nvfortran compiler requires paths of modules that are used indirectly to be
present in the module search path provided with the -I... option. To support
compilation of Noah-MP code in MPAS-Atmosphere with the nvfortran compiler, this
commit (1) adds Noah-MP source directories to FCINCLUDES, and (2) adds the path
to the esmf_time_f90 code when compiling Noah-MP code.
* Add 'sf_noahmp_in' package to Noah-MP static fields in atmosphere core registry
Prior to this commit, the Noah-MP static fields 'soilcomp', 'soilcl1',
'soilcl2', 'soilcl3', and 'soilcl4' did not have any packages attached to them,
and as a result, even model simulations that didn't use the Noah-MP scheme would
attempt to read these fields from input files. If attempting to run a simulation
with the default 'mesoscale_reference' physics suite and an old (<= v8.1.0)
input file, the model run would fail when trying to inquire about the
nSoilComps dimension:
ERROR: At least one fields to be read from the 'input' stream is dimensioned
ERROR: by 'nSoilComps', but the 'nSoilComps' dimension is not defined
ERROR: in the file <input stream filename>.
CRITICAL ERROR: Please check the input file(s) to be read by the 'input' input stream.
Since these static fields are only needed by the Noah-MP scheme, this commit
attaches the existing 'sf_noahmp_in' package to these five fields so that they
will only be active, and therefore, only read, if config_lsm_scheme = 'sf_noahmp'
in the &physics namelist group.
* Add missing dependency among physics in MPAS-A main physics Makefile
In order to permit correct parallel builds of the MPAS-A physics, the
core_physics target in src/core_atmosphere/physics/Makefile should depend
not only on core_physics_wrf but also on core_physics_noahmp.
Note that core_physics_wrf depends on core_physics_mmm, and so it isn't
necessary to also list core_physics_mmm as a dependency for core_physics.
* Split physics source compilation from library updates for parallel builds
This commit splits the compilation of source files in MPAS-A physics from the
updating of libphys.a to support parallel builds of MPAS-A. Previously,
compilation of source files in physics_wrf, physics_noahmp, and physics_mmm
included updates of libphys.a with the resulting object files. When physics_wrf,
physics_noahmp, and physics_mmm are compiled in parallel, this could result in
race conditions when updating libphys.a.
Now, Makefiles have been modified so that source files in physics_wrf,
physics_noahmp, and physics_mmm can be compiled in parallel first, and as a
subsequent step, object files from these directories are added to libphys.a in
a serial fashion.
Note that since the physics_mmm directory is populated from the external
MMM-physics repository, this commit updates the MMM-physics tag to
20240626-MPASv8.2, which includes updates to the MMM-physics Makefile.mpas.
* Ignore the entire physics_mmm subdirectory of src/core_atmosphere/physics
The physics_mmm directory is populated from the external MMM-physics repository
and should be ignored in its entirety.
* Don't include FCINCLUDES in build options checked for build compatibility
With commit c7401502, the FCINCLUDES variable is modified with core-specific
options (for the atmosphere core specifically) and so including FCINCLUDES in
the set of build options that are checked for compatibility in the top-level
Makefile leads to a failed check, forcing a clean between the compilation of the
atmosphere core and other cores.
The FCINCLUDES variable contains very similar information to the CPPINCLUDES
variable, which is included in compatibility checks, and in order to enable the
atmosphere core to be compiled after compilation of another core, this commit
removes FCINCLUDES from the variables that are checked for build compatibility.
* Add 'sf_noahmp' to list of possible_values for config_lsm_scheme
This commit adds 'sf_noahmp' to the list of possible options for the
config_lsm_scheme namelist variable in the atmosphere core's Registry.xml file.
Also included in this commit is an update of the name of the 'noah' option to
'sf_noah' under possible_values for config_lsm_scheme.
* Add 'mp_thompson_aerosols' to list of possible_values for config_microp_scheme
This commit adds 'mp_thompson_aerosols' to the list of possible options for the
config_microp_scheme namelist variable in the atmosphere core's Registry.xml file.
* Fix instances of incorrect quotes in init_atmosphere and atmosphere registries
This commit fixes several instances of unbalanced or badly nested quotes in
the init_atmosphere and atmosphere Registry.xml files. These changes result
in different registry-generated code. For example, the generated code for
rnifampten attributes looked like:
call mpas_add_att(r2Ptr % attLists(1) % attList, 'long_name', 'tendency of ')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'ice-friendly"', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'aerosol', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'number', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'concentration', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'due', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'to', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'microphysics"', '')
before the changes in this commit, whereas it now looks like:
call mpas_add_att(r2Ptr % attLists(1) % attList, 'long_name', 'tendency of ice-friendly aerosol number concentration due to &
µphysics')
The latter is clearly correct while the badly nested quotes led to the former
incorrect code.
* Update comments
* Update src/core_atmosphere/physics/mpas_atmphys_driver.F
* Update src/core_atmosphere/physics/mpas_atmphys_driver.F
* fix duplicated lines and chars
* fix indentation
* Fix merge conflicts with v8.2
* Fix merge conflicts with MP driver for v8.2
* Debugging for compile
* fixing surface layer conflicts by adding a new version that doesn't conflict with MMMs version.
* Fixing more merge conflicts
* remove sf_mynn.F that prevent checkout_externals
* Remove duplicate flag line
* Add 2 idealized cases for microphysics
* Remove writes and fix subroutine name
* Refactor and bugfixes for TEMPO MP -> v8.2
* Update pointer to TEMPO
* Add TEMPO to radiation
* case select for TEMPO options
* re-implementation of the MYNN-EDMF
* Fully separate TEMPO
* Fix Registry conflict
* finishing the MYNN surface layer re-implementation and a fix ing the PBL driver
* Bug fixes
* Bug fix for non-aerosol-aware TEMPO
* fix merge error that removed ruc init
* Bug fixes and clean up for MYNN-EDMF
* Merged pull request from AndersJensen and removed 'use ccpp_kind_types' statement from line 102 of bl_ugwp.F
* Add aerosol mixing for TEMPO + MYNN
* update UGWP hash
---------
Co-authored-by: Laura Fowler <[email protected]>
Co-authored-by: Michael Duda <[email protected]>
Co-authored-by: Jim Wittig <[email protected]>
Co-authored-by: G. Dylan Dickerson <[email protected]>
Co-authored-by: Kuan-Chih Wang <[email protected]>
Co-authored-by: Andy Stokely <[email protected]>
Co-authored-by: Anders Jensen <[email protected]>
Co-authored-by: Joseph.B.Olson <[email protected]>
Co-authored-by: Michael Barlage <[email protected]>
Co-authored-by: Michael Toy <[email protected]>
* Fix memory leaks due to duplicate `allocate` calls on pointers
The `mpas_pool_create_pool` subroutine already allocates the pointer in its arguments.
Therefore, calling `allocate` before `mpas_pool_create_pool` actually causes memory leaks.
* * Made a couple of corrections in subroutines advance_scalars and atm_advance_scalars_mono_work
so that they can be called with state arrays different than "scalars":
-> in subroutine advance_scalars, modified the line:
call mpas_pool_get_dimension(state, 'num_scalars', num_scalars)
to
call mpas_pool_get_dimension(state, 'num_'//trim(field_name), num_scalars)
to expand the definition of num_scalars to state arrays different than "scalars".
-> in subroutine atm_advance_scalars_mono_work, modified the line:
call exchange_halo_group(block % domain, 'dynamics:scalars_old')
to
call exchange_halo_group(block % domain, 'dynamics:'//trim(field_name)//'_old')
so that halo exchange can be applied to state arrays different than "scalars".
* Add routine for reporting compile- and run-time settings to the log file
The new mpas_framework_report_settings routine writes information about compile-
time options and run-time settings to the log file.
A call to mpas_framework_report_settings has been added to the test core's
test_setup_log routine, so that information like the following is written to the
log file for the test core:
Output from 'git describe --dirty': v8.1.0-dirty
Compile-time options:
Build target: intel
OpenMP support: no
OpenACC support: no
Default real precision: single
Compiler flags: optimize
I/O layer: SMIOL
Run-time settings:
MPI task count: 32
* Use mpas_framework_report_settings in atm_setup_log for atmosphere core
This commit replaces code in the atmosphere core's atm_setup_log routine for
writing compile-time options and run-time settings with a single call to the
framework routine mpas_framework_report_settings.
* Use mpas_framework_report_settings in init_atm_setup_log for init_atmosphere core
This commit replaces code in the init_atmosphere core's init_atm_setup_log
routine for writing compile-time options and run-time settings with a single
call to the framework routine mpas_framework_report_settings.
* Include OpenACC device info in mpas_framework_report_settings output
If MPAS is compiled with OPENACC=true, the mpas_framework_report_settings
routine now calls a new private routine, report_acc_devices, in mpas_framework
to report information about OpenACC devices to the log. The report_acc_devices
routine uses the OpenACC runtime library to inquire about the number of
available devices, the device number for the calling MPI task, and info about
the device type and driver. As an example from Derecho with the NVHPC 24.3
compilers, the following information is reported in the log file:
OpenACC configuration:
Number of visible devices: 1
Device # for this MPI task: 0
Device vendor: NVIDIA
Device name: NVIDIA A100-SXM4-40GB
Device driver version: 12020
If MPAS is not compiled with OPENACC=true, then the report_acc_devices routine,
and the call to it from mpas_framework_report_settings, are pre-processed out.
* Add "-check all" to LDFLAGS_DEBUG for 'intel' build target
Regarding the '-check all' option, the Intel 2024.1 manual states:
Caution
Files compiled with option check all should also be
linked with this same option, or the link step may fail.
* Move 21 additional fields into the "invariant" stream in the atmosphere core
This commit moves 21 additional time-invariant fields from the "input" and
"restart" streams into the "invariant" stream for the atmosphere core. These
additional fields are time-invariant terrestrial fields that are used by various
physics parameterization schemes, and as they were in the "input" and "restart"
streams, they are conditionally included in the "invariant" stream with the
DO_PHYSICS preprocessing macro.
Specifically, the 21 fields now included in the "invariant" stream are:
isltyp
ivgtyp
mminlu
isice_lu
iswater_lu
landmask
shdmin
shdmax
snoalb
albedo12m
greenfrac
var2d
con
oa1
oa2
oa3
oa4
ol1
ol2
ol3
ol4
* Add stream_function and velocity_potential fields to the diag struct.
* Add da_state input/output stream.
This stream is only used by MPAS-JEDI.
This stream is associated with the "jedi_da" package, and so it will only be active if config_jedi_da = true.
* * In ./src/core_atmosphere/physics/physics_mmm, changes all the file extension .F to .F90.
Modified Makefile accordingly.
* * In ./src/core_atmosphere/physics/physics_mmm, updated bl_gwdo.F90 with additional
CCPP compliance requirements.
* * In ./src/core_atmosphere/physics/physics_mmm, updated bl_ysu.F90 with additional
CCPP compliance requirements.
Modified ./src/core_atmosphere/physics/physics_wrf/module_bl_ysu.F to accomodate
changes in ./physics_mmm/bl_ysu.F90.
* * In ./src/core_atmosphere/physics, updated ./physics_mmm/cu_ntiedtke.F90 and
./physics_wrf/module_cu_ntiedtke.F.
* * In ./src/core_atmosphere/physics/physics_wrf, finished to update module_cu_ntiedtke.F.
Added modules cu_ntiedtke_pre.F and cu_ntiedtke_post.F to comply with CCPP requirements,
and to initialize and finalize variables used in ./physics_mmm/cu_ntiedtke.F90. Modified
Makefile accordingly.
* * In ./src/core_atmosphere/physics:
-> in physics_mmm/sf_sfclayrev.F90, updated the revised surface layer scheme.
-> in physics_wrf, updated module_sf_sfclayrev.F and added the new module
sf_sfclayrev_pre.F. Modified aMakefile accordingly.
-> corrected the call to to sfclayrev.
* * In ./src/core_atmosphere/physics/physics_mmm, updated bl_mynn.F90 with CCPP-compliance
requirements for declarations of arrays with intent "in", "inout", and "out".
* * In ./src/core_atmosphere/physics/physics_mmm/cu_ntiedtke.F90, added doxygen comments
in subroutine cu_ntiedtke_run.
* * In ./src/core_atmosphere/physics/physics_mmm, updated mp_wsm6.F90 with CCPP-compliance
requirements for declarations of arrays with intent "in", "inout", and "out".
* * In ./src/core_atmosphere/physics:,
-> renamed module ccpp_kinds.F to ccpp_kind_types.F.
-> in ./physics_wrf, changed "use ccpp_kinds" to "use ccpp_kind_types" in modules
bl_mynn_post.F, bl_mynn_pre.F, cu_ntiedtke_post.F, sf_mynn_pre.F, sf_sfclayrev_pre.
-> in ./physics_mmm, changed "use ccpp_kinds" to "use ccpp_kind_types" in modules
bl_gwdo.F90, cu_ntiedtke.F90, mp_radar.F90, mp_wsm6.F90, sf_sfclayrev.F90, and
sf_mynn.F90.
-> in ./physics_mmm, changed "use mpas_kind_types" to "use ccpp_kind_types" in
modules bl_mynn.F90, bl_mynn_subroutines.F90, and mynn_shared.F90.
-> in ./physics_mmm, also updated mp_wsm6_effectRad.F90, with CCPP-compliance
requirements for declarations of arrays with intent "in", "inout", and "out".
-> Modified Makefile accordingly.
* * As an intermediary step to moving ./physics_mmm to its own repository and then removing
./physics_mmm/Makefile which is currently specific to MPAS, we first renamed Makefile
to Makefile.mpas. We modified ./physics/Makefile accordingly.
* Remove Fortran only flag from LDFLAGS_DEBUG for NV targets
The -Mchkptr flag is only valid for Fortran compilation. If included in
a C build with NVHPC compilers (like the check with openacc.c does), the
build fails.
* Add optional CMake build for atmosphere and init_atmosphere core of MPAS-Model
Introduces an optional CMake build for the atmosphere and init_atmosphere core
of the MPAS-Model, designed to improve support for MPAS-JEDI which employs
ecbuild/CMake. The intention is to enhance compatibility and facilitate
integration with MPAS-JEDI, while retaining the default build method using GNU
make. The CMake build addition is based on contributions from the NSF NCAR
Mesoscale and Microscale Meteorology (MMM) Laboratory and the Joint Center for
Satellite Data Assimilation (JCSDA).
The CMake build does not support SMIOL and requires PIO2. SMIOL support for the
CMake build is planned for future updates. Outside of the PIO2 requirement, the
only additional dependency for the CMake build is CMake itself.
To build with CMake, make sure CMake, netcdf-c, netcdf-fortran, parallel-netcdf,
and parallelio are installed and discoverable. If using a module package
environment, load the modules for the above libraries. Create a build directory
and enter it: mkdir <build_dir>; cd <build_dir>. Configure the build with
CMake by running cmake <mpas_src_dir> <cmake_flags>, where mpas_src_dir is
the MPAS source directory and cmake_flags is a string of flags used to set
various build options.
CMAKE_BUILD_TYPE: Specifies the build type. Possible values are Release,
Debug, and RelWithDebInfo. Defaults to Release.
MPAS_DOUBLE_PRECISION: Controls if MPAS is built with single or double
precision. Possible values are ON and OFF. Defaults to ON.
MPAS_CORES: Controls which MPAS Cores are built. Possible values are atmosphere,
init_atmosphere atmosphere, and init_atmosphere.
Next, run make -j<number_of_jobs> where number_of_jobs is the number of jobs
GNU Make will execute in parallel.
* Copy invariant fields used by scalar transport via OpenACC data directives
This commit adds code and OpenACC directives in the MPAS_atm_dynamics_init
routine to transfer invariant fields used by scalar transport code to the
device, and it adds code and directives in the MPAS_atm_dynamics_finalize
routine to delete these invariant fields.
The new code and directives are surrounded by tests on the MPAS_OPENACC
preprocessing macro, and so the changes in this commit have no effect unless
MPAS-A is compiled with OPENACC=true.
Since the MPAS_atm_dynamics_init routine is called once during model startup and
the MPAS_atm_dynamics_finalize once during model shutdown, the invariant fields
will remain resident on the device across all dynamics time steps.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_pbl.F, commented print
statements in subroutine init_pbl.
* * In ./src/core_init_atmosphere:
-> added the module mpas_init_atm_thompson_aerosols.F to interpolate climatological
monthly-mean Gocart-based water-friendly and ice-friendly aerosols for use with
the aerosol-aware Thompson cloud microphysics scheme.
-> in Registry.xml, added climatological, and first-guess water- and ice-friendly
aerosols for use with the aerosol-aware Thompson cloud microphysics scheme.
-> in mpas_init_atm_cases.F, added call to initialize first-guess water- and ice-
friendly aerosols.
-> modified Makefile accordingly.
* * In ./src/core_atmosphere/physics/physics_wrf, updated module_mp_thompson.F to WRF-v4.1.4.
* * In ./src/core_atmosphere/physics, started the implementation of the aerosol-aware
option of the Thompson cloud microphysics parameterization:
-> in mpas_atmphys_control.F, added the option mp_thompson_aerosols in subroutine
physics_namelist_check to trigger the aerosol-aware option of the microphysics
scheme.
-> in mpas_atmphys_packages.F, added the option mp_thompson_aerosols to activate
the package mp_thompson_in in subroutine atmphys_setup_packages.
-> in mpas_atmphys_manager.F, added the option mp_thompson_aerosols when initializing
the microphysics timestep dt_microp, and to activate the computation of effective
radii in subroutine physics_run_init. also edited a few print statements.
* * In ./src/core_atmosphere/Registry.xml, ./src/core_atmosphere/dynamics/mpas_atm_time_integration.F,
and ./src/core_atmosphere/physics, finished the implementation of the Thompson cloud microphysics
parameterization:
-> in Registry.xml, added the prognostic variables and associated tendencies to run the aerosol-
aware option, i.e. nifa, nwfa, and nc. also added surface emissions of nwfa (nwfa2d) and nifa
(nifa2d).
-> in mpas_atm_time_integration.F, added tend_physics to the argument list of the call to subroutine
driver_microphysics to compute cloud microphysics tendencies.
-> in all other modules, modified sourcecode to run the aerosol-aware option of the thompson cloud
microphysics scheme when we use the option config_microp_scheme = "mp_thompson_aerosols".
* * In ./src/core_atmosphere/physics, added the option "mp_thompson_aerosols" to compute
the radiative effective radii of cloud liquid water, cloud ice, and snow in modules
mpas_atmphys_driver_radiation_lw.F and mpas_atmphys_driver_radiation_sw.F.
* * In ./src/core_atmosphere/physics, cleaned-up mpas_atmphys_todynamics.F.
* * In ./src/core_atmosphere/physics, added the PBL and free-tropospheric mixing of
the cloud liquid water number concentration, and water-friendly and ice-friendly
Thompson_aerosols when the option mp_thompson_aerosols is set to true.
* * In ./src/core_atmosphere/physics, added the RRTMG shortwave aerosol optical properties of
the Thompson water-friendly and ice-friendly aerosols.
-> in ./physics_wrf, added module_mp_thompson_aerosols.F and module_ra_rrtmg_sw_aerosols.F
to compute the RRTMG aerosol optical properties (AOPs)of the Thompson aerosols. added the
aerosol effects in module_ra_rrtmg_sw.F.
-> in mpas_atmphys_vars.F, added local variables of the AOPs of the Thompson aerosols.
-> in mpas_atmphys_driver_radiation_sw.F, added calls to subroutine gt_aod and subroutine
calc_aerosol_rrtmg_sw to compute the AOPs of the Thompson aerosols.
* * In ./src/core_atmosphere/physics/mpas_atmphys_finalize, added the option
mp_thompson_aerosols in subroutine atmphys_finalize. Ths option is used
to deallocate all arrays used in the "MP_THOMPSON" tables.
* * In src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, corrected the
initialization of the array sorted_arr in the vertical interpolation of nifa
and nwfa in subroutine init_vinterp_gocart.
* * In ./src/core_atmosphere/physics/mpas_atmphys_init_microphysics.F, added the
computation of the surface emission of water friendly aerosols (nwfa2d).
* Add OpenACC directives throughout atm_advance_scalars_work
This commit adds OpenACC directives throughout the atm_advance_scalars_work
routine.
Also as part of this commit, macros have been defined at the top of
mpas_atm_time_integration.F to activate timers for OpenACC data transfers when
MPAS-A is compiled with OPENACC=true. For the data transfers in the
atm_advance_scalars_work routine, the new timer
atm_advance_scalars [ACC_data_xfer]
will be written to the timer summary in log files.
With the NVHPC 24.3 compilers and flags provided in the 'nvhpc' target, results
for a regional test case are bitwise identical when the original code and code
in this commit are both compiled with OPENACC=true.
* Add OpenACC directives throughout atm_advance_scalars_mono_work
This commit adds OpenACC directives throughout the atm_advance_scalars_mono_work
routine. At present, halo exchanges in the monotonic scalar transport routine
require fields to be transferred to the host before the halo exchange, and then
back to the device afterward.
For the data transfers in the atm_advance_scalars_mono_work routine, the new
timer
atm_advance_scalars_mono [ACC_data_xfer]
will be written to the timer summary in log files.
With the NVHPC 24.3 compilers and flags provided in the 'nvhpc' target, results
for a regional test case are bitwise identical when the original code and code
in this commit are both compiled with OPENACC=true.
* * Initial implementation of the Noah-MP land surface scheme (LSM):
The Noah-MP LSM sourcecode is copied from the Noah-MP github repository and corresponds
to release-v5.0.1 (at hash 12a0b51aa52f6f5189525009ff249dfcbc95d29a), i.e.:
commit 12a0b51aa52f6f5189525009ff249dfcbc95d29a
Merge: 9fd80cb 843a742
Author: Cenlin_He <[email protected]>
Date: Thu Jan 4 20:21:19 2024 -0700
Merge pull request #108 from NCAR/develop
* * In ./src/core_atmosphere/physics and ./src/core_atmosphere/physics/physics_noahmp,
modified sourcecode to compile the original Noah-MP sourcecode inside the MPAS
framework:
-> in ./physics_noahmp/utility, modified Machine.F90 to become dependent to
mpas_kind_types.F.
-> in ./physics_noahmp/utility, ./physics_noahmp/src, and ./physics_noahmp/drivers/hrldas,
modified Makefile to compile Noah-MP inside the MPAS framework.
-> in src/core_atmosphere/physics/physics_noahmp/drivers/hrldas/NoahmpDriverMainMod.F90,
removed the WRF modules module_ra_gfdleta and module_data_gocart_dust.
-> modified Makefile to compile all physics_noahmp subdirectories.
* * In ./src/core_atmosphere/Makefile, now link ./physics/physics_noahmp/parameters/NoahmpTable.TBL
as we do for all the files in ./physics/physics_wrf. NoahmpTable.TBL is need to run the Noah-MP
land surface scheme.
* * In ./src/core_atmosphere/physics:
-> in /physics_noahmp/drivers, renamed the directory hrldas to mpas.
-> in Makefile, changed all instance of hrldas to mpas.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas, removed all instances of the J dimension
and all references to WRF-related dimensions.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas, added NoahmpIOVarFinalizeMod.F90
to deallocate all arrays first allocated in NoahmpIOVarInitMod.F90. Modified Makefile accordingly.
* * In ./src/core_init_atmosphere, added the initialization of the static variables
soilcomp, soilcl1, soilcl2, soilcl3, and soilcl4 needed to run the Noah-MP land
surface scheme.
* * In ./src/core_atmosphere, starting the first modifications to run the Noah-MP
land surface scheme:
-> added the Noah-MP Registry file ./physics/Registry_noahmp.xml to Registry.xml.
That file includes all the extra arrays to run Noah-MP.
-> in ./physics/mpas_atmphys_control.F, added the option sf_noahmp as an alternate
to sf_noah.
-> in ./physics/mpas_atmphys_manager.F, added the outputs month and day as outputs
to mpas_get_time. month and day are needed in ./physics_noahmp.
-> in ./physics/mpas_atmphys_packages, added the package sf_noahmp_in.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas/NoahmpDriverMainMod.F90,
added print statements for NoahmpIO%itimestep and NoahmpIO%xice_threshold.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas/NoahmpInitMainMod.F90,
remove the initialization of NoahmpIO%areaxy.
* * In ./src/core_atmosphere/physics/physics_noahmp/drivers/mpas/NoahmpIOVarFinalizeMod.F90,
removed the sourcecode that initializes NoahmpIO variables since NoahmpIOVarFinalizeMod.F90
is only called to deallocate variables at the end of forecasts.
* * In ./src/core_atmosphere:
-> in Registry.xml, added Noahmp variables to the input and restart streams to run
the Noah-MP land surface scheme.
-> added ./physics/Registry_noahmp.xml which contains all the Noah-MP variables
needed to run the Noah-MP land surface scheme.
* Add module to exercise OpenACC to the test core
Right now this just contains a stub routine, but this gets some of the
"plumbing work" done.
* Add mpas_test_openacc to build and test_core_run
Guard the call to mpas_test_openacc routine with ifdef MPAS_OPENACC so
the routines are only run if the test_core is built with OpenACC
enabled.
* Add new variables to test_core Registry for OpenACC tests
* Add representative array routines to OpenACC tests
This test carries out some calculations on the CPU and the GPU based on
patterns in the dynamics of core_atmosphere. If the difference between
the CPU results and the GPU results is 0, the test succeeds.
* Remove OpenACC stub test, fix a comment
* * In src/core_atmosphere/physics/checkout_data_files.sh, updated mpas_vers from 8.0 to
8.2 to access the latest version of the MPAS-Data repository. When mpas_vers="8.2",
the file CCN_ACTIVATE_DATA needed to run the aerosol-aware Thompson cloud microphysics
scheme is downloaded to /physics/physics_wrf/files.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, commented extra
print statements at the beginning and end of each subroutine to clarify the logfile.
* * In ./src/core_atmosphere/physics/physics_wrf/module_ra_rrtmg_sw_aerosols.F:
-> in subroutine calc_aerosol_rrtmg_sw, moved the declaration of dimensions ims,ime,...
above the declaration of variables p,t3d,dz8w,qv3d.
-> this allows module_ra_rrtmg_sw_aerosols.F to be compiled with gcc/12.2.0.
-> commented extra print statements related to the definition of the AOPs.
* * In src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, corrected typos
at lines 697 and 700: replaced "\" with "&".
* * In ./src/core_atmosphere/Registry.xml, corrected the description of package
mp_thompson_aers_in. mp_thompson_aers_in refers to the aerosol-aware version
of the Thompson cloud microphysics scheme, NOT the scale-aware version.
* * In ./src/core_atmosphere, added the subdirectory "tools". At present, tools include the
subdirectory manage_externals.
manage_externals is part of the Earth System Model Computational Infrastructure (ESMCI)
and is a management utility used to check out groups of externals from revision control
based on an externals description file (Externals.cfg).
the master repository for manage_externals is https://github.com/ESMCI/manage_externals.
we checked out manage_externals at the hash:
commit f545b7ba8341e1da79dd0550c6a800f4972c8531
Merge: 876b344 2c4174b
Author: Jim Edwards <[email protected]>
Date: Tue Jan 2 06:19:40 2024 -0700
Merge pull request #210 from jedwards4b/defer_svn_access_error
defer svn access to github error to actual checkout point, this will …
* * In ./src/core_atmosphere/physics,removed the directory physics_mmm.
* * In ./src/core_atmosphere/physics:
-> Modified Makefile to compile the physics_mmm directory accessed from the
github repository https://github.com/NCAR/MMM-physics.
-> Modified .gitignore to ignore the files LICENSE and README.md in physics_mmm.
* * In ./src/core_atmosphere, added the file Externals.cfg. This file is used
to access the github repository https://github.com/NCAR/MMM-physics and
download and compiled the shared physics directory ./physics/physics_mmm.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, renamed
"QNWFA_QNIFA_Monthly_Sigma.dat" to "QNWFA_QNIFA_SIGMA_MONTHLY.dat".
* * In ./src/core_init_atmosphere, do not include the Thompson water- and ice-friendly
aerosols in the initial conditions and lateral boundary conditions files when the
when the GOCART input "QNWFA_QNIFA_SIGMA_MONTHLY.dat" is not present in the run
directory.
-> In Registry.xml, added package mp_thompson_aers_in.
-> In mpas_init_atm_core_interface.F, added the initialization of mp_thompson_aers_in.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, added subroutine
init_atm_thompson_aerosols_lbc to compute the lateral boundary conditions for the
water-friendly and ice-friendly aerosols in the aerosol-aware Thompson cloud
microphysics.
The lateral boundary conditions for the water-friendly and ice-friendly aerosols
are computing using the monthly mean aerosol climatology interpolated to the MPAS
mesh and available in the regional initial conditions.
* When running the 'parse' tool, provide Registry file as command-line argument
The 'parse' tool for generating code based on a Registry file can either read
the Registry from stdin or from a file whose name is supplied as a command-line
argument. This tool assumes that if its command-line argument count is 2, then
the argument is a Registry filename (the argument count is 1 if no command-line
arguments are given).
Since future development will add processing for an arbitrary number of
command-line arguments to the parse tool, deciding whether to read the Registry
from stdin based on the argument count is not a viable strategy, so with this
commit, each core's Makefile now passes a filename as the only argument to the
parse tool, and the parse tool itself has been modified to require a
command-line argument.
* * In ./src/core_init_atmosphere, continued the implementation of the lateral boundary
conditions for nc, nifa, and nwfa.
-> in Registry.xml, added the fields nifa_gocart_clim, nwfa_gocart_clim, and
pwif_gocart_clim to the output stream so that they can be read later from
the initial conditions file when building the lateral boundary conditions.
-> in Registry.xml, added the fields nofa_gocart_clim, nwfa_gocart_clim, and
pwif_gocart_clim to the input stream so that they can be read using the
initial conditions file when building the lateral boundary conditions.
-> in mpas_init_atm_core_interface.F, revised the initialization of package
mp_thompson_aerosols.
-> in mpas_init_atm_cases.F, add call to subroutine init_atm_thompson_aerosol_lbc
to build the lateral boundary conditions lbc_nc, lbc_nifa, and lbc_nwfa after
call to subroutine init_atm_case_lbc.
* * In ./src/core_atmosphere/physics:
-> added mpas_atmphys_lsm_noahmpinit.F to initialize Noah-MP variables within
the MPAS framework.
-> added mpas_atmphys_driver_lsm_noahmp.F to run the Noah-MP land surface scheme
within the MPAS framework.
* * In ./src/core_atmosphere/physics:
-> in mpas_atmphys_vars.F, added the structure mpas_noahmp of type NoahmIO_type
that contains all the Noah-MP variables and arrays needed for Noah-MP to
communicate with the MPAS physics.
-> in mpas_atmphys_init.F, added call to init_lsm_noahmp to initialize Noah-MP.
-> in mpas_atmphys_driver.F, added call to driver_lsm_noahmp to run Noah-MP.
-> Modified Makefile accordingly.
-> in ./../mpas_atm_core.F, modified call to physics_init to include the pools
diag_physics_noahmp and output_noahmp.
* * In ./src/core_atmosphere/physics/Registry_noahmp.xml, added the argument missing_value
to the definition of Noah-MP variables. missing_value is set to -9999 for integer and
-9999.0 to real variables, as defined in Machine.F90 used in Noah-MP.
* * In ./src/core_atmosphere/physics/mpas_atmphys_lsm_noahmpinit.F, removed variables needed in
the crop model. The crop model is currently not available and related variables are not
used.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_lsm_noahmp.F, removed variables needed
in the ground water parameterization.The ground water model is currently not available and
related variables are not used.
* * In src/core_atmosphere/physics/physics_noahmp/drivers/mpas:
-> in NoahmpInitMod.F90, removed the initialization of xland, xice, and
xice_threshold, and move all three variables to ./physics.
-> in NoahmpDriverMainMod.F90, rearranged print statements.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_lsm_noahmp.F, added the
initialization of xland to subroutine lsm_noahmp_fromMPAS.
* * In Makefile, removed the option "-fp-model precise" for the ifort option.
* * In ./src/core_atmosphere/Registry.xml:
-> moved the variables soilcomp, soilcl1, soilcl2, and soilcl3 from the input stream
to the invariant stream.
-> added the variable soilcl4 to the input stream.
* * In ./src/core_atmosphere/physics:
-> in ./physics_noahmp/drivers/mpas/NoahmpDriverMainMod.F90, commented extra
print statements.
-> in mpas_atmphys_driver_lsm_noahmp.F, commented extra print statements.
* * In src/core_atmosphere/Registry.xml, removed the static fields needed to run Noah-MP
from the restart stream since they are available in the "invariant" stream and the
invariant stream is included in the "restart" stream.
* * In ./src/core_atmosphere/physics/mpas_atmphys_lsm_noahmpinit.F, commented extra
print statements originally written during the initialization of Noah-MP.
* * In ./src/core_atmosphere/physics/Registry_noahmp.xml, corrected the description
of a few Noah-MP variables.
* * In ./src/core_atmosphere/physics/mpas_atmphys_lsm_noahmpinit.F, removed extra
print statements before and after call to subroutine NoahmpInitMain.
* * In ./src/core_atmosphere/physics, deallocate all Noah-MP arrays needed to run
the Noah-MP land surface scheme.
* Change URL for MMM-physics external to use https rather than git
So that the checkout of the MMM-physics repository will work even for those
who don't have SSH keys set up on GitHub, this commit changes the URL for
the MMM-physics external to use https rather than git.
* Add new function 'parse_macros' to registry tool utility code
The parse_macros function takes an array of strings representing C
pre-processing macro definitions (as would be used on the command-line when
invoking cpp or a C/C++ compiler) and parses the macro name and macro value,
passing those as two arguments to a user-provided callback function.
At present, the parse_macros function is not used.
* Generate core_variables.inc and domain_variables.inc using run-time macros
Previously, the 'parse' tool used macro definitions at the time it was compiled
to set the content that it writes to the core_variables.inc and
domain_variables.inc files. This has the drawback that the parse tool needs to
be recompiled any time a different core is being compiled.
With this commit, the parse tool now takes additional command-line arguments in
the form of C pre-processor macro definitions to set the values of variables
that are written to core_variables.inc and domain_variables.inc at runtime.
The following macros must now be defined with command-line arguments of the form
-D<macro>=<value> when running the parse tool:
MPAS_EXE_NAME
MPAS_GIT_VERSION
MPAS_BUILD_TARGET
MPAS_NAMELIST_SUFFIX
For example, when compiling the atmosphere core, the parse tool might be invoked
as
parse Registry.xml -DMPAS_EXE_NAME=atmosphere_model \
-DMPAS_GIT_VERSION=v8.1.0 \
-DBUILD_TARGET=gnu \
-DNAMELIST_SUFFIX=atmosphere
Accordingly, this commit also modifies the main Makefile for each core so that
the content of $(CPPFLAGS) is passed as command-line arguments to the parse
tool.
Other content written to the core_variables.inc file by the parse tool is still
based on the contents of the Registry.xml file that is supplied as the first
command-line argument.
* Implement new Makefile logic for sharing framework builds across cores
This commit updates the top-level Makefile (and also the Makefile in src/) to
enable reuse of existing builds of either the shared infrastructure or a core
when all build options in the current build are compatible with those used to
previously compile the infrastructure or core.
Previously, after compiling one core but before compiling another, an
intermediate clean step was required (or the AUTOCLEAN=true option could be
specified, which automatically executed the clean step). However, the
infrastructure code in src/framework, src/operators, and src/external is
core-independent and can be reused by multiple MPAS cores, provided build
options are the same.
Now when compiling, build options are saved to the files .build_opts.$(CORE)
and .build_opts.framework. If either of these files already exist, the build
options saved in them are compared with the current build options to determine
whether the core or the shared infrastructure needs to be recompiled.
The AUTOCLEAN=true option now cleans the core, shared infrastructure, or both as
needed.
Also in this commit, old logic in the top-level Makefile for determining whether
cleaning was necessary has been removed and the .gitignore file has been updated
to ignore the new .build_opts.* files.
* * In ./src/core_atmosphere/dynamics/mpas_atm_time_integration.F, apply subroutine
scalars_driving to nc, nifa, nwfa for regional forecasts using the cloud
microphysics option mp_thompson_aerosols.
* * In ./src/core_atmosphere, modified sourcecode so that the option mp_thompson_aerosols
works in both the "convection permitting" and "mesoscale_reference" suites:
-> in Registry.xml, removed package bl_mynn_in from the definition of nifa and nwfa,
and tend_nifa and tend_nwfa.
-> in mpas_atmphys_interface.F, expand the initialization of nifa_p and nwfa_p outside
of simply using bl_mynn. this change allows the computation of the aerosol optical
properties in mpas_atmphys_driver_radiation_sw.F with the mesoscale_reference suite.
* * In ./src/core_atmosphere/dynamics/mpas_atm_time_integration.F, corrected calls to
subroutine scalars_driving for scalars nc, nifa, and nwfa when config_apply_lbcs
is true in subroutine atm_srk3.
* * In ./src/core_atmosphere/physics/mpas_atmphys_control.F, increased the format A10 to
format A20 when checking the name of each physics parameterization.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F:
-> in subroutine init_hinterp_gocart, replaced the calls to subroutine
read_gocart_init with calls to subroutine read_met_init.
-> removed subroutine read_gocart_init.
* Parallelize the remapping of Noah-MP static fields in init_atmosphere core
Following the strategy employed for the parallel remapping of other static
fields in the init_atmosphere core, this commit reworks the code for remapping
soilcomp, soilcl1, soilcl2, soilcl3, and soilcl4 so that no special graph
partition files are needed in order to get bit-wise identical values for these
fields when using any number of MPI tasks.
The soilcomp field is handled as a multi-layer, continuous field, while
soilcl{1,2,3,4} are handled as categorical fields (although they are declared
in the Registry as "real" fields).
* * In ./src/core_init_atmosphere, removed the variables nifa_gocart_clim, nwfa_gocart_clim,
pwif_gocart_clim in the input and output streams when computing the lateral boundary
conditions when mp_thompson_aerosols is set to true (note that we get the exact same
boundary conditions as earlier):
-> in Registry.xml, removed the variables nifa_gocart_clim, nwfa_gocart_clim, and
pwif_gocart_clim in the input and output streams.
-> in mpas_init_atm_cases.F, added the timeStart timestamp in the call to subroutine
init_atm_thompson_aerosols_lbc.
-> in mpas_init_atm_thompson_aerosols.F, added a call to subroutine init_hinterp_gocart
to read the binary file QNWFA_QNIFA_SIGMA_MONTHLY.dat and interpolate the variables
nifa_gocart_clim, nwfa_gocart_clim, and pwif_gocart_clim to the MPAS mesh when the
timestamp is the same as timestart.
* * In ./src/core_atmosphere/physics:
-> in mpas_atmphys_driver_microphysics.F, commented extra print statements in subroutine
init_microphysics.
-> in mpas_atmphys_init_microphysics.F, commented extra print statements in subroutine
init_thompson_aerosols_forMPAS.
* * In ./src/core_atmosphere/physics/mpas_atmphys_control.F, replaced the MM5 surface layer
scheme option (sf_monin_obukhov) with the revised MM5 surface layer scheme option
(sf_monin_obukhov_rev) in the definition of the mesoscale_reference suite.
* Work around Noah-MP compilation issues with GNU -std=f2008 flag
At present, the Noah-MP code fails to compile with the GNU compilers using the
-std=f2008 flag, which is included in the FFLAGS set for the 'gnu' build
target:
RunoffSurfaceDynamicVicMod.F90:205:4:
151 | goto 1003
| 2
......
205 | 1003 if ( DepthYTmp <= 0.0 ) DepthYTmp = 0.0
| 1
Error: Legacy Extension: Label at (1) is not in the same block as the GOTO statement at (2)
RunoffSurfaceDynamicVicMod.F90:205:4:
175 | goto 1003
| 2
......
205 | 1003 if ( DepthYTmp <= 0.0 ) DepthYTmp = 0.0
| 1
Error: Legacy Extension: Label at (1) is not in the same block as the GOTO statement at (2)
As a work-around for this issue, this commit adds logic to the Noah-MP Makefile
to remove '-std=f2008' if it is present in FFLAGS. The modified FFLAGS are only
used when compiling Noah-MP, and so Fortran 2008 compliance is still enforced
for other code when building with the 'gnu' target.
* Add ONLY clause to USE statements for the 'mpi_f08' module
Besides being a generally good practice, the use of only those entities from the
'mpi_f08' module that are actually needed eliminates link-time issues with debug
builds using the Intel oneAPI compilers (with the Cray MPICH library).
* Fix issue in destroying halo groups when there are multiple domain instances
When there exist multiple domain instances, and atm_destroy_halo_groups and
mpas_deallocate_domain are called in a particular order for the domains, the
config_halo_exch_method module variable in the mpas_atm_halos module may be
invalid, leading to an error when destroying halo groups. This results in the
error message
ERROR: Failed to destroy halo exchange groups.
in the MPAS-Atmosphere log file.
Consider a situation in which we have two domain instances, domain_0 and
domain_1. If the atm_build_halo_groups routine and the atm_destroy_halo_groups
plus mpas_deallocate_domain routines are called in "last-in, first-out" order,
i.e.,
call atm_build_halo_groups(domain_0, ierr)
call atm_build_halo_groups(domain_1, ierr)
...
call atm_destroy_halo_groups(domain_1, ierr)
call mpas_deallocate_domain(domain_1)
call atm_destroy_halo_groups(domain_0, ierr)
call mpas_deallocate_domain(domain_0)
the module variable config_halo_exch_method would be first set to point to
memory owned by domain_0 and then re-set to point to memory owned by domain_1 in
the calls to atm_build_halo_groups. Upon calling atm_destroy_halo_groups for
domain_1 followed by a call to mpas_deallocate_domain, the
config_halo_exch_method pointer would become invalid, leading to an error when
calling mpas_destroy_halo_groups for domain_0.
Since the domain instance is passed as an argument to atm_destroy_halo_groups
routine, rather than using a module variable set when halo exchange groups were
built, we can simply retrieve config_halo_exch_method from the domain argument,
thereby avoiding the original problem.
* Improve check on deallocation of hash table in mpas_geotile_mgr_finalize
In some cases (typically with the Intel oneAPI compilers), parallel remapping
of static fields in the init_atmosphere core will fail with the message
ERROR: Problem deallocating the geotile hash table
for some MPI ranks. There is apparently a problem in deallocating the 'hash'
member of mpas_geotile_mgr_type instances in mpas_geotile_mgr_finalize.
This commit improves the checks on the deallocation of 'mgr % hash' in the
mpas_geotile_mgr_finalize routine, making them more stringent. With the
modifications to the deallocation checks, the deallocation errors no longer
occur, suggesting that they were entirely spurious.
* Update version number to 8.2.0
* Add module search paths to support compilation of Noah-MP code with nvfortran
The nvfortran compiler requires paths of modules that are used indirectly to be
present in the module search path provided with the -I... option. To support
compilation of Noah-MP code in MPAS-Atmosphere with the nvfortran compiler, this
commit (1) adds Noah-MP source directories to FCINCLUDES, and (2) adds the path
to the esmf_time_f90 code when compiling Noah-MP code.
* Add 'sf_noahmp_in' package to Noah-MP static fields in atmosphere core registry
Prior to this commit, the Noah-MP static fields 'soilcomp', 'soilcl1',
'soilcl2', 'soilcl3', and 'soilcl4' did not have any packages attached to them,
and as a result, even model simulations that didn't use the Noah-MP scheme would
attempt to read these fields from input files. If attempting to run a simulation
with the default 'mesoscale_reference' physics suite and an old (<= v8.1.0)
input file, the model run would fail when trying to inquire about the
nSoilComps dimension:
ERROR: At least one fields to be read from the 'input' stream is dimensioned
ERROR: by 'nSoilComps', but the 'nSoilComps' dimension is not defined
ERROR: in the file <input stream filename>.
CRITICAL ERROR: Please check the input file(s) to be read by the 'input' input stream.
Since these static fields are only needed by the Noah-MP scheme, this commit
attaches the existing 'sf_noahmp_in' package to these five fields so that they
will only be active, and therefore, only read, if config_lsm_scheme = 'sf_noahmp'
in the &physics namelist group.
* Add missing dependency among physics in MPAS-A main physics Makefile
In order to permit correct parallel builds of the MPAS-A physics, the
core_physics target in src/core_atmosphere/physics/Makefile should depend
not only on core_physics_wrf but also on core_physics_noahmp.
Note that core_physics_wrf depends on core_physics_mmm, and so it isn't
necessary to also list core_physics_mmm as a dependency for core_physics.
* Split physics source compilation from library updates for parallel builds
This commit splits the compilation of source files in MPAS-A physics from the
updating of libphys.a to support parallel builds of MPAS-A. Previously,
compilation of source files in physics_wrf, physics_noahmp, and physics_mmm
included updates of libphys.a with the resulting object files. When physics_wrf,
physics_noahmp, and physics_mmm are compiled in parallel, this could result in
race conditions when updating libphys.a.
Now, Makefiles have been modified so that source files in physics_wrf,
physics_noahmp, and physics_mmm can be compiled in parallel first, and as a
subsequent step, object files from these directories are added to libphys.a in
a serial fashion.
Note that since the physics_mmm directory is populated from the external
MMM-physics repository, this commit updates the MMM-physics tag to
20240626-MPASv8.2, which includes updates to the MMM-physics Makefile.mpas.
* Ignore the entire physics_mmm subdirectory of src/core_atmosphere/physics
The physics_mmm directory is populated from the external MMM-physics repository
and should be ignored in its entirety.
* Don't include FCINCLUDES in build options checked for build compatibility
With commit c7401502, the FCINCLUDES variable is modified with core-specific
options (for the atmosphere core specifically) and so including FCINCLUDES in
the set of build options that are checked for compatibility in the top-level
Makefile leads to a failed check, forcing a clean between the compilation of the
atmosphere core and other cores.
The FCINCLUDES variable contains very similar information to the CPPINCLUDES
variable, which is included in compatibility checks, and in order to enable the
atmosphere core to be compiled after compilation of another core, this commit
removes FCINCLUDES from the variables that are checked for build compatibility.
* Add 'sf_noahmp' to list of possible_values for config_lsm_scheme
This commit adds 'sf_noahmp' to the list of possible options for the
config_lsm_scheme namelist variable in the atmosphere core's Registry.xml file.
Also included in this commit is an update of the name of the 'noah' option to
'sf_noah' under possible_values for config_lsm_scheme.
* Add 'mp_thompson_aerosols' to list of possible_values for config_microp_scheme
This commit adds 'mp_thompson_aerosols' to the list of possible options for the
config_microp_scheme namelist variable in the atmosphere core's Registry.xml file.
* Fix instances of incorrect quotes in init_atmosphere and atmosphere registries
This commit fixes several instances of unbalanced or badly nested quotes in
the init_atmosphere and atmosphere Registry.xml files. These changes result
in different registry-generated code. For example, the generated code for
rnifampten attributes looked like:
call mpas_add_att(r2Ptr % attLists(1) % attList, 'long_name', 'tendency of ')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'ice-friendly"', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'aerosol', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'number', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'concentration', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'due', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'to', '')
call mpas_add_att(r2Ptr % attLists(1) % attList, 'microphysics"', '')
before the changes in this commit, whereas it now looks like:
call mpas_add_att(r2Ptr % attLists(1) % attList, 'long_name', 'tendency of ice-friendly aerosol number concentration due to &
µphysics')
The latter is clearly correct while the badly nested quotes led to the former
incorrect code.
* Update version number to 8.2.1
* Use more mpi_f08 features in mpi_f08_test for odd cases where certain
MPI implementations not compiled with mpi_f08 enabled still pass the check.
Using OpenMPI 4.0.0 and gfortran 11.4.1 compiling without mpi_f08
features the MPI wrapper will still link and supply mpi_f08 (-lmpi_usempif08)
despite not being valid or usable. This causes the `use mpi_f08` and some
features to compile fine, but other mpi_f08 features to not exist and fail
compilation.
* Check for libpnetcdf library existence rather than just directory
This follows the same logic as before with the added check that a libpnetcdf.*
file exists in the checked directory. The change in logic will now account for
situations where PNETCDF is set to a directory containing both lib/ and lib64/
directories and libpnetcdf.* exists in just the lib/ folder. Should libpnetcdf.*
exist in both directories, the lib64/ still maintains precedence.
* Remove debugging prints around calls to sfclayrev in driver_sfclayer
This commit removes some debugging log writes around calls to sfclayrev in the
driver_sfclayer subroutine. These writes led to messages like the following
appearing in the log file for every model timestep:
--- enter subroutine sfclayrev:
--- end subroutine sfclayrev:
--- enter subroutine sfclayrev seaice:
--- end subroutine sfclayrev seaice:
* Add mpas_dmpar_bcast_real4s routine for broadcasting real(kind=R4KIND) arrays
This commit adds a new routine to the dmpar module for broadcasing
real(kind=R4KIND) arrays. The new mpas_dmpar_bcast_real4s routine mirrors the
mpas_dmpar_bcast_reals routine, with the dummy array argument being of R4KIND
rather than RKIND kind.
* Correct double-precision build failures of mpas_atmphys_init_microphysics.F
The mpas_atmphys_init_microphysics module contains a routine, table_ccnAct, that
broadcasts an R4KIND real array, tnccn_act, from the Thompson microphysics
scheme. This broadcast was previously performed by a call to
mpas_dmpar_bcast_reals through the macro DM_BCAST_MACRO. However, because the
mpas_dmpar_bcast_reals routine broadcasts arrays of kind RKIND, when MPAS-A is
compiled with double-precision reals, the mpas_dmpar_bcast_reals routine can no
longer be used with the tnccn_act array.
With this commit, the DM_BCAST_MACRO in mpas_atmphys_init_microphysics.F has
been defined to mpas_dmpar_bcast_real4s, which broadcasts arrays of R4KIND
reals. This works for both single- and double-precision builds of MPAS-A, since
the kind type of the tnccn_act array is independent of the MPAS-A build
precision.
* * In ./src/core_atmosphere/physics/physics_wrf/Makefile, corrected all instances when
spaces were used instead of tabs (lines 41-43, and lines 104, 108, 111, and 114.)
* Restore CMake support for building the atmosphere core
This commit restores CMake support for the atmosphere core, which broke
due to changes made prior to the version 8.2 release. The breaking
changes included moving the physics_mmm directory to its own git
repository. This is accounted for in CMake by using FetchContent to clone
the correct version of the physics_mmm source code directory based on
the MPAS model git version. Additionally, this commit builds the noahmp
library that was added to the code base. Previously, this library only
supported being built with a hardcoded GNU Makefile. This commit also
accounts for the new registry preprocessing arguments that make the
registry processing core-agnostic. These changes re-enable a seamless
build process for the atmosphere core and improve modularity and
maintainability.
* Restore CMake support for building the init_atmosphere core
This commit restores CMake support for the atmosphere core, which broke
due to changes made prior to the version 8.2 release. The breaking
changes were a result of the addition of mpas_init_atm_thompson_aerosols.F
to the init_atmosphere core without updating the CMakeLists.txt. To fix
this problem, this file was appended to the list of source files in the
init_atmosphere core's CMakeLists.txt.
* * In ./src/core_atmosphere//physics/mpas_atmphys_driver_microphysics.F, corrected
the calculation of the height Above Ground Level (AGL) prior to computing radar
in subroutine compute_radar_reflectivity.
* Check for the libnetcdf library existence similar to pnetcdf
As the netcdf library logic mirrors the pnetcdf one, it would suffer
from the same issues noted in 0ca4208 about lib/lib64 checking.
This updates the corresponding issue in the netcdf library detection
before it becomes an issue.
* * In ./src/core_atmosphere:
-> removed package bl_mynn_in from packages included for nc and tend_nc. the
array declaration for nc, nifa, nwfa, tend_nc, tend_nifa, and tend_nwfa
now includes the same packages.
-> in ./physics/mpas_atmphys_interface.F, added arrays nifa_p, and nwfa_p for
the PBL option "bl_mynn".
-> in ./physics/mpas_atmphys_todynamics.F, only updates tend_scalars(index_nc,
tend_scalars(index_nifa, and tend_scalars(index_nwfa due to PBL processes
when the microp_scheme option is "mp_thompson_aerosols".
These changes are needed to run the MYNN PBL scheme without the aerosol-aware
option of the Thompson cloud microphysics scheme.
* Update comments
* Update src/core_atmosphere/physics/mpas_atmphys_driver.F
* Update src/core_atmosphere/physics/mpas_atmphys_driver.F
* Add core_physic-specific dependencies to mpas_atmphys_lsm_noahmpfinalize.o
Add the object dependencies in the makefile for
mpas_atmphys_lsm_noahmpfinalize.o specific to the objects created under
core_physics, i.e. any use statement that reference modules compiled as part of
the core_physics target.
Modules originating from NoahMP and framework are ignored as they should be
handled separately before the core_physics target is started.
* fix duplicated lines and chars
* fix indentation
* Fix MPAS-A dycore-only build
When building MPAS-A as a dycore, all physics-related components are disabled.
An example scenario of this is for use with CAM/CAM-SIMA.
However, the MPAS-A registry file in version 8.2.0 contains the following
regressions, which cause the dycore-only build to break:
* The MPAS-A registry file introduced a new stream, `da_state`, which
unconditionally includes variables that exist only when physics are enabled.
* The MPAS-A registry file unconditionally includes a new registry file from
Noah-MP, which is applicable only when physics are enabled.
Fix MPAS-A dycore-only build by guarding the above regions with the
`DO_PHYSICS` macro.
* * In ./src/core_init_atmosphere/mpas_init_atm_thompson_aerosols.F, changed print statements
"enter/end subroutine init_atm_gocart" with "enter/end subroutine init_atm_thompson_aerosols.
* * In ./src/core_atmosphere/physics/mpas_atmphys_driver_radiation_sw.F, removed
extra deallocate for variable tao5503d_p in subroutine deallocate_radiation_sw.F.
* Fix merge conflicts with v8.2
* Fix merge conflicts with MP driver for v8.2
* Debugging for compile
* fixing surface layer conflicts by adding a new version that doesn't conflict with MMMs version.
* Fixing more merge conflicts
* remove sf_mynn.F that prevent checkout_externals
* Remove duplicate flag line
* Add 2 idealized cases for microphysics
* Remove writes and fix subroutine name
* Refactor and bugfixes for TEMPO MP -> v8.2
* Update pointer to TEMPO
* Add TEMPO to radiation
* Update version number to 8.2.2
* Fix portability issue in parse_macros function used by the MPAS registry
The parse_macros function that is used by the MPAS registry 'parse' tool
previously made use of the state/saveptr/lasts argument to strtok_r when
parsing command-line macro definitions. This was not portable and resulted
in incorrectly generated output on some systems.
This commit reworks the parse_macros function so that it no longer uses the
state/saveptr/lasts argument to strtok_r and instead relies only on the return
value of strtok_r to provide portability across systems with different strtok_r
implementations.
* case select for TEMPO options
* re-implementation of the MYNN-EDMF
* Fully separate TEMPO
* Fix Registry conflict
* finishing the MYNN surface layer re-implementation and a fix ing the PBL driver
* Bug fixes
* Bug fix for non-aerosol-aware TEMPO
* fix merge error that removed ruc init
* Bug fixes and clean up for MYNN-EDMF
* Add brtemp and cldmask fields to atmosphere Registry.xml file for MPAS-JEDI
These fields are required by MPAS-JEDI and are associated with the `jedi_da` package,
which is active only if `config_jedi_da = true`.
Without setting `config_jedi_da = true` this commit should have no impact on
memory usage for stand-alone MPAS-A applications.
* Merged pull request from AndersJensen and removed 'use ccpp_kind_types' statement from line 102 of bl_ugwp.F
* Remove redundant query of nCellsSolve dimension in physics_run_init
The physics_run_init routine in the mpas_atmphys_manager module contained a
redundant query of the nCellsSolve dimension from the mesh pool. This commit
simply removes the extra query.
* Add aerosol mixing for TEMPO + MYNN
* update UGWP hash
* update UGWP hash
* update ugwp pointer
* minor bug fixes and cleanup
* more minor bug fixes
* Revert redundant psfc initialization + some other fixes
* Coupling aerosols to radiation and removing unnecesary variables from te MYNN packages
* corrections to PR
* finishing request from Anders
* various bug fixes for the v8.2.2_merge branch
* fix for icloud_bl undeclared/initialized when running MMM's MYNN.
---------
Co-authored-by: Kuan-Chih Wang <[email protected]>
Co-authored-by: Michael Duda <[email protected]>
Co-authored-by: Laura Fowler <[email protected]>
Co-authored-by: Jim Wittig <[email protected]>
Co-authored-by: G. Dylan Dickerson <[email protected]>
Co-authored-by: Andy Stokely <[email protected]>
Co-authored-by: Anthony Islas <[email protected]>
Co-authored-by: Anders Jensen <[email protected]>
Co-authored-by: Joseph.B.Olson <[email protected]>
Co-authored-by: Michael Barlage <[email protected]>
Co-authored-by: Byoung-Joo (BJ) Jung <[email protected]>
Co-authored-by: Michael Toy <[email protected]>
…Model into v8.2.2_merge_UGWP_update
…odel into v8.2.2_gsl_develop_merge_UGWP_update
…mosphere/Registry.xml
…into v8.2.2_gsl_develop_merge_UGWP_update
|
@mdtoyNOAA have you merged the current gsl/develop to your branch? I see a lot of commits from ncar and others at gsl that I wouldn't expect here. |
I thought I had. I'll need to double check. |
…into v8.2.2_gsl_develop_merge_UGWP_update
…into v8.2.2_gsl_develop_merge_UGWP_update
…into v8.2.2_gsl_develop_merge_UGWP_update
jderrico-noaa
requested review from
joeolson42,
tanyasmirnova and
AndersJensen-NOAA
tanyasmirnova
approved these changes
Nov 13, 2024
joeolson42
reviewed
Nov 13, 2024
| ! Variables that are INTENT(OUT) or INTENT(INOUT): | ||
|
|
||
| ! 3d, layers: | ||
| do k = kts,kte | ||
| do i = its,ite | ||
| rublten(i,k,j) = rublten_hv(i,k) | ||
| rvblten(i,k,j) = rvblten_hv(i,k) | ||
| rthblten(i,k,j)= rthblten_hv(i,k) |
There was a problem hiding this comment.
It's not your fault, but this design (using PBL tendencies to capture the GWD tendencies) is not good for meaningful analyses. This is a WRF flaw adopted by MPAS (thoughtlessly). We need to bring this issue up at the next MPAS meeting and push them to separate PBL from GWD tendencies.
There was a problem hiding this comment.
plus, it looks like you are overwriting the tendency arrays, not adding to them, which to me is a bug.
There was a problem hiding this comment.
I agree it looks like a bug, but fortunately it isn't because in UGWP/bl_ugwp.F we are adding to the tendencies, not overwriting:
! rotate tendencies from zonal/meridional back to model grid
!
do k = kts,kte
do i = its,ite
rublten(i,k) = rublten(i,k)+dudt(i,k)*cosa(i) + dvdt(i,k)*sina(i)
rvblten(i,k) = rvblten(i,k)-dudt(i,k)*sina(i) + dvdt(i,k)*cosa(i)
joeolson42
approved these changes
Nov 13, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updated the Unified Gravity Wave Physics (UGWP) scheme:
Added the non-stationary gravity wave drag (GWD) scheme used in the UFS, i.e., UGWPv1.
Updated the orographic GWD scheme to the version used in the UFS UGWPv1 suite. Also, improved the scale-aware aspects of the scheme by eliminating the 'cdmbgwd' namelist tuning parameter, making the scheme suitable for the MPAS variable-resolution mesh.
Changed default value of 'config_native_gwd_gsl_static' to 'false' in src/core_init_atmosphere/Registry.xml.
Added 'netCDF' flags to Makefiles. NOTE that this is a temporary arrangement that will only work on Jet. Need to un-hardwire this so that it works on all platforms.
Added 'dummy' definition records to nlcat and nscat dimensions in src/core_atmosphere/Registry.xml to prevent model termination if these dimensions are not part of an input stream.
The non-stationary GWD input lookup table, ugwp_limb_tau.nc, is read in using MPAS streams. The file was added to the src/core_atmosphere/physics/physics_wrf/files directory.
Added .gitignore file to UGWP submodule.
The code compiles (both core_init_atmosphere and core_atmosphere) and runs successfully at both global (240km) with NCAR physics and 3km_conus with latest GSL physics.