Minutes for 2024 meetings
Welcoming Ted Mansell to the group!
Dustin Swales Discussed Instrument Simulators
- CFMIP Observational Simulator Package (COSP) 11/06/2024
- Instrument simulators are diagnostic tools that map the model state into synthetic observations
- Simulators
- International Satellite Cloud Climatology Project (ISCPP)
- Multi-angle Imaging SpectroRadiometer (MISR)
- Moderate Resolution Imaging Spectroradiometer (MODIS)
- Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO)
- Cloudsat
Frequency of Meeting brought up, should it change?
- Meeting has been going for four years, meetings every other week
- Discussed changing to once a month
Chats:
Grant Firl - NOAA Affiliate Agree with Lisa. I'd rather see coupling with microphysics generalized. It's not impossible to replace " if (microphysics_scheme_X)" with "if (single_moment), if (microphysics_condition 2), ..." The same comment applies to radiation/cloud interaction.
Grant Firl - NOAA Affiliate Can you share these slides (and put a link to them on the meeting notes) please?
Dustin Swales - NOAA Federal https://docs.google.com/presentation/d/19p6nErkRQhSLDBrA1LGZta_eaXBAQ_TRIEc4dUKBJew/edit?usp=drivesdk
CCPP-related activities at NSF NCAR CGD by Peter Lauritzen: Physics schemes in CESM/CAM are hard to work with due to the complicated code structure. Two CAM repos: ESCOMP CAM and ESCOMP CAM-SIMA. Moving forward, CAM-SIMA will be the only one. CAM-SIMA infrastructure: constituents object and dycore port. Testing/validation: snapshots of state before and after physics from cam_development and CAM-SIMA. Idealized tests. Time sinks and lessons learned: CCPP standard naming, debugging constituents, restructure code for CCPP'ization...
Generalized thermodynamic infrastructure moving forward: WGNE questionnaire on energy budget in ESM:
New rep from GSL: Mike Toy. New rep from NSSL Ted Mansell.
Discussion: Lisa: How do the CAM-CCPP repos connect with the NCAR root CCPP repo? Is it a long-term goal to sync with root repo? Ligia: Currently no connection. Jesse: Yes, ultimately it is the goal. Dustin: Constituent object as part of the capgen mainly developed by CGD. Runtime flexibility to deal with tracers.
Questions on PUMAS: Both NOAA and NCAR may work on this together. Peter: How does NOAA organize the PUMAS code? Lisa: Will be the same official PUMAS and pull it as sub-module. Dustin:
Peter: How the root repo coordinate with all kinds of physics development and management? Ligia:
Jesse: Frequent checkout instead of the entire repo. Separate tuning from main physics updates. Dustin:
Ligia: DTC will learn from the CAM-SIMA document page.
Chats: Jimy Dudhia 2:01 PM depends what type of mushrooms Grant Firl - NOAA Affiliate 2:04 PM also a CSU alum! Adam Herrington 2:16 PM Here's a blurb I put together yesterday on simple physics in cam-sima: https://sima.ucar.edu/what-we-do/technical-developments Lisa Bengtsson - NOAA Federal 2:31 PM Is there an English version of that? :-) Adam Herrington 2:31 PM I have a flow chart showing how these repos connect (it currently assumes ufs is using capegen tho!). Same link above, just scroll to the bottom. Feel free to email me questions. Lisa Bengtsson - NOAA Federal 2:31 PM Thanks! Fanglin Yang - NOAA Federal 2:37 PM To answer the question why there is a separate CCPP ufs_dev repo: one of the reasons is to have a relatively stable version before and during NCEP's major model upgrade. Michael Kavulich 2:39 PM This does assume a relatively stable CCPP framework which I think is at least a few years away Lisa Bengtsson - NOAA Federal 2:39 PM I will have to sign off, but will follow up with some of you in our next meeting. Grant Firl - NOAA Affiliate 2:40 PM Some other reasons for separate forks are "streamlined"/faster PRs within institutions and separation of testing responsibilities. Dustin Swales - NOAA Federal 2:42 PM @mike I think we are close on a unified framework. Michael Kavulich 2:43 PM Unified yes, but we are also making changes fairly rapidly, which affects the ability to "desync" physics and framework Sorry, I didn't mean to imply capgen was years away :D Grant Firl - NOAA Affiliate 2:44 PM and NEPTUNE, right? Jimy Dudhia 2:44 PM RRTMGP and NoahMP are also separate Dustin Swales - NOAA Federal 2:45 PM We need to have a meeting on tuning! Grant Firl - NOAA Affiliate 2:46 PM We (DTC folks) should add some stuff in CCPP documentation regarding best practices for "walling off" tuning stuff within schemes so that different hosts can control them OUTSIDE of the physics code. Peter Hjort Lauritzen 2:48 PM https://escomp.github.io/CAM-SIMA-docs/ Adam Herrington 2:48 PM tik tok Jesse Nusbaumer 2:48 PM we need a special dance for that Fanglin Yang - NOAA Federal 2:49 PM Grant was a star in those u-tube videos Grant Firl - NOAA Affiliate 2:50 PM Looks nice! Dustin Swales - NOAA Federal 2:50 PM Consider it stolen! Jesse Nusbaumer 2:50 PM Thanks All! Credit should go to Courtney Peverley and Michael Waxmonsky on my team Dustin Swales - NOAA Federal 2:52 PM In theory RRTMGP could be used on Mars. Someone should build a suite Michael Kavulich 2:52 PM My masters thesis was on the martian atmosphere, I care! Grant Firl - NOAA Affiliate 2:54 PM I've seen some interesting atmospheric modeling on Titan too. Michael Kavulich 2:56 PM And coal-powered! D:
- NRL colleagues request for watching the UFS fork of CCPP.
- UFS: how to handle physics development in pre-operational implementation periods. More specifically, what level of review is warranted in pre-operational implementation branches and how developers/codeowners should be involved.
Discussion: NRL would like to know code changes before PRs to the CCPP root repo. Being a codeowner of the UFS dev fork can identify code changes applied to the broad community.
Joe: No knowledge about the impacts of physics code changes in different dycores. A testing framework is needed to test the impacts.
Fanglin: Where are RTs being done? Too many RTs now.
Dustin: Parameters changes or tunings in a scheme can be managed more easily than algorithmic changes.
Mike K.: Namelist options for parameter changes can impact runtime efficiency.
Grant: It seems that the way we manage the code and repos is not appropriate for such a situation.
Alex: A easier way to know the changes helps.
Fanglin: Does NRL want to be the code managers or watchers?
Lisa: Adding non-developer as code owners is a bit stretching.
Fanglin and Lisa: Code changes happen after thorough testing.
Ligia: Propose to let Dom watch the UFS dev and provide comments.
Joe: Reconsider CCPP's goals and scopes.
Lisa: Not only diff dycores but also diff apps are challenging.
Fanglin: Physics will never work in a P&P fashion.
Mike K.: The code owner does not imply any privileges. Don't agree with Joe in terms of CCPP's goals.
Alex: Propose to add 2 codeowners to the UFS dev fork. Any other proposal
Chats: Dustin Swales - NOAA Federal 2:11 PM
Host/application specific physics modifications could be either algorithmic or parametric. Parametric changes are easy to mitigate during the PR process, algorithmic changes are tougher.
Jimy Dudhia 2:13 PM
Seems you just need to inform interested groups for a particular scheme when a PR is being proposed, so they can test it themselves.
Dustin Swales - NOAA Federal 2:18 PM
@mike True. With configurability and generalization often there is a cost. So we need to be careful.
Jimy Dudhia 2:23 PM
PR's should be separated by developer and not combined.
Michael Kavulich 2:24 PM
I think "Codeowner" is different from "Code manager"
Fanglin Yang - NOAA Federal 2:25 PM
yes, Mike
Jimy Dudhia 2:25 PM
Reviewer is another mechanism
Dustin Swales - NOAA Federal 2:28 PM
IMO, the whole root and fork setup might be a bit ambitious, or ahead of where we are with CCPP as a community. This was built on the idea that all CCPP enabled hosts would use the root and contribute back. At the moment we have three CCPP hosts, UFS/NEPTUNE use the same physics, CAM-SIMA has their own physics repo. MMM also has their own "CCPP compliant" physics repo.
Michael Kavulich 2:28 PM
Codeowner is much more granular
Dustin Swales - NOAA Federal 2:30 PM
Collaborators has write access to the repo, no more notifications
Soren Rasmussen 2:30 PM
We could make a pull request template to tag a user everytime a PR is created
Grant Firl - NOAA Affiliate 2:30 PM
Don't codeowners need to have write privileges too? In general, I don't know if that will work for all forks. Dom is a special case where folks are probably OK with it, but it requires a lot of trust to add write access / CODEOWNER.
Dustin Swales - NOAA Federal 2:31 PM
@grant That's right. All CODEOWNERS are collaborators with repo write access
Jimy Dudhia 2:34 PM
Yes, testing is separate from reviewing.
Dustin Swales - NOAA Federal 2:38 PM
I don't see how introducing a new dycore will make the physics more complicated?
Grant Firl - NOAA Affiliate 2:41 PM
@Mike In order for the CODEOWNERS to get automatic review requests, I think that the GitHub users need to have write access.
Soren Rasmussen 2:42 PM
We could add a Github Action rule to message @Dom'sGithubUsername or email his directly everytime a PR is created:
https://github.com/marketplace/actions/send-email
Soren Rasmussen 2:43 PM
(might be a different github action than that one)
Michael Kavulich 2:44 PM
This is a good point Lisa, I was only speaking to technical aspects of the CODEWOWNERS file.
Joseph Olson - NOAA Federal 2:45 PM
@dustin, the physics won't get more complicated with additional dycores, just multi-dycore tuning gets more complicated.
Dustin Swales - NOAA Federal 2:46 PM
Correct, which is not a physics problem.
Grant Firl - NOAA Affiliate 2:47 PM
In order for multiple dycores and applications to effectively share the physics, CCPP schemes need to do a better job of documenting and compartmentalizing parts of the code that need to change between dycores/applications so that host-controlled namelists can do the work for us.
Alex Reinecke 2:47 PM
Not necessarily me.
Michael Kavulich 2:48 PM
I want this slack bot! 😆
Grant Firl - NOAA Affiliate 2:49 PM
Another option is to start PRs in the root repo as soon as they're started in the fork repos.
Lisa Bengtsson - NOAA Federal 2:49 PM
The scientific testing is done much before the PR is created, so maybe such a notification is also too late?
Dustin Swales - NOAA Federal 2:50 PM
We have schemes in the CCPP that are shared across UFS applications, and some that are shared across hosts. So this is not an unsolvable problem, nor is it unique.
Jimy Dudhia 2:50 PM
@Lisa yes time needs to be allowed for other organizations to test PRs
Dustin Swales - NOAA Federal 2:51 PM
I second that Lisa! The onus is on NRL to participate in the development process from the nebulous, just like any UFS application
Alex Reinecke 2:51 PM
let me log on/off
Fanglin Yang - NOAA Federal 2:53 PM
Jim Doyle has already been invited to a few of our physics working groups, including GWD, microphysics, and some convection discussions We can add more from NRL
- Mike Barlage leads the discussion "Status and Plans for NoahMP Code Unification in UFS, MPAS and CCPP" (https://docs.google.com/presentation/d/1knqGBYcKer6lLfL2-VEZ_8acds5IkhPqVY2FRo8gYx0/edit?usp=drivesdk).
History and roadmap of Noah MP:
- NCAR/NoahMP (NCAR refactored NCARr and non-refactored NCARnr versions). Future development goes in NCARr
- CCPP version shared history a long, long time ago (pre-dates github repo); CFS project
- NOAA-EMC/NoahMP (EMC). Essentially a copy of CCPP used in UFS noahmp land component, contains NUOPC interfaces, technically a fork of NCAR/noahmp but actually not.
UFS land infrastructure
UFS Component Model - NoahMP
Land-only Driver: UFS land driver that plugs directly into CCPP Physics repo. Needs two dependencies: Fortran compiler and NetCDF library. Being used by DA systems.
Non-exhaustive others: WRF NCARnr, MPAS NCARr (non-submodule), NASA-LIS, and others
Path forward to unification: Short-term (~ 1 mon) UFS/CCPP submodule EMS in ccpp-physics and component model; move NoahMP code from upper levels of ccpp-physics; remove redundancy; no answer changes. Mid-term may move to NCARr repo that needs new interfaces for new data structures; transfer to ccpp-physics/develop have to wait for GFS17.
Final thoughts: should have created the NCARnr submodule in 2020, but there were reasons why we didn't. Ultimately, we want all parent models to use a submodule of NCARr or a fork of it since that is where all new development happens.
Laura: In MPAS V8.2.1, downloaded NCARr and populated into physics directory. Decided not to use the submodule approach. Reasons: source code is hardwired to high-res model, figured out the mods need for flexibility usage in MPAS. Makefile, no need for DA, no need for groundwater, etc. Configure I/O for reduced dimensions. Implementation is not complicated. Compiler limitation of GNU. Specific to GNU complier or not is not clear yet. After making the NoahMP compliant to MPAS, test runs have been done. Future: relax most hardwired components.
Discussion:
Lisa has a question about the ccpp-physics submodule in land-only driver repo. Two copies of the same CCPP physics code in the atmosphere component and EMC component land model physics.
Lisa asked why NoahMP works as a component model.
Lisa asked about GFDL land model as a component.
Helin asked if the development at EMC can go into NCARr easily.
Adam asked where the heat fluxes are calculated (LSM or PBL)? Mike B. responded that fluxes are calculated in LSM. Dive deep into the call order of physics for different setup of NoahMP. From the CESM perspective, the order matter when these terms are calculated.
Chats:
You 2:04 PM Welcome to the meeting, Helin!
Grant Firl - NOAA Affiliate 2:17 PM There is a python script to copy over the CCPP code
Lisa Bengtsson - NOAA Federal 2:18 PM ok, it is clear now, thanks
Dustin Swales - NOAA Federal 2:18 PM Yeah what a birds nest!
Dustin Swales - NOAA Federal 2:31 PM Could you put the non-refactored code in the submodule as a branch?
Grant Firl - NOAA Affiliate 2:31 PM @Dustin ... and make the branch read-only?
Dustin Swales - NOAA Federal 2:32 PM Exactly
Adam Herrington 2:36 PM Yes SCAM runs the entire CESM model.
Dustin Swales - NOAA Federal 2:40 PM Is it fair to say that SCAM is a configuration of CESM?
Dustin Swales - NOAA Federal 2:41 PM ^@Adam
Adam Herrington 2:41 PM Sure, SCAM is a "compset" or selection of CESM components (ATM+LND/DOCN). (I guess CICE too if the p0int is over sea-ice)
Dustin Swales - NOAA Federal 2:42 PM Great thanks! DOCN? Data(?) Ocean
Adam Herrington 2:42 PM yes
Michael Barlage - NOAA Federal 2:47 PM https://docs.google.com/presentation/d/1knqGBYcKer6lLfL2-VEZ_8acds5IkhPqVY2FRo8gYx0/edit?usp=drivesdk My slides if you want to take a look
Adam Herrington 2:49 PM Unrelated question. How long has MOM been a component of UFS?
Lisa Bengtsson - NOAA Federal 2:52 PM Since 2020, MOM6 has always been the ocean model in the UFS global applications
Lisa Bengtsson - NOAA Federal 2:53 PM It was added as a component from the beginning, same as CICE6, WW3 etc Adam Herrington
2:54 PM Thanks, Lisa! We've been "tuning up" MOM6 for CESM3 for at least a year now. It's been ... tricky. Lisa Bengtsson - NOAA Federal
2:54 PM I can imagine!
- EMC introduce RhaeSung Kim, new co-code manager for the UFS fork of CCPP.
- NRL transition to FNMOC is in early 2025. Can change code until approx Oct 2024. May use CCPP v6, or CCPP v7, or later code.
- Dustin Swales (DTC) gave an update on the status of the upcoming CCPP v7 release (slides).
Welcome RhaeSung! Lisa Bengtsson - NOAA Federal 2:04 PM Welcome RhaeSung! Michael Kavulich 2:04 PM Yes, I will add you to the Framework meetings! We meet on Thursday mornings. Grant Firl - NOAA Affiliate 2:05 PM I just added RhaeSung to the UFS/NCAR fork meeting on Wed. afternoons. Dustin Swales - NOAA Federal 2:06 PM We will need to add him as Admin to the repositories RhaeSung Kim - NOAA Affiliate 2:07 PM Thanks all for the warm welcome! Grant Firl - NOAA Affiliate 2:21 PM Note that the "native" forcing will not be bit-for-bit between SCM and UFS because UFS output isn't for every timestep and the SCM vertical coordinate is Eulerian vs UFS's (FV3) semi-lagrangian vertical coordinate. Grant Firl - NOAA Affiliate 2:23 PM If UFS history files and ICs were on the same grid, we'd have a better chance at bit-for-bit. Fanglin Yang - NOAA Federal 2:24 PM There is also the debate of Cp vs Cv issue Grant Firl - NOAA Affiliate 2:25 PM The "native" forcing for short cases should be pretty darn close to being able to reproduce most things. Fanglin Yang - NOAA Federal 2:27 PM For FV3, from physics to dynamics heating tendencies are scaled by Cp/Cv. Should the dynamics tendency be scaled by Cv/Cp in SCM ? This might worth testing Grant Firl - NOAA Affiliate 2:29 PM @Fanglin. This is definitely worth looking at. I guess it depends when the scaling is done in relation to when the non-physics state tendencies are calculated. Fanglin Yang - NOAA Federal 2:30 PM I need to jump to a different meeting. Thank you all for welcome RhaeSung to the team. Any help you can offer for RhaeSung to spin up is appreciated Jesse Nusbaumer 2:30 PM ls Dustin Swales - NOAA Federal 2:31 PM https://docs.google.com/presentation/d/1zAbNLNdMNi3DuJsn_K7Mxs1eoA6pDqXyThLBZ3Zrkts/edit?usp=drivesdk Grant Firl - NOAA Affiliate 2:34 PM But, after the release, anything special in the release branch should go to "develop" Jimy Dudhia 2:35 PM If you want to back later and figure out what you checked out it might be tricky Jimy Dudhia 2:36 PM More related to repeatability Grant Firl - NOAA Affiliate 2:38 PM Another release-like snapshot should be an upcoming "HR4" tag... Lisa Bengtsson - NOAA Federal 2:39 PM yes, the tags are good.
Topic: CCPP release updates and naming of CCPP suites implementation plan
Ligia provided an update of the CCPP V7 release status. Mike K. led the discussion of the naming of suites plan.
Discussions:
Reorganization of the directory structure is in place, which needs to be included in the release information.
Grant suggests that anyone who are interested in dynamics-physics coupling can reach out to Grant, Dustin and Soren. They will attend the workshop on this topic in Exeter UK next month.
Chats:
Lisa Bengtsson - NOAA Federal 2:10 PM
I know Amy Solomon has worked on a MOSAiC SCM case, is that different from what you're adding here? Which organization is adding MOSAiC?
Dustin Swales - NOAA Federal 2:14 PM
@Lisa I believe that it is from Amy, brought in via Tracy at the DTC. Lisa Bengtsson - NOAA Federal 2:14 PM
One other question, there is a PR to the ufs weather model currently for HR4, with an updated gravity wave drag scheme because of the performance of ugwpv1_gsldrag significantly degrades the ACC score and biases in HR3 compared to HR2. Could this updated gravity wave drag scheme be part of the release instead as it will be the scheme used in GFSv17? @Dustin, OK I will check with Amy why she's not adding it herself, thanks.
Grant Firl - NOAA Affiliate 2:17 PM
Can bird names be more scientific? E.g. there are over 120 types of warblers. As a bird person, this is offensive! :)
Ligia Bernardet - NOAA Federal 2:19 PM
@Lisa, possibly we can do HR4, but we need to keep to a timeline, so we need to check on that. I know there is already a HR4 (draft?) PR @Lisa, My understanding is that MOSAIC is from Amy via Tracy (DTC)
Jimy Dudhia 2:22 PM
Looks like the suites will have version numbers?
Lisa Bengtsson - NOAA Federal 2:24 PM
@Ligia, the main physics update is the PR you see committed (from Jongil). The major update from HR3 to HR4 is going to be coupled DA which is outside of this discussion.
Grant Firl - NOAA Affiliate 2:25 PM
@Jimy I think that the version number listed in the SDFs corresponds to the version of the SDF XML scheme file. It doesn't correspond to public releases. This is perhaps misleading.
Michael Kavulich 2:26 PM
Yes Grant; this actually doesn't have any functionality right now with CCPP prebuild, but capgen does verify the xml format against the suite.xsd schema file, so in the future this version tag may be useful
Michael Kavulich 2:27 PM
We do hope to address versioning of suites and schemes in the not-too-distant future, that will hopefully be the next project after the suite names are settled!
Jimy Dudhia 2:28 PM
This brings up the point of how to add or update suites between v7 and v8
Dustin Swales - NOAA Federal 2:29 PM
https://github.com/ufs-community/ccpp-physics/pull/207
Michael Kavulich 2:30 PM
I feel like this is dangerous if we are dead-set on meeting a June release deadline
Michael Kavulich 2:31 PM
Also is there a chance there are more changes before the prototype is implemented? Or is that already frozen?
Grant Firl - NOAA Affiliate 2:32 PM
I feel like the moving parts are more on the SCM side. It's not too difficult to update the physics. We should re-run tests, etc., but if we want the release to be maximally useful, we should try to get it in.
Lisa Bengtsson - NOAA Federal 2:33 PM
Thanks Grant.
Grant Firl - NOAA Affiliate 2:36 PM
There doesn't have to be a lag. It's up to us. If we prioritize it, it will happen quickly.
Adam Herrington 2:37 PM
There are over 4000 official mineral names (feldspar, olivine, ...)
Dustin Swales - NOAA Federal 2:38 PM
We could add colors? Say Red Robin, yuuum
Jimy Dudhia 2:40 PM
I see it in main
Dustin Swales - NOAA Federal 2:42 PM
Optional arguments are now allowed in the CCPP again. Which could be included as part of this release.
Topic: Naming of CCPP suites
Slides: https://docs.google.com/presentation/d/1KJ4-dShCtjhUl-gpTz3IonnUYcn2iZ-h/edit#slide=id.p1
Notes from Mike K.
Joe Olson: can we get rid of suites all together??
Fanglin Yang - NOAA Federal 2:32 PM R1L2C2M1
Idea is that you just have a letter followed by option number for each parameterization
How to deal with grouping?
How to deal with subcycling
Kate Zhang: Can we start with a base file and specify modifications Would be really nice to have tool to easily manipulate suite files, but too much complexity?
Jim Doyle: Some comments Dates not good What about when parameterizations are removed? No renumber What about double digits
Lulin: we maybe have too much information in names with Fanglin’s proposal Maybe a tool that gives a hash based on your sdf, namelist, model hash, and that can then be decoded to give full information about how it was generated
Multiple comments about “dates bad”
Comments:
Lisa Bengtsson - NOAA Federal 2:05 PM
In the UFS, the CCPP suite used in prototype HR3 is called: FV3_GFS_v17_coupled_p8_ugwpv1
Dustin Swales - NOAA Federal 2:08 PM
@Lisa. In the SCM, just recently the "HR3" suffix was adopted by the SDF name, instead of. "_coupled_p8_ugwpv1". See https://github.com/NCAR/ccpp-scm/pull/459
Lisa Bengtsson - NOAA Federal 2:09 PM
ok... maybe that should be consistent with what we use in the UFS? To avoid confusion
Dustin Swales - NOAA Federal 2:09 PM
Fair point.
Lisa Bengtsson - NOAA Federal 2:16 PM
I agree with the challenges you present. In the UFS we use TAGS to avoid the problem on differences in source code. It seems that since the CCPP release v7 only pertains to the SCM, there's increasing divergence from the UFS. I think that is unfortunate if you rename things in the SCM, but not in the UFS. Also, better to tag model versions that go with the suites, instead of releasing a once a year snapshot of the code?
Ligia Bernardet - NOAA Federal 2:20 PM
@Lisa, I can address some of your questions/concerns during the discussion
Lisa Bengtsson - NOAA Federal 2:24 PM
Use the GFSv16 model TAG!!
Dustin Swales - NOAA Federal 2:25 PM
And correct namelist and SDF.
Lisa Bengtsson - NOAA Federal 2:32 PM
I unfortunately have to go early, I don't know who the users of a release are, but would like to learn. I think this divergence from the UFS is unfortunate, in the UFS we have workflow tags and model tags so you know what you run. Sorry I can't stick around to follow up, but hope to have a chance to provide input before you make this change.
Jimy Dudhia 2:32 PM
If the suites go with versions the date could be replaced with v7 etc.
Fanglin Yang - NOAA Federal 2:32 PM
R1L2C2M1
Fanglin Yang - NOAA Federal 2:39 PM
less automation is preferred
Joseph Olson - NOAA Federal 2:39 PM
Fanglin's idea isn't bad in my opinion, if we can't delete them altogether.
Ligia Bernardet - NOAA Federal 2:40 PM
Are there any suites in CAM/CESM? If so, what are they called?
Jimy Dudhia 2:41 PM
Maybe just need to be able to append a chem SDF to a physics SDF
Fanglin Yang - NOAA Federal 2:42 PM
some of the features Kate mentioned can be controlled by namelist options
Jimy Dudhia 2:44 PM
+PBL, shallow, surface-layer = 7
Joseph Olson - NOAA Federal 2:44 PM
+ocean=8
Dustin Swales - NOAA Federal 2:45 PM
The SDF has the "buildtime" physics configuration (groups, ordering, subcycling) The NML has runtime configuration. The could be many NML combinations for a given SDF.
Jimy Dudhia 2:45 PM
+strat ?
Ligia Bernardet - NOAA Federal 2:46 PM
strat? do you mean ozone and H2O photolysis? ice?
Jimy Dudhia 2:49 PM
yes, strat physics options Supporting operational suites would need tagging separate from versions of CCPP
Jimy Dudhia 3:00 PM
The description in the xml could be formalized to name all the options
Fanglin Yang - NOAA Federal 3:01 PM
I agree with Dustin. SDF saves a lot of pain for layman developer
Topic: Evolution of the Internal Physics State
Slides: https://drive.google.com/file/d/1ZY5eZj8_zhpoe16vhV3p8AtnffF-OljA/view?usp=drive_link
- Note that physics ordering in MPAS standalone differs from ordering in CAM/MPAS
- Note that I/O sometimes happen after a certain physics (not generally after all parameterizations are called)
- Fanglin thinks is it is a reasonable request to developers to return tendencies. This will help reordering flexibility.
- Grant sees this as an interoperability issue. Schemes that modify the state do not support interoperability. All schemes should return tendencies. Then state updates can be done by interstitials or the Framework. We should not lose sight of performance hits - schemes that today update state, would output tendencies and then we'd update the state. So, more calculations. Recommends that schemes return tendencies.
- Joseph: Agree to output tendencies, not a big ask of physics developers. Ordering of updates have more impact for climate models that have longer timesteps. Dustin: Yes, that is why CGD needs to have this flexibility. This helps with the interoperability.
- Dom Heinzeller: Agree with requirement of returning tendencies. It is an overhead that provides clarity. UFS will want to support FV3 and MPAS dycores ways of updating state, so we need the parameterizations to output tendencies.
- Lisa Bengtsson: Today in UFS we compute a tendency for diagnostics for schemes that update state. This is an opportunity for cleanup. For SFS C96 (100 km) long timesteps, investigating these topics can be interesting research
- Dustin: If the Framework is doing the state updates, it can handle a lot of the diagnostics.
- Grant: In the UFS, there are mismatches in that not all aspects of the state are in the same timestep. E.g., geopotential is not updated after the state is updated, so next physics parameterization uses updated state but old geopotential. Is this a significant error? Joe: It is worth to do this right to help reduce model errors. We are in an era of diminishing returns and we need to pursue little gains. Ligia: Next steps? Ask developers to return tendencies. For UFS, just physics and convection. Fanglin: Good opportunity to retire some of teh convection schemes that are not being maintained.
Topic: Scientific Documentation
Slides: https://docs.google.com/presentation/d/1xgPltCz6DeYHA8GFQmRzwnBmx82L6MPP/
Attendees: Soren and
Topic: New member intro: Larissa Reames from NSSL
Schemes/suites/capabilities for SCM+CCPP v7 release
https://docs.google.com/presentation/d/1BC7xnbCR9LNypPFN-t74gJ6Ya5aQ2Cg3YpilRy-ms5g/edit#slide=id.p1
DTC will support a past release for a limited time DTC offers tags corresponding to major UFS events, such as prototypes
This release will be paired only with SCM not with UFS
SCM updates: Additional cases, Spack-stack, replay mode, single precision for certain suites, GPU (TBD)
Naming of Suites:
Submodules: Single source for use in CCPP and non-CCPP models. DTC will be CODEOWNERS of the repos to watch. Place to host these submodules: NOAA, NOAAgov, ufs-community, NCAR, ESCOMP...
Discussions: Andrew: will the release impact codes currently used by other operational systems? Ligia: No.
Fanglin: suite name is often confusing.
The schemes in different suites for discussion: https://docs.google.com/spreadsheets/d/1Ak4K2P-8Ml9MqewG1D7iQGUcGk58DKn0mBHqX0mim24/edit#gid=0
Joe: To keep the supported number of suites minimum. The test of the top of the trunk is important for development and engagement.
Fanglin: Should provide tutorial for users to create their own suites.
Lisa: The suites are associated with hosts. The supported ones are called out in the release. Other suites are still there for users to work with. Do we have data to support how the released CCPP codes are used?
Larissa: NSSL embraces MPAS and physics for CA forecasts.
Ligia: gravity drag coverage
Fanglin: Ideally, we should move to the newer version of the drag schemes.
Joe: Can we add something in the release codes that stop the code and encourage users to use newer released code.
Chats:
Ligia Bernardet - NOAA Federal 2:04 PM
https://docs.google.com/presentation/d/1BC7xnbCR9LNypPFN-t74gJ6Ya5aQ2Cg3YpilRy-ms5g/edit#slide=id.p1
Lisa Bengtsson - NOAA Federal 2:21 PM
Are there no non UFS hosts supported?
Dustin Swales - NOAA Federal 2:21 PM
No
Jimy Dudhia 2:21 PM
link to spreadsheet https://docs.google.com/spreadsheets/d/1Ak4K2P-8Ml9MqewG1D7iQGUcGk58DKn0mBHqX0mim24/edit#gid=0
Dustin Swales - NOAA Federal 2:24 PM
Begin shameless plug: Why not include RRTMGP in a SDF? It was tested with P8 and works, but its slow in 3D apps, not the SCM. End plug:
Lisa Bengtsson - NOAA Federal 2:30 PM
Top or the trunk, or a tested host model tags are more relevant I think.
Jimy Dudhia 2:34 PM
Supported could apply to regularly compiled/tested suites or all of them could be regularly tested
Fanglin Yang - NOAA Federal 2:39 PM
Let users (outside of the UFS community) to define their own SDF with different parameterization options allow the wider community to explore new ideas and experiment with new parameterizations we rarely test.
Jimy Dudhia 2:40 PM
GFSv17-pre
Jimy Dudhia 2:47 PM
More of an internal question for DTC is which suites get tested to cover the ccpp_physics and also which dycores.
Dustin Swales - NOAA Federal 2:49 PM
The SCM regression tests use the supported suites. Currently the ccppv6 SDFs
Lisa Bengtsson - NOAA Federal 2:50 PM
haha who owns NOAA?
Dustin Swales - NOAA Federal 2:50 PM
I could not find the owner!
Fanglin Yang - NOAA Federal 2:52 PM
I'd vote for NCAR
Dustin Swales - NOAA Federal 2:53 PM
NCAR makes the most sense
Jimy Dudhia 2:54 PM
Is there a problem with mixing github accounts for hosts?
Fanglin Yang - NOAA Federal 2:55 PM
using an umbrella repo structure is also a viable option
Lisa Bengtsson - NOAA Federal 2:56 PM
Isn't the idea that MPAS will be moved into UFS, which is using CCPP?
Ligia Bernardet - NOAA Federal 2:56 PM
MPAS dycore will be connected to UFS. MPAS standloane will live on outside of UFS
Jimy Dudhia 2:57 PM
@Lisa yes. Maybe its physics development would occur elsewhere
Lisa Bengtsson - NOAA Federal 2:57 PM
and they are not interested in CCPP? (standalone)?
Dustin Swales - NOAA Federal 2:58 PM
MPAS dycore is going into the UFS, which couples to the CCPP.
CCPP is not going into MPAS
Lisa Bengtsson - NOAA Federal 2:59 PM
But Laura is here to couple MPAS to CCPP?
Attendees:
Topic: Submodule discussion: https://docs.google.com/presentation/d/1vCpLNng3qg_VAH2BqpkLY-TwQKg5xUaKbIuLRbhkofw/edit#slide=id.p1
Dustin went through the slides.
Questions from Lisa: How do we develop the scheme as part of a suite in a ccpp enabled host?
What if a host introduces a change not wanted by another host?
Who will be responsible for merging updates from the stand alone repository into their respective hosts? And vice versa…
How does Regression Testing (RT) work if someone updates the scheme outside of the UFS, without requirement of RT testing on their end?
George: C3 and GF schemes have developers from different countries, which made it difficult to update the scheme.
Lisa: Not concern about the technical part of the scenario but the scientific implications of the development. How do we insure the development of the physics is coordinated?
George: Don't have to take care of all other hosts.
Lisa: Need to evaluate all developments towards the scheme, which needs a lot of efforts. Priority is to improve UFS performance.
Ligia: A lot of concerns came up. But many of them are hypothetical. UFS community still has control of most of the physics development after the submodule approach is adopted. The main point is to figure out if using submodule provide benefit to the development and management processes.
Laura: Haven't thought about the problems Lisa raised. MMM always has this situation with updating new developments. Testing and evaluations are needed from MMM. Want to try SAS but feel too much work is needed for MMM models. Submodule is helpful for applications in other hosts.
Jim: Like the concept of using submodule for code management and development.
Adam: CGD uses a lot of externals and submodules in their models (PUMAS and CLUBB). Having these capabilities is beneficial for the general code management. Communicating with developers is critical.
Lisa: Different groups tuned the physics or suites differently. Need to make sure all new developments do not change UFS's goal. Need to establish who is responsible for scientific evaluations.
George: It is not a big issue. Things are going this way already.
Mike K.: More benefits from using the submodule approach such as the versioning.
Anders: Codes developed to be used in different hosts. We also want to write the code the way that consistent core part stays stable with out-facing codes being adjustable. Having the physics work in different hosts is beneficial to the overall health of physics development.
Fanglin: The concept of CCPP is to improve interoperability of physics. Submodules should be in a centralized location (CCPP repo).
Joe: Valid concerns from the group. But
Dustin Swales - NOAA Federal3:01 PM
There is no difference if development gathers in a personal fork or a submodule branch. When it's time to come into the ccpp, it's tested
Joseph Olson - NOAA Federal3:03 PM
gotta go, sorry
Chats:
You 2:01 PM
https://docs.google.com/presentation/d/1vCpLNng3qg_VAH2BqpkLY-TwQKg5xUaKbIuLRbhkofw/edit#slide=id.p1
Fanglin Yang - NOAA Federal 2:21 PM
Suggestion: 1) create a CCPP community under, for instance, https://github.com/ufs-community/CCPP, to host all "scheme module" repositories, 2) assign CCPP code managers as the owner of individual scheme module instead of "developers" Fanglin Yang - NOAA Federal2:23 PM or under https://github.com/NCAR/CCPP
Ligia Bernardet - NOAA Federal 2:23 PM
@Fanglin, Noah-MP and RRTMGP will not be in this location. I am not sure we can fully control where the schemes are.
Fanglin Yang - NOAA Federal 2:26 PM
@Ligia, in particular for RRTMGP we should discuss with the developer and see if we can move it within ufs-community or NCAR
Dustin Swales - NOAA Federal2:38 PM
@fanglin I agree. We should do for GP whatever we decide to do for the other schemes under consideration.
Grant Firl - NOAA Affiliate2:40 PM
RE: the question of where we want submodules to be stores -- if we can't convince folks to use a single CCPP repository for sharing physics, why would we have any better success asking folks to put their scheme submodules under one organization?
*stored
Dustin Swales - NOAA Federal2:41 PM
Excellent point!
Grant Firl - NOAA Affiliate2:45 PM
Speaking to Jim's points, I think that submodules allows for easier versioning/tagging of individual schemes and different hosts can pull in specific versions as they see fit. This is fundamentally different than having one top-level repo with versioning/tagging on the entire physics repository level. I see this as helping hosts to keep up as they can and reducing the load -- they don't have to do all scheme updates at once. This is also beneficial for scientific testing, of course, testing one thing at a time
Ligia Bernardet - NOAA Federal2:47 PM
@Lisa, I think your concerns are not about submodules but about joint development for multiple hosts. It has been the premise of the CCPP that interoperability can take us all further than separate development for each host.
James Doyle2:47 PM
Thanks @Grant. I agree with you and my impression is that this is a really nice way to go forward.
Dustin Swales - NOAA Federal2:48 PM
one fire at a time Georg
James Doyle2:49 PM
Agree with @Ligia (and @Lisa) that joint development for multiple host models is a challenge. Perhaps a follow-on topic for another meeting.
Fanglin Yang - NOAA Federal2:49 PM
We may have to use the NCAP/ccpp to facilitate the submodule development path, and use ufs_community/ccpp_physics for UFS specific tuning, testing and evaluaiton.
NCAP/ccpp and ufs_community/ccpp will be synced less frequently (if it is all possible)
Lisa Bengtsson - NOAA Federal2:54 PM
An opposite approach is the Integrated Forecast System, IFS, where physics, dynamics and data assimilation is developed in an integrated way. They have been quite successful in doing it this way.
Georg Grell - NOAA Federal2:55 PM
But there are still some physics routines used in other places too
Attendees: Jordan, Andrew, Soren, Lisa, Laura, Fanglin, Dustin, Jesse, Michael K., Qingfu, Adam, Jim, Weiwei, Joe, Ligia, and Lulin
Topic 1: Discussions about the origins of the code for Tiedtke convection and the question that was raised about possible restrictions of its use in certain scenarios, such as operations.
Fanglin: https://docs.google.com/document/d/13i2knlHCxWNYU1DT9eaNZw_o7HMTTaixei4LCid0IdE/edit.
In short, there is no concern using it for operations. The updated version of the convection scheme from ECMWF will be released open-source (apache-2) soon.
MPAS version with the scale-awareness is ready. We need to discuss how to unify in the CCPP version. Wei Wang will be back around mid January. We should discuss with Wei about this unification effort.
Laura: The MPAS version is somewhat CCPP compliant but not fully ready CCPPized.
Jim: Modifications are still needed to adapt the scheme for NEPTUNE.
Fanglin:
Ligia: Question to Laura about keeping different code bases.
Laura: Since differences exist among different groups using the scheme, it is difficult to have just one unified code base.
Fanglin: Question to Qingfu about which version EMC is testing. Qingfu responded as the PR from Matus.
Fanglin: Question to Jim about NRL's plan of moving forward using the scheme for operations. Jim: Will work with the CCPP code management.
Both Fanglin and Jim found biases in the tropic induced by the scheme.
Jesse: CGD is CCPPizing a certain version of the scheme from MMM. Is it the same in the PR or a different version? Laura: Some developments from MMM were made to the code since it was handed to CGD. The core of the scheme should be model agnostic.
Dustin: The goal is to make the physics schemes host agnostic, which needs efforts and resources.
Jesse: Will help to move towards to goal.
Laura: How to tag and version different codes is an important aspect.
Ligia: Move forward with Matus' PR soon.
Dustin: Question to Laura: How was MMM shared physics brought into the CCPP? Weiwei confirmed that it is done through the submodule. This can be a way moving forward integrating MMM physics with the CCPP.
Ligia: Need community buy-in to have a centralized repo with community contributions and development to the physics schemes.
Laura: PRs are mainly allowed from developers.
Fanglin: Second Dustin's point about not having many forks and submodules.
Jim: Updates from the repo are not very transparent for different hosts. Can relevant parties be involved in the review process? Ligia: should be able to. Lisa: Challenges associated with modes of using the physics.
Dustin: Poor coding habits led to many problems we are seeing now!
Chat: Fanglin Yang - NOAA Federal2:02 PM
https://docs.google.com/document/d/13i2knlHCxWNYU1DT9eaNZw_o7HMTTaixei4LCid0IdE/edit
Andrew Hazelton - NOAA Affiliate2:11 PM
Once the latest (scale-aware) version of scheme is in CCPP, is it fairly straightforward to bring it into UFS applications (like HAFS) or is it still fairly complicated?
Dustin Swales - NOAA Federal2:19 PM
For a centralized repository to work, the code needs to be model agnsostic, which requires work.
Fanglin Yang - NOAA Federal2:19 PM
https://docs.google.com/document/d/13i2knlHCxWNYU1DT9eaNZw_o7HMTTaixei4LCid0IdE/edit pasted one more time for those came late
Ligia Bernardet - NOAA Federal2:22 PM
This is the PR from Matus: https://github.com/NCAR/ccpp-physics/pull/1025
Qingfu Liu - NOAA Federal2:23 PM
https://github.com/NCAR/ccpp-physics/pull/1025
Lisa Bengtsson - NOAA Federal2:31 PM
It has many similarities to ECMWFs operational version, it even references the latest cycles from IFS r36, copied straight from operational code (or open IFS) using the exact code formulation for updraft entr/detr revisions that hasn't been published. I think we need to acknowledge the developers at ECMWF!
Fanglin Yang - NOAA Federal2:32 PM
@Lisa, are you referring to the CCPP version or the PMAS/WRF version ?
Adam Herrington2:32 PM
when we say "noaa repo" -- we're talking about github.com/NCAR/ccpp-physics?
Dustin Swales - NOAA Federal2:33 PM
This one https://github.com/ufs-community/ccpp-physics
Lisa Bengtsson - NOAA Federal2:33 PM
This version: https://github.com/ufs-community/ccpp-physics/blob/31a99de05048187b61a2ea8381417b8dbd8db7e0/physics/cu_ntiedtke.F90
Dustin Swales - NOAA Federal2:35 PM
The physics repo was recently modified to distinguish interstitial and scheme code, https://github.com/ufs-community/ccpp-physics/tree/ufs/dev/physics.
Weiwei Li2:43 PM
Yes @Dustin
James Doyle2:45 PM
I agree with @Lisa regarding making sure we acknowledge ECMWF and Peter Bechtold (and developers).
Fanglin Yang - NOAA Federal2:47 PM
@jim@Lisa I second
Ligia Bernardet - NOAA Federal2:53 PM
@Lisa, where do you think acknowledgements need to be made? Within the code? Or when we make presentations? Or...?
Jesse Nusbaumer2:54 PM
Does this raise the issue about how parameters should be managed in general? For example should most (all) parameters be namelist variables that the host model is responsible for? Not sure that's an ideal solution, just a strawman
Ligia Bernardet - NOAA Federal2:55 PM
Jesse and all: see recommendations at https://docs.google.com/document/d/1uYnxbw_KsuzBGhnvhSW-hhk3c6PbpTCPTNOdV6Z8TVc/edit These recommendations were discussed in the last 1-2 CCPP Physics meetings
Jesse Nusbaumer2:56 PM
Thanks Ligia!
Ligia Bernardet - NOAA Federal2:59 PM
We'll need to wrap up in a min
Adam Herrington2:59 PM
magic number =. a hard coded number and not a parameter?
Dustin Swales - NOAA Federal3:00 PM
exactly