The configuration of CAM7 for the next CMIP7 DECK/SSPs

[dan800]

At the CAM7 dev. meeting the question arose of what model configuration be used to deliver output for CMIP7 DECK, SSPs and critical mips. This is a thread to discuss what that should be and what is on the critical path to getting to those configurations (and possibly setting a lower priority for other configurations).

@JulioTBacmeister: 80km (i.e., L93) w/ full CHEM for DECK simulations. For LE this may not be practical, in which case add L58 w/o full CHEM if we can show climate sensitivity is similar to CHEM+L93. (note, L58 will Use specified constituents from the DECK sims).

Potential issues:

• no L93 w/o CHEM required for CMIP7, but useful for S2S.
• 80km (i.e., L93) w/ full CHEM still needs upper boundary specified. Where will that come from? Additional WACCM simulations?

[islasimpson]

I'm not sure that we should consider the L58 version to be the one we'd use for large ensembles. The whole point of this exercise was that we didn't think we had a good enough stratosphere in CAM and that there were features that are of relevance to climate variability and change that we don't have, such as a QBO. I think if we truly believe that's the case, then we should be aiming to have these features in projects such as large ensembles. So, I think for this reason we'd need to have a specified chemistry of L93 as well. If we don't think this is important enough to include in large ensemble type experiments, then I think we have to question whether the vertical resolution choice, that is very much geared toward having a good QBO, is the right one. I definitely agree that an SC version of the model would be relevant for S2S prediction and in fact there's a group at Scripps that's starting out a project with FV at L83 to investigate this. @dan800 - maybe we could talk at some point whether you foresee any issues with boundary conditions for this project?

[dan800]

This is using the same setup as what has been used in all other 80km top simulations.
Yes, but this is new territory and I'm not sure when or what was decided when the model was first set up.

[islasimpson]

Yeah, I was just using this to clarify to Julio what information was needed as I think he set it up for all other 80km top simulations

[dan800]

@JulioTBacmeister @adamrher @islasimpson: So I had a look at a FWscHIST.ne30_L83 case to see what's in the name list and chemistry. I think this is at least part of how it's set up in 'sc'

Molecular diffusion of species is off:
do_molec_press = 0.1D0 and the top level is 0.0081 hPa (0.8Pa)

non-LTE cooling calcs are on (I think) and use constituents in the WACCM forcing file as part of the calculation - CO2, O, O2 and O3 if I recall correctly.

The only prognostic constituents are H2O, CH4, N2O, CFC11, CFC12, H2O2, H2SO4, SO2 & DMS. The chemistry (if you can call it that) for CH4, N2O, CFC11 and CFC12 is just a simple loss mechanism - water is produced from the loss of CH4.
DMS is lost with reactions with OH and NO3 (both specfied) to produce SO2 which then reacts with OH to produce H2SO4. Not sure what H2O2 does in the model - probably something to do with the aerosols (its reaction with OH is a source of water but I doubt it's significant).

Looks like there's 20 aerosol species in total.

So not sure what switching off SC for a low top model gets you unless it drops some of these tracers. I can't imagine reading in a heating rate and calling Fomichev cost 10% of the model. But we can certainly see if turning off those two things really changes the 50 to 80 km region much. If it doesn't we can just ditch it for all CAM configurations.

[adamrher]

@islasimpson I may be mistaken, but I thought that one of the reasons for the L58 grid was to provide a better representation of the QBO, primarily getting the right phase/magnitude ~20km, so that we can have greater confidence we are capturing top-down impacts of QBO on the troposphere. Am I mis-remembering your work on this w/ Peter?

[islasimpson]

The L58 grid isn't going to improve the representation of anything in the stratosphere because it still has the same model lid height as the low top model. It doesn't have a QBO. There's really not much to gain in the stratosphere by having dz=500, without raising the model lid height. I should have thought of this when you said that you want to look at the QBO in these runs - they won't have a QBO. My work with Peter was more about polar vortex variability and, yes - polar vortex variability can have important impacts on tropospheric variability and change. Many others have shown this too. CAM is currently behind the times in having the model lid height where it is. But the L58 model is not improving on the model lid height. The L93 model is. For polar vortex impacts, I don't think dz=500 is necessary. We just need a higher model lid. The primary reason for choosing dz=500 was to get some detailed features of the QBO correct. I'm skeptical of the importance of this for the troposphere, but there are studies that show that the QBO has impacts on the MJO. However, we find that even with the improved representation of the QBO in this model, we still don't see the connection with the MJO.

So, long story short - you're right that the primary reason for having dz=500 was to get some features of the QBO right, in particular the amplitude in the lower stratosphere. But that's not going to happen in the L58 grid. It's only going to happen in the L93 grid.

[swrneale]

I think the decision that we will have a L93/80km model with (detailed?) chemistry for CMIP is done. Do we want to open that back up - I doubt it. Although that will be the end point it doesn't mean development is always done with that version of the model. In fact if you look at the remaining cpu-hrs in AMWG projects this year, there is simply NO WAY we can do that. I think the chemists have to answer the question of (1) How much chemistry is good enough and (2) is there any alternative to simply running full chemistry first to deliver 'specified chemistry' model datasets. I think for the predominantly tropospheric physics developments: PUMAS/EDMF/CLUBB we have to run L58 with specified chemistry (where the SC datasets come from in the short term might be a question). We just don't need L93 or chemistry to start to understand the model with these changes in (precipitation, clouds, aerosol interactions, surface coupling). When we decide on chemistry and how to deal with the lid/radiation in L93 then we will test with that configuration.

[dan800]

@islasimpson @swrneale I think you both make good points.

I think we are agreed that we need a tuned up high top model with chemistry (complexity tbd) the can be submitted to the next DECK. here I think there are open questions about what to do with physics and boundary conditions above the stratopause. A low top version without chemistry will be useful to tune tropospheric physics in the near term.

The question remains, once that physics is settled, do we focus on a L58 or L93 version (without chemistry) for other MIPS where the cost is prohibitive to run with interactive chemistry. What are we aiming to deliver to the other working groups in 18 - 24 months for CMIP7 is the question to be answered. It doesn't mean science can't be done with a low top model, but I would argue (and I think so would Isla) that for the next CMIP now is the time to commit to L93 - you can't look at tropospheric variability without a "high-top" model that resolves the stratosphere completely. A L58 for the LE will be scientifically compromised. We've been lagging other centers in this regard for years and I think I'd rather participate in fewer mips but do them the best we can.

[islasimpson]

I think with the L58 we're doubling the cost without gaining much. We're setting dz=500 in the troposphere and the only real reason to do that, from all the vertical resolution tests that we did, was to get the QBO as good as can be and in L58 there won't be a QBO. I think the only thing we gain here is with the higher boundary layer resolution and having dz=500 is a bit pointless except that it allow the same tuning to be done as for L93. I agree that we really need to figure out what is computationally feasible and what is optimal. If I had it my way with infinite human resources I would say that preferable to the current L58 would be a model with a similar number of levels but an 80km top, so coarser resolution than dz=500. But the difficulty with that is that this would very likely require different tuning than the L93, due to the different vertical resolution, so that may not be feasible. Although it would have the advantage that our water vapor issue would be less severe. I think that L58 can be useful for people who aren't that interested in climate variability and change and just want to focus on something that's pretty tropospheric without impacts from the polar vortex, but I agree with Dan that we are really behind the times when it comes to the model lid height and that we need to be changing this. That being said, I am not sure we need to have the most perfect QBO in the world or that we should be centering our decisions around this and I'm somewhat regretful that I didn't push this point harder when the vertical resolution decision was being made. But I realize that we have already made this decision and we probably need to just carry on, but I do think a coarser dz with an 80km top is a potential fall back, if it's computationally infeasible to do any actual science with this model.

[dan800]

if it's computationally infeasible to do any actual science with this model.

That's not stopping Earthworks! ;-)

[JulioTBacmeister]

I strongly disagree that we may not gain something at L58. After a decade an a half or more of tuning it shouldn't be a shock that the L32 model looks OK. I'd argue that coarse vertical resolution (and coarse time resolution for that matter) force the model physics to incorporate limiters and fixers that make its behavior difficult to understand in physical terms if not actually producing unphysical behavior. Tuning with bad resolution has also certainly led to many instances of getting an OK-looking answer for the wrong reasons. Why should I trust what we learn about connections between tropospheric, stratospheric or other variability if vertical wave propagation needs to compensate for massive implicit numerical diffusion/dispersion? And, it now seems to me the only reason water vapor in WACCM ever looked OK was because of high numerical diffusion. Why should I believe anything that model says about how stratospheric water will change in the future?

I do agree that for many applications that involve studying variability (especially extratropical) there isn't much point in going with the L58 configuration. On the other hand there are also variability questions MJO, MJO/TC connections, ENSO and other air-sea coupled modes that can be studied in L58.

[islasimpson]

Ok, fair points. I was going by the fact that there wasn't anything that we looked at that really changed in our FV tests between dz=700/800 and dz=500, except for the QBO. But it's true that stratospheric water vapor did, and maybe we just didn't look at the right things to see the impacts of limiters and fixers that you're talking about.

[islasimpson]

But you also wouldn't trust how stratospheric water vapor is going to change with a model lid height at 40km

[JulioTBacmeister]

No argument. I also agree that focussing on the QBO to the exclusion of other things is not ideal - especially given how sensitive the QBO is to parameterized GW.

[dan800]

Not sure we need to revisit the vertical grid again - I think we are down to deciding on a no-chemistry workhorse model lid L93 (80 km - sponge) vs L58 (35 km - sponge). I believe these have the same resolution in the lower atmosphere. Please correct me if I a wrong.

And, it now seems to me the only reason water vapor in WACCM ever looked OK was because of high numerical diffusion. Why should I believe anything that model says about how stratospheric water will change in the future?

@JulioTBacmeister The issue remains that there is a dry bias in the present model and we need to fix it. I would say you're not going to get the UTLS water vapor correct at all latitudes (and hence it's radiative forcing for climate) without accounting for methane oxidation or having an ozone profile that is consistent with the cold point tropopause (see other thread for how the met office adjusts ozone to match the cold point to avoid a heating mismatch). Regarding WACCM strat trends, as with all our model predictions, we look at how well we do under recent past (mostly satellite era) to see if we trust future predictions - there has been a ton of work done validating past trends in the stratosphere for SPARC CCMI / CCMVal by Doug, Bill R. and countless postdocs.

[JulioTBacmeister]

@dan800 I am sorry for my vituperation. Of course I know WACCM trends have been validated by looking at the recent past. This is all we can do, and there is every reason to expect for "linear" changes the model is behaving as nature does.

[adamrher]

Not sure we need to revisit the vertical grid again - I think we are down to deciding on a no-chemistry workhorse model lid L93 (80 km - sponge) vs L58 (35 km - sponge). I believe these have the same resolution in the lower atmosphere. Please correct me if I a wrong.

Yes this is true. Where L93 and L58 overlap, the grids are identical.

I think we are down to deciding on a no-chemistry workhorse model lid L93 (80 km - sponge) vs L58 (35 km - sponge).

Does this need to be an either or? Can we not have no-chemistry "workhorses" ... Since the grids overlap, the tunings will produce similar tropospheric climatologies, for features that both grids can realistically simulate (e.g., not extra-tropical variability). I don't view L58 as just a tool for tuning up L92 cheaply. I'm worried when I hear statement such as you need a high-top to get climate change right ... this is of course true, but it's a hammer of an argument. What is the state of the art in your field of interest? If it's clear that you can make significant advances w/o getting climate change exactly right, and you need a cheaper model to make progress in this field, e.g., sea-level rise, than you will get more mileage out of a low top model that has a similar ECS. How different is the CESM2 ECS in L70 WACCM vs. L32 CAM?

If it's not entirely clear at this point, I am personally optimistic about the L58 low top grid. It's increased BL resolution gives realistic looking Sc to Cu transitions (see here). Gains in resolution in the UTLS is yielding more accurate vertical transport (from a numerics perspective), but of course now exposes biases that only exacerbate the long-standing dry bias in CAM that Dan is referring to. And so agree with Dan that I think we need to pay close attention to the way we are treating methane and matching of ozone to the cold point, in these specified chemistry configurations.

[JulioTBacmeister]

I see the L58 as a tool we are providing to the community (including ourselves) for development of PBL physics, deep convection schemes etc. , i.e., things that control tropospheric climate to O(1) and are extremely unlikely to be affected in a significant way by whether we have a top near 45km or 85km or 150km. I would argue (less confidently due to aerosol processes) that the same holds for chemistry - we can tune/study basic tropospheric climate in L58 w/out chemistry. It is less straightforward to argue for DZ=500m. As Adam has pointed stratus deck cross-sections and their transitions "look" nicer in the DZ=500m grid. This is likely to reflect a more realistic representation of physical processes. At some level I'd say we should increase free-topo vertical resolution because it CAN'T be the wrong thing to do, and we can't really say beforehand what benefits a tuned-up model with DZ will yield. Otherwise, why not work on a really cheap model with a mixed layer PBL, and 10,5,3,2, ... additional layers?

As far as the L93 without chemistry I don't think it's a decision we need to make now. But I tend to agree with Adam, why can't we provide both? WACCM-SC is regularly used to study and tune middle atmospheric dynamics. I don't know exactly what the "specified chemistry" is but whatever it is, it is 5x faster isn't it. So why wouldn't we provide something similar for the L93 configuration?

[dan800]

I think we are confusing the model configurations we want with those needed for CMIP7. We have to guess what the MIPS will be that are in addition to the DECK+SSPs (which we'll need to do with chemistry at least once). If it for Dynvar MIP then it's likely we want a L93-SC. But what about the other MIPS? if a L58-SC is the best tool for those then we'll need to develop that one as well.

But that doesn't preclude developing L58-SC for other scientific endeavors. Adam and Julio may have things outside of CMIP7 that would benefit from such a model. So agreed it's not either/or.

What we want to figure out is what's the minimal number of configurations and how best to develop them. I already see things possibly out of order - lots of sims being done but skipping over getting the rad forcing / boundary conditions right. It might be easier to run first L93+chem to get radiatively active gases, then chop it to get the boundary conditions for water and temperature in the middle strat and compare L58 to L93 SCs. As long as we don't try to chuck a lot of new physics in I think we can take our time and get this right. I am still at the point of asking 'what exactly is SC for CAM?' and I co-wrote the WACCM-SC paper! Can anyone tell me what the UBC conditions for T, CH4, N2O and H2O are on either L93 or L58? We also need a doc on what is the same and what differs (e.g., how momentum is dumped from non-oro gw). In doing so, we may find ways to optimize the models for efficiency - seems we can toss at least two tracers (HORIZ and VERT), but that means a code change I believe. We may also be able to ditch non-LTE cooling in both models.

[dan800]

BTW, anyone figured out how we are going to handle the QBO signal in stratospheric ozone, which will be prescribed in SC, now that the QBO is not nudged and so will vary from ensemble member to ensemble member? Note, you can't point out that the tail isn't wagging the dog if the tail is taped to the dogs hind leg.

[dan800]

Another point re. what's in and what's out in CAM-SC. In the section &waccm_forcing_nl the waccm file (/glade/p/cesmdata/cseg/inputdata/atm/waccm/waccm_forcing/SCWACCM_forcing_zm_L70_1849-2015_CMIP6ensAvg_c181011.nc) has the following fields:
float PS(time, lat) ;
float CO2(time, lev, lat) ;
float H(time, lev, lat) ;
float NO(time, lev, lat) ;
float O(time, lev, lat) ;
float O2(time, lev, lat) ;
float O3(time, lev, lat) ;
float QRS_TOT(time, lev, lat) ;
NO is in there because NO radiative cooling is important above ~100 km. I hope it's not being used or calculated in CAM-SC with any lid that is below this altitude. If it is, it's another potential cost saving.

[dan800]

@JulioTBacmeister @adamrher @cecilehannay

I had a look at the env_build.xml in the most recent case listed in "issues" /glade/p/cesmdata/cseg/runs/cesm2_0/f.e21.FWscHIST.ne30_L48_BL10_cam6_3_046_control.hf.001/

<entry id="CAM_CONFIG_OPTS" value="-phys cam_dev -age_of_air_trcs -chem waccm_sc_mam4 -nlev 58">

I don't think you want -age_of_air_trcs on for these tests. It's just adding to the cost of the run.

[dan800]

Why not remove the add_default calls on those tracers? It keeps things cleaner - I'm not a fan of add_default, unless it's for PS, U, V, T, OMEGA and Z3.

[adamrher]

That does appear to work. Setting

history_chemistry = .false.
history_chemspecies_srf = .false.
history_waccm = .false.

Gets rid of all those aerosol species in the h0 file (among other things). To see what's left on the h0 file execute this command on cheyenne:

ncdump -h /glade/scratch/aherring/cam6_3_043_FWscHIST_ne30pg2_ne30pg2_mg17_L59dev_mct_600pes_220209_nodefhist0/run/cam6_3_043_FWscHIST_ne30pg2_ne30pg2_mg17_L59dev_mct_600pes_220209_nodefhist0.cam.h0.1850-01.nc

The namelist history_amwg is still defaulted to true, so I think that means everything needed by the amwg diagnostics is still there.

[adamrher]

We should also consider adding -microphys mg2 to the CONFIG_OPT as well, since that will remove the two graupel tracers ... Andrew says graupel doesn't matter at these resolutions.

[cecilehannay]

@adamrher: I really would like to try the radical road: Do empty tapes and add the variables we look at. There are hundreds of plots in the AMWG diags and I am sure that there are a bunch of plots we don't look at. I would rather collect the list of plots people look at and slowly build from there. The AMWG diags would still run. It would be just that the plots are missing.

I have output Pbytes of data in the last year and I am sure that only a very fraction was looked at.
If it doesn't work, we could always go back and do something like:

history_amwg= .true.
history_chemistry = .false.
history_chemspecies_srf = .false.
history_waccm = .false.

But I would prefer to try the radical approach. Then we would know what people really look at.

[adamrher]

But I wonder if we're putting too much emphasis on h0 files as the cause of our problems. For ten years there are 12 months * 10 years * nvars number of snapshots in the h0 files ... whereas daily has 365 times more time-samples than the h0 files ... and then another factor 4 for 6-hourly output.