cam6_4_019: New CLUBB External to fix GPU problem #1086
Conversation
…nto cam_derecho_test
Katetc
requested review from
sjsprecious,
cacraigucar,
adamrher,
nusbaume and
bstephens82
There was a problem hiding this comment.
The current CLUBB GPU tag runs successfully on Derecho's GPU but fails the ensemble consistency test. The UWM group is debugging the GPU code now but we (CISL group) are fine with moving on with this PR and including the new CLUBB GPU tag in the CAM regression test.
|
@Katetc I'm under the impression that these new clubb externals remove the ghost point from the thermodynamic 'zt' grid. But I'm not seeing a reduction in the size of the zt arrays in clubb_intr in this PR. What am I missing? |
.gitmodules
Outdated
There was a problem hiding this comment.
The ccs_config tag needs to be updated to ccs_config_cesm1.0.0 so that the CLUBB GPU code can be compiled correctly on the GPU.
There was a problem hiding this comment.
The current head of cam_development points to ccs_config_cesm1.0.0 so when this branch is merged up, the external will be correctly versioned.
There was a problem hiding this comment.
Thanks Kate. That is great!
src/physics/cam/clubb_intr.F90
Outdated
| @@ -1810,18 +1850,25 @@ subroutine clubb_ini_cam(pbuf2d) | |||
| call addfld ( 'edmf_qtflx' , (/ 'ilev' /), 'A', 'W/m2' , 'qt flux (EDMF)' ) | |||
| end if | |||
|
|
|||
| #ifndef SILHS | |||
There was a problem hiding this comment.
Couldn't this be wrapped in a if ( trim(subcol_scheme) == 'SILHS' ) then? The only reason to put code like this inside a #ifndef directive is to hide it completely from the compiler. I believe that hm_metadata and the two fields always are declared.
There was a problem hiding this comment.
Seems like it should be fine. I'm updating to just a straight "if" here.
There was a problem hiding this comment.
Should probably use /= instead of .ne. to adhere to the CAM coding standards (it is a should and not a must)
|
@adamrher and @bstephens82 - @Katetc requested reviews from both of you. Please either remove yourself as a reviewer or do a review when you can. |
There was a problem hiding this comment.
I'm approving this PR, but would like to understand why all the arrays on the thermodynamic grid in clubb_intr.F90 are still retaining their ghost points. It was my understanding that this PR is removing the ghost point? Is the plan to just do that in the clubb externals, not the CAM interface?
For example, rho_zt is still of size dimension(state%ncol,pverp+1-top_lev) in clubb_intr.F90; it has the ghost point.
This code merely changes some lower BCs to one-sided derivatives within CLUBB core (and fixes some minor bugs involving full-level variables being placed on the wrong (half-level) grid levels). This PR doesn't change the clubb_intr, but UWM hopes to do that at some point. |
|
@adamrher, do you think that the changes in the solution look "large", or do they look like what you'd expect from truncation error (numerics)? |
|
Thanks for getting back to me on the ghost point. All is good then.
The global mean turbulent fluxes and radiative fluxes look ~identical, so that suggests round-off level changes. There's some differences in the lat-lon map plots but those are what you would expect from comparing to two, 2 year long runs. |
src/physics/cam/clubb_intr.F90
Outdated
| @@ -1810,18 +1850,25 @@ subroutine clubb_ini_cam(pbuf2d) | |||
| call addfld ( 'edmf_qtflx' , (/ 'ilev' /), 'A', 'W/m2' , 'qt flux (EDMF)' ) | |||
| end if | |||
|
|
|||
| #ifndef SILHS | |||
There was a problem hiding this comment.
Should probably use /= instead of .ne. to adhere to the CAM coding standards (it is a should and not a must)
I had a minor bug fix too, so I fixed this "if" statement as well. |
cacraigucar
changed the title
Add GPU regression testing
Small answer changes, but no new CLUBB science. Diagnostics can be found here:
https://webext.cgd.ucar.edu/F2000climo/newCLUBBtesting/larson_tag_20240605.katemerge.062724-0201.F2000dev.f09_f09_mg17_1_2_vs_larson_tag_control.cam6_3_162.062724-1359.F2000dev.f09_f09_mg17_1_2/website/index.html
Ran 6 year ne30 BLT1850 simulations to look at differences in applicable and longer runs. Had some trouble with diagnostics but the results can be found here:
https://webext.cgd.ucar.edu/FHIST/clubb_tests/larson_tag_20240605.katemerge.071924-1633.FLTHIST.ne30pg3_g17_1979_1984_vs_larson_tag_control.cam6_4_007.071924-1639.FLTHIST.ne30pg3_g17_1979_1984/website/
Differences seem to be minimal.
Fixes #1036
closes #1048