Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stage atmospheric backgrounds and UFS cubed-sphere history files #2792

Merged
Merged
Show file tree
Hide file tree
Changes from 109 commits
Commits
Show all changes
122 commits
Select commit Hold shift + click to select a range
b5eb345
Use rename_atm of jcb parm erpos
danholdaway May 29, 2024
3a8f527
Initial commit
DavidNew-NOAA May 31, 2024
6904982
Update atm_berror_bump_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
0b37fe5
Update atm_berror_gsibec_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
d770b63
Update atm_berror_identity_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
0187bba
Update atm_bkg_fix.yaml.j2
DavidNew-NOAA Jun 3, 2024
4dcd47a
Rename some things
DavidNew-NOAA Jun 3, 2024
e61963d
Merge branch 'feature/stage_from_yaml' of https://github.com/DavidNew…
DavidNew-NOAA Jun 3, 2024
f91297d
Rename some things
DavidNew-NOAA Jun 3, 2024
983d720
Add changes to analysis python scripts
DavidNew-NOAA Jun 3, 2024
9436f4b
update hashes and config
danholdaway Jun 4, 2024
97a63a7
Merge remote-tracking branch 'upsteam/develop' into feature/rename_atm
danholdaway Jun 4, 2024
6e44a5f
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 4, 2024
2490049
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 4, 2024
3022393
Minor fix
DavidNew-NOAA Jun 4, 2024
cc1a3b6
Minor update
DavidNew-NOAA Jun 4, 2024
2097116
Minor update
DavidNew-NOAA Jun 4, 2024
5327711
Coding norms
DavidNew-NOAA Jun 4, 2024
2702ef0
Remove jcb from GW submodules
danholdaway Jun 4, 2024
b6456be
Remove JCB
danholdaway Jun 4, 2024
2a5f301
adjust link to point to new jcb location in gdasapp
danholdaway Jun 4, 2024
9f425a6
Update GDASapp Hash
danholdaway Jun 4, 2024
4d36caf
Move some stuff around
DavidNew-NOAA Jun 4, 2024
55358e5
Merge remote-tracking branch 'upsteam/develop' into feature/rename_atm
danholdaway Jun 4, 2024
9e357f9
gdas hash
danholdaway Jun 4, 2024
ba2d12d
Add time loop for some staging
DavidNew-NOAA Jun 5, 2024
d0d4429
bump gdas
danholdaway Jun 6, 2024
ee1aa30
bump gdas
danholdaway Jun 6, 2024
2138df1
Merge branch 'NOAA-EMC:develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 6, 2024
09ec021
hashes
danholdaway Jun 6, 2024
a5db89e
update gdas hash
danholdaway Jun 6, 2024
b71d285
Merge remote-tracking branch 'upstream/develop' into feature/move_jcb
danholdaway Jun 6, 2024
1bafbe8
bring russ change in
danholdaway Jun 6, 2024
e52e0de
follow russ for GSI utils hash
danholdaway Jun 6, 2024
4d3150c
revert gsi utils
danholdaway Jun 7, 2024
9c4f77b
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 7, 2024
f70b0ad
Remove all references to mytask.config and mytask.runtime_config
DavidNew-NOAA Jun 7, 2024
3f8ba42
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 7, 2024
49f8593
Update wxflow hash
DavidNew-NOAA Jun 7, 2024
2e56698
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 11, 2024
b6213a5
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 11, 2024
2bd9949
Tab out Jinja2 templates for readability
DavidNew-NOAA Jun 11, 2024
46f81e5
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 12, 2024
458aa34
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 12, 2024
eebeb4b
Merge branch 'develop' into feature/move_jcb
danholdaway Jun 12, 2024
488a3de
update gdas hash
danholdaway Jun 13, 2024
d9501c5
Merge branch 'feature/move_jcb' of https://github.com/danholdaway/glo…
danholdaway Jun 13, 2024
2cb6664
Use replace_tmpl filter
DavidNew-NOAA Jun 13, 2024
bd7f770
Update wxflow hash
DavidNew-NOAA Jun 13, 2024
b714cca
Merge remote-tracking branch 'upstream/develop' into feature/move_jcb
danholdaway Jun 13, 2024
4de3199
Merge branch 'develop' into feature/move_jcb
WalterKolczynski-NOAA Jun 13, 2024
f76bef4
update to latest gdas app
danholdaway Jun 13, 2024
23971d2
mod
danholdaway Jun 13, 2024
d5cb822
add back ufsutils
danholdaway Jun 13, 2024
f8a18be
Update wxflow hash and fix big in lgetkf staging file
DavidNew-NOAA Jun 14, 2024
4b21d8a
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 17, 2024
c6223c4
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 17, 2024
18c596c
Merge branch 'feature/stage_from_yaml' of https://github.com/DavidNew…
DavidNew-NOAA Jun 17, 2024
2c8feb5
Simplify Jinja2 templates a bit
DavidNew-NOAA Jun 17, 2024
2c5ef59
Replace CDUMP with RUN
DavidNew-NOAA Jun 17, 2024
dcfc40f
Fix bug
DavidNew-NOAA Jun 17, 2024
a3d0d06
Merge branch 'feature/rename_atm' of https://github.com/danholdaway/g…
RussTreadon-NOAA Jun 18, 2024
584762a
update gdas.cd hash to bring in updated jcb, jcb-gdas, and jcb-algori…
RussTreadon-NOAA Jun 18, 2024
ee4397a
Merge branch 'NOAA-EMC:develop' into feature/rename_atm
RussTreadon-NOAA Jun 19, 2024
39e719d
Merge branch 'NOAA-EMC:develop' into feature/rename_atm
RussTreadon-NOAA Jun 20, 2024
9d1c2ec
Merge branch 'develop' into feature/stage_from_yaml
DavidNew-NOAA Jun 20, 2024
dd6613b
remove path specific logic to identify MSU machines (#2695)
RussTreadon-NOAA Jun 20, 2024
826c3c3
add ocnanalecen to HERCULES.env (#2699)
RussTreadon-NOAA Jun 20, 2024
c14607f
fix shellcheck error in detect_machine.sh (#2695)
RussTreadon-NOAA Jun 20, 2024
9973c70
fix pycodestyle error in hosts.py (#2695)
RussTreadon-NOAA Jun 20, 2024
ddcd376
Updates
DavidNew-NOAA Jun 20, 2024
7aa041e
Refactor aero_prepobs.py like other tasks
DavidNew-NOAA Jun 20, 2024
e175b25
Update wxflow hash
DavidNew-NOAA Jun 20, 2024
b00d31e
Merge remote-tracking branch 'upstream/feature/stage_from_yaml' into …
RussTreadon-NOAA Jun 21, 2024
7cc86d9
update gdas.cd to resolve wxflow issues and update jcb hashes (#2699)
RussTreadon-NOAA Jun 21, 2024
f053ea1
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 21, 2024
8663123
add wxflow to PYTHONPATH for ufsda jobs; update prep_snow_obs script …
RussTreadon-NOAA Jun 22, 2024
076a59d
simply wxflow path for ufsda jobs (#2654)
RussTreadon-NOAA Jun 23, 2024
19f35e9
update gdas.cd hash to fix issues with aero and soca DA (#2700)
RussTreadon-NOAA Jun 23, 2024
3768b9d
do not set wxflowPATH in rocoto j-job drivers, add wxflowPATH to load…
RussTreadon-NOAA Jun 24, 2024
32a2002
update gdas.cd hash to simplify wxflowpath in GDASApp ctests (#2700)
RussTreadon-NOAA Jun 24, 2024
d1d88a6
Merge branch 'develop' into feature/rename_atm
aerorahul Jun 24, 2024
8fc02e2
add g-w python to PYTHONPATH on WCOSS2 in upp.sh (#2700)
RussTreadon-NOAA Jun 24, 2024
bc33c2d
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 25, 2024
8da0821
Merge branch 'develop' into feature/rename_atm
aerorahul Jun 26, 2024
3e84035
Merge branch 'NOAA-EMC:develop' into feature/rename_atm
RussTreadon-NOAA Jun 26, 2024
d70afac
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 26, 2024
72bf7d8
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 27, 2024
c1ef4b3
comment out find lines for purge_every_days (#2719)
RussTreadon-NOAA Jun 28, 2024
f551018
Update scripts/exglobal_cleanup.sh
aerorahul Jun 28, 2024
dcec081
Merge branch 'develop' into feature/rename_atm
RussTreadon-NOAA Jun 28, 2024
cc99a5a
Update
DavidNew-NOAA Jun 28, 2024
994215b
Merge branch 'feature/rename_atm' into feature/cubed_sphere_bkg
DavidNew-NOAA Jun 28, 2024
2ff0e28
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Jul 1, 2024
0335921
Merge branch 'NOAA-EMC:develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Jul 18, 2024
16eaef3
Update GDAS hash
DavidNew-NOAA Jul 18, 2024
a3928d2
Revert "Address issues in creating XML for GFS forecast-only with app…
DavidNew-NOAA Jul 18, 2024
9c7a5b6
Update GDAS hash
DavidNew-NOAA Jul 18, 2024
b6c4316
Revert "Revert "Address issues in creating XML for GFS forecast-only …
DavidNew-NOAA Jul 18, 2024
0fdd889
Revert "Update GDAS hash"
DavidNew-NOAA Jul 18, 2024
d4257a6
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Jul 18, 2024
41f5043
Update GDAS hash
DavidNew-NOAA Jul 19, 2024
f32bb45
Update GDAS hash
DavidNew-NOAA Jul 19, 2024
5c31302
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Jul 25, 2024
b1fc756
Update GDAS hash
DavidNew-NOAA Jul 25, 2024
3e30c8b
Update atm_var_bkg.yaml.j2
DavidNew-NOAA Jul 25, 2024
018f24b
Update atm_var_fv3ens.yaml.j2
DavidNew-NOAA Jul 25, 2024
6e85636
Temporarily disable METplus jobs (#2796)
DavidHuber-NOAA Jul 25, 2024
48442e9
Update GDAS hash
DavidNew-NOAA Jul 26, 2024
ca54020
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Jul 26, 2024
8f7e91b
Update GDAS hash
DavidNew-NOAA Jul 30, 2024
1141774
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Jul 30, 2024
c0ab2e7
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 1, 2024
586800c
Update GDAS hash
DavidNew-NOAA Aug 1, 2024
b76c68d
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 6, 2024
352b8c2
Update GDAS hash
DavidNew-NOAA Aug 6, 2024
0197f3f
Update GDAS hash
DavidNew-NOAA Aug 6, 2024
fb9909b
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 6, 2024
904d38a
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 6, 2024
6fa309e
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 7, 2024
ed500db
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 7, 2024
4784d24
Merge branch 'develop' into feature/cubed_sphere_bkg
DavidNew-NOAA Aug 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions parm/config/gfs/config.base
Original file line number Diff line number Diff line change
Expand Up @@ -453,6 +453,8 @@ export binary_diag=".false."

# Verification options
export DO_METP="@DO_METP@" # Run METPLUS jobs - set METPLUS settings in config.metp
# TODO Reenable METplus jobs when issue #2790 is resolved
export DO_METP="NO"
export DO_FIT2OBS="YES" # Run fit to observations package
export DO_VRFY_OCEANDA="@DO_VRFY_OCEANDA@" # Run SOCA Ocean and Seaice DA verification tasks

Expand Down
27 changes: 9 additions & 18 deletions parm/gdas/staging/atm_lgetkf_bkg.yaml.j2
Original file line number Diff line number Diff line change
@@ -1,6 +1,3 @@
{% set ftype_list = ['fv_core.res', 'fv_srf_wnd.res', 'fv_tracer.res', 'phy_data', 'sfc_data'] %}
{% set time_list = [current_cycle] %}

mkdir:
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
Expand All @@ -14,19 +11,13 @@ mkdir:
- '{{ COM_ATMOS_ANALYSIS_TMPL | replace_tmpl(tmpl_dict) }}'
{% endfor %}
copy:
{% for time in time_list %}
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': 'enkfgdas',
'${YMD}': previous_cycle | to_YMD,
'${HH}': previous_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.coupler.res', '{{ DATA }}/bkg/{{ memchar }}/']
{% for ftype in ftype_list %}
{% for itile in range(1,7) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.{{ ftype }}.tile{{ itile }}.nc', '{{ DATA }}/bkg/{{ memchar }}/']
{% endfor %}
{% endfor %}
{% endfor %}
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': 'enkfgdas',
'${YMD}': previous_cycle | to_YMD,
'${HH}': previous_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- ['{{ COM_ATMOS_HISTORY_TMPL | replace_tmpl(tmpl_dict) }}/enkf{{ GPREFIX }}cubed_sphere_grid_atmf006.nc', '{{ DATA }}/bkg/{{ memchar }}/']
- ['{{ COM_ATMOS_HISTORY_TMPL | replace_tmpl(tmpl_dict) }}/enkf{{ GPREFIX }}cubed_sphere_grid_sfcf006.nc', '{{ DATA }}/bkg/{{ memchar }}/']
{% endfor %}
13 changes: 2 additions & 11 deletions parm/gdas/staging/atm_var_bkg.yaml.j2
Original file line number Diff line number Diff line change
@@ -1,14 +1,5 @@
{% set ftype_list = ['fv_core.res', 'fv_srf_wnd.res', 'fv_tracer.res', 'phy_data', 'sfc_data'] %}
{% set time_list = [current_cycle] %}

mkdir:
- '{{ DATA }}/bkg'
copy:
{% for time in time_list %}
- ['{{ COM_ATMOS_RESTART_PREV }}/{{ time | to_fv3time }}.coupler.res', '{{ DATA }}/bkg/']
{% for ftype in ftype_list %}
{% for itile in range(1,ntiles+1) %}
- ['{{ COM_ATMOS_RESTART_PREV }}/{{ time | to_fv3time }}.{{ ftype }}.tile{{ itile }}.nc', '{{ DATA }}/bkg/']
{% endfor %}
{% endfor %}
{% endfor %}
- ['{{ COM_ATMOS_HISTORY_PREV }}/{{ GPREFIX }}cubed_sphere_grid_atmf006.nc', '{{ DATA }}/bkg/']
- ['{{ COM_ATMOS_HISTORY_PREV }}/{{ GPREFIX }}cubed_sphere_grid_sfcf006.nc', '{{ DATA }}/bkg/']
27 changes: 9 additions & 18 deletions parm/gdas/staging/atm_var_fv3ens.yaml.j2
Original file line number Diff line number Diff line change
@@ -1,24 +1,15 @@
{% set ftype_list = ['fv_core.res', 'fv_srf_wnd.res', 'fv_tracer.res', 'phy_data', 'sfc_data'] %}
{% set time_list = [current_cycle] %}

mkdir:
{% for imem in range(1,NMEM_ENS+1) %}
- '{{ DATA }}/ens/{{ 'mem%03d' | format(imem) }}'
{% endfor %}
copy:
{% for time in time_list %}
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': 'enkfgdas',
'${YMD}': previous_cycle | to_YMD,
'${HH}': previous_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.coupler.res', '{{ DATA }}/ens/{{ memchar }}/']
{% for ftype in ftype_list %}
{% for itile in range(1,ntiles+1) %}
- ['{{ COM_ATMOS_RESTART_TMPL | replace_tmpl(tmpl_dict) }}/{{ time | to_fv3time }}.{{ ftype }}.tile{{ itile }}.nc', '{{ DATA }}/ens/{{ memchar }}/']
{% endfor %}
{% endfor %}
{% endfor %}
{% for imem in range(1,NMEM_ENS+1) %}
{% set memchar = 'mem%03d' | format(imem) %}
{% set tmpl_dict = ({ '${ROTDIR}': ROTDIR,
'${RUN}': 'enkfgdas',
'${YMD}': previous_cycle | to_YMD,
'${HH}': previous_cycle | strftime('%H'),
'${MEMDIR}': memchar }) %}
- ['{{ COM_ATMOS_HISTORY_TMPL | replace_tmpl(tmpl_dict) }}/enkf{{ GPREFIX }}cubed_sphere_grid_atmf006.nc', '{{ DATA }}/ens/{{ memchar }}/']
- ['{{ COM_ATMOS_HISTORY_TMPL | replace_tmpl(tmpl_dict) }}/enkf{{ GPREFIX }}cubed_sphere_grid_sfcf006.nc', '{{ DATA }}/ens/{{ memchar }}/']
{% endfor %}
2 changes: 2 additions & 0 deletions ush/forecast_postdet.sh
Original file line number Diff line number Diff line change
Expand Up @@ -209,6 +209,8 @@ EOF
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atmf${FH3}.nc" "atmf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.sfcf${FH3}.nc" "sfcf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.atm.logf${FH3}.txt" "log.atm.f${FH3}"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.cubed_sphere_grid_atmf${FH3}.nc" "cubed_sphere_grid_atmf${FH3}.nc"
${NLN} "${COMOUT_ATMOS_HISTORY}/${RUN}.t${cyc}z.cubed_sphere_grid_sfcf${FH3}.nc" "cubed_sphere_grid_sfcf${FH3}.nc"
if [[ "${WRITE_DOPOST}" == ".true." ]]; then
${NLN} "${COMOUT_ATMOS_MASTER}/${RUN}.t${cyc}z.master.grb2f${FH3}" "GFSPRS.GrbF${FH2}"
${NLN} "${COMOUT_ATMOS_MASTER}/${RUN}.t${cyc}z.sfluxgrbf${FH3}.grib2" "GFSFLX.GrbF${FH2}"
Expand Down
4 changes: 2 additions & 2 deletions ush/parsing_model_configure_FV3.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,15 +31,15 @@ local WRITE_GROUP=${WRITE_GROUP:-1}
local WRTTASK_PER_GROUP=${WRTTASK_PER_GROUP:-24}
local ITASKS=1
local OUTPUT_HISTORY=${OUTPUT_HISTORY:-".true."}
local HISTORY_FILE_ON_NATIVE_GRID=".false."
local HISTORY_FILE_ON_NATIVE_GRID=".true."
local WRITE_DOPOST=${WRITE_DOPOST:-".false."}
local WRITE_NSFLIP=${WRITE_NSFLIP:-".false."}
local NUM_FILES=${NUM_FILES:-2}
local FILENAME_BASE="'atm' 'sfc'"
# OUTPUT_GRID
local OUTPUT_FILE="'${OUTPUT_FILETYPE_ATM}' '${OUTPUT_FILETYPE_SFC}'"
local ZSTANDARD_LEVEL=0
local IDEFLATE=1 # netCDF zlib lossless compression (0-9); 0: no compression
local IDEFLATE=0 # netCDF zlib lossless compression (0-9); 0: no compression
local QUANTIZE_NSD=${QUANTIZE_NSD:-0} # netCDF compression
local ICHUNK2D=$((4*restile))
local JCHUNK2D=$((2*restile))
Expand Down
Loading