Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/get arch adds an empty archive job to GEFS system #2772

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
5f444c9
added an arch_test.sh to global-workflow/jobs/rocoto
AntonMFernando-NOAA Jul 16, 2024
b81155e
Added config.arch file to global-workflow/parm/config/gefs
AntonMFernando-NOAA Jul 16, 2024
9262051
modified global-workflow/parm/config/gefs
AntonMFernando-NOAA Jul 16, 2024
50fc7d8
modified workflow/applications/gefs.py
AntonMFernando-NOAA Jul 16, 2024
976a7dd
modified global-workflow/workflow/rocoto/gefs_tasks.py
AntonMFernando-NOAA Jul 16, 2024
566b851
modified (removed blank line) global-workflow/workflow/rocoto/gefs_ta…
AntonMFernando-NOAA Jul 16, 2024
a13a53a
modified modified global-workflow/workflow/rocoto/gefs_tasks.py
AntonMFernando-NOAA Jul 16, 2024
c999ed4
added arch_test.sh to global-workflow/jobs/rocoto
AntonMFernando-NOAA Jul 16, 2024
084be61
removed arch_test.sh (error)
AntonMFernando-NOAA Jul 16, 2024
2f2ed22
added arch_test.sh
AntonMFernando-NOAA Jul 16, 2024
5c17345
modified arch_test.sh
AntonMFernando-NOAA Jul 16, 2024
c1e55ef
modified parm/config/gefs/config.resources
AntonMFernando-NOAA Jul 17, 2024
9b2343d
Modified /work2/noaa/global/antonf/global-workflow/workflow/rocoto/ge…
AntonMFernando-NOAA Jul 18, 2024
1e0cc66
modified /work2/noaa/global/antonf/global-workflow/workflow/rocoto/ge…
AntonMFernando-NOAA Jul 18, 2024
a95e2a0
modified global-workflow/workflow/rocoto/gefs_tasks.py
AntonMFernando-NOAA Jul 18, 2024
6b7cf11
Modified gefs_tasks.py
AntonMFernando-NOAA Jul 18, 2024
5ac0234
modified gefs_tasks.py
AntonMFernando-NOAA Jul 18, 2024
ffc6d83
modified gefs_tasks.py
AntonMFernando-NOAA Jul 18, 2024
9d15072
Modified gefs_task.py (added envars)
AntonMFernando-NOAA Jul 20, 2024
907dd5f
removed f in the dependencies
AntonMFernando-NOAA Jul 20, 2024
b71316f
removed f
AntonMFernando-NOAA Jul 20, 2024
a42eddd
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Jul 22, 2024
d9bfb67
gefs_tasks.py modified to reflect correct task names
AntonMFernando-NOAA Jul 24, 2024
db3f7a5
Merge branch 'feature/get-arch' of https://github.com/AntonMFernando-…
AntonMFernando-NOAA Jul 24, 2024
d83678a
resolved conflicts
AntonMFernando-NOAA Jul 31, 2024
ca45cae
Merge branch 'develop' into feature/get-arch
AntonMFernando-NOAA Jul 31, 2024
cf4708f
updated config.resources
AntonMFernando-NOAA Aug 1, 2024
af28b91
Merge branch 'feature/get-arch' of https://github.com/AntonMFernando-…
AntonMFernando-NOAA Aug 1, 2024
73e209b
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Aug 1, 2024
541b1ea
updated arch_test.sh
AntonMFernando-NOAA Aug 1, 2024
8cce2c4
Merge branch 'feature/get-arch' of https://github.com/AntonMFernando-…
AntonMFernando-NOAA Aug 1, 2024
c8eaccf
updated arch_test.sh
AntonMFernando-NOAA Aug 1, 2024
9f7ee8c
update gefs.py
AntonMFernando-NOAA Aug 1, 2024
c832d08
updated config.resources
AntonMFernando-NOAA Aug 1, 2024
c72c3d2
updated config.resources
AntonMFernando-NOAA Aug 1, 2024
a260f22
updated gefs_tasks.py
AntonMFernando-NOAA Aug 1, 2024
fe4d0f2
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Aug 2, 2024
9c4802a
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Aug 2, 2024
c408fab
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Aug 6, 2024
d0c216f
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Aug 6, 2024
faaac8b
modified arch_test.sh
AntonMFernando-NOAA Aug 6, 2024
b623210
Merge branch 'feature/get-arch' of https://github.com/AntonMFernando-…
AntonMFernando-NOAA Aug 6, 2024
6fb6cbf
Make arch_test.sh executable
Aug 6, 2024
0a90507
Merge branch 'NOAA-EMC:develop' into feature/get-arch
AntonMFernando-NOAA Aug 7, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions jobs/rocoto/arch_test.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
#! /usr/bin/env bash
###############################################################
exit 0
15 changes: 15 additions & 0 deletions parm/config/gefs/config.arch
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
#! /usr/bin/env bash

########## config.arch ##########
# Archive specific

echo "BEGIN: config.arch"

# Get task specific resources
. "${EXPDIR}/config.resources" arch

export ARCH_GAUSSIAN="YES"
export ARCH_GAUSSIAN_FHMAX=${FHMAX_GFS}
export ARCH_GAUSSIAN_FHINC=${FHOUT_GFS}

echo "END: config.arch"
7 changes: 7 additions & 0 deletions parm/config/gefs/config.resources
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,13 @@ case ${step} in
export is_exclusive=False
;;

"arch")
export walltime="06:00:00"
export ntasks=1
export tasks_per_node=1
export threads_per_task=1
export memory="4096M"
;;
*)
echo "FATAL ERROR: Invalid job ${step} passed to ${BASH_SOURCE[0]}"
exit 1
Expand Down
4 changes: 3 additions & 1 deletion workflow/applications/gefs.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ def _get_app_configs(self):
"""
Returns the config_files that are involved in gefs
"""
configs = ['stage_ic', 'fcst', 'atmos_products']
configs = ['stage_ic', 'fcst', 'atmos_products', 'arch']

if self.nens > 0:
configs += ['efcs', 'atmos_ensstat']
Expand Down Expand Up @@ -79,4 +79,6 @@ def get_task_names(self):
if self.do_extractvars:
tasks += ['extractvars']

tasks += ['arch']

return {f"{self._base['RUN']}": tasks}
41 changes: 41 additions & 0 deletions workflow/rocoto/gefs_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -546,3 +546,44 @@ def extractvars(self):
task = rocoto.create_task(member_metatask_dict)

return task

def arch(self):
deps = []
dep_dict = {'type': 'metatask', 'name': 'atmos_prod'}
deps.append(rocoto.add_dependency(dep_dict))
dep_dict = {'type': 'metatask', 'name': 'atmos_ensstat'}
deps.append(rocoto.add_dependency(dep_dict))
if self.app_config.do_ice:
dep_dict = {'type': 'metatask', 'name': 'ice_prod'}
deps.append(rocoto.add_dependency(dep_dict))
if self.app_config.do_ocean:
dep_dict = {'type': 'metatask', 'name': 'ocean_prod'}
deps.append(rocoto.add_dependency(dep_dict))
if self.app_config.do_wave:
dep_dict = {'type': 'metatask', 'name': 'wave_post_grid'}
deps.append(rocoto.add_dependency(dep_dict))
dep_dict = {'type': 'metatask', 'name': 'wave_post_pnt'}
deps.append(rocoto.add_dependency(dep_dict))
if self.app_config.do_wave_bnd:
dep_dict = {'type': 'metatask', 'name': 'wave_post_bndpnt'}
deps.append(rocoto.add_dependency(dep_dict))
dep_dict = {'type': 'metatask', 'name': 'wave_post_bndpnt_bull'}
deps.append(rocoto.add_dependency(dep_dict))
dependencies = rocoto.create_dependency(dep=deps, dep_condition='and')

resources = self.get_resource('arch')
task_name = 'arch'
task_dict = {'task_name': task_name,
'resources': resources,
'envars': self.envars,
'cycledef': 'gefs',
'dependency': dependencies,
'command': f'{self.HOMEgfs}/jobs/rocoto/arch_test.sh',
'job_name': f'{self.pslot}_{task_name}_@H',
'log': f'{self.rotdir}/logs/@Y@m@d@H/{task_name}.log',
'maxtries': '&MAXTRIES;'
}

task = rocoto.create_task(task_dict)

return task
Loading