Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Separate tracker and genesis jobs from vrfy job #2058

Merged
merged 24 commits into from
Nov 14, 2023
Merged

Separate tracker and genesis jobs from vrfy job #2058

merged 24 commits into from
Nov 14, 2023

Commits on Nov 3, 2023

  1. Create rocoto job scripts for tracker/genesis 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    eae8120 View commit details
    Browse the repository at this point in the history

  2. Add tracker/genesis settings to config.base 

    - Add flags to turn on/off the tracker and genesis jobs
- Add HOMEens_tracker path setting
- Settings previously found on config.vrfy

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    3b0035d View commit details
    Browse the repository at this point in the history

  3. Remove tracker/genesis settings from config.vrfy 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    65526d2 View commit details
    Browse the repository at this point in the history

  4. Remove tracker/genesis from vrfy rocoto job script 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    0a325c6 View commit details
    Browse the repository at this point in the history

  5. Create configs for tracker/genesis jobs 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    91b9aad View commit details
    Browse the repository at this point in the history

  6. Add tracker/genesis to resource config 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 3, 2023
    Configuration menu
    Copy the full SHA
    ec25fa9 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 6, 2023
    Configuration menu
    Copy the full SHA
    d794dde View commit details
    Browse the repository at this point in the history

  2. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 6, 2023
    Configuration menu
    Copy the full SHA
    d1c0794 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 7, 2023
    Configuration menu
    Copy the full SHA
    7211e69 View commit details
    Browse the repository at this point in the history

  2. Add tracker and genesis jobs to setup 

    When jobs turned on:
- Add jobs to rocoto mesh.
- Add configs to list of configs to source.
- Add jobs to gfsarch dependency list.

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    b26dad4 View commit details
    Browse the repository at this point in the history

  3. Reduce vrfy job resources 

    - Drop walltime to 10 mins.
- Drop node usage to 1.
- Remove exclusive flag, no longer needed.

The job needs very little resources now ahead of the final
pieces being removed.

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    8b86919 View commit details
    Browse the repository at this point in the history

  4. Update jjob_header arguments for tracker/genesis 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    ea3f4ca View commit details
    Browse the repository at this point in the history

  5. Change tracker_ver to ens_tracker_ver 

    Change version variable to match what JJOB scripts use

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    64bfe75 View commit details
    Browse the repository at this point in the history

  6. Create and use config.tropcy 

    - Create new config for common tracker and genesis settings.
- Update tracker and genesis configs to source config.tropcy.

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    9311843 View commit details
    Browse the repository at this point in the history

  7. Tropcy config updates 

    - Move HOMEens_tracker setting to config.tropcy.
- Delete COMINsyn setting from config.tropcy (will use
setting from config.base).

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    f0b9ecd View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    88a749a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    db8c460 View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 9, 2023
    Configuration menu
    Copy the full SHA
    f16996d View commit details
    Browse the repository at this point in the history

  2. Remove extra line in config.resources 

    Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    91ad811 View commit details
    Browse the repository at this point in the history

  3. Add tracker/genesis to forecast-only mode 

    Also correct syntax for adding jobs to tasks.py

Refs NOAA-EMC#1988
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    5f4e66e View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 14, 2023
    Configuration menu
    Copy the full SHA
    1ef9611 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/develop' into feature/tracker_… 

    …jobs_separate

* upstream/develop:
  Refine fit2obs job in rocoto mesh (NOAA-EMC#2047)
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    7d53fc7 View commit details
    Browse the repository at this point in the history

  3. Update post dependencies for archive job 

    - Adjust post metatask dependency to always be added, instead of
only when there aren't other dependencies.
- Add postanl job to archive dependencies.

Refs NOAA-EMC#2048
    @KateFriedman-NOAA
    KateFriedman-NOAA committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    07f97a8 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'NOAA-EMC:develop' into feature/tracker_jobs_separate

    @KateFriedman-NOAA
    KateFriedman-NOAA authored Nov 14, 2023
    Configuration menu
    Copy the full SHA
    baebcfb View commit details
    Browse the repository at this point in the history