Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Assign machine- and RUN-specific resources #2672

Merged
merged 59 commits into from
Jun 25, 2024
Merged

Assign machine- and RUN-specific resources #2672

merged 59 commits into from
Jun 25, 2024

Commits on Jun 10, 2024

  1. Add run-specific resource variables

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    8f0185a View commit details
    Browse the repository at this point in the history

  2. Add machine-specific resource files

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    420edf8 View commit details
    Browse the repository at this point in the history

  3. Force npe_node_eobs to be set properly NOAA-EMC#2092

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    8376ee6 View commit details
    Browse the repository at this point in the history

  4. Use run-specific variables

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    aacecb0 View commit details
    Browse the repository at this point in the history

  5. Fix tabbing

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    cf0e397 View commit details
    Browse the repository at this point in the history

  6. Make nth_anal independent of run

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    463c931 View commit details
    Browse the repository at this point in the history

  7. Fix gdas FHMAX/OUT variable

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    a9ebe3b View commit details
    Browse the repository at this point in the history

  8. Correct enkf task variable name

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    e5084f9 View commit details
    Browse the repository at this point in the history

  9. Fixed shellcheck issues

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    6bed0d1 View commit details
    Browse the repository at this point in the history

  10. Fix more shellcheck issues

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    b716671 View commit details
    Browse the repository at this point in the history

  11. Remove unused case

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 10, 2024
    Configuration menu
    Copy the full SHA
    e2e2421 View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. Make additional fcst, efcs, anal variables RUN-specific

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    7fd5f0e View commit details
    Browse the repository at this point in the history

  2. Strip enkf from RUN when reading config.anal

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    ffec316 View commit details
    Browse the repository at this point in the history

  3. Add double quotes

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    b80814b View commit details
    Browse the repository at this point in the history

  4. Fix fcst/efcs RUNs in variable declarations

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    63b97dc View commit details
    Browse the repository at this point in the history

  5. Fix C768 analysis case in config.resources

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    552ddda View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Corrected machine-specific resource config names

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    27cdab4 View commit details
    Browse the repository at this point in the history

  2. Add memory requirement for WCOSS2 prep job

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    12b880b View commit details
    Browse the repository at this point in the history

  3. Adjust epos and analcalc high res time limits/memory upward

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    41134d5 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. Increase memory for UPP @C384 on WCOSS2.

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    7271bfe View commit details
    Browse the repository at this point in the history

  2. Make wtime for waveprep RUN-agnostic

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    4e28b2c View commit details
    Browse the repository at this point in the history

  3. Declare runtime job resource variables based on RUN

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    9166934 View commit details
    Browse the repository at this point in the history

  4. Clean up env files

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    10a75b9 View commit details
    Browse the repository at this point in the history

  5. Add {} around EXPDIR

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    d9e2f32 View commit details
    Browse the repository at this point in the history

  6. Add double quotes to nth_

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    824c16f View commit details
    Browse the repository at this point in the history

  7. Fix resource tests

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    b21f736 View commit details
    Browse the repository at this point in the history

  8. Merge remote-tracking branch 'emc/develop' into feature/machine_res

    David Huber committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    e14a0e1 View commit details
    Browse the repository at this point in the history

  9. Use -n instead of ! -z

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    24946a6 View commit details
    Browse the repository at this point in the history

Commits on Jun 14, 2024

  1. Declare one variable per line

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    803add5 View commit details
    Browse the repository at this point in the history

  2. Use the environment bash for new config files

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    dae27f0 View commit details
    Browse the repository at this point in the history

  3. Declare one variable per line; unset dynamic variables

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    3204824 View commit details
    Browse the repository at this point in the history

  4. Add comment on sourcing machine resources file

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    7c4e095 View commit details
    Browse the repository at this point in the history

  5. Merge remote-tracking branch 'emc/develop' into feature/machine_res

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 14, 2024
    Configuration menu
    Copy the full SHA
    a80e008 View commit details
    Browse the repository at this point in the history

Commits on Jun 17, 2024

  1. Merge remote-tracking branch 'emc/develop' into feature/machine_res

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    93248b3 View commit details
    Browse the repository at this point in the history

  2. Rename mem/node variable, move to config.resources.

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    0286dd0 View commit details
    Browse the repository at this point in the history

  3. Ignore unused shellcheck warnings

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 17, 2024
    Configuration menu
    Copy the full SHA
    238ae98 View commit details
    Browse the repository at this point in the history

Commits on Jun 18, 2024

  1. Apply suggestions from Walter's code review 

    Co-authored-by: Walter Kolczynski - NOAA <[email protected]>
    @DavidHuber-NOAA @WalterKolczynski-NOAA
    DavidHuber-NOAA and WalterKolczynski-NOAA authored Jun 18, 2024
    Configuration menu
    Copy the full SHA
    e5680e8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'feature/machine_res' of github.com:DavidHuber-NOAA/glob… 

    …al-workflow into feature/machine_res
    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    c499a96 View commit details
    Browse the repository at this point in the history

  3. Increase memory requirement on S4 for ediag

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 18, 2024
    Configuration menu
    Copy the full SHA
    ac0778d View commit details
    Browse the repository at this point in the history

Commits on Jun 19, 2024

  1. Adjusted available memory on kjet to what is accepted by the job sche… 

    …duler, 93GB is rejected
    @InnocentSouopgui-NOAA
    InnocentSouopgui-NOAA committed Jun 19, 2024
    Configuration menu
    Copy the full SHA
    c5659e3 View commit details
    Browse the repository at this point in the history

Commits on Jun 20, 2024

  1. Merge branch 'feature/machine_res' of github.com:DavidHuber-NOAA/glob… 

    …al-workflow into feature/machine_res
    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    0562ef0 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'emc/develop' into feature/machine_res

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    03496d5 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'feature/machine_res' of github.com:InnocentSouopgui-NOA… 

    …A/global-workflow into feature/machine_res

bringing changes from Jet.
    @InnocentSouopgui-NOAA
    InnocentSouopgui-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    71704d3 View commit details
    Browse the repository at this point in the history

  4. Prevent OpenMP over-allocation of resources on S4

    @InnocentSouopgui-NOAA
    InnocentSouopgui-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    1caf612 View commit details
    Browse the repository at this point in the history

  5. Add array length check for older bash versions.

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    7cc2753 View commit details
    Browse the repository at this point in the history

  6. Recalculate analysis npe_node when redefining threads for machines

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    895d018 View commit details
    Browse the repository at this point in the history

  7. Correct rocoto version in hercules setup module.

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    b901c47 View commit details
    Browse the repository at this point in the history

  8. Only run analyses @ 8 threads at C768 on WCOSS2.

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    fdd0850 View commit details
    Browse the repository at this point in the history

  9. Merge branch 'feature/machine_res' of https://github.com/DavidHuber-N… 

    …OAA/global-workflow into feature/machine_res

Pulling new updates from Dave
    @InnocentSouopgui-NOAA
    InnocentSouopgui-NOAA committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    7ba6006 View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. fix typo preventing a S4 to check array size

    @InnocentSouopgui-NOAA
    InnocentSouopgui-NOAA committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    859da0a View commit details
    Browse the repository at this point in the history

  2. High res wallclock/memory adjustments

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    f34bdd6 View commit details
    Browse the repository at this point in the history

  3. Correct max mem requestable on WCOSS2

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    4b21dbc View commit details
    Browse the repository at this point in the history

  4. Merge branch 'feature/machine_res' of https://github.com/DavidHuber-N… 

    …OAA/global-workflow into feature/machine_res

Merging new commit from Dave
    @InnocentSouopgui-NOAA
    InnocentSouopgui-NOAA committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    88e7f50 View commit details
    Browse the repository at this point in the history

  5. Merge pull request #13 from InnocentSouopgui-NOAA/feature/machine_res 

    Adapt Feature/machine res to Jet and S4
    @DavidHuber-NOAA
    DavidHuber-NOAA authored Jun 21, 2024
    Configuration menu
    Copy the full SHA
    11556d0 View commit details
    Browse the repository at this point in the history

  6. Merge remote-tracking branch 'emc/develop' into feature/machine_res

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    68063b0 View commit details
    Browse the repository at this point in the history

  7. Merge branch 'feature/machine_res' of github.com:DavidHuber-noaa/glob… 

    …al-workflow into feature/machine_res
    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    cb03a52 View commit details
    Browse the repository at this point in the history

Commits on Jun 23, 2024

  1. Fix atmensanlfv3inc resource declarations

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 23, 2024
    Configuration menu
    Copy the full SHA
    184d235 View commit details
    Browse the repository at this point in the history

Commits on Jun 24, 2024

  1. Remove reference to non-existent RUN-specific gempak variables

    @DavidHuber-NOAA
    DavidHuber-NOAA committed Jun 24, 2024
    Configuration menu
    Copy the full SHA
    9917143 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'develop' into feature/machine_res

    @DavidHuber-NOAA
    DavidHuber-NOAA authored Jun 24, 2024
    Configuration menu
    Copy the full SHA
    fe85950 View commit details
    Browse the repository at this point in the history