Add hash to CI experiment paths to avoid collisions #1735

TerrenceMcGuinness-NOAA · 2023-07-12T15:11:48Z

Description

When creating experiments we add a HASH tag onto the pslot name so that we can get unique behavior when moving archive information around.

Also, the "extra" pslot name is removed down from the ../RUNTESTS/ path

Type of change

Please delete options that are not relevant.

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Clone, Build, and Ran case C48_S2S on Orion

Checklist

My code follows the style guidelines of this project
I have performed a self-review of my own code
I have commented my code, particularly in hard-to-understand areas
My changes need updates to the documentation. I have made corresponding changes to the documentation
My changes generate no new warnings
New and existing tests pass with my changes

no op change for test

…orkflow into develop

…nness-NOAA/global-workflow into feature_1732_add-pslot-tag

…te experiment

…till there (bug fix)

ci/scripts/check_ci.sh

  # since all successfull ones where previously removed
-  if [[ "${num_cases}" -eq 0 ]] && [[ -d "${pr_dir}/RUNTESTS" ]]; then
+  if [[ -z $(ls -A "${pr_dir}/RUNTESTS/EXPDIR") ]] ; then


…nness-NOAA/global-workflow into feature_1732_add-pslot-tag

updated edate

can not use || ture because needs to be false when dir is empty

aerorahul

I have tried to clear up the confusion between a case and pslot.
A case is not a pslot as the pslot now contains a git SHA.

ci/scripts/driver.sh

ci/scripts/run_ci.sh

ci/scripts/clone-build_ci.sh

ci/scripts/check_ci.sh

Co-authored-by: Rahul Mahajan <[email protected]>

aerorahul

Looks good. Though I have a few more suggestions for clean up that uses the variables defined rather than creating paths from variables that need to be known.

ci/scripts/check_ci.sh

ci/scripts/clone-build_ci.sh

ci/scripts/run_ci.sh

ci/scripts/check_ci.sh

Co-authored-by: Rahul Mahajan <[email protected]>

TerrenceMcGuinness-NOAA · 2023-07-13T03:50:31Z

Machine: Orion
Start: Wed Jul 12 16:54:17 CDT 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout:                      *SUCCESS*
Checkout: Completed at Wed Jul 12 16:56:34 CDT 2023
Build:                         *SUCCESS*
Build: Completed at Wed Jul 12 17:26:03 CDT 2023
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 12 17:26:11 CDT 2023 for experiment C48_S2S_61d242ea
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 12 17:26:15 CDT 2023 for experiment C96_atm3DVar_61d242ea
Created experiment:            *SUCCESS*
Case setup: Completed at Wed Jul 12 17:26:20 CDT 2023 for experiment C96C48_hybatmDA_61d242ea
Experiment C48_S2S_61d242ea completed: *SUCCESS*
Experiment C48_S2S_61d242ea Completed at Wed Jul 12 18:06:08 CDT 2023
with 18 successfully completed jobs
Experiment C96_atm3DVar_61d242ea completed: *SUCCESS*
Experiment C96_atm3DVar_61d242ea Completed at Wed Jul 12 19:36:09 CDT 2023
with 86 successfully completed jobs
Experiment C96C48_hybatmDA_61d242ea completed: *SUCCESS*
Experiment C96C48_hybatmDA_61d242ea Completed at Wed Jul 12 20:18:12 CDT 2023
with 148 successfully completed jobs

TerrenceMcGuinness-NOAA · 2023-07-13T03:52:12Z

With final updates debugged all cases pass.

aerorahul

Looks great.

TerrenceMcGuinness-NOAA and others added 30 commits June 7, 2023 09:40

Update driver.sh

9ba6c00

no op change for test

Merge branch 'NOAA-EMC:develop' into develop

03e3e39

Merge branch 'NOAA-EMC:develop' into develop

5e4d690

Merge branch 'NOAA-EMC:develop' into develop

d792096

Update driver.sh

e6fb30b

no op change for test

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

9f7352f

…orkflow into develop

Merge branch 'NOAA-EMC:develop' into develop

38e0917

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

6101b70

…orkflow into develop

Merge branch 'NOAA-EMC:develop' into develop

fe1dcde

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

fa9e3e0

…orkflow into develop

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

4bfa873

…orkflow into develop

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

d0bdf58

…orkflow into develop

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

79f06b7

…orkflow into develop

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

5eef690

…orkflow into develop

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

19891d5

…orkflow into develop

Merge branch 'develop' of github.com:TerrenceMcGuinness-NOAA/global-w…

44b687e

…orkflow into develop

removed two cases for testing

3a5d137

added switch to create experment to tag pslot name

f3b6403

Merge branch 'feature_1732_add-pslot-tag' of github.com:TerrenceMcGui…

24064ee

…nness-NOAA/global-workflow into feature_1732_add-pslot-tag

updates for adding has to pslot name

a1aaf6e

Merge branch 'NOAA-EMC:develop' into feature_1732_add-pslot-tag

3707a4a

fine tuning pslot with hash name

115c799

Merge branch 'feature_1732_add-pslot-tag' of github.com:TerrenceMcGui…

633f9fc

…nness-NOAA/global-workflow into feature_1732_add-pslot-tag

forgot : on pslot line in casefile

f2eaee7

removed -c -g -u from checkout to reduce time to build

3a47b41

fixed extra pslot removal in cron checks

2361130

Merge branch 'feature_1732_add-pslot-tag' of github.com:TerrenceMcGui…

bcd9b45

…nness-NOAA/global-workflow into feature_1732_add-pslot-tag

case loop updates pslot bug fix in driver

25508ec

still needed pslot at the end of path to EXPDER for setup_xml in crea…

7f84442

…te experiment

had to add EXPDIR onto the path where number of remaining cases are s…

21cd42d

…till there (bug fix)

github-advanced-security bot found potential problems Jul 12, 2023

View reviewed changes

ci/scripts/check_ci.sh Fixed Show fixed Hide fixed

github-advanced-security bot found potential problems Jul 12, 2023

View reviewed changes

TerryMcGuinness-NOAA and others added 4 commits July 12, 2023 10:21

added back two other cycled case files with updates

3658c32

Merge branch 'feature_1732_add-pslot-tag' of github.com:TerrenceMcGui…

ff5c181

…nness-NOAA/global-workflow into feature_1732_add-pslot-tag

Update C96C48_hybatmDA.yaml

f19dc02

updated edate

Update check_ci.sh

60df7e9

can not use || ture because needs to be false when dir is empty

aerorahul requested changes Jul 12, 2023

View reviewed changes

TerrenceMcGuinness-NOAA and others added 9 commits July 12, 2023 11:55

Update ci/scripts/driver.sh

96cf183

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/driver.sh

aee3c60

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/run_ci.sh

0f34d7c

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/run_ci.sh

a4d0922

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/run_ci.sh

2b6d64c

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/clone-build_ci.sh

10233c2

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/check_ci.sh

310348a

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/check_ci.sh

4e139c6

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/check_ci.sh

93b1f06

Co-authored-by: Rahul Mahajan <[email protected]>

WalterKolczynski-NOAA approved these changes Jul 12, 2023

View reviewed changes

WalterKolczynski-NOAA changed the title ~~Feature 1732 add pslot tag~~ Add hash to CI experiment paths to avoid collisions Jul 12, 2023

aerorahul approved these changes Jul 12, 2023

View reviewed changes

aerorahul reviewed Jul 12, 2023

View reviewed changes

ci/scripts/check_ci.sh Outdated Show resolved Hide resolved

TerrenceMcGuinness-NOAA and others added 6 commits July 12, 2023 17:17

Update ci/scripts/check_ci.sh

67a01f3

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/check_ci.sh

ff6429f

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/clone-build_ci.sh

b33e394

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/run_ci.sh

888ceb4

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/run_ci.sh

2a82708

Co-authored-by: Rahul Mahajan <[email protected]>

Update ci/scripts/check_ci.sh

61d242e

Co-authored-by: Rahul Mahajan <[email protected]>

aerorahul approved these changes Jul 13, 2023

View reviewed changes

aerorahul merged commit 631fe61 into NOAA-EMC:develop Jul 13, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add hash to CI experiment paths to avoid collisions #1735

Add hash to CI experiment paths to avoid collisions #1735

TerrenceMcGuinness-NOAA commented Jul 12, 2023 •

edited

Loading

aerorahul left a comment

aerorahul left a comment

TerrenceMcGuinness-NOAA commented Jul 13, 2023

TerrenceMcGuinness-NOAA commented Jul 13, 2023

aerorahul left a comment

Add hash to CI experiment paths to avoid collisions #1735

Add hash to CI experiment paths to avoid collisions #1735

Conversation

TerrenceMcGuinness-NOAA commented Jul 12, 2023 • edited Loading

aerorahul left a comment

Choose a reason for hiding this comment

aerorahul left a comment

Choose a reason for hiding this comment

TerrenceMcGuinness-NOAA commented Jul 13, 2023

TerrenceMcGuinness-NOAA commented Jul 13, 2023

aerorahul left a comment

Choose a reason for hiding this comment

TerrenceMcGuinness-NOAA commented Jul 12, 2023 •

edited

Loading