Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[DO NOT MERGE] Weekly CI Tests Friday Nov 17, 2023 #2081

Closed
wants to merge 1 commit into from
Closed

[DO NOT MERGE] Weekly CI Tests Friday Nov 17, 2023 #2081

wants to merge 1 commit into from

Conversation

emcbot
Copy link

@emcbot emcbot commented Nov 17, 2023

[DO NOT MERGE] Weekly CI Tests Friday Nov 17, 2023

@emcbot emcbot marked this pull request as draft November 17, 2023 22:44
@emcbot emcbot added CI/CD Issue related to CI/CD CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Hera-Ready **CM use only** PR is ready for CI testing on Hera CI-Hera-Building **Bot use only** CI testing is cloning/building on Hera labels Nov 17, 2023
@emcbot
Copy link
Author

emcbot commented Nov 17, 2023

Automated global-workflow Testing Results:

Machine: Hera
Start: Fri Nov 17 22:49:10 UTC 2023 on hfe05
---------------------------------------------------
Checkout: Completed at Fri Nov 17 22:50:54 UTC 2023
Build: Completed at Fri Nov 17 23:36:54 UTC 2023
Case setup: Completed for experiment C384C192_hybatmda_b4577edb
Case setup: Completed for experiment C384_S2SWA_b4577edb
Case setup: Completed for experiment C384_atm3DVar_b4577edb

@emcbot emcbot added CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress and removed CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Nov 17, 2023
@emcbot
Copy link
Author

emcbot commented Nov 17, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Fri Nov 17 16:48:18 CST 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout: Completed at Fri Nov 17 16:49:13 CST 2023
Build: Completed at Fri Nov 17 17:39:44 CST 2023
Case setup: Completed for experiment C384_atm3DVar_b4577edb
Case setup: Completed for experiment C384C192_hybatmda_b4577edb
Case setup: Completed for experiment C384_S2SWA_b4577edb

@emcbot emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Nov 17, 2023
@emcbot
Copy link
Author

emcbot commented Nov 17, 2023 

Experiment C384_S2SWA_b4577edb Terminated: *** FAILED *** on Orion
Experiment C384_S2SWA_b4577edb Terminated with 1 tasks failed at Fri Nov 17 17:56:20 CST 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/2081/RUNTESTS/COMROT/C384_S2SWA_b4577edb/logs/2016070100/gfsfcst.log

@WalterKolczynski-NOAA
Copy link
Contributor

I think this might've been a hardware issue:

  11: [1700265124.353915] [Orion-19-05:298423:0]         address.c:1588 UCX  ERROR failed to parse address: number of ep addresses exceeds 6
  12: [1700265124.353916] [Orion-19-05:298424:0]         address.c:1588 UCX  ERROR failed to parse address: number of ep addresses exceeds 6
  16: [1700265124.353914] [Orion-19-05:298428:0]         address.c:1588 UCX  ERROR failed to parse address: number of ep addresses exceeds 6
  17: [1700265124.353909] [Orion-19-05:298429:0]         address.c:1588 UCX  ERROR failed to parse address: number of ep addresses exceeds 6
  19: [1700265124.353915] [Orion-19-05:298431:0]         address.c:1588 UCX  ERROR failed to parse address: number of ep addresses exceeds 6

Try rewinding the forecast job in the failed experiment.

@emcbot
Copy link
Author

emcbot commented Nov 18, 2023

Experiment C384C192_hybatmda_b4577edb SUCCESS Sat Nov 18 02:27:22 UTC 2023

@emcbot
Copy link
Author

emcbot commented Nov 18, 2023

Experiment C384_atm3DVar_b4577edb SUCCESS Sat Nov 18 02:36:21 UTC 2023

@emcbot
Copy link
Author

emcbot commented Nov 18, 2023

Experiment C384_S2SWA_b4577edb SUCCESS Sat Nov 18 04:06:20 UTC 2023

@emcbot emcbot added CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully and removed CI-Hera-Running **Bot use only** CI testing on Hera for this PR is in-progress labels Nov 18, 2023
@emcbot
Copy link
Author

emcbot commented Nov 18, 2023

All CI Test Cases Passed:

Experiment C384C192_hybatmda_b4577edb **SUCCESS** at Sat Nov 18 02:27:22 UTC 2023
Experiment C384_atm3DVar_b4577edb **SUCCESS** at Sat Nov 18 02:36:21 UTC 2023
Experiment C384_S2SWA_b4577edb **SUCCESS** at Sat Nov 18 04:06:20 UTC 2023

@WalterKolczynski-NOAA WalterKolczynski-NOAA added CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion and removed CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed labels Nov 19, 2023
@emcbot emcbot added CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion and removed CI-Orion-Ready **CM use only** PR is ready for CI testing on Orion CI-Orion-Building **Bot use only** CI testing is cloning/building on Orion labels Nov 19, 2023
@emcbot emcbot added the CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress label Nov 19, 2023
@emcbot
Copy link
Author

emcbot commented Nov 19, 2023

Automated global-workflow Testing Results:

Machine: Orion
Start: Sun Nov 19 14:41:04 CST 2023 on Orion-login-1.HPC.MsState.Edu
---------------------------------------------------
Checkout: Completed at Sun Nov 19 14:41:59 CST 2023
Build: Completed at Sun Nov 19 15:30:04 CST 2023
Case setup: Completed for experiment C384_atm3DVar_b4577edb
Case setup: Completed for experiment C384C192_hybatmda_b4577edb
Case setup: Completed for experiment C384_S2SWA_b4577edb

@emcbot emcbot added CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed and removed CI-Orion-Running **Bot use only** CI testing on Orion for this PR is in-progress labels Nov 19, 2023
@emcbot
Copy link
Author

emcbot commented Nov 19, 2023 

Experiment C384C192_hybatmda_b4577edb Terminated: *** FAILED *** on Orion
Experiment C384C192_hybatmda_b4577edb Terminated with 1 tasks failed at Sun Nov 19 16:06:13 CST 2023
Error logs:
/work2/noaa/stmp/GFS_CI_ROOT/PR/2081/RUNTESTS/COMROT/C384C192_hybatmda_b4577edb/logs/2023040200/enkfgdaseupd.log

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA Awaiting requested review from TerrenceMcGuinness-NOAA TerrenceMcGuinness-NOAA will be requested when the pull request is marked ready for review TerrenceMcGuinness-NOAA is a code owner

@WalterKolczynski-NOAA WalterKolczynski-NOAA Awaiting requested review from WalterKolczynski-NOAA WalterKolczynski-NOAA will be requested when the pull request is marked ready for review WalterKolczynski-NOAA is a code owner

Assignees
No one assigned
Labels
CI/CD Issue related to CI/CD CI-Hera-Passed **Bot use only** CI testing on Hera for this PR has completed successfully CI-Orion-Failed **Bot use only** CI testing on Orion for this PR has failed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@emcbot @WalterKolczynski-NOAA @TerryMcGuinness-NOAA