Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ACCESS-ESM1.5: QA Tests #41

Merged
merged 14 commits into from
Aug 9, 2024
Merged

ACCESS-ESM1.5: QA Tests #41

merged 14 commits into from
Aug 9, 2024

Conversation

CodeGat
Copy link
Collaborator

@CodeGat CodeGat commented Aug 6, 2024

Add QA tests for ACCESS-ESM1.5 to the model-config-tests package. Requires bump to 0.0.3 when merged.

In this PR:

  • Move all test_*_config.py files into new qa directory
  • Added test_access_esm1p5_config.py - see ACCESS-ESM1.5 testing #21 (comment)
  • Added access_esm1p5 QA pytest marker
  • Bumped version to 0.0.3

Closes #21

Pinging @jo-basevi so she can follow along on my progress 😄

@CodeGat CodeGat added priority:blocker type:qa-check Quality Assurance Checks labels Aug 6, 2024
@CodeGat CodeGat self-assigned this Aug 6, 2024
Copy link

codecov bot commented Aug 6, 2024

Codecov Report

Attention: Patch coverage is 52.00000% with 36 lines in your changes missing coverage. Please review.

Project coverage is 73.58%. Comparing base (a14002a) to head (d859828).
Report is 1 commits behind head on main.

Files Patch % Lines
...model_config_tests/qa/test_access_esm1p5_config.py 50.00% 36 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #41      +/-   ##
==========================================
- Coverage   76.29%   73.58%   -2.71%     
==========================================
  Files          15       15              
  Lines         658      723      +65     
==========================================
+ Hits          502      532      +30     
- Misses        156      191      +35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@CodeGat
Copy link
Collaborator Author

CodeGat commented Aug 6, 2024

Note access-esm1.5-configs will require a (not yet released 0.0.3 model-config-tests package to function due to ACCESS-NRI/access-esm1.5-configs@d37311a

@CodeGat
Copy link
Collaborator Author

CodeGat commented Aug 7, 2024

Might need a bit of a sanity check on the use of parameterization in the pytests.
The code coverage is also failing...Do I need to make tests for my tests...?

@CodeGat CodeGat marked this pull request as ready for review August 7, 2024 03:15
@CodeGat
Copy link
Collaborator Author

CodeGat commented Aug 7, 2024

Tested this on Gadi with ACCESS-NRI/access-esm1.5-configs@9191fd9 pre-industrial configuration. A bunch of errors because no metadata.yaml, and the failures seem expected. Might need to check the relevance of the 'configs' test marker tests...
Also still don't know what to do with the code coverage for the tests themselves...

@CodeGat CodeGat requested a review from jo-basevi August 8, 2024 03:58
Copy link
Member

@aidanheerdegen aidanheerdegen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Soz, a couple of changes and some questions/comments.

src/model_config_tests/qa/test_access_esm1p5_config.py Outdated Show resolved Hide resolved
src/model_config_tests/qa/test_access_esm1p5_config.py Outdated Show resolved Hide resolved
src/model_config_tests/qa/test_access_esm1p5_config.py Outdated Show resolved Hide resolved
src/model_config_tests/qa/test_access_esm1p5_config.py Outdated Show resolved Hide resolved
@aidanheerdegen aidanheerdegen self-requested a review August 9, 2024 03:50
aidanheerdegen
aidanheerdegen previously approved these changes Aug 9, 2024
Copy link
Member

@aidanheerdegen aidanheerdegen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

…t function from `src/model_config_tests` directory
aidanheerdegen
aidanheerdegen previously approved these changes Aug 9, 2024
Copy link
Member

@aidanheerdegen aidanheerdegen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll approve, but if you think we should make the changes to the module use path now, then feel free to do so and ask for yet another review

RELEASE_MODULE_LOCATION = "/g/data/vk83/modules/access-models"

aidanheerdegen
aidanheerdegen previously approved these changes Aug 9, 2024
Copy link
Member

@aidanheerdegen aidanheerdegen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG(reat)TM

Copy link
Member

@aidanheerdegen aidanheerdegen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice catch.

@CodeGat CodeGat merged commit d24f258 into main Aug 9, 2024
6 of 8 checks passed
@CodeGat CodeGat deleted the 21-esm1.5-qa-tests branch August 9, 2024 06:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:blocker type:qa-check Quality Assurance Checks
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ACCESS-ESM1.5 testing
2 participants