[PR]: Enable skipna for spatial and temporal mean operations

[lee1043]

Description

Enable skipna parameter for mean operation.

• Closes [Feature]: skipna parameter for averager #582

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• My changes generate no new warnings
• Any dependent changes have been merged and published in downstream modules

If applicable:

• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass with my changes (locally and CI/CD build)
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have noted that this is a breaking change for a major release (fix or feature that would cause existing functionality to not work as expected)

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 100.00%. Comparing base (1cfd369) to head (b770b91).

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #655   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           15        15           
  Lines         1555      1556    +1     
=========================================
+ Hits          1555      1556    +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[lee1043]

Test code attached: Archive.zip

[tomvothecoder]

Thanks for this PR @lee1043! I will more closely either some time at the end of this week or early next week.

[pochedls]

@lee1043 – Thanks for taking on this PR. This seems like a handy option to have in xcdat. I just took a quick look at this – it seems like skipna should be added to the docstring (since it is now available in the public API). I'm also curious if you can copy a couple existing unit tests, add a np.nan value somewhere, and ensure that the NaN value propagates to the final result (I think this is the expected behavior).

[lee1043]

@pochedls thanks for the suggestion. I will work on these and ping you and @tomvothecoder for review once ready.

[tomvothecoder]

I'm assuming that we want to add the skipna arg to group_average(), climatology(), and departures() (in addition to average()). Is this correct @lee1043 and @pochedls? I made these updates in this commit 7f9d2e3 (#655) along with adding unit tests.

[pochedls]

I'm assuming that we want to add the skipna arg to group_average(), climatology(), and departures() (in addition to average()). Is this correct @lee1043 and @pochedls? I made these updates in this commit 7f9d2e3 (#655) along with adding unit tests.

I think so. This probably needs to be tested on real data (I can't think of unintended consequences, but it is worth stress testing a little).

[tomvothecoder]

I'm assuming that we want to add the skipna arg to group_average(), climatology(), and departures() (in addition to average()). Is this correct @lee1043 and @pochedls? I made these updates in this commit 7f9d2e3 (#655) along with adding unit tests.

I think so. This probably needs to be tested on real data (I can't think of unintended consequences, but it is worth stress testing a little).

Can you upload one of your model validation scripts in the xcdat-validation repo? I would like to re-use it for testing.

[pochedls]

Getting back to this. I don't have an appropriate validation script for this, but may be able to help with this if there is an obvious minimum validation example. For example, is there an equivalent set of CDAT commands that xcdat should be able to match here? I think we would need some reference code to develop a larger model validation script.

[tomvothecoder]

11/6/24 Testing Requirements

Test skipna which skips missing values (as marked by NaN). It should only skip missing values for float dtypes.

The average should be calculated using non-NaN values or just return NaN for the average when there are all NaN values in a group being averaged over.

Prerequisites

• Find equivalent set of CDAT commands to test against skipna feature
• Develop testing with PCMDI Metrics (observation data)

APIs to test

• spatial.spatial_avg()
• temporal.average()
• temporal.group_average()
• temporal.climatology()
• temporal.departures()

Results

TBD

[tomvothecoder]

Just rebased this branch on the latest main. Please update your local branch accordingly.