Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What should happen to non-loaded coordinates etcetera? #5068

Open
Tracked by #5165
trexfeathers opened this issue Nov 15, 2022 · 1 comment
Open
Tracked by #5165

What should happen to non-loaded coordinates etcetera? #5068

trexfeathers opened this issue Nov 15, 2022 · 1 comment

Comments

@trexfeathers
Copy link
Contributor

trexfeathers commented Nov 15, 2022

Sub-task of #5165

#5054 introduces the ability for the following objects to be ignored during NetCDF load, if there is a failure when trying to attach them to the loaded Cube:

  • DimCoord
  • AuxCoord
  • CellMeasure
  • AncillaryVariable

In these cases, the objects have been successfully created, but are discarded when they can't be added to the Cube. A big argument for more tolerant loading is the ability for users to fix problems in their data within Iris, so it would be more helpful to preserve the objects that can't be added, allowing the user to examine them and in some cases modify them to fix problems.

The priority of this is unknown, as non-loaded objects are new territory (rather than just raising an Exception). We can perhaps wait for user reports, and votes on this issue, before deciding if we should act.

It's also uncertain how we would achieve this. Possibilities:

  • Convert the objects to Cubes and include them in the loaded CubeList.
  • Enhance the concept of a CubeList so it can contain these other objects.
  • Loading returns an extra Iterable/Mapping containing non-loaded objects.
  • Save the unloaded objects to disk (might be an appropriate case for pickling).
  • Make whatever behaviour optional via a context manager / loading parameter.
@rcomer
Copy link
Member

rcomer commented Nov 15, 2022

Ages ago there was a feature request to load coordinates by themselves from NetCDF - #814.

trexfeathers added a commit to trexfeathers/iris that referenced this issue Nov 16, 2022
stephenworsley pushed a commit that referenced this issue Nov 16, 2022
…5054)

* More tolerant NetCDF loading by skipping invalid coords etc.

* Tolerant loading tests.

* What's New entry.

* self.monkeypatch comments.

* Add integration test.

* Include TODO comment.

* Use monkeypatch context manager.

* Mention CannotAddError in the Whats New.

* Another reference to #5068.
@pp-mo pp-mo added the Dragon 🐉 https://github.com/orgs/SciTools/projects/19?pane=info label Jul 10, 2023
@trexfeathers trexfeathers added Dragon Sub-Task 🦎 https://github.com/orgs/SciTools/projects/19?pane=info and removed Dragon 🐉 https://github.com/orgs/SciTools/projects/19?pane=info labels Jul 10, 2023
@scitools-ci scitools-ci bot removed this from 🚴 Peloton Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

3 participants