Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added functional timestamps #6125

Draft
wants to merge 23 commits into
base: main
Choose a base branch
from
Draft
Changes from 2 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
13e7ee1
added functional timestamps
ESadek-MO Aug 19, 2024
7b32abe
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 19, 2024
824c8ae
wip
ESadek-MO Aug 20, 2024
9abb8e2
wip
ESadek-MO Aug 20, 2024
96a2000
Merge branch 'mesh' of github.com:ESadek-MO/iris into mesh
ESadek-MO Aug 20, 2024
75a471f
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 20, 2024
378e642
a couple of small changes
ESadek-MO Aug 20, 2024
6c2d9bf
Merge branch 'mesh' of github.com:ESadek-MO/iris into mesh
ESadek-MO Aug 20, 2024
56f994c
fixed super calls
ESadek-MO Aug 20, 2024
092e354
easy review comments
ESadek-MO Aug 20, 2024
3cbe9c0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 20, 2024
0fc0099
Implemented metadata property. Currently fails on circular
ESadek-MO Aug 20, 2024
67c9ed0
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 20, 2024
d8ad4dd
bounds are circular looping
ESadek-MO Aug 22, 2024
a97a177
Merge branch 'mesh' of github.com:ESadek-MO/iris into mesh
ESadek-MO Aug 22, 2024
050d47b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Aug 22, 2024
9c0b61d
fixed bounds issue
ESadek-MO Oct 3, 2024
d584439
merge conflicts
ESadek-MO Oct 3, 2024
99c995a
merge conflicts
ESadek-MO Oct 3, 2024
ab3072b
Merge branch 'main' into mesh
ESadek-MO Oct 3, 2024
adbb97b
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Oct 3, 2024
73e0062
merge conflict
ESadek-MO Oct 4, 2024
9b4f204
Merge branch 'mesh' of github.com:ESadek-MO/iris into mesh
ESadek-MO Oct 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
169 changes: 101 additions & 68 deletions lib/iris/mesh/components.py
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -1138,6 +1138,10 @@ def all_connectivities(self):
"""All the :class:`~iris.mesh.Connectivity` instances of the :class:`MeshXY`."""
return self._connectivity_manager.all_members

@property
def timestamp(self):
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
return MAX(self._coord_manager.timestamp, self._connectivity_manager.timestamp)
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

@property
def all_coords(self):
"""All the :class:`~iris.coords.AuxCoord` coordinates of the :class:`MeshXY`."""
Expand Down Expand Up @@ -2001,6 +2005,7 @@ def __init__(self, node_x, node_y, edge_x=None, edge_y=None):
# optional coordinates
self.edge_x = edge_x
self.edge_y = edge_y
self.timestamp = datetime.now()
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

def __eq__(self, other):
# TBD: this is a minimalist implementation and requires to be revisited
Expand Down Expand Up @@ -2031,6 +2036,7 @@ def __str__(self):
return f"{self.__class__.__name__}({', '.join(args)})"

def _remove(self, **kwargs):
self.timestamp = datetime.now()
result = {}
members = self.filters(**kwargs)

Expand All @@ -2045,6 +2051,7 @@ def _remove(self, **kwargs):
return result

def _setter(self, element, axis, coord, shape):
self.timestamp = datetime.now()
axis = axis.lower()
member = f"{element}_{axis}"

Expand Down Expand Up @@ -2138,6 +2145,7 @@ def node_y(self, coord):
self._setter(element="node", axis="y", coord=coord, shape=self._node_shape)

def _add(self, coords):
self.timestamp = datetime.now()
member_x, member_y = coords._fields

# deal with the special case where both members are changing
Expand Down Expand Up @@ -2377,6 +2385,7 @@ def __init__(self, *connectivities):
self.ALL = self.REQUIRED + self.OPTIONAL
self._members = {member: None for member in self.ALL}
self.add(*connectivities)
self.timestamp = datetime.now()

def __eq__(self, other):
# TBD: this is a minimalist implementation and requires to be revisited
Expand Down Expand Up @@ -2423,6 +2432,7 @@ def add(self, *connectivities):
# manager.
# No warning is raised for duplicate cf_roles - user is trusted to
# validate their outputs.
self.timestamp = datetime.now()
add_dict = {}
for connectivity in connectivities:
if not isinstance(connectivity, Connectivity):
Expand Down Expand Up @@ -2564,6 +2574,7 @@ def remove(
contains_edge=None,
contains_face=None,
):
self.timestamp = datetime.now()
removal_dict = self.filters(
item=item,
standard_name=standard_name,
Expand Down Expand Up @@ -2724,74 +2735,7 @@ def __init__(
raise ValueError(msg)
# Held in metadata, readable as self.axis, but cannot set it.
self._metadata_manager.axis = axis

points, bounds = self._construct_access_arrays()
if points is None:
# TODO: we intend to support this in future, but it will require
# extra work to refactor the parent classes.
msg = "Cannot yet create a MeshCoord without points."
raise ValueError(msg)

# Get the 'coord identity' metadata from the relevant node-coordinate.
node_coord = self.mesh.coord(location="node", axis=self.axis)
node_metadict = node_coord.metadata._asdict()
# Use node metadata, unless location is face/edge.
use_metadict = node_metadict.copy()
if location != "node":
# Location is either "edge" or "face" - get the relevant coord.
location_coord = self.mesh.coord(location=location, axis=axis)

# Take the MeshCoord metadata from the 'location' coord.
use_metadict = location_coord.metadata._asdict()
unit_unknown = Unit(None)

# N.B. at present, coords in a MeshXY are stored+accessed by 'axis', which
# means they must have a standard_name. So ...
# (a) the 'location' (face/edge) coord *always* has a usable phenomenon
# identity.
# (b) we still want to check that location+node coords have the same
# phenomenon (i.e. physical meaning identity + units), **but** ...
# (c) we will accept/ignore some differences : not just "var_name", but
# also "long_name" *and* "attributes". So it is *only* "standard_name"
# and "units" that cause an error if they differ.
for key in ("standard_name", "units"):
bounds_value = use_metadict[key]
nodes_value = node_metadict[key]
if key == "units" and (
bounds_value == unit_unknown or nodes_value == unit_unknown
):
# Allow "any" unit to match no-units (for now)
continue
if bounds_value != nodes_value:

def fix_repr(val):
# Tidy values appearance by converting Unit to string, and
# wrapping strings in '', but leaving other types as a
# plain str() representation.
if isinstance(val, Unit):
val = str(val)
if isinstance(val, str):
val = repr(val)
return val

nodes_value, bounds_value = [
fix_repr(val) for val in (nodes_value, bounds_value)
]
msg = (
f"Node coordinate {node_coord!r} disagrees with the "
f"{location} coordinate {location_coord!r}, "
f'in having a "{key}" value of {nodes_value} '
f"instead of {bounds_value}."
)
raise ValueError(msg)

# Don't use 'coord_system' as a constructor arg, since for
# MeshCoords it is deduced from the mesh.
# (Otherwise a non-None coord_system breaks the 'copy' operation)
use_metadict.pop("coord_system")

# Call parent constructor to handle the common constructor args.
super().__init__(points, bounds=bounds, **use_metadict)
self._load_points_and_bounds()

# Define accessors for MeshCoord-specific properties mesh/location/axis.
# These are all read-only.
Expand All @@ -2808,6 +2752,31 @@ def location(self):
def axis(self):
return self._metadata_manager.axis

@property
def points(self):
"""The coordinate points values as a NumPy array."""
if self.timestamp < self._mesh.timestamp or self.timestamp is None:
self._load_points_and_bounds()
return self._values
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

@points.setter
def points(self, value):
if value:
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
msg = "Cannot set 'points' on a MeshCoord."
raise ValueError(msg)

@property
def bounds(self):
if self.timestamp < self._mesh.timestamp or self.timestamp is None:
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
self._load_points_and_bounds()
return self.bounds
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

@bounds.setter
def bounds(self, value):
if value:
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
msg = "Cannot set 'bounds' on a MeshCoord."
raise ValueError(msg)
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

# Provide overrides to mimic the Coord-specific properties that are not
# supported by MeshCoord, i.e. "coord_system" and "climatological".
# These mimic the Coord properties, but always return fixed 'null' values.
Expand Down Expand Up @@ -3017,6 +2986,70 @@ def summary(self, *args, **kwargs):
result = "\n".join(lines)
return result

def _load_points_and_bounds(self):
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
points, bounds = self._construct_access_arrays()
if points is None:
# TODO: we intend to support this in future, but it will require
# extra work to refactor the parent classes.
msg = "Cannot yet create a MeshCoord without points."
raise ValueError(msg)

# Get the 'coord identity' metadata from the relevant node-coordinate.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something missed in #4757 and in previous discussions: the metadata such as units, standard_name, etcetera is also subject to change.

Thankfully it is quick to recalculate, so we don't need to protect it behind a date check. But we do need a way to make sure it is recalculated on the fly.

Metadata classes a tricksy and it's probably best if we don't make any changes there. Instead we can put self._metadata_manager behind a `@property' so that we can regenerate it every time before returning it:

1

- self._metadata_manager = metadata_manager_factory(MeshCoordMetadata)
+ self._some_other_name = metadata_manager_factory(MeshCoordMetadata)

2

@property
def _metadata_manager(self):
    # An explanatory comment.
    
    self._some_other_name.standard_name = something
    # Etcetera for all standard coord metadata
    # THIS INCLUDES DETERMINING THE CORRECT VALUE, AS IN
    #  THE CURRENT BLOCK WITHIN _load_points_and_bounds

    return self._some_other_name

3

I don't think this will be necessary since I don't think there will be any calls to set self._metadata_manager?

@_metadata_manager.setter
def _metadata_manager(self, value):
    self._some_other_name = value

node_coord = self.mesh.coord(include_nodes=True, axis=self.axis)
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
node_metadict = node_coord.metadata._asdict()
# Use node metadata, unless location is face/edge.
use_metadict = node_metadict.copy()
if location != "node":
# Location is either "edge" or "face" - get the relevant coord.
kwargs = {f"include_{location}s": True, "axis": axis}
location_coord = self.mesh.coord(**kwargs)
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

# Take the MeshCoord metadata from the 'location' coord.
use_metadict = location_coord.metadata._asdict()
unit_unknown = Unit(None)

# N.B. at present, coords in a Mesh are stored+accessed by 'axis', which
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
# means they must have a standard_name. So ...
# (a) the 'location' (face/edge) coord *always* has a usable phenomenon
# identity.
# (b) we still want to check that location+node coords have the same
# phenomenon (i.e. physical meaning identity + units), **but** ...
# (c) we will accept/ignore some differences : not just "var_name", but
# also "long_name" *and* "attributes". So it is *only* "standard_name"
# and "units" that cause an error if they differ.
for key in ("standard_name", "units"):
bounds_value = use_metadict[key]
nodes_value = node_metadict[key]
if key == "units" and (
bounds_value == unit_unknown or nodes_value == unit_unknown
):
# Allow "any" unit to match no-units (for now)
continue
if bounds_value != nodes_value:

def fix_repr(val):
# Tidy values appearance by converting Unit to string, and
# wrapping strings in '', but leaving other types as a
# plain str() representation.
if isinstance(val, Unit):
val = str(val)
if isinstance(val, str):
val = repr(val)
return val

nodes_value, bounds_value = [
fix_repr(val) for val in (nodes_value, bounds_value)
]
msg = (
f"Node coordinate {node_coord!r} disagrees with the "
f"{location} coordinate {location_coord!r}, "
f'in having a "{key}" value of {nodes_value} '
f"instead of {bounds_value}."
)
raise ValueError(msg)
super().__init__(points, bounds=bounds, **use_metadict)
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-initialising the instance every time is a very surprising thing to do. I'm not sure of the possible consequences of doing this. Have you looked this up?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually forget known consequences. Even if there are none, a big problem is that the parent classes will have been written assuming that __init__ will only be run once. Even if it works now, someone could break it any time by making a seemingly 'safe' change. It might even happen during an edit to MeshCoord, since it's not obvious from looking at MeshCoord.__init__ that this is happening.

Can you come up with an alternative?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've had a look through the parent 'stack' and I reckon if you can work out a way to update the points, the bounds, and the metadata (MeshCoordMetadata) then you'll be in the clear.

self.timestamp = self._mesh.timestamp
trexfeathers marked this conversation as resolved.
Show resolved Hide resolved

def _construct_access_arrays(self):
"""Build lazy points and bounds arrays.

Expand Down
Loading