-
Notifications
You must be signed in to change notification settings - Fork 284
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bounds of derived variable are not read correctly #3678
Comments
Just been asked to look at this. The original example in the CF conventions does include the bounds attributes. I believe they are only "optional" in the sense that the file isn't actually invalid (by CF rules) without them. If I put the file into the NERC online CF checker , it gives ...
So, I read that as attempting to interpret all the "_bnds" variables as primary data variables, and then complaining that they have no standard_name or units. Without the linking bounds attributes, I think we cannot safely associate "A_bnds" with "A" (i.e. automatically in code): The only way to do it is to rely on a naming convention which, apart from being a rather weak and fragile approach, is definitely not in the CF conventions. So we would really not be keen on adding that to Iris. I must say you're not alone in encountering files like this, even in standard model output or even archive data. I was shown some files in the CMIP archive which do not pass the current CF checker -- though they probably did pass the earlier version that was in place when they were submitted. ( I think it may have been invalid units in that case ?). |
Hi @pp-mo, thanks for taking a look at this! Unfortunately, I am afraid it is a bit more than the usual data problems and really is the intended way of encoding things according to CF, not just leniency. There are extensive discussions on the topic at the ESMValTool repo, the CMOR repo, the old CF trac and probably other places. The long and short of it is that the formula term variables don't have Instead, the connection comes from the Does this make sense? |
Thanks for explaining @schlunma Anyway, I just got a chance to look at this again + I can see you are 100% right 💐 I've re-categorised this release "Iris 3,1" and label "CF 1.6/1.7" for now (though that label could do with a revamp). |
Hi @bjlittle, this is still relevant for us, but only with a medium priority (we, the ESMValTool devs, assigned priorities to each issue with At the moment we have a (lenghty) custom solution for this that does the trick, but it would be of course much nicer and cleaner if this would be handled by iris 🚀 Thanks!! |
Thanks @schlunma. |
No worries at all @pp-mo! Thanks for all your support on ESMValTool-related issues, we really appreciate it 👍 |
@zklaus , @schlunma how important would it be to preserve the form in which the bounds are attached to the variable when round tripping load/save?
However, within Iris, there is just one way in which the bounds are attached (which structurally mirrors 1. at the moment). In order to solve this as simply as possible, I would expect that a cube loaded from a file of form 2 would save the same as a cube loaded from a file of form 2 currently does. It may be a seperate issue to make sure that this saved form aligns as closely to the best possible interpretation of CF, but would this be acceptable as a solution for the time being? |
Hi @stephenworsley, thanks for working on this, much appreciated! In my opinion, the priority for ESMValTool would be to be able to read files of form 2 without additional code. So yes, I think your proposed solution would definitely work for us! 👍 It would of course be nice if a CF-compliant file of form 2 actually saves in the same form, but this is only has a secondary priority for us. |
@schlunma @bouweandela would you say that our own When we applied the changes in #6190, Iris no longer loads |
Yes, the text here suggests that this is indeed invald CF: "If a parametric coordinate variable with a This seems to be not the case for
Here is an example of a valid file (taken from the link above):
|
As discussed today @SciTools/peloton (and after)@pp-mo wanted to make clear that I don't think it is "bad CF" for the dependencies to have bounds connected via a "bounds" attribute, as for coords.
So, it is wrong to do that if the parametric coordinate variable itself has a 'bounds' attribute -- which is the case here. So, I think that :
|
What about the sample
|
Experimented with this. It looks like the dataset from which "hybrid_height.nc" was called "theta_and_orog_subset.pp", and PP data of that name still exists in iris-test-data. Example:
That is ...
|
( The "other question" from above ) In the v1.7 spec, new text was added in section "7.1. Cell Boundaries" to the effect stated above
This makes clear there that there are two means of recording bounds of formula terms. But it seems to insist that a separate bounds variable with its own formula_terms attribute would still always exist.
The example given there is somewhat similar to what Iris produces,
So in this example, 'eta' is one of it's own terms, but 'eta_bounds' points at another variable with a 'formula_terms' describing all the term bounds, instead of just a simple bounds variable for the 'eta' term (which is what Iris does). So What ?This presents a real problem, since we probably want to retain the ability to use existing "wrong" behaviour, purely because it is so longstanding in Iris output |
Amazing work @pp-mo 🏅 |
Hi guys,
in ESMValTool we found the following issue concerning files with derived variables (in particular
atmosphere_hybrid_sigma_pressure_coordinate
, see ESMValGroup/ESMValCore#543):The bounds of the derived variable of model output that is consistent with the CF conventions cannot be read correctly with
iris
. Here is a minimal example using the nc file that is used on the CF convention webpage (without the optional attributes and the units ofA
replaced by1
):Reading this file with
iris
gives
As you can see, the
air_pressure
coordinate does not have bounds. When the optional attributesA:bounds = "A_bnds"
andB:bounds = "B_bnds"
are added to the file,iris
is able to read the bounds correctly.The text was updated successfully, but these errors were encountered: