Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use of default fill value #145

Open
timhultberg opened this issue Feb 25, 2022 · 1 comment
Open

Use of default fill value #145

timhultberg opened this issue Feb 25, 2022 · 1 comment
Labels
question Further information is requested or discussion invited

Comments

@timhultberg
Copy link

NUG: "It is not necessary to define your own _FillValue attribute for a variable if the default fill value for the type of the variable is adequate"
CFC: "The NUG conventions for missing data changed significantly between version 2.3 and version 2.4. Since version 2.4 the NUG defines missing data as all values outside of the valid_range, and specifies how the valid_range should be defined from the _FillValue (which has library specified default values) if it hasn’t been explicitly specified. If only one missing value is needed for a variable then we recommend that this value be specified using the _FillValue attribute. Doing this guarantees that the missing value will be recognized by generic applications that follow either the before or after version 2.4 conventions."

Concerning the use an explicit _FillValue attribute (when its value equals the default value) NUG says it is not necessary but the CFC recommends it. So far so good. Now my understanding is that this recommendation is mainly to safeguard against generic applications which do not (yet) follow the latest (NUG) conventions and that the correct behaviour of generic applications now is to treat values equal to the default fill value as missing (when no _FillValue attribute is present). Is this understanding correct?

@timhultberg timhultberg added the question Further information is requested or discussion invited label Feb 25, 2022
@czender
Copy link

czender commented Jun 9, 2022

That is my understanding. NCO was written under the "old" NUG guidance, and required explicitly set _FillValue to indicate the presence of missing values. The "new" NUG guidance requires that all arithmetic loops now check for the presence of the default fill value. It will take a while to migrate to this convention, and the resulting arithmetic will always be slower because of the check. So the CFC guidance helps to ensure that NCO (and some other generic applications) will operate as intended on your dataset.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested or discussion invited
Projects
None yet
Development

No branches or pull requests

2 participants