Handle more second subdivisions

[st-bender]

Describe the bug

Some netcdf files produced by python's xarray have time units nanoseconds since <datetime>.
Those are not yet handled by NCDataset via CFTime (Alexander-Barth/NCDatasets.jl#192).
Supporting additional second subdivisions in timedecode() would be appreciated, such as microsecond and nanosecond. This would also be consistent with python's handling, see Alexander-Barth/NCDatasets.jl#181 (comment).

To Reproduce

julia> using CFTime

julia> timedecode(1e9, "nanoseconds since 2000-01-01 00:00:00.001", "proleptic_gregorian")
ERROR: unknown units nanoseconds
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] timeunits(#unused#::Type{DateTimeProlepticGregorian}, units::String)
   @ CFTime ~/.julia/packages/CFTime/n09Um/src/CFTime.jl:446
 [3] timedecode(#unused#::Type{DateTimeProlepticGregorian}, data::Float64, units::String)
   @ CFTime ~/.julia/packages/CFTime/n09Um/src/CFTime.jl:494
 [4] timedecode(data::Float64, units::String, calendar::String; prefer_datetime::Bool)
   @ CFTime ~/.julia/packages/CFTime/n09Um/src/CFTime.jl:545
 [5] timedecode(data::Float64, units::String, calendar::String)
   @ CFTime ~/.julia/packages/CFTime/n09Um/src/CFTime.jl:543
 [6] top-level scope
   @ REPL[6]:1

Expected behavior

Same result as with seconds:

julia> timedecode(1, "seconds since 2000-01-01 00:00:00.001", "proleptic_gregorian")
2000-01-01T00:00:01.001

Environment

• operating system: Ubunutu 18.04
• Julia version: 1.6.7 and 1.8.3
• CFTime: 0.1.2 and 0.1.3 (master)

[Alexander-Barth]

So far, the smallest time units is milliseconds as internally that is the precision of the time stamp. We follow the approach of the Dates module in Julia.

I am wondering if in your use-case, sub-millisecond precision is important?

As far as I can see, we could:

1. simply round the time information to the closest millisecond
2. implement different internal time units as it is done in numpy.
3. integrate with different packages able to handle sub-millisecond precisions (like https://github.com/JeffreySarnoff/TimesDates.jl)
4. declare it out-of-scope 😞

[st-bender]

Hi,

Thanks for your thoughts.

So far, the smallest time units is milliseconds as internally that is the precision of the time stamp. We follow the approach of the Dates module in Julia.

I am wondering if in your use-case, sub-millisecond precision is important?

At least for my use case it is not important, but it may be important for others. Netcdf files are produced that way by at least one package and maybe other tools. It would just be nice to be able to read them into Julia without too much fiddling. NCDatasets.jl (and Rasters.jl using the former) are the two I found that are at least starting points for having something similar to xarray. Both currently depend on CFTime.

As far as I can see, we could:

1. simply round the time information to the closest millisecond

Fine for me, and probably the easiest way until someone complains about missing time resolution. 🤷

2. implement different internal time units as it is done in [numpy](https://numpy.org/doc/stable/reference/arrays.datetime.html).

I guess that may be where it comes from, quite often the times in numpy are listed as datetime[ns].

3. integrate with different packages able to handle sub-millisecond precisions (like https://github.com/JeffreySarnoff/TimesDates.jl)

Python's xarray also has two options, datetime and cftime. But that depends on how much time and resources you want to spent on implementing the solution.

4. declare it out-of-scope disappointed

That would mean:

5. Leave all the preprocessing steps to python/xarray, separately or via PyCall.

[st-bender]

Hi there,
It's been a while, any more ideas on it?
Meanwhile, I found out that numpy uses internally a 64-bit integer representation, hence datetime64[ns], and counting starts on 1970-01-01. This leaves about +/- 290 years that can be addressed that way. Using 64-bit integer milliseconds like Dates then gives about +/- 290_000_000 years.

Anyway, regardless of the internal representation, it would be nice if CFTime could support all SI prefixes. The CF convention states:

The acceptable units for time are listed in the udunits.dat file. The most commonly used of these strings (and their abbreviations) includes day (d), hour (hr, h), minute (min) and second (sec, s).

The prefixes can be found in https://docs.unidata.ucar.edu/udunits/current/udunits2-prefixes.xml. The easiest way would be to round to the nearest millisecond then.

[Alexander-Barth]

Here are some experiments that I did a while ago:
https://github.com/JuliaGeo/CFTime.jl/blob/flexible-resolution/test/flexible_resolution.jl

The idea is to just wrap the time as-is in a struct and use units and time origin as type parameters. Time units and origin would be flexible and we can use the type specialization of the julia compiler.