Cluster CDFs from CSA sometimes have variables incorrectly marked non-record-variant #939
Labels
bug
Something isn't working
CDF
Cluster
Data Servers
Issues with remote data servers (SPDF, MMS, MAVEN, JAXA, etc)
pytplot
Issues involving the pytplot package
While troubleshooting a crash loading Cluster data, I discovered a problem with the CDFs themselves: if the CDF contains only a single sample for an array-valued variable, it can lose its leading [time] dimension and is marked as "non-record-variant".
In this example:
we load moments data for all four probes. C1 has multiple samples falling into the selected time range, but C3 only has a single timestamp. C4 seems to have nothing available. Here are some log messages from loading the C3 data, with multiple errors:
The scalar-valued variables seem to have degenerated from 1-element arrays, as they should be, to bare scalars.
The array-valued variables (velocity and pressure_tensor) have also lost their leading dimension. Using the cdfeditor command-line utility directly on the CDFs returned by the CSA query (so, bypassing any possible issues PySPEDAS, PyTplot, or cdflib) shows that the C3 variables are marked non-record-variant, which is incorrect.
The same variables for the C1 CDF, with multiple samples, load correctly and are correctly marked as record-variant.
Other than adding some defensive programming to cdf_to_tplot to avoid outright crashes, there's not going to be a clean way to handle this. We can't just add the time dimension back, because that would break applications that rely on correctly marked NRV variables having the expected number of dimensions.
The text was updated successfully, but these errors were encountered: