-
Notifications
You must be signed in to change notification settings - Fork 48
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Consider adding a programatic way to add leap seconds #43
Comments
Just out of curiosity, is it a feature that has been requested upstream? |
cc @scottransom for the whether or not it has been requested for SOFA not to put the leapseconds in a hard-coded part of a c file... |
Nope. I have not seen or heard about this request I don't think.... I could try and pass it on, though. |
@scottransom - I think that would be good indeed. It does seem strange to have to reinstall or recompile a whole package just because of a new leap second. |
@scottransom - I think we had some discussions about this years ago and got a "no", but I could be mis-remembering! So let us know what you can find out. Ideally this would be a bit of a back-and-forth if SOFA is amenable to it, though. That is, it would be good for us to have a look at any proposed solution so as to see if it works in the astropy et al. use case. |
I think the main objection was one of code portability for the C parsing the text file, and, in some sense, the hard-wired numbers can never be wrong for dates earlier than the file was created (so it's important for the code to know when the table was correct). It might make sense to have a SOFA/ERFA API that lets people add new leap seconds to the internal table. Then SOFA would not have to worry about file I/O and Astropy could have its own table for loading newer leap seconds at package import. This would require the code inside SOFA to be reorganized a bit. |
@timj - exactly what I was thinking: have a way to add leap seconds, but the ERFA/SOFA side would just be |
@timj Huh. I have to admit that I don't really see the utility of this. Maybe I'm missing something. @eteq says in #5783 that the issue is that a bugfix release has to be made. Why is that? Why can't you simply update the ERFA code that astropy ships with, anticipating the new SOFA release? Or does changing C-code somehow mandate a bugfix release? We know the format and way that those changes work. And we are usually given months of warning before a leap second comes in to effect.... |
@scottransom - yes, that's exactly the problem: to update the C-code we have to do a bugfix release because it's all compiled into astropy in the standard mode. The other option is for a user to use a system-installed ERFA and update ERFA itself... But the point in astropy/astropy#5783 is that we already download the information needed to update leap seconds dynamically at runtime for other reasons. So we want to be able to make that consistent with what ERFA thinks the leap seconds are. That's good both for the "getting the latest leap second" use case and the "I want to reproduce a prediction I made years ago before the most recent leap seconds came out" case. |
Hmmm... OK. I'm still not sure this is enough ammunition for me to try to get SOFA changed, though. A bigger benefit (IMO) would be that users forced to use not-up-to-date astropy's (and ERFAs) would always have correct leap seconds since they are determined dynamically by the downloaded IERS info. That would probably prevent errors. The reasons about avoiding bugfix releases seem like only (sorry!) convenience for the astropy maintainers -- and given the infrequent leap second additions, not a huge improvement in convenience at that. Note: Am just trying to figure out how to spin this to the SOFA board |
@scottransom - Oh, yes, sorry, the way you said it there is exactly what I see as the most important point. It's not that I so much mind issuing a bugfix, but rather that I have no faith that users will install said bugfix. Also, from a science reproducibility standpoint I think it really is important that there's a way to specify the leap seconds... So I guess that's call to also have a |
@eteq - why is reproducibility an issue? Since leap seconds aren't retroactive, they shouldn't affect anything current or past. It would only affect future predictions across leap sec boundaries. Or is that exactly what you are talking about? |
@scottransom - Yep, exactly. For example, I write a code that tells me where some asteroid is going to be in ~1 year because I got an observing program approved or something, but then a leap second is inserted before the observations happen. I want to be able to go back and ask where I though the object was at a particular time before I knew there'd be a leap second so that I can figure out why I put that in the proposal/paper/whatever. Almost all practical cases I work on this is a very small effect... but it might matter for e.g. VLBI or the like...? (Although I presume you would know a lot better @scottransom) |
(I should clarify that I think the reproducibility argument is a lot less important than the other issue, though. If |
I agree it is really about ensuring that people running slightly old code don't get it wrong and publish a paper on a mysterious change in, say, orbital period (has happened...). To me, easiest would be if SOFA provided a routine to access and possibly replace the whole leap second table; we can deal with inserting one at the end or removing one. But I'm not sure that is more or less of a hassle between different compilers than just putting the leap seconds in an ascii file and read it... Anyway, implementation detail... |
Leap seconds tables come in two formats: Reading the zic(8) man page, it seems one of these two formats is used to compile the tzfile(5) data base of Linuxes. |
I think that the main issue is one of control of the leap second. If the
leap second is
hardcoded it is very well controlled. This is especially true for
industrial applications.
On the other hand, if you can retrieve the list of leap seconds that are
already applied,
those can easily be changed in a wrapper while not affecting the integrity
or the
control of the time keepers' software. That differs from having a SOFA
library routine
that allows the user to change the leap seconds being applied, and thus
give a wrong
time. It will then be up to the programmer and errors there may be serious
and go
unnoticed, or industry may come up with its own 'leap second' scheme while
claiming
they still are using the SOFA libraries.
Though it may be useful to have that discussion, I am not convinced that it
is going
to lead to a change in SOFA.
…On Tue, May 23, 2017 at 6:52 PM, R. J. Mathar ***@***.***> wrote:
Leap seconds tables come in two formats:
ftp://ftp.iana.org/tz/code/leapseconds
https://www.ietf.org/timezones/data/leap-seconds.list
Reading the zic(8) man page, it seems one of these two formats is used to
compile the tzfile(5) data base of Linuxes.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#43 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AE67CJ6m10RfLggyhSfxqcqk4uIjPaCUks5r8xzvgaJpZM4L8b8e>
.
--
* * * * * * * * http://www.mssl.ucl.ac.uk/~npmk/ * * * *
Dr. N.P.M. Kuin ([email protected])
phone +44-(0)1483 (prefix) -204111 (work)
mobile +44(0)7908715953 skype ID: npkuin
Mullard Space Science Laboratory – University College London –
Holmbury St Mary – Dorking – Surrey RH5 6NT– U.K.
|
So just to let you all know, SOFA is still discussing this and something will almost certainly change. We don't exactly know what, yet, though. Right now, I believe that the leading contender is the simplest -- just change the licensing of the leap sec file so that it may be changed as needed by the users and still compiled into the SOFA library. That would keep the default behavior for the vast majority of users, and allow others to change things as appropriate for them. |
Just wanted to check if anything has happened here. I think the user should have some warning as to the staleness of the leap second file. This is the reason we had to write our own leap second management code for Barycorrpy, where we maintain a separate leap second file downloaded and a log file which has the date of last download. We compare this to check staleness and then download a new file, if the user has permitted updates. |
Leap seconds are already maintained by the operating system -- for example, Please, can we -- instead of adding one more "own leap management system" -- agree on a canonical way to handle leap seconds? |
Within astropy itself, we have access to updated leap seconds via IERS-A; that has the advantage of not being OS-dependent. So, at least adding the warning should be easy (PR welcome; I won't have time in the near future). Logically, though, as a "standard of astronomy", this is handled by SOFA/ERFA... - @scottransom - any news in that regard? @olebole - since astropy currently gets its leap seconds from SOFA, for Debian the current scheme is actually reasonably good, correct? You just have a special update for ERFA and all works? |
Hey, yes we can do that. Would be ideal to have it OS independent. IERS-A has the date of the last bulletin. However, do we not need something which has a date-wise break up of when each leap second was added? |
Not in practice: Unless I create a special update for our released Debian (and Ubuntu) versions, the ERFA package there stays at the version that way current on the Debian release day. For example, in Debian Stretch, we still have version 1.3.0. Any leap second introduced after this date (none in the moment, however) will not be there. And I am just not able to update all packages containing leap-second data for all supported platforms on a leap second announcement. |
@olebole - OK, that makes sense. I agree that an external list would be better for ERFA; possibly, we need to deviate from SOFA and just built that in ourselves. @eteq, @taldcroft - what do you think? @shbhuk - one can infer the leap seconds from the IERS-A list (which we download already) quite trivially, by looking for jumps in UT1-UTC. But possibly we should just download the file you point at. |
Yes, we (SOFA) discussed this issue quite a bit last year. The committee didn't like the idea of changing the source code, but we are changing the license for the file so that users can modify the leap second routines as they see fit. See the 2017 Aug 25 note here http://www.iausofa.org There is not really a great solution to this problem since sometimes users will want to do non-standard things with leap seconds (i.e. test codes without proper leap seconds in place, or make predictions for the future). |
@scottransom Would it at least be possible to make Then, it would be simpler for |
@jwoillez I can ask about this with the SOFA folks. Basically you would just like it if those enums became (global?) ints in dat.c? |
Is there a reason to not implement it as a stand alone TAI-UTC file, with automatic updates which one can turn off? For the majority who want a system which updates itself, they don't have to bother. For the ones who want to play with it, they can always modify that .txt file or delete a row, as they deem fit. |
@shbhuk - the text files essentially exist already - the question really is on how to allow one to use these inside p.s. Should perhaps add in defense of the hard-coding that, generally, one better is up-to-date with the code as well - things do change beyond leap seconds. But not remotely as often... |
Which file is this? Till we fix this issue, can we at least add a warning in astropy documentation for this in the meanwhile to create awareness? |
@scottransom: I take back what I asked. It should be OK to keep the leap seconds array All: See #49 for a proposal. |
We've been silent on this for a while, but I was just talking to @luojing1211 (PINT) and one thing that became clear is that on many observatory computers, one cannot count on continuous access to the internet, so updates are done sporadically. In that sense, a fallback to a system leap second file may also be useful (if that one is not OK, everything is probably lost anyway): |
Also, should probably investigate whether #50 has changed the situation to make this all easier to do... The changelog seems to say it's just the actual "terms and conditions" not the code itself that has changed, but I could be mis-understanding (haven't looked at the diff yet). |
Its only the permission that has changed, so that one can now substitute one's own file even in SOFA. For ERFA, I guess this is not as relevant (though we might as well be the one that provides a useful standard substitute). |
It is worth noting that /usr/share/zoneinfo/leap-seconds.list is an OS-dependent convention. It is good that the software packages the table, because the file might not always be available/updated. Assuming /usr/share/zoneinfo/leap-seconds.list is always available/updated is just that, an assumption. My recommendation would be to make the software look for an environment variable (document it, of course) that indicates the location of "/usr/share/zoneinfo/leap-seconds.list", and then prefer that over the hard-coded table IF it is supplied. This would be the system independent way of solving the problem, yields great flexibility, and at least makes full automation without build updates a possibility. |
A function that could be called from within dat.c that searches for a leap-seconds.list file may look as follows:
Such files apparently only exist on Ubuntu by default, not openSUSE or CentOS. On the latter systems the observer would need to copy the leap-seconds list himself into the local computer's file system. |
Nice, thanks for the starter code. Maybe we will find some time to work on this soon, the importance is going to rise for us shortly. Makes me wonder if we should open issues in openSUSE and CentOS about it too, but we could also solve the problem in our pipeline. Our software pipeline can stage the file for us on IaC-created or PaaS created hosting, and then inject an environment variable indicating its location that the software can use. |
Sorry to have not followed-up in this thread - I wasn't watching because of the follow-on discussion in #49 . And then #58 ended up implementing the base requirement of this issue, so I'm closing this. But the last few comments above are deeply tied to #61 so I suggest we move further discussion there. |
Currently leap seconds are hard-coded into the C, as SOFA does. However, there are use cases (e.g. astropy/astropy#5783) where it would be useful to have the option for leap seconds to be added at runtime. This option should be added.
The main debate here is whether or not this is acceptable, as it's a break from how SOFA does it. I think yes, because it's a pretty important use case - users should be able to get leap seconds from online sources and update at runtime without having to recompile...
The text was updated successfully, but these errors were encountered: