Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Packaging libpll-2 for Debian Med #17

Open
shayandoust opened this issue Jun 18, 2020 · 15 comments
Open

Packaging libpll-2 for Debian Med #17

shayandoust opened this issue Jun 18, 2020 · 15 comments

Comments

@shayandoust
Copy link

Hello,

Just a question about libpll and libpll-2. The Debian-Med packaging team has already packaged libpll[1], however due to some softwares needing libpll-2 as a dependency instead (such as Raxml-NG), we could consider basing the source-code of libpll to libpll-2.

As libpll-2 is a fork of libpll, please could you confirm whether softwares written using libpll as a dependency can also use libpll-2? Are the functions preserved and do they have the same intended output?

Kind regards,
Shayan Doust
[1]: https://salsa.debian.org/med-team/libpll
@computations
Copy link
Collaborator

Hello Shayan,

libpll-2 is intended to be a super set of lipll. Libpll-2 came about as a fork to implement some fairly large optimizations that were deemed "too different" than the original library. So, a fork was made. Realistically, libpll-2 has replaced libpll, so it should be find to package libpll2 instead.

One thing to note before you end up doing a bunch of work: most of the products that use libpll-2 build and link it statically. I know this is true for my software, and I think that it is the default for raxml-ng. @amkozlov can speak further on this, but I personally have never linked with libpll-2 dynamically. Don't let this stop you from packaging libpll-2 though, just be aware of it.

Best
Ben

@amkozlov
Copy link
Collaborator

Dear Shayan,

first of all, thanks for your efforts packaging our software, and sorry for the version confusion that we created! :(

As you can see from the commit history, libpll-2 evolved quite a bit since branching from libpll, so it's hard to guarantee that all software based on libpll will work seamlessly with libpll-2 (at least the might be "trivial" issues like unit tests failing due to minor numeric deviations in results, or an API function that got an extra "optional" parameter). But then, how many packages are using libpll at the moment?

Since you mentioned raxml-ng: I plan to release raxml-ng 1.0 in July, and this would be a good opportunity to introduce clean versioning for libpll-2 and pll-modules. We were a bit unsure about the future of both projects previously, but now it's clear that the codes are not being maintained/developed outside from our lab. So we shall have an internal discussion with everyone involved and implement whatever decision we come up with.

So my suggestion would be to wait with packaging just a little bit more.

Best,
Alexey

@computations
Copy link
Collaborator

computations commented Jun 19, 2020 •
edited
Loading

I decided to diff the last commit of libpll and the current head of libpll-2 for pll.h to see what the API difference was. Most of the changes are minor but here is a list

  • Many structs and functions relating to site repeats.
  • The functions pll_utree_query_tipnodes and pll_utree_query_innernodes have been removed.
  • pll_utree_every and pll_utree_every_const have had their arguments changed. Previously it had been a call back function that took 1 node, and now it is a callback function that takes 2 nodes.
  • const unsigned int* -> const pll_state_t* for arguments labeled tipmap.
  • pll_core_update_sumtable_ti_4x4 has lost its tipmap argument.
  • pll_core_likelihood_derivatives gained 2 additional arguments: unsigned int parent_ids and unsigned int child_ids.
  • A utility function to parse phylip files has been added
  • RNG functions have been added.
  • Error codes for fasta and phylip files have gone from 1xx to 2xx.
  • Some new fields in pll_partition_t: nodes, asc_additional_sites and repeats.
  • Some new fields in pll_utree_t: binary and vroot.

I am not sure if any of those changes constitute a breaking change, but since the list was small enough I figured it would be worthwhile to enumerate the changes.

@shayandoust
Copy link
Author

shayandoust commented Jun 19, 2020 •
edited
Loading

Hello,

First of all, I'd like to thank you both for warm praise of packaging your softwares and your prompt responses.

Libpll-2 came about as a fork to implement some fairly large optimizations that were deemed "too different" than the original library

Ahh, alright. Thanks for the explanation.

Don't let this stop you from packaging libpll-2 though, just be aware of it.

If there are any issues or concerns during packaging, I'll certainly raise them as a new issue here.

As you can see from the commit history, libpll-2 evolved quite a bit since branching from libpll, so it's hard to guarantee that all software based on libpll will work seamlessly with libpll-2 (at least the might be "trivial" issues like unit tests failing due to minor numeric deviations in results, or an API function that got an extra "optional" parameter)

In response to this and the list of differences @computations has kindly made, I think the best course of action to prevent any sort of "breakage" would be for us to simply package libpll-2 separately. This will ensure simultaneous versions that downstream and users can rely on (should their softwares link or take precedence with a version over the other).

But then, how many packages are using libpll at the moment?

This is quite a difficult question. Our simple best estimates is looking at popcon1 (though this is more of the "worst case" as not everyone opts to share data for popcon) or through sources.debian.org and finding where "libpll" has been referenced as a (build) dependency for all libraries and softwares.

As for raxml-ng, thanks for the insight. As you suggested, I might just hold back on the packaging until raxml-ng (1.0) is released, although raxml-ng is listed on our covid-19 packages that need work2, so there is a bit of urgency with packaging.

Thanks a lot for both of your efforts!

Shayan Doust

@shayandoust
Copy link
Author

Hello computations et. al.,

Just a something that I've stumbled across. I've realised that libpll-2 generates libraries under the same name as libpll (as well as installing headers under /usr/include/libpll too). This will introduce conflict if a user decides to install both versions of libpll and libpll-2.

Is there a possibility of preventing this from happening within CMakeLists? Preferably, renaming to something like "libpll2" as opposed to "libpll" (which would mean libpll2-*.so files and headers installed under /usr/include/libpll2)? If it would make things easier with regards to not having a drastic upstream change, maybe a variable inside CMakeLists to allow for this to happen?

Kind regards,
Shayan Doust

@computations
Copy link
Collaborator

We can (and should) change the name. The big hold up on our end is that we would want to update all of our products to the use the new name. It mostly just ends up being a time problem. Maybe this isn't as big of a deal as I am thinking, because the speed of development on this lib is pretty slow, so we actually have a large amount of time before we really need to change the name in dependent products. Nonetheless, it is a time cost.

If it would make things easier with regards to not having a drastic upstream change, maybe a variable inside CMakeLists to allow for this to happen?

Maybe I am misunderstanding this, but I think that having something that overloads include paths from libpll2 to libpll in the build process is just asking for bugs. My preference is to just rip the band-aid off and change the name wholesale, including all of our products.

In terms of time frame @shayandoust, what were you expecting (on the order of dats vs weeks vs months)?

@shayandoust
Copy link
Author

Hello computations,

We can (and should) change the name. The big hold up on our end is that we would want to update all of our products to the use the new name. It mostly just ends up being a time problem. Maybe this isn't as big of a deal as I am thinking, because the speed of development on this lib is pretty slow, so we actually have a large amount of time before we really need to change the name in dependent products. Nonetheless, it is a time cost.

That does make sense, with a name change, you eradicate any possible conflicts further down the line when more softwares depend on libpll-2.

My preference is to just rip the band-aid off and change the name wholesale, including all of our products.

This would ideally be the best method.

In terms of time frame @shayandoust, what were you expecting (on the order of dats vs weeks vs months)?

There's no pressure of a timeframe, although of course the kind occasional upstream nudge to get an idea of progress :). Therefore, do carry out a name change whenever you find it feasible to do so. Although, the quicker we package this software and make it available in the repository, the more softwares we can support that depend on libpll-2 (for instance, raxml-ng which depends on pll-modules, which depends on libpll-2) and the less of a "blocked" state packages are.

Kind regards,
Shayan Doust

@tillea
Copy link

tillea commented Oct 20, 2021

Hi,
I intend to continue Shayan's work. I confirm I consider it unfortunate to keep the same version as the old libpll as well as the same SOVERSION (which is set to 0). The latter means libpll-2 is a plugin replacement for libpll with full ABI compatibility. Can you confirm that this is the case?
Kind regards, Andreas.

@xflouris
Copy link
Owner

Hi @tillea ,
the two libraries have diverged substantially. The cleanest solution would be to have libpll2 as a separate package, in the same way for example beast-mcmc and beast2-mcmc are maintained in Debian.
The API differs, and ABI is also incompatible. There is some software using libpll but most new programs use libpll-2.
Best regards,
Tomas

@tillea
Copy link

tillea commented Oct 20, 2021 via email

@amkozlov
Copy link
Collaborator

Hi @tillea,

many thanks for working on this and sorry for the confusion with libpll/libpll-2. We are now aiming to finally resolve it by replacing libpll-2 with a new library called coraxlib:

https://codeberg.org/Exelixis-Lab/coraxlib

We expect to switch our main tools (raxml-ng, generax etc.) to this library in the course of next year.

So from our perspective, it would make sense to wait until then and package coraxlib instead of libpll-2.

Best,
Alexey

@tillea
Copy link

tillea commented Oct 25, 2021 via email

@computations
Copy link
Collaborator

We will try to get this done sooner rather than later, but barring that an intermediate solution would be to build raxml-ng statically. There is already an option to do this in the raxml-ng cmake: STATIC_BUILD. Maybe this is enough to hold us over until coraxlib is ready for shipment

@tillea
Copy link

tillea commented Oct 26, 2021 via email

@tillea
Copy link

tillea commented Jan 18, 2022

Any news about a release of this library?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@xflouris @shayandoust @amkozlov @tillea @computations