Open Questions on Xmlsec Integration

[maths22]

As I am working on getting the native builds of #2888 into a healthy state (see https://github.com/maths22/nokogiri/tree/nokogiri-xmlsec-testing for what I've been working on), I've realized it opens up some questions.

xmlsec itself requires one of the crypto libraries listed on this page: https://www.aleksey.com/xmlsec/download.html , and as seen here: https://www.aleksey.com/xmlsec/xmldsig.html#algorithms , different crypto libraries end up supporting a combination of different formats and different algorithms, which ends up affecting the behavior exposed in ruby. (The current tests implicitly assume an openssl backend.) Most ruby distributions will be on platforms with openssl available since ruby itself bundles the openssl extension, but that's not a guarantee, and we certainly can't build on one system and expect an ABI-compatible openssl to be available on the other end. This leaves us with a few options best I can see:

• Default to having the xmlsec features disabled, and require an explicit flag to enable them at build time
• Have the xmlsec features disabled if using the native gem, but if using the compiled-on-your-system gem try to install xmlsec features if possible (and either fail if it can't or log a warning and just have the features disabled)
• Maybe have multiple versions of the native gem (not sure how specifying them would work) that are precompiled against each of openssl 1.1.0, openssl 3.x, and no xmlsec/openssl
• Something else I haven't thought of? (everything else I'm currently thinking of ends up actually requiring nokogiri export symbols, and the whole goal here is to not export symbols)

[flavorjones]

@maths22 Thanks for bringing up these points. I should have some headspace this weekend to respond more thoughtfully than I can at the moment. Thanks for your patience with me!

[flavorjones]

@maths22 So the heavyweight option you didn't list is to statically link against a crypto library when we precompile. If we're not exporting symbols, then the code within nokogiri.so should call the symbols we've statically linked and it shouldn't matter what version of, say, openssl the user has installed on their system.

We do something similar already with libiconv and zlib. Don't see why it wouldn't work for the crypto library. It will make the gem bigger, and we'll end up having to cut releases when the dependency is updated -- but that's probably OK.

If we wanted to explore this option, we should also probably make a decision about which crypto library (or libraries) we would prefer. I'd prefer something that's licensed under MIT or Apache or BSD (not GPL or LGPL) if possible. Smaller is (slightly) better. And preferably something that's stable and doesn't get frequent security updates.

WDYT?

[maths22]

I think exploring this seems reasonable. From a licensing perspective, I think openssl is really the only reasonable option (openssl 3.0 is Apache), and it's by far the most featureful backend. I suspect it will make build times atrocious, but I don't think there is really a workaround there. I'll give it a shot at least and see how much it hurts both our build time and our gem size, but I like the idea in theory.

[flavorjones]

Also - I think we should create an issue to track all the xmlsec-related conversations and work, if that's OK. I'd like to be transparent, but also I like having a canonical hub that links out to all the other useful information.

Something like #2204 but for xmlsec? If you want to create it, cool, otherwise I'll do it in the next day or two.

[maths22]

A tracker issue would sound like a good idea.

[maths22]

Filed #2902 ; please review/edit as appropriate, as I just took the nokogumbo one and fairly lightly edited it with my opinions of what I think we do/don't want to do differently here and sprinkled in some questions.

[flavorjones]

@maths22 - apologies for not replying on any of the issues you've created, I've had to triage my OSS time recently. Hoping to dig back in over the next two weeks.

[flavorjones]

Also it may be worth taking a look at ruby/prism#1149 which is another possible approach to solving this integration problem.