[Python][Packaging] PyArrow wheel building is failing because of disabled vcpkg install of liblzma

[jorisvandenbossche]

Because of the "xz backdoor" situation (https://tukaani.org/xz-backdoor/, https://boehs.org/node/everything-i-know-about-the-xz-backdoor), the vcpkg port of liblzma is currently not working (it's pointing to github sources which are currently online, see microsoft/vcpkg#37839 / microsoft/vcpkg#37957).

While Arrow C++ itself doesn't depend on liblzma directly, it is being built because of one of our (recursive) dependencies (note: in all built wheels we have always used older, not-affected versions of liblzma, given that we are building with an older snapshot of vcpkg). But that means that our vcpkg install is failing to build Arrow C++ in the wheel building pipeline.

[raulcd]

I am going to try updating to the newest vcpkg version and also the workaround suggested on the linked issues.

 $ git diff
diff --git a/ports/liblzma/portfile.cmake b/ports/liblzma/portfile.cmake
index 0fdc38d..338d151 100644
--- a/ports/liblzma/portfile.cmake
+++ b/ports/liblzma/portfile.cmake
@@ -1,6 +1,6 @@
 vcpkg_from_github(
     OUT_SOURCE_PATH SOURCE_PATH
-    REPO tukaani-project/xz
+    REPO bminor/xz
     REF "v${VERSION}"
     SHA512 c28461123562564e030f3f733f078bc4c840e87598d9f4b718d4bca639120d8133f969c45d7bdc62f33f081d789ec0f14a1791fb7da18515682bfe3c0c7362e0
     HEAD_REF master

[kou]

It seems that https://github.com/tukaani-project/xz is enabled again.
So we don't need #41106.

[kou]

Hmm. curl -o xz-5.4.4.tar.gz --fail -L https://github.com/tukaani-project/xz/archive/v5.4.4.tar.gz works on local but doesn't work on GitHub Actions...

https://github.com/ursacomputing/crossbow/actions/runs/8613582243/job/23605169513#step:7:1939

#19 252.0 [DEBUG] 1000: execute_process(curl --fail -L https://github.com/tukaani-project/xz/archive/v5.4.4.tar.gz --create-dirs --output /opt/vcpkg/downloads/tukaani-project-xz-v5.4.4.tar.gz.15077.part)
#19 252.0 [DEBUG] 1000: cmd_execute_and_stream_data() returned 22 after   112854 us
#19 252.0 error: Failed to download from mirror set
#19 252.0 error: https://github.com/tukaani-project/xz/archive/v5.4.4.tar.gz: curl failed to download with exit code 22
#19 252.0   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
#19 252.0                                  Dload  Upload   Total   Spent    Left  Speed
#19 252.0 
#19 252.0 curl: (22) The requested URL returned error: 404 

[kou]

It seems that this is working on GitHub Actions too...

[raulcd]

We can close this one as GitHub has re-enabled the repo and it's not failing with liblzma anymore. I am opening other issues related to the current wheel failures both on Windows and macOS.