Rework the release build scripts

[joeraz]

The release scripts have been causing some significant trouble lately.

I've been doing what I can to keep them running, but unfortunately, environment configuration has never been a strong suit of mine. I've had to use several hacks just to keep them working. On more than one occasion, this led to some partially broken builds, and while I've been able to get them somewhat stable for the moment, I'm reaching a point where the number of workarounds and occasional issues is starting to add up to the point where I fear I might eventually be unable to do any more releases.

Here are the recent issues that I'm aware of:

• The newly released Pillow 10.0 no longer offers 32-bit wheels, which was an intentional decision by the Pillow team. This causes AppVeyor's build to fail. If I update to use a 64-bit Python build, the Windows installer will generate, but after installing it, I get an error "PyInstaller: FormatMessageW failed" when I try to run the installed version. I have to force it to use Pillow version 9.5 as a workaround.
• PyGame has recently stopped working on the Windows installed version, so the music won't play. It works fine when running from source code on the same machine.
• The Mac release is still using Python 3.9. Would like to update it to Python 3.11 (issue Build Mac OS packages using Python 3.11 #322). I tried to update it before, but this created a broken build, and I don't have any ability to test Mac packages, so changes to that script are pretty much trial and error.
• Issue Issue with SourceForge cardset downloads for MacOS package #246 has resurfaced a couple times recently, where SourceForge tacks on the ?viasf=1 to the end of the filename of the cardsets package the script downloads. This happens sometimes, but other times, it doesn't - I think it might be a bad mirror, but I'd like to try and find a more permanent fix, so I'm not constantly changing file names in the GitHub action script as a workaround.
• Not as urgent as the above items, but I'd also like to get the scripts set up so I can choose to generate separate builds of the package with the full cardsets package in addition to the normal builds, for releases.

Pillow 10.0.0 created a compatibility issue. I resolved it, but it's going to need a release, as I already had a couple people bring it up. I don't feel comfortable even offering a beta 2.21 release with the current scripts. At least if I can get PyGame running right, I could make do with the workarounds on the other items for a 2.21, and we can do a 2.22 (or 3.0) release later with better packages. But for now, I can't really keep these scripts going as-is.

[joeraz]

Sorry if my initial post was a bit too much of a vent - right now, it just feels like a lot of the scripts are falling apart at once, and it's getting a bit frustrating.

[kingjon3377]

For #246, if --content-disposition isn't enough to get wget to use the filename provided in the Content-Disposition header, adding the --trust-server-names option may help. You could also (instead) take the same approach for the cardsets as you do for the MacOS Python runtime, specifying the filename with -O.

[joeraz]

For #246, if --content-disposition isn't enough to get wget to use the filename provided in the Content-Disposition header, adding the --trust-server-names option may help. You could also (instead) take the same approach for the cardsets as you do for the MacOS Python runtime, specifying the filename with -O.

That worked. For now.

I also noticed the PyGame issue seems to have stopped, for the moment at least. The workarounds for the remaining issues are in place, so the release is no longer blocked.

But we'll still need a proper fix for the remaining items.

[joeraz]

So these are the items from this list that are still an issue:

• The newly released Pillow 10.0 no longer offers 32-bit wheels, which was an intentional decision by the Pillow team. This causes AppVeyor's build to fail. If I update to use a 64-bit Python build, the Windows installer will generate, but after installing it, I get an error "PyInstaller: FormatMessageW failed" when I try to run the installed version. I have to force it to use Pillow version 9.5 as a workaround.
• The Mac release is still using Python 3.9. Would like to update it to Python 3.11 (issue Build Mac OS packages using Python 3.11 #322). I tried to update it before, but this created a broken build, and I don't have any ability to test Mac packages, so changes to that script are pretty much trial and error.
• Not as urgent as the above items, but I'd also like to get the scripts set up so I can choose to generate separate builds of the package with the full cardsets package in addition to the normal builds, for releases.

The first two of these items need to be resolved for me to be comfortable calling any future releases stable. Number three is less important, but will simplify the release process a lot.

[joeraz]

Should update to Python 3.12 now. Need to confirm there aren't any regressions.

[joeraz]

@jnumm Don't know if you're still around or would know anything that might help with this?

[joeraz]

So there was an update on the Pillow front. Apparently, the 32-bit wheels were recreated for Windows, so that is no longer an issue.

But now AppVeyor's environment updated and the make script is unable to find the xgettext command. I set it to use the previous environment, which works for now. But this will likely only work until the next update. I tried manually installing gettext, among other things. No luck.

Does anyone know if any of the version updates described in https://www.appveyor.com/updates/2024/06/13/ could affect gettext?