Build TuxGuitar on non-x86_64 architectures

[helge17]

In the tg-multiarch branch I am trying to make the build system multi-arch capable, at least for Linux, maybe later also for FreeBSD and MacOS.

This would make it easier to build TuxGuitar on non-x86_64 architectures, for upstream and hopefully also for the Linux distributions and FreeBSD (in fact, I had this idea while working on the "official" Debian package). On MacOS we have to support the ARM architecture sooner or later.

It was a bit harder than expected, but I think it was worth it in the end.

Currently I can build the Linux SWT version on ppc64le and aarch64 with misc/build_tuxguitar_from_source.sh, in addition to the x86_64 builds, which should work as before.

Basically, I'm facing two problems:

Availability of SWT and JFX

The repos archive.eclipse.org and repo.maven.apache.org only support a few architectures. So if you want to build TuxGuitar on other architectures, you have to use the SWT/JFX provided by the operating system. I think I could modify the build script to accomplish this on Linux and maybe on FreeBSD.

Different names for the same architectures

My idea is not to maintain a separate pom.xml file for each individual architecture, but to have a single pom.xml (and a build script) that automatically detects which architecture it is currently running on.

Unfortunately, there are (at least?) three sources for the architecture names:

• The Maven ${os.arch} variable
• $ uname -m
• $ dpkg --print-architecture (for Debian packaging)

Many names are the same, but there are differences: x86_64 is sometimes called amd64, aarch64 is the same as arm64, ppc64le=ppc64el. That's why in different places I use different sources for the architecture name, partly because that name is needed at that point to get things working, partly because it's the only name available at the current location (e.g. how can I read the output of uname -m into a Maven variable at the very beginning of the build process without ugly workarounds?).

Next steps

My multi-arch implementation is quite invasive, might introduce new errors and clash with other changes. So I ask for some feedback: Do you think it's worth including these changes in master? Maybe there is a better way to support more architectures? Do you have any suggestions on what could be done better?

If your agree, I would gradually merge my changes into master. From time to time I might also make some non-x86_64 packages available, but not regularly, as it is too time consuming to keep all build systems up to date.

[profOnno]

Sorry for mentioning this, cause I'm coming back to the versions plugin. 😟 .
You can modify pom.xml files with it, also adjust properties. This way you can use one pom.xml file and let mvn do modifications on it for architecture specific futures (swt versions?).

maven set property
There is also a build helper plugin for maven, build helper

if (helpfull) {
   use = true;
} else {
   use = false;
   ignore = true;
}

[helge17]

Don't worry, we all (especially me ;-) ) learn a lot in this project.

I'm not against using the version plugin or any other new plugin to build TuxGuitar, if it really helps us.

But as mentioned in #272, in this particular case, the overhead of using this plugin is simply much greater than the benefit: Why should we use a fairly complex plugin to partially solve a problem that I can completely solve with two lines of code in the build script? Keep in mind that most of us are not professional programmers, but just deal with TuxGuitar (and programming in general) in our spare time. If I need to update the version string of TuxGuitar in a few weeks, I will have to re-read the plugin documentation just to make a change that I can do with vi or sed in a few seconds. It's not worth it.

In this multi-arch context, my problem is not to modify the POM files, but to dynamically control the build process depending on the underlying architecture, without modifying the POM files and without setting architecture-specific parameters, profiles, etc. in the POM files. The idea is to have a static set of POM files (and a build script) that detects the architecture automatically and "does the right things".

I have found this way to set the architecture in the DEBIAN/control file from the POM file. This method can also be used to set the version number. But the method is very limited as it only appends something to the file, it can't really edit the file.

[profOnno]

Thanks for your reply,
I'm not an experienced java programmer, but now I understand some more about the problem. I was looking at the pom.xml as a static template, and then the build script could modify the template for it's special needs. Not sure about the overhead of the plugin, maybe i'll play around with it some more in the future.

I'll try keep an eye on your multi arch work, interesting. For the 'most of us are not professional programmers' I've started a discussion, showing were I went wrong and how to prevent that.

[helge17]

The most invasive part of the tg-multiarch branch is the renaming of the directories and the artifacts: For Linux, FreeBSD and MacOS, I removed the "x86_64" from the names since they have at least two architectures they could run on. For Windows, I've kept the "x86_64" in the names as I can't think of any other architectures we could realistically support.

All other changes are small: a few "uname -m" and "${os.arch}" in a few places, update of the build script, slightly changed package names, that's it.

All x86_64 builds should work as before, some new architectures should work out of the box.

The renaming might clash with some PRs or forks, so I didn't want to merge the changes directly into master. Of course, feedback from @guiv42 and everyone else is very welcome.

[guiv42]

OK, I'll have a look (when I find some time!)
About the PRs: none of the 3 current PRs are mature enough to be merged. And re-creating them after tg-multiarch is merged should be quite easy I think. So I would not consider this a blocking point.
I did not look precisely, but I anticipate some complexity on my side for the multi-arch rpm packaging currently done in openSuse (with many patches for different architectures). Nothing that can't be solved, but my main problem here is that currently I can only test x86-64. Might need to setup a VM in another arch for testing purposes, that could take some time.

[guiv42]

After a first test: this looks really good :)
I successfully built and ran tuxguitar in an aarch64 VM.
Unfortunately I won't be able to test this more seriously, my 10+ years old machine does not correctly handle qemu/aarch64 overhead (about 12 hours processing to install a Linux distro and all dependencies required to build).

I checked (not tested) the existing multi-arch rpm packaging of openSuse. One single issue: in my environment, ${os.arch} from maven returns amd64, which can cause some problems. I don't know how openSuse build service will handle that.

So: why not simply delete the arch name from the output folder name? See example below for linux swt (could be done also for jfx, deb, freebsd and mac):

-		<project.finalName>tuxguitar-${project.version}-linux-swt-${os.arch}</project.finalName>
+		<project.finalName>tuxguitar-${project.version}-linux-swt</project.finalName>

Basically, I would prefer to delete all instances of ${os.arch} if this is possible.

Then I would be in favor of merging tg-multiarch into master.

[helge17]

Thank you very much for your feedback! I will remove all ${os.arch} from the pom files. I need to slightly customize the build script for this, but it's quite easy.

[helge17]

Commit 8811ba3 removes ${os.arch} from the pom files.

If there are no objections, then I would merge tg-multiarch into master in the next few days.

[helge17]

The tg-multiarch branch is now merged into master. Today's pre-release also contains packages for arm64 and ppc64le.