Corrupted download cache on builders

[barthalion]

          @barthalion @hfiguiere this is so annoying, could someone please just clear the clones/caches on masterworker 3 and 5 please?

Deleting /var/lib/buildbot-master/master/workers/MasterWorker5/Builds/build/flatpak-module-qpdf-pikepdf and /var/lib/buildbot-master/master/workers/MasterWorker3/Builds/build/flatpak-module-qpdf-pikepdf should be enough. I hope they will get re-generated / re-cloned as necessary. I'm pretty sure the problem is that the change of remote url/scheme never gets applied on the third level of submodules.

See also #211

Originally posted by @dreua in flathub/com.github.jeromerobert.pdfarranger#222 (comment)

[dreua]

I meant second level of submodules (pdfarranger -> flatpak-module-qpdf-pikpdf -> flatpak-builder-tools).
The linked issue should be flathub/com.github.jeromerobert.pdfarranger#211

Thanks for bringing this here and let me know if there are any further questions!

[dreua]

Another fail on MasterWorker5: https://buildbot.flathub.org/#/builders/6/builds/129471

Note that submodule sync does not have the --recursive flag set, but submodule update runs with it.

[dreua]

Another failed build on MasterWorker5, restarting until it gets put on a builder without the issue as a workaround is getting annoying. What can I do to bring this forward?

Failed build PR: flathub/com.github.jeromerobert.pdfarranger#249

[dreua]

@barthalion I'm done waiting for this and being annoyed by failing builds. You have two weeks to come up with some meaningful response here. (Just ssh into the builders, clean the cache and we won't get on each others nerves anymore 🤷)
Otherwise I'll need to reconsider my involvement with flathub, probably ceasing to update the application or removing it from flathub.

[barthalion]

I don't see how your passively aggressive comment is supposed to make anyone happy to be helping with anything.

[barthalion]

What's even funnier, the PR that triggered your entitled response built just fine. The simplest fix would be to stop referencing flatpak-builder-tools as git submodule in your submodule repo. This is bound to be breaking the more submodules you nest, especially if they may also be used by other apps.

(And on that note, while your app has been grandfathered, we no longer support external submodules like yours.)

[dreua]

I've asked nicely three times in this issue alone and have opened a PR against buildbot, the issue on the recipe repo and asked on the matrix channel even earlier. All nice and calm but nothing happened. Now I am annoyed because there is something broken which is not in my control and I can do nothing to fix it myself.

I don't think I am being passive agressive or entitled because I stated very clearly how I feel and what I want to happen and the consequences if that is not going to happen. After all I am not doing this because I love flatpaks or flathub so much but rather because users asked for a flatpak on flathub. (That might be a general issue in the dynamics between app developers and you as infrastructure maintainer. We are not here firstly to please you but to please our users. Of course I always try to keep things nice and until yesterday I believe I put in the effort and got nothing in return.) I'm sorry for putting pressure on you here and I understand if that feels unjust. On the other hand you are the only person I know of who could fix this problem and I don't want to keep being annoyed by the failing builds.

The build I mentioned this issue in is not affected by this, you are right and I'm sorry for the confusion. In this case the buildbot took over one and a half hour to respond to the PR which is longer than it usually takes but a totally different issue, maybe not an issue at all. It just reminded human me about the other issue I had and led me to write the comment here.
Yesterday's build on master worker 8 ran just fine because this issue only affects master worker 3 and 5. The builds I linked previously all failed exactly because of this issue.

The simplest fix would be to stop referencing flatpak-builder-tools as git submodule in your submodule repo. This is bound to be breaking the more submodules you nest, especially if they may also be used by other apps.

I didn't come up with the submodules myself, let me quote the flatpak-builder-tools readme:

The intended usage of the generators is as a submodule used as part of your build process to generate manifests.

I use the builder tools to regularly regenerate the pikepdf dependencies which is necessary to catch changes in the set of dependencies which could go beyond simple version updates.

Placing the builder tools as submodule inside the qpdf/pikepdf repository is the only place where it makes sense to me and that in turn needs to be placed inside the recipe repository resulting in a second level submodule, unless there is a better way I am not aware of.

(And on that note, while your app has been grandfathered, we no longer support external submodules like yours.)

This is the first time I'm hearing this, thanks for making me aware. However, I do wonder if there is a different or better way for collaboration between app maintainers. I created the external submodule in an open source spirit so that others can easily use it. Furthermore it allows me a great deal of automation in dependency updates in order to keep dependencies minimal and up to date, so that installation size and the attack surface are minimized.

Honestly I begin to wonder if I'm the only maintainer who came up with a setup like this to allow both collaboration and automation of dependency updates. If there are better and less submodule heavy ways to achieve those two goals I'm happy to implement them.