Fixes and improvements to scripts for working with cyclic dependencies

[Maxython]

close #20338 - it is not possible to compile all packages using build-all.sh because buildorder.py cannot create build orders with circular dependencies.

There is also a solution to part of problem #20500:

Subpackages cause cycles in the build order.

There are also minor changes, all details are in the commit.

CC @tstein @TomJo2000

[tstein]

It took me an hour to review this and I'm still not clear on what the new behavior is. We should not merge this until the code and comments are clear enough that reading it in one pass leaves you with a truthy understanding of the build order computation.

(I think it's a mistake to keep hacking on this without stepping back to think about how this code should be and how your changes get us closer. Not trying to block, because I think it's possible to get this to a state where it's better to merge it than not, but then we should have that conversation right after this.)

[TomJo2000]

Nice work Max.

[robertkirkman]

Hello, I greatly appreciate the work that Maxython has done in this PR to enable the build-all.sh to begin building packages, rather than the current vanilla behavior of build-all.sh as it currently exists in the unmodified repo (which is to print a large error about "cyclic dependencies" and do nothing). I am not intending to be rude or belittling towards the quality of the code in this PR, on the contrary I would like to help it be as complete and useful a tool as possible to implement a functioning build-all.sh for use in the pinned issue until a time when the vanilla repository gets its own functioning build-all.sh implementation, whatever form that might end up taking, and this PR has served well for a very large amount of packages consecutively, without behaving too strangely for most of the process. I would just like to explain something that happens to me every time I attempt to build the libepoxy package currently using the command scripts/run-docker.sh ./build-package.sh libepoxy in any clone of the repository that has this PR applied to it. In the vanilla repository, these commands complete successfully:

docker container stop termux-package-builder
docker container rm termux-package-builder
docker image rm ghcr.io/termux/package-builder:latest
git clone https://github.com/termux/termux-packages.git
cd termux-packages/
scripts/run-docker.sh ./build-package.sh libepoxy

On the other hand, unfortunately, it appears that there might be a problem when using the same command in a clone of the repository that has this PR applied to it (with the exception of the glib package which conflicts currently, but seems unrelated and shouldn't impact this because glib is not a dependency of libepoxy)

docker container stop termux-package-builder
docker container rm termux-package-builder
docker image rm ghcr.io/termux/package-builder:latest
git clone https://github.com/termux/termux-packages.git
cd termux-packages/
curl https://patch-diff.githubusercontent.com/raw/termux/termux-packages/pull/20513.diff | git apply -v --exclude=packages/glib/build.sh
scripts/run-docker.sh ./build-package.sh libepoxy

since when I use those commands, I see this:

[7/13] Compiling C object src/libepoxy.so.p/dispatch_egl.c.o
FAILED: src/libepoxy.so.p/dispatch_egl.c.o 
aarch64-linux-android-clang -Isrc/libepoxy.so.p -Isrc -I../src/src -Iinclude -I../src/include -Iinclude/epoxy -I/data/data/com.termux/files/usr/include -fdiagnostics-color=always -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -std=gnu99 -Oz -g -fstack-protector-strong -Oz -fPIC -Wpointer-arith -Wmissing-declarations -Wformat=2 -Wstrict-prototypes -Wmissing-prototypes -Wnested-externs -Wbad-function-cast -Wold-style-definition -Wdeclaration-after-statement -Wunused -Wuninitialized -Wshadow -Wmissing-noreturn -Wmissing-format-attribute -Wredundant-decls -Werror=implicit -Werror=nonnull -Werror=init-self -Werror=main -Werror=missing-braces -Werror=sequence-point -Werror=return-type -Werror=trigraphs -Werror=array-bounds -Werror=write-strings -Werror=address -Werror=int-to-pointer-cast -Werror=pointer-to-int-cast -fno-strict-aliasing -Wno-int-conversion -fvisibility=hidden -MD -MQ src/libepoxy.so.p/dispatch_egl.c.o -MF src/libepoxy.so.p/dispatch_egl.c.o.d -o src/libepoxy.so.p/dispatch_egl.c.o -c ../src/src/dispatch_egl.c
In file included from ../src/src/dispatch_egl.c:28:
In file included from ../src/src/dispatch_common.h:46:
../src/include/epoxy/glx.h:36:10: fatal error: 'X11/Xlib.h' file not found
   36 | #include <X11/Xlib.h>
      |          ^~~~~~~~~~~~
1 error generated.

It looks to me like the libx11 and libglvnd-dev build dependencies of libepoxy end up not being completely detected, causing them not to be installed when using the code from this PR, as opposed to the behavior of the vanilla repository where they are correctly built automatically before the libepoxy package is built. This can be confirmed by testing this command, which works in both the current vanilla repository, and the current repository with this PR applied to it.

scripts/run-docker.sh ./build-package.sh libx11 libglvnd libepoxy

I will troubleshoot this and try to find if there is any alternative or workaround possible to this situation in order to automatically detect the required dependencies at all times.

[TomJo2000]

This review is only pertaining to commit c7976ec.

This is pretty much unreviewable.
843 lines changed in a single commit is one hell of a lot to ask.
The specific pieces I have commented on in this review are by no means exhaustive, but I do not have 2-3 hours to review one commit.
I may be coming back to this later and do more incremental review,
but the amount of nesting in the code makes it exceedingly difficult to follow.

Please consider inverting some of the deeply nested conditionals into guard clauses so they can be mentally discarded after they are passed.

[Maxython]

Please consider inverting some of the deeply nested conditionals into guard clauses so they can be mentally discarded after they are passed.

Perhaps some local functions could be created in some places, but is it worth it to make the code more readable? It is worth noting that the commit highlights a large number of changed lines due to new conditions added to large algorithms (but in essence they did not change that much) for the correct operation of the collector.