Update CI, mostly MSYS1 (now using CI tarball instead of checkout)

[GitMensch]

ci adjustments

• ubuntu:
  • only run jobs for "coverage" and "additional warnings" if the main ci build works
    and use its generated tarball in both cases
  • add two new artifacts: test binaries and windows source
• msys1+ubuntu:
  • upload config.log after the build - because we may need it to debug build issues
  • always upload the testsuite.log (additional build documentation)
• msys1:
  • GMP url changes, building it again for performance reasons
  • building BDB with all relevant patches from MSYS2
  • using msys-build instead of building Bison (only necessary for GC4)
  • drop GC install log step and therefore extra prefix
  • drop extra CFLAGS previously necessary for local cJSON (fixed in 3.x)
    and/or ignoring failing NIST) --> as after last upstream update everything works
  • enable NIST85 (+ comment-code in case we ever need to skip something there
    and/or ignoring failing NIST) --> as after last upstream update everything works
  • ci cache adjustment:
    • remove split per matrix
    • split per software, enabling smaller updates
  • use CI tarball like for the minimal build
  • drop workflow specific expected failures that now work fine
  • move env to MSYS job
• integrate msys1.yml into ubuntu.yml, renaming to build_nightly.yml

[ddeclerck]

As for the MSYS1 workflow : maybe I missed something ; why is it necessary to trigger it from the Ubuntu workflow ?

[GitMensch]

If the workflow is separated then we can only get the CI tarball (MSYS1 is too old to build with the new dependencies in GC 3.3) by knowing the run-id of the job; we can either trigger it from Ubuntu to do so (but this misses the output in PRs.. and anywhere outside the "Action" tab) or do some more complicate API calling with timeout in the MSYS1.

... or we just combine them into one, and save all the struggle...

[ddeclerck]

... or we just combine them into one, and save all the struggle...

That might be a good idea. But considering you already went through the struggle, might as well keep it as you made it.

[GitMensch]

I think I'll give that a shot soonTM, maybe its as easy as I've thought :-)

[GitMensch]

It would be really good to know which of those tests hang under MSYS2.

While CI runs take longer that way it could be reasonable doing an explicit make test first (multi-job), then a non-parallel run of the internal testsuite - and if there are still hanging results check if we see "regular" issues at the same set of tests ?

[ddeclerck]

It would be really good to know which of those tests hang under MSYS2.

I tried to push the CI a bit ; also checked older logs, looking for missing test results. Looks like this is not tied to any specific test (either this, or most of the tests are affected ?), because each time it was a different one. Here are a few of them (numbers not relevant):

58: COPY: recursive
198: ASSIGN to device-name
343: USE FOR DEBUGGING syntax-checks (1) 
514: LISTING directive free-form reference-format
691: LOCAL-STORAGE (2)
950: EXTFH: Indexed with FH--FCD
957: EXTFH: INDEXED with multiple keys
1010: RETURN-CODE moving

Most of the time this is accompanied with a WFMO error in the logs, which hints at a locking issue.

As you suggest, we might have resort to runing the testsuite sequentially here (takes ~15-20 min, while in parallel it is more like ~10-12 min). I guess that's acceptable...

[GitMensch]

Thanks for checking. I'll leave the single-job test adjustment for MSYS2 to you (at least for now I'd suggest to reorder and have NIST in multi-job run first). I'll try to combine the other two into one in the meantime, if it works as good as I think, then the result will be better than what we had before.

[ddeclerck]

MSYS1 is too old to build with the new dependencies in GC 3.3

Was curious about it, so I just tried installing autoconf 2.70 from source in MinGW, and it happens to build and pass the testsuite (cf PR 188). Don't know if that could be useful.

[GitMensch]

for some reason the push closed this... so we're going on in #189 now (note: the MSYS1 autoconf update is in as a comment [and split per its "new way] in that PR, but shouldn't be needed).