Meson functional tests

[Ericson2314]

Motivation

Run the functional test suite with Meson

Context

#2503

Depends on #11176
Depends on #11177
Depends on #11178
Depends on #11179

Priorities and Process

Add 👍 to pull requests you find important.

The Nix maintainer team uses a GitHub project board to schedule and track reviews.

[fricklerhandwerk]

Triaged in Nix team meeting:

• @roberth tried to work around some failures and also had design questions
  • it should probably follow the convention we already have for the unit tests, even if we're not building much
  • @Ericson2314: instead of copying the sources to the build directory we could actually "install" them
• @roberth: this also needs to work in the VM test
  • can definitely be done
  • would be fantastic if we could just package the tests, but maybe it's just not how it works
  • @Ericson2314: we could absolutely do both, but there's still a lot of reworking to get the functional tests how we want them

[nixos-discourse]

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-07-10-nix-team-meeting-minutes-160/49101/1

[Ericson2314]

@GoldsteinE might you be interested in helping out with this? I think I just (hackily) fixed the CI eval issue so the actual interesting failures will show up.

The differences between the Meson and Make bash environments for the functional tests is...very subtle! :( It makes debugging these failures hard.

[Ericson2314]

Fishing out some non-test-suite files like this is no good with the broken-up build. If we want to test .. we should do it a different way (e.g. make some dirs in $TEST_ROOT and cd inside there).

[GoldsteinE]

I’ll take a look, but I don’t really know Meson.

Unrelated, but I think one of the end goals here should be rewriting functional tests to something that is not bash. Writing logic in bash is extremely error-prone and I think replacing it with e.g. Python would make all of this logic much less brittle.

[Ericson2314]

@GoldsteinE Thank you!!

Unrelated, but I think one of the end goals here should be rewriting functional tests to something that is not bash.

I agree! I wrote up #10823 for this.

[Ericson2314]

The PRs that this depends on should be reviewed separately, but then it's good to go!