Source filtering and disabling tests (to later run them externally)

[roberth]

Motivation

• Prevent unnecessary rebuilds by including only relevant files. This speeds up the builds of anything that depends on the main build

  • e.g. Experimental python bindings #7735 or the docker image

• Provide a solid foundation for working on tests that are run in a test environment that is outside the build of nix itself, catching bugs and preventing regressions that can't be detected there.

  • especially Run test suite on NixOS #7778

Context

Additive or negative filtering

The source filtering approach is constructive instead of based on the removal of specific files. This is a best practice because it does not allow newly added files to accidentally cause unnecessary rebuilds. While a ~20 item list doesn't look great, it is hardly worse than the alternative, which is the removal of 8 files and 4 directory. Furthermore, new additions will typically not need inclusion into the main build, as the main build's tooling is already established and most additions of tooling will be for items that aren't part of the main build, such as the python bindings, or the docker image earlier.

Checklist for maintainers

Maintainers: tick if completed or explain if not relevant

• agreed on idea
• agreed on implementation strategy
• tests, as appropriate
  • functional tests - tests/**.sh
  • unit tests - src/*/tests
  • integration tests - tests/nixos/*
• documentation in the manual
• code and comments are self-explanatory
• commit message explains why the change was made
• new feature or incompatible change: updated release notes

[edolstra]

Not sure about the source filtering. That seems like a fair amount of complexity for a somewhat uncommon use case. It would be nice when hacking on the VM tests (usually I use nix.package = toDerivation /nix/store/... to avoid rebuilds).

The ability to disable tests looks useful, maybe that could be a separate PR?

[edolstra]

Sorry, didn't mean to hit "close".

[Ericson2314]

@edolstra See also #7876. I do think we have some leeway between moving things around and filtering, in that the more we do of the one, the less we need to do of the other. But I don't think there is a sustainable option of "not filtering and not moving around files" --- we need to do something.

[Ericson2314]

Based on the discussion in #7847 last time, where we mentioned doing filtering as a prelude to moving things around to avoid needing the filtering, I think we definitely want to do this next.

[roberth]

While I want to agree, as any solution to the rebuilds problem would be helpful, I think we did agree on the goal of moving things, in which case this PR is a bit redundant. My interpretation of the meeting is that we minimize the amount of work done during the move, but we do perform the move (soon?).

[Ericson2314]

Now that we have positive source filtering, should this be rebased?

[roberth]

Had to reindent the filter, so here's a link to view the files with whitespace changes hidden.

[roberth]

Not enabled by default because it does duplicate the compilation work if you want to run both VM and non-VM tests. (I figured I'd be over-explaining by adding that sentence to the code)