Skip to content

Commit

Permalink
Problem: We have no measurements of how slow testing is
Browse files Browse the repository at this point in the history
Solution: Add 'time -p' to the fractalide-tests* commands.

Result:

pkgs$ nix-build -A fractalide-tests 2>/dev/null | xargs nix-store --read-log | tail -n 3
real 99.86
user 216.53
sys 155.04

$ nix-build -A fractalide-tests-flat 2>/dev/null | xargs nix-store --read-log | tail -n 3
real 16.67
user 48.75
sys 10.58

Each time a tested file requires a dependency, racket has to chase
through 200 directories for it. This is made worse by the fact that
we start racket once for each file we find.

Without modifying 'raco test' itself, we could perhaps make our own
wrapper around the test runner to keep everything in one racket
process.

Flattening fractalide is probably not an option. It takes so much
longer to build, and the derivations for the dependencies cannot
be reused. It would negate any speed advantage for the tests.

The real improvement would probably be to solve
fractalide/racket2nix#158 : Build separate derivations for all
packages, then build a racket environment that links them all in
one folder.
  • Loading branch information
clacke committed Aug 15, 2018
1 parent 0ad25ce commit 536df30
Showing 1 changed file with 12 additions and 3 deletions.
15 changes: 12 additions & 3 deletions pkgs/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -60,13 +60,22 @@ pkgs {
});

fractalide-tests = self.runCommand "fractalide-tests" {
buildInputs = [ fractalide-tests-pkg.env ];
buildInputs = [ fractalide-tests-pkg.env self.parallel self.time ];
} ''
racket -l- raco test ${fractalide-tests-pkg.env}/share/racket/pkgs/*/modules/rkt/rkt-fbp/agents
# If we do raco test <directory> the discovery process will try to mkdir $HOME.
# If we allow raco test to run on anything in agents/gui it will fail because
# requiring gui fails on headless.
find ${fractalide-tests-pkg.env}/share/racket/pkgs/*/modules/rkt/rkt-fbp/agents \
'(' -name gui -prune ')' -o '(' -name '*.rkt' -print ')' |
time -p parallel -n 1 -j ''${NIX_BUILD_CORES:-1} -I {} \
'racket -l- raco test {} |& grep -v -e "warning: tool .* registered twice" -e "@[(]test-responsible"; exit ''${PIPESTATUS[0]}' |
tee $out
exit ''${PIPESTATUS[1]}
'';

fractalide-tests-flat = fractalide-tests.overrideAttrs (oldAttrs: {
buildInputs = [ (fractalide.override { flat = true; }) ];
buildInputs = [ (fractalide.override { flat = true; }).env self.parallel self.time ];
});
})
];
Expand Down

0 comments on commit 536df30

Please sign in to comment.