Skip to content

Commit

Permalink
Problem: We are not sure what is causing fractalide-tests to be slower
Browse files Browse the repository at this point in the history
Building fractalide-tests is 2-3x slower than non-nix raco test with
just full racket and raco-linked fractalide. Is it because if the huge
links.rktd?

If so, flattening should help.

Solution: Add a flattened version of fractalide-tests to find out.

Add 'time -p' to the fractalide-tests* commands.

Result:

```
pkgs$ nix-build -A fractalide-tests 2>/dev/null | xargs nix-store --read-log | tail -n 3
real 99.86
user 216.53
sys 155.04
```

```
pkgs$ nix-build -A fractalide-tests-flat 2>/dev/null | xargs nix-store --read-log | tail -n 3
real 16.67
user 48.75
sys 10.58
```

Each time a tested file requires a dependency, racket has to chase
through 200 directories for it. This is made worse by the fact that
we start racket once for each file we find.

Here's a run entirely nix-less:

```
modules/rkt/rkt-fbp/agents$ /usr/bin/time -p raco test *.rkt cardano-wallet compiled fvm hyperflow IO math mesg plumbing test
[ . . . ]
real 79.57
user 69.78
sys 9.97
```

That actually took longer (wall-clock time) than the flattened run
inside Nix! Apparently raco doesn't parallelize. Here's with parallel:

```
modules/rkt/rkt-fbp/agents$ nix-shell -p parallel --run "find . '(' -name gui -prune ')' -o '(' -name '*.rkt' -print ')' | /usr/bin/time -p parallel -j 4 bash /nix/store/4b1lh27jv1wvm0g7gpif0sxxihgdmi7n-raco-test.sh"
[ . . . ]
real 38.65
user 131.71
sys 15.29
```

I have no explanation for the user time being higher than flattened fractalide.

Possible solutions:

 - Without modifying 'raco test' itself, we could perhaps make our own
   wrapper around the test runner to keep everything in one racket
   process.

 - The real improvement would probably be to solve
   fractalide/racket2nix#158 : Build separate derivations for all
   packages, then build a racket environment that links them all in
   one folder.

Non-solution:

 - Sadly, flattening fractalide.

   It takes so much longer to build, and the derivations for the
   dependencies cannot be reused. It would negate any speed advantage
   for the tests.

   Building flat fractalide takes over an hour CPU time[0]. On our
   Hydra it doesn't even build because it runs out of RAM.

[0] On an Intel Core i3-7100U 2.4 GHz
  • Loading branch information
clacke committed Aug 15, 2018
1 parent 1596294 commit 5ebfc1e
Showing 1 changed file with 7 additions and 3 deletions.
10 changes: 7 additions & 3 deletions pkgs/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,15 @@ pkgs {
});

fractalide-tests = let
# parallel cannot quite handle full inline bash, and destroys quoting, so we can't use bash -c

# parallel cannot quite handle full inline bash, and destroys quoting, so we can't use bash -c
racoTest = builtins.toFile "raco-test.sh" ''
racket -l- raco test "$@" |&
grep -v -e "warning: tool .* registered twice" -e "@[(]test-responsible"
exit ''${PIPESTATUS[0]}
'';
in self.runCommand "fractalide-tests" {
buildInputs = [ fractalide-tests-pkg.env self.parallel ];
buildInputs = [ fractalide-tests-pkg.env self.parallel self.time ];
inherit racoTest;
} ''
# If we do raco test <directory> the discovery process will try to mkdir $HOME.
Expand All @@ -76,10 +76,14 @@ pkgs {
find ${fractalide-tests-pkg.env}/share/racket/pkgs/*/modules/rkt/rkt-fbp/agents \
'(' -name gui -prune ')' -o '(' -name '*.rkt' -print ')' |
parallel -n 1 -j ''${NIX_BUILD_CORES:-1} bash $racoTest |
time -p parallel -n 1 -j ''${NIX_BUILD_CORES:-1} bash $racoTest |
tee $out
exit ''${PIPESTATUS[1]}
'';

fractalide-tests-flat = fractalide-tests.overrideAttrs (oldAttrs: {
buildInputs = [ (fractalide.override { flat = true; }).env self.parallel self.time ];
});
})
];
}

0 comments on commit 5ebfc1e

Please sign in to comment.