Include Worker ID in env similar to Jest

[PaulMest]

Is there any way to get the worker ID within the test or a test helper? The goal would be to use this to automatically set up a different port or database.

Jest offers this as process.env.JEST_WORKER_ID (jestjs/jest#2284, jestjs/jest#5494).

[nabilfreeman]

+1, so I can run 1 database per worker

[PaulMest]

FWIW, I've migrated all of my projects to Jest and have been very happy.

[CoderSpinoza]

@PaulMest How long did it took for you to migrate? This feature is just essential for parallel tests...

[PaulMest]

@CoderSpinoza
I think it all depends on how many tests you have to convert, how much can be done with a codemod, and if you have any other tools in your environment that don't support Jest.

For the most complicated project that involved dozens of tests and hundreds of assertions, it took a couple of days. Bulk of the work was done in ~4 hours.

Once we switched to Jest and larger test suites were running in under 30s, people started writing more tests because it wasn't seen as a pain.

[nabilfreeman]

That's cool, I didn't think about using Jest on the back-end.

We actually wrote our own process management using child_process and inside the child files we passed in a database name and a test file to run. So now each suite runs in its own thread. The child process is running plain Mocha in require mode so this library wasn't necessary.

This actually had some neat advantages because the test suites were always running a fresh DB migration and completely separate, something we've had to be really careful about in teardown hooks in the past.

On the DB part itself I wrote a script that creates N databases in parallel (company-test-167) corresponding to each suite that will be run.

Finally some batch code using async-promise-pool that runs 3x the system cores in parallel (this was best setting for performance).

Really happy with the result but obviously highly custom. Our test times went down by 90% locally and like 99% on Codeship because you get a 128-core machine to run things on.

Same as Paul, our number of tests written has exploded since building this in March - the fast feedback loop is really critical for adoption IMO. We are also finally doing TDD because booting up the tests is finally fast.

[CoderSpinoza]

@nabilfreeman The result sounds great. Did you also have a chance to get aggregated coverage report from parallel tests?

[nabilfreeman]

Great question @CoderSpinoza. I don't know how to do coverage in terms of nyc yet but we built our own coverage checker which counts:
% API routes tested
% ORM model helpers tested
% Utility methods tested

And returns an aggregate. Currently our coverage is still very bad and we're just focusing on getting API routes to 90%. 3 months work has taken us from <40% to 61%.