Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On CI, multi-versions tests should be executed in parallel #75

Closed
bmuschko opened this issue Dec 13, 2018 · 4 comments · Fixed by #147
Closed

On CI, multi-versions tests should be executed in parallel #75

bmuschko opened this issue Dec 13, 2018 · 4 comments · Fixed by #147

Comments

@bmuschko
Copy link
Contributor

Executing the full set of integration tests including all versions currently takes a long time (~30 mins). We should think about a way to run them in parallel. The multi-version test runner already offers a way to run these tests for a given version.

  • Option 1: Rename all multi-version thats with the suffix *MultiVersionIntegrationTest. On CI run the task integrationTest and provided a --test filter + the system property that define the version.
  • Option 2: Move all multi-version tests to a new source set. A new test task can run these tests with a specific version by providing the system property.

Use Travis CI's parallel execution feature. We already do this to run integration tests and documentation tests in parallel.

@bmuschko bmuschko changed the title Multi-versions tests should be executed in parallel on CI On CI, multi-versions tests should be executed in parallel Dec 13, 2018
@big-guy
Copy link
Member

big-guy commented Dec 19, 2018

As a quick and dirty solution, could we also try running the integration tests with maxParallelForks > 1?

@bmuschko
Copy link
Contributor Author

bmuschko commented Dec 19, 2018

We already do but I guess it doesn't help much as a build job on Travis CI only has 1 CPU with 2 cores.

@marracuene
Copy link
Contributor

We already do but I guess it doesn't help much as a build job on Travis CI only has 1 CPU with 2 cores.

Please see these timings I have just posted... indeed the long-runnning docTest is multi-version and in parallel... still takes ages. #136 (comment)

@cstroe
Copy link
Contributor

cstroe commented Sep 12, 2020

Using Travis CI's Matrix build, we can break out the build by Play version. The fix is in #147 , see the Travis CI build here

See the .travis.yml changes for details.

I didn't check if the individual builds are completely deduplicated, the per-build tasks might still be running non-parametrized tests with every build.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants