test/CMakeLists.txt: updated the test fixture costs #5565
Conversation
firewave
changed the title
|
Actually to properly calculate the costs locally you should not have any values set. need to re-generate the order. |
|
The stored costs can be found under |
firewave
changed the title
|
This already takes the improvements from #5566 into consideration. |
| # To collect data to update this list remove "<cmake-build-folder>/Testing/Temporary/CTestCostData.txt", | ||
| # disable the fixture_cost() statements below and run a Debug build with "ctest -j1" several times. | ||
| # Afterwards run it with "ctest -j11" and immediately cancel the run and update the list accordingly to the | ||
| # first eleven tests chosen. |
There was a problem hiding this comment.
Why only the first 11? We should be using the top slowest tests.
There was a problem hiding this comment.
Yeah, it is kind of arbitrarily - it was because my machine has 12 threads. I also didn't want to have a list which is too long. But that is also about the amount of tests which run longer than the rest and delivers the desired result with -j2 (what is what we have in the CI).
This should only be applied when using the CI since locally CTest will track the actual times it takes and re-order them. Unfortunately I have seen those costs getting messed up pretty easily so they became useless. That's why I didn't make it optional yet. Still need to file bug reports about it.
No description provided.