Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test/CMakeLists.txt: bumped CTest timeout to accommodate macos-* runner slowness #6704

Merged
merged 1 commit into from
Aug 17, 2024

Conversation

firewave
Copy link
Collaborator

The macos-* runners are much slower than the other leading to sporadic timeouts when running CTest with a debug build in the CI.

@firewave
Copy link
Collaborator Author

firewave commented Aug 16, 2024

5/107 Test  #15: TestCondition ........................***Timeout  90.07 sec
[...]
6/107 Test  #68: TestTokenizer ........................***Timeout  90.04 sec

The same test on from the same workflow on ubuntu-22.04:

6/107 Test  #15: TestCondition ........................   Passed   32.07 sec
[...]
5/107 Test  #68: TestTokenizer ........................   Passed   26.92 sec

Most of the time in tests is spent because of copying the settings (actually it is copying the Library within). This will be addressed in the future in the prerequisite work towards automatic library detection within the application.

@firewave firewave changed the title test/CMakeLists.txt: bumped CTEST_TIMEOUT to accommodate macos-* slowness test/CMakeLists.txt: bumped CTEST_TIMEOUT to accommodate macos-* runner slowness Aug 16, 2024
@firewave firewave force-pushed the ctest-timeout branch 2 times, most recently from e27323c to 99ec831 Compare August 16, 2024 12:07
@firewave firewave changed the title test/CMakeLists.txt: bumped CTEST_TIMEOUT to accommodate macos-* runner slowness test/CMakeLists.txt: bumped CTest timeout to accommodate macos-* runner slowness Aug 16, 2024
@firewave
Copy link
Collaborator Author

I removed the timeout to get the actual runtime but the test finished in the expected amount of time:

5/107 Test  #15: TestCondition ........................   Passed   24.06 sec
6/107 Test  #68: TestTokenizer ........................   Passed   25.26 sec

So I can only bump the timeout based on the factor (seems to be x5 at most) other tests were slower in the failing run.

  3/107 Test   #5: TestAutoVariables ....................   Passed   62.45 sec
[...]
  4/107 Test  #59: TestStl ..............................   Passed   83.60 sec
[...]
  7/107 Test #102: cfg-std_c ............................   Passed   51.21 sec
[...]
  8/107 Test  #73: TestUninitVar ........................   Passed   72.39 sec
 3/107 Test   #5: TestAutoVariables ....................   Passed   12.26 sec
[...]
 4/107 Test  #59: TestStl ..............................   Passed   18.99 sec
[...]
 7/107 Test #102: cfg-std_c ............................   Passed   15.09 sec
[...]
 8/107 Test  #73: TestUninitVar ........................   Passed   20.29 sec

@firewave firewave marked this pull request as ready for review August 16, 2024 12:23
@firewave firewave merged commit 97277b0 into danmar:main Aug 17, 2024
63 checks passed
@firewave firewave deleted the ctest-timeout branch August 17, 2024 11:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants