Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check Run Appears Under Wrong Workflow Run When Test Runner is Run Twice on the Same Commit #222

Open
trudeaua21 opened this issue Jun 9, 2023 · 0 comments
Labels
bug Something isn't working

Comments

@trudeaua21
Copy link
Contributor

trudeaua21 commented Jun 9, 2023

Bug description

If you run the test runner multiple times on the same commit, the Check Run with the test results seems to only populate under the first workflow run for that commit.

Please observe the below images which describe the problem in more detail.

TestRunnerBugReport1

There are 3 workflow runs - 73, 74, and 75, all of which were run on the same commit, d3665f3. 73 and 75 were run on the branch `main`, whereas 74 was run on the branch `test-unity-test-runner-check-run`.

TestRunnerBugReport2

Neither workflow run 74 nor 75 have the "Test Results" Check Run.

TestRunnerBugReport3

3 "Test Results" Check Runs for Workflow Runs 73, 74, and 75 have all populated within the jobs for Workflow Run 73.

How to reproduce

  1. Make a workflow file using the test runner which provides the githubToken input to ensure that the test results check run will show up.
  2. Using a workflow_dispatch trigger (other triggers are fine, this will make it easier though), run a workflow file which runs the test runner (resulting in a workflow run that we'll call workflow run #1).
  3. Observe that the Test Results check run shows up properly for workflow run #1
  4. Manually run the workflow file on that same branch again (not a re-trigger of workflow run #1, but a new run of the workflow file, resulting in workflow run #2).
  5. Observe that the Test Results check run has not shown up properly for workflow run #2, and that there are now 2 separate Test Results check runs under workflow run #1.

4a. Alternatively, after step 3, you could also trigger the bug by making a branch off of the branch you ran the workflow file on in step 2, then running the workflow file on the new branch (only if the new branch is still on the same commit as the base branch).

Expected behavior

Whenever the test runner is run (even if the test runner is being run on the same commit but in a different workflow run), the Check Run with the test results should appear in the workflow run where the tests were run (rather than in the first workflow run for a given commit).

Additional details

This may or may not be fixable - a cursory Google search of the problem led me to this issue, which may or may not be talking about the same problem. I'm a bit too unfamiliar with the inner workings of the Check Run mechanism to determine whether or not it their findings apply here, so I figured I'd open an issue just in case anyone else runs into this problem.

@trudeaua21 trudeaua21 added the bug Something isn't working label Jun 9, 2023
@trudeaua21 trudeaua21 changed the title Check Run Appears Under Wrong Workflow Run When Action is Run Twice on the Same Commit Check Run Appears Under Wrong Workflow Run When Test Runner is Run Twice on the Same Commit Jun 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant