Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test coverage report incomplete for apex test retries #35

Open
aspagni opened this issue Mar 8, 2024 · 3 comments
Open

Test coverage report incomplete for apex test retries #35

aspagni opened this issue Mar 8, 2024 · 3 comments
Assignees
Labels
bug Something isn't working

Comments

@aspagni
Copy link

aspagni commented Mar 8, 2024

Describe the bug
When running the sfp apextests trigger command, if there are LOCKED_ROW errors due to parallel testing, the failed tests rerun in serial mode (and that works fine).
Issue is that, within the .test results folder, only the test-result-[runId]-coverage.json for the second serial run is saved.
This does not allow to have a full picture of the code coverage report (i.e. to send it to sonarqube).

Other files in the .testresults folder (junit and JSON) keep track of both test runs, only the coverage one loses track of the original parallel run (see screenshot)

To Reproduce
Run sfp apextests trigger in a repository where LOCKED_ROW error happens and gets retried.
Check generated coverage file in .testresults folder

Expected behavior
Either merge the coverage for both files into a single one, or at least persist the original one.

Screenshots
image.png
The XML file shows that two runs happened (and they are somehow merged into a single one, see the two IDs in the name file)

Platform Details:

  • OS: Windows
  • Version sfp/36.0.9
  • Salesforce CLI(sfdx cli) Version: 2.22.7
  • CI Platform: locally

Additional context
Add any other context about the problem here.

@azlam-abdulsalam azlam-abdulsalam added bug Something isn't working and removed analysis labels Mar 11, 2024
@dieffrei
Copy link
Collaborator

@azlam-abdulsalam code coverage result from the first run is merged into second file per design.
Is there any special reason for that?
Or shall we keep them separated?
Screenshot 2024-05-27 at 07 18 47

@aspagni
Copy link
Author

aspagni commented May 28, 2024

Just be clear, the issue is not with the files being merged, but with the actual content of the file, which only contains test results from the second run

@Schuchie
Copy link

Schuchie commented Sep 3, 2024

@dieffrei I think I've made a fix in the past for that merging. The reason for that was, that only the first or second (?) was taken, where then previous successfully runs were ignored completely. Maybe it should always take successfully runs and the runs which are only contained in one of these lists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

4 participants