Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: gracefully handle race condition between parallel builds #10

Merged
merged 1 commit into from
Apr 6, 2024
Merged

fix: gracefully handle race condition between parallel builds #10

merged 1 commit into from
Apr 6, 2024

Conversation

levibostian
Copy link
Contributor

I am receiving build failures when I run 2 parallel builds that both run setup-mint when they startup. You can reproduce this scenario by creating 2 workflows that get triggered when a PR is opened and both workflows run setup-mint action. 1 will succeed with saving the cache, the other will fail early and give error message "Unable to reserve cache with key -macOS-X64-irgaly/setup-mint-master, another job may be creating this cache."

I believe this is a race condition between the parallel builds where they both attempt to reserve the github actions cache using the same cache key. One way to handle this scenario is to not fail the action when this race condition occurs, since you only need 1 of the parallel builds to save the cache. The other build can skip saving and move on.

References:
oven-sh/setup-bun#45 - experiencing the same issue as this person actions/cache#485 (comment) - explaining the idea of how caches are reserved

I am receiving build failures when I run 2 parallel builds that both run setup-mint when they startup. You can reproduce this scenario by creating 2 workflows that get triggered when a PR is opened and both workflows run setup-mint action. 1 will succeed with saving the cache, the other will fail early and give error message "Unable to reserve cache with key -macOS-X64-irgaly/setup-mint-master, another job may be creating this cache."

I believe this is a race condition between the parallel builds where they both attempt to reserve the github actions cache using the same cache key. One way to handle this scenario is to not fail the action when this race condition occurs, since you only need 1 of the parallel builds to save the cache. The other build can skip saving and move on.

References:
oven-sh/setup-bun#45 - experiencing the same issue as this person
actions/cache#485 (comment) - explaining the idea of how caches are reserved
@levibostian levibostian marked this pull request as draft April 3, 2024 12:35
@levibostian
Copy link
Contributor Author

Converting to draft PR while I finish testing. I am running the fork against a project of mine to see if this fixes the problem I was experiencing.

@levibostian
Copy link
Contributor Author

Testing concluded and successful!

How I tested

  • I have 2 github action jobs. Both jobs run in parallel when a PR is opened and both run the setup-mint action first.

  • I modified my github workflows to run this PR's fix.

  • I clear all github actions caches.

  • I open a PR which runs the 2 github jobs in parallel.

  • The github jobs both compile mint and both run mint bootstrap for the first time. When it's time for the jobs to save the cache, you will notice that 1 of the jobs will display a warning but will continue running. The other job will successfully save the cache.

  • After the 2 jobs finish running successfully for the first time (and successfully saving a cache for the first time), I re-ran the jobs to see if they restore the caches successfully. Both of the 2 jobs successfully restore the previous cache and both run bootstrap using this cache. The 2 runners run super fast now! 1 job, 2nd job.


Re-opening PR since testing is successful. Ready for review.

@levibostian levibostian marked this pull request as ready for review April 3, 2024 13:10
@irgaly irgaly self-requested a review April 6, 2024 08:31
@irgaly
Copy link
Owner

irgaly commented Apr 6, 2024

@levibostian
Thank you for your report and careful testing!
It seems a nice approach to key conflict.

I'll merge this and release new one.

@irgaly irgaly merged commit 6f9cd90 into irgaly:main Apr 6, 2024
irgaly added a commit that referenced this pull request Apr 6, 2024
@irgaly irgaly mentioned this pull request Apr 6, 2024
irgaly added a commit that referenced this pull request Apr 6, 2024
@irgaly
Copy link
Owner

irgaly commented Apr 6, 2024

I published v1.4.0 with this PR's fix. Please check it with irgaly/setup-mint@v1 that is automatically point v1.4.0.
Thanks!

@levibostian levibostian deleted the skip-cache-mintbin branch April 6, 2024 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants