-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Surface browser version change information #94
Conversation
87f6bd2
to
41f3acd
Compare
Co-authored-by: Danny Gleckler <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is what the file we'll be checking into each repo would look like. Open to other suggestions around naming, etc.
🎉 This PR is included in version 0.21.0 🎉 The release is available on: Your semantic-release bot 📦🚀 |
Sorry, somehow totally missed this PR. So when do we commit the updates to this file? Only when tests have failed? |
I was hoping it would get committed as part of the visual-diff action just like a change to a golden? |
But yeah... that make me think we don't want to update it unless the |
Locally, I guess we never want to update it because we don't want people accidentally committing it? Then in CI, if the tests passed the action isn't currently set up to commit anything (even orphaned goldens) because those would just be handled next time there's an actual failure or update needed. But in this case, it's a bit more awkward - you could run into a scenario where the goldens have passed for a few weeks and the next failure jumps up a few browser versions. In practice, maybe this would never ever happen - a new browser version always causes a failure. And technically, the data is still accurate - the previous goldens were generated at that browser version from weeks ago. So maybe what we want is:
? |
I think that makes sense! I'll put up a PR to at least handle the local scenario. |
This adds a tracker
.vdiff.json
file to the root of the repo-under-test that is intended to get checked into source control. In there we can put whatever we like, but for now it contains the browsers and versions of those browsers from the previous test run.We can then use that information to surface any changes in the versions in the report -- both in the side panel but also when viewing a particular browser's results: