discuss: potential direction for refactoring differential tests

[anna-carroll]

Motivation

In conversations with @jameswenzel , he said that Foundry had fixed the issue discussed here #809 - that rolling back chain state using vm.snapshot no longer erases failed test assertions.

If this is true, the stateless modifier is no longer necessary, and it is possible to refactor Differential tests to use a simpler setup.

Solution

Here I've put forth a potential direction for refactoring Differential tests. I can extend to other tests currently using stateless if we align on the approach.

[0age]

This does seem much cleaner to me!

[anna-carroll]

This does seem much cleaner to me!

Update on my side: not sure vm.revertTo is very stable foundry-rs/foundry#6355

[Evalir]

Hey hey, just an update here @anna-carroll / @0age : we're aware of the issues with snapshot/revertTo—@mattsse tracked this particular bug down and the fix is on foundry-rs/foundry#6451 — let us know if this is sufficient to unblock this, and if not let us know what else should we fix 🙏

[anna-carroll]

Hey hey, just an update here @anna-carroll / @0age : we're aware of the issues with snapshot/revertTo—@mattsse tracked this particular bug down and the fix is on foundry-rs/foundry#6451 — let us know if this is sufficient to unblock this, and if not let us know what else should we fix 🙏

This is amazing @Evalir I think I've been able to verify that the fix works reliably!

I've started in on deprecating the stateless modifier in favor of using snapshots.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (09d431e) 99.77% compared to head (0971769) 99.70%.
Report is 4 commits behind head on main.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1293      +/-   ##
==========================================
- Coverage   99.77%   99.70%   -0.08%     
==========================================
  Files          22       22              
  Lines        1361     1363       +2     
  Branches      220      220              
==========================================
+ Hits         1358     1359       +1     
- Misses          3        4       +1     

Flag	Coverage Δ
foundry	74.90% <100.00%> (-0.20%)	⬇️
production	100.00% <ø> (ø)
reference	100.00% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

This PR has been automatically marked as stale because it has not had recent activity. It will be closed in 14 days if no further activity occurs. Thank you for your contributions. If you believe this was a mistake, please comment.