Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve Reftests #146

Open
hexaeder opened this issue Nov 15, 2023 · 1 comment
Open

Improve Reftests #146

hexaeder opened this issue Nov 15, 2023 · 1 comment

Comments

@hexaeder
Copy link
Collaborator

In #145, I just needed to update a bunch of reftest images again. We somehow need to change how we handle them. Problems:

  1. All assets are stored in the git repo. This is not ideal since the git repo will grow over time.
  2. Currently, the reftests are numbered based on their ordering in the containing file; therefore, if you want to introduce another reference test in between, all the following tests change their name.
  3. There is no "interactive" way of going through the tests and accepting updates.

On the other hand, I like the simplicity of the system, and I like that I only need to mark plot commands in the docs with @reftest to test them. Some time ago, I created a experimental reftest package based on those ideas in GraphMakie, https://github.com/hexaeder/PlotReferenceTests.jl which solves 2 by changing the syntax to something like

@reftest "reference name" plot(...) 

Problem 3 is also easy to solve by implementing some tooling.
However, I don't really know how to deal with 1, which blocks further improvements. Ideas?

@asinghvi17
Copy link
Member

I believe that we upload tarballs as attachments to version tags in Makie, you might find some use for the ReferenceTools and ReferenceUpdater folders in the main repo!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants