Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing suite for builder #10

Open
bertmiller opened this issue Nov 23, 2022 · 1 comment
Open

Testing suite for builder #10

bertmiller opened this issue Nov 23, 2022 · 1 comment

Comments

@bertmiller
Copy link
Member

Rationale

To allow anyone to easily make changes to the builder, measure their impact, and ensure that they are safe.

Implementation

  • Provide standardized tools and data (sample bundles)for testing
  • Provide a framework for testing (e.g. performance metrics)
  • Create guide to deploying and testing the builder

One way to approach this is to create a mass conflicting bundles at random and test merging against those. But thinking it through we might want a standardized dataset that allows for more precise measurement of performance (e.g. variance in the randomized generation of bundles doesn't affect performance)

@bertmiller
Copy link
Member Author

I was thinking we need these things:

  • provide a guide to setting up the builder for local development
  • provide a test script to run
  • provide testing data (bundles and txs)

the output should be something like:

  • hey, the code all works as intended and these X number of checks were met!
  • XX bundles were merged
  • XX bundles were simulated
  • in XX time XX block templates were generated
  • the most profitable block was XY ETH with XX bundles merged alongside XX transactions

ideally this is somehow standardized with the same inputs so that you can directly compare 2 versions of the code and say "hey, this version performed better than the other, which performs better than the main branch"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant