Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

discussion: x-client fork prep and test patterns #84

Open
meowsbits opened this issue Jun 19, 2019 · 11 comments
Open

discussion: x-client fork prep and test patterns #84

meowsbits opened this issue Jun 19, 2019 · 11 comments
Labels
status:0 wip ECIP is still work in progress and shall not be merged. type: std-ecbp ECIPs of the type "ECBP" - describing best practices rather than protocols.

Comments

@meowsbits
Copy link
Member

meowsbits commented Jun 19, 2019

🚧 🐠 I would like to see and help work toward describing plans and accountable standards for

A. Test network testing protocol

As cody speaks to, testnet outcomes are only as meaningful as the rigors they are subjected to. We ought to be clear and collaborative about the design and implementation of these "live" testnet protocols. Who will participate in this? Who will take responsibility, and for which aspects? What are standards that we expect for these protocols? ... reproducability? ... regression-readiness?

  • Kensington
  • Kotti
  • Morden

B. Cross client integration tests

Like establishing a shared and meaningful idea of what "not breaking" means above, there are clear limitations to the implications of even successful testnet behavior (for example eth-classic/go-ethereum#67 would not have been caught).

This points to the value of shared cross-client test suites, which can prove out edge cases and other important simulations that may not be possible or practical on a test network. ethereum/tests is an example of this, albeit in a very one-trick-pony kind of way.


Since the teams "owning" client development are either independent and/or volunteer-based, and that participation in the network is entirely in the hands of the client operator in the first place, the standards that might be developed here could not be considered with any more earnest than suggested guidelines for client software and operator review, albeit developed with common aims and shared skin in the game.

@meowsbits
Copy link
Member Author

Rel B. Cross client integration tests

At ETC Labs, we've been developing an "extrapolated" test suite based on ethereum/tests that extends relevance for some of these "sub-suites" thru the anticipated Atlantis configuration. So far -- albeit experimentally -- classic geth and multigeth have shown consistent behavior for custom Difficulty and GeneralStateTests and we're working on adoption for Parity as well.

Work on this at multi-geth can be found here: https://github.com/etclabscore/multi-geth/tree/feat/parity-chainspec-tests-fill-atlantis-2, and an example Atlantis-filled difficulty test can be found here: https://gist.github.com/whilei/cf5700910b41207a37d9f7ca58d0fd3d

@soc1c
Copy link
Contributor

soc1c commented Jun 19, 2019

Kotti Hardfork is tonight!

https://stats.kotti.goerli.net/

@soc1c
Copy link
Contributor

soc1c commented Feb 3, 2020

are you still working on this @meowsbits?

now we only have multi geth, parity ethereum, and hyperledger besu left.

@bobsummerwill
Copy link
Member

@meowsbits Also bear in mind that we now have Whiteblock Genesis at our disposal for exactly this kind of cross-client ETC testing need. It would be great if we could make use of that capability for ETC Aztlan/Phoenix testing:

https://genesis.whiteblock.io/

@YazzyYaz has been giving it a whirl. We are also proposing Yaz as a Vice-Chair for the EEA Testing WG with @atoulme and @zscole at the helm.

ETC Cooperative can also take on running instances of ethereum/tests, HIVE, ChainSafe amenome, whatever. I want us to have the same degree of coverage as there is for ETH. Union of "all the things", including ethstats and forkmon instances.

CC @GregTheGreek, @edwardmack.

@bobsummerwill
Copy link
Member

And of course, we welcome OTHERS within the ETC ecosystem to "run all the things" as well.

@atoulme
Copy link
Contributor

atoulme commented Feb 3, 2020

We have an example with multigeth here: https://github.com/whiteblock/genesis-examples/tree/master/etc

We also have examples featuring besu and parity. Putting all 3 together is just a matter of creating the right configuration.

What would be useful to test?

@GregTheGreek
Copy link

We use anemone + some internal infrastructure. Happy to chat about it further. It allows for us to test against the clients

@soc1c
Copy link
Contributor

soc1c commented Feb 5, 2020

we need to test aztlan on mordor with whiteblock genesis or chainsafe anemone

@TheEnthusiasticAs
Copy link
Member

Can this issue be closed and new issue opened with the refer to here? Mixing Aztlan and Atlantis topics is not the best way to follow the work progress in my opinion. Thx

@meowsbits
Copy link
Member Author

How about I'll change the name to remove the Atlantis pigeon-hole, broadening to something like "fork prep" ?

@TheEnthusiasticAs
Copy link
Member

it is also ok for me ). Also, pls lable this issue. Thx

@meowsbits meowsbits changed the title discussion: x-client Atlantis (ECIP 1054) prep and testing protocols discussion: x-client fork prep and test patterns Feb 11, 2020
@q9f q9f added status:0 wip ECIP is still work in progress and shall not be merged. type: std-ecbp ECIPs of the type "ECBP" - describing best practices rather than protocols. labels Aug 17, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status:0 wip ECIP is still work in progress and shall not be merged. type: std-ecbp ECIPs of the type "ECBP" - describing best practices rather than protocols.
Projects
None yet
Development

No branches or pull requests

8 participants
@atoulme @bobsummerwill @GregTheGreek @meowsbits @soc1c @TheEnthusiasticAs @q9f and others