Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable ingestion via CDP in horizon verify-range docker image #5553

Open
tamirms opened this issue Dec 11, 2024 · 0 comments
Open

Enable ingestion via CDP in horizon verify-range docker image #5553

tamirms opened this issue Dec 11, 2024 · 0 comments

Comments

@tamirms
Copy link
Contributor

tamirms commented Dec 11, 2024

There is a verify-range docker image which is used to detect ingestion regressions in horizon release candidates. It works by ingesting a range of ledgers and then dumping the contents of the horizon history tables to disk. Then it repeats the same workflow but this time using the horizon release candidate. Finally, the output from both runs are compared. The job succeeds if the output from both runs are identical.

The verify-range job is effective in detecting regressions but it takes very long to run on full history. Also, the verify-range job often fails on tasks because it consumes too much memory. Both of these issues can be solved if we update the verify-range docker image to ingest from a GCS datastore of ledgers instead of using captive core.

@tamirms tamirms moved this from Backlog to To Do in Platform Scrum Dec 11, 2024
@tamirms tamirms added this to the platform sprint 54 milestone Dec 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: To Do
Development

No branches or pull requests

1 participant