Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

System Test: Nightly invocation for soroban test cases #10

Open
sreuland opened this issue Jan 9, 2023 · 4 comments
Open

System Test: Nightly invocation for soroban test cases #10

sreuland opened this issue Jan 9, 2023 · 4 comments

Comments

@sreuland
Copy link
Contributor

sreuland commented Jan 9, 2023

What problem does your feature solve?

Currently, system-tests are invoked automatically only on open PR's in soroban-tools.

What would you like to see?

Nightly runs system test with latest main/master branch source versions.
Notification via slack channel when the nightly run fails.
Run tests against multiple soroban enabled target networks:

  1. Futurenet?
  2. Local(i.e. standalone with inlined compiled core from source tag version of rc release)
  3. Testnet(configured against the remote sdf hosted rpc instance)

What alternatives are there?

Manually invoke the system tests on command line with targer package or source code versions:

docker run --platform linux/amd64 --rm -it "stellar/system-test" --SorobanCLICrateVersion "0.4.0" --SorobanRPCDebianVersion "0.4.0-10" --SorobanExamplesGitHash "v0.4.1" --SorobanExamplesRepoURL "https://github.com/stellar/soroban-examples.git" --RustToolchainVersion "1.66.0" --CoreDebianVersion "19.6.1-1158.c0ad35aa1.focal~soroban" --HorizonDebianVersion "2.22.0~soroban-323" TestFilter "^TestDappDevelop$/^.*$" --VerboseOutput true 

@sreuland sreuland moved this from Backlog to Next Sprint Proposal in Platform Scrum Jan 9, 2023
@sreuland sreuland changed the title Automate test invocation for release cycle Automate test invocation for soroban release cycle Jan 11, 2023
@sreuland
Copy link
Contributor Author

need to identify the release version of components that need to be invoked for release, could be hard code to start/fallback, but automated retrieval design would be ideal.

@sreuland
Copy link
Contributor Author

include nightly cron job of tests with main/master latest builds of all components.

@sreuland sreuland changed the title Automate test invocation for soroban release cycle Nightly test invocation for soroban system test Jan 27, 2023
@sreuland
Copy link
Contributor Author

after further triage in sprint poker, we narrowed the scope of this to just be a new nightly scheduled run of system tests.

@paulbellamy paulbellamy moved this from Next Sprint Proposal to Current Sprint in Platform Scrum Jan 31, 2023
@mollykarcher mollykarcher moved this from Current Sprint to Next Sprint Proposal in Platform Scrum Mar 13, 2023
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Backlog in Platform Scrum Mar 28, 2023
@mollykarcher mollykarcher moved this from Backlog to Next Sprint Proposal in Platform Scrum Mar 28, 2023
@sreuland sreuland moved this from Next Sprint Proposal to Current Sprint in Platform Scrum Mar 28, 2023
@mollykarcher mollykarcher moved this from Current Sprint to Backlog in Platform Scrum Apr 17, 2023
@mollykarcher mollykarcher moved this from Backlog to Next Sprint Proposal in Platform Scrum May 23, 2023
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Backlog in Platform Scrum Jul 13, 2023
@mollykarcher mollykarcher added this to the Soroban Pubnet Release milestone Sep 26, 2023
@mollykarcher mollykarcher moved this from Backlog to Next Sprint Proposal in Platform Scrum Sep 26, 2023
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Backlog in Platform Scrum Sep 26, 2023
@mollykarcher mollykarcher moved this from Backlog to Next Sprint Proposal in Platform Scrum Sep 27, 2023
@sreuland
Copy link
Contributor Author

for consideration, move this up to the Soroban Stable P20 Release milestone, to have it in place sooner and providing earlier visibility on test coverage against latest rc's, that may be changing regularly during testnet's discovery phase.

@sreuland sreuland changed the title Nightly test invocation for soroban system test System Test: Nightly invocation for soroban test cases Sep 27, 2023
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Current Sprint in Platform Scrum Nov 7, 2023
@mollykarcher mollykarcher moved this from Current Sprint to Next Sprint Proposal in Platform Scrum Jan 2, 2024
@mollykarcher mollykarcher moved this from Next Sprint Proposal to Backlog in Platform Scrum Jan 30, 2024
@mollykarcher mollykarcher moved this from Backlog to Next Sprint Proposal in Platform Scrum Feb 1, 2024
@stellarsaur stellarsaur self-assigned this Feb 2, 2024
@stellarsaur stellarsaur moved this from Next Sprint Proposal to Current Sprint in Platform Scrum Feb 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Backlog
Development

No branches or pull requests

3 participants