Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(test): Run test matrix on test_programs #6429

Merged
merged 13 commits into from
Nov 12, 2024

Conversation

aakoshh
Copy link
Contributor

@aakoshh aakoshh commented Nov 1, 2024

Description

Problem*

Resolves #6390

Summary*

Runs the test_programs with -Inf, 0 and +Inf --inliner-aggressiveness before we can change the default from +Inf (inline everything) to 0.

There are some test failures triggered by this, which need to be fixed before merging this:

Additional Context

A specific test can be executed like so:

cargo test -q -p nargo_cli --test execute <test-name>

Consistent test failures are (created tickets for them, see above):

    tests::execution_success::test_brillig_cow::false_i64_min_expects
    tests::execution_success::test_brillig_cow::true_i64_min_expects
    tests::execution_success::test_closures_mut_ref::true_0_expects
    tests::execution_success::test_closures_mut_ref::true_i64_min_expects
    tests::execution_success::test_eddsa::true_0_expects
    tests::execution_success::test_eddsa::true_i64_min_expects
    tests::execution_success::test_missing_closure_env::true_0_expects
    tests::execution_success::test_missing_closure_env::true_i64_min_expects
    tests::execution_success::test_references::true_i64_min_expects
    tests::execution_success::test_slice_dynamic_index::true_i64_min_expects

Note that instead of the _brillig suffix we have ::true_ indicate which tests are run with --force-brillig.

All failures are under execution_success, so I switched off the inlining matrix for the other test types; doing so impoves the runtime by 15-30%.

Another bug independent from inliner aggressiveness was #6445 This existed in the codebase before and I have seen it locally, but it never seemed to have affected CI; it probably uses less threads. I fixed it in this PR; since it affected test code generation I didn't think a separate PR would be helpful.

Documentation*

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

@aakoshh aakoshh changed the title Run test matrix on test_programs feat(test): Run test matrix on test_programs Nov 1, 2024
@aakoshh aakoshh changed the title feat(test): Run test matrix on test_programs chore(test): Run test matrix on test_programs Nov 1, 2024
@aakoshh aakoshh marked this pull request as draft November 1, 2024 16:59
tooling/nargo_cli/build.rs Outdated Show resolved Hide resolved
@aakoshh aakoshh requested a review from asterite November 8, 2024 12:17
@TomAFrench TomAFrench added this pull request to the merge queue Nov 12, 2024
Merged via the queue into master with commit 82046be Nov 12, 2024
49 checks passed
@TomAFrench TomAFrench deleted the 6390-inliner-test-matrix branch November 12, 2024 12:30
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 12, 2024
…st` (noir-lang/noir#6502)

feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 12, 2024
…ng/noir#6502)

feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 13, 2024
…-lang/noir#6506)

fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 13, 2024
)

fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 14, 2024
feat: avoid unnecessary ssa passes while loop unrolling (noir-lang/noir#6509)
fix(tests): Use a file lock as well as a mutex to isolate tests cases (noir-lang/noir#6508)
fix: set local_module before elaborating each trait (noir-lang/noir#6506)
fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 14, 2024
feat: avoid unnecessary ssa passes while loop unrolling (noir-lang/noir#6509)
fix(tests): Use a file lock as well as a mutex to isolate tests cases (noir-lang/noir#6508)
fix: set local_module before elaborating each trait (noir-lang/noir#6506)
fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 14, 2024
…g/noir#6449)

chore(ssa): Skip array_set pass for Brillig functions (noir-lang/noir#6513)
chore: Reverse ssa parser diff order (noir-lang/noir#6511)
chore: Parse negatives in SSA parser (noir-lang/noir#6510)
feat: avoid unnecessary ssa passes while loop unrolling (noir-lang/noir#6509)
fix(tests): Use a file lock as well as a mutex to isolate tests cases (noir-lang/noir#6508)
fix: set local_module before elaborating each trait (noir-lang/noir#6506)
fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 14, 2024
chore(ssa): Skip array_set pass for Brillig functions (noir-lang/noir#6513)
chore: Reverse ssa parser diff order (noir-lang/noir#6511)
chore: Parse negatives in SSA parser (noir-lang/noir#6510)
feat: avoid unnecessary ssa passes while loop unrolling (noir-lang/noir#6509)
fix(tests): Use a file lock as well as a mutex to isolate tests cases (noir-lang/noir#6508)
fix: set local_module before elaborating each trait (noir-lang/noir#6506)
fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Nov 14, 2024
…me intepreter (noir-lang/noir#6514)

fix: disallow `#[test]` on associated functions (noir-lang/noir#6449)
chore(ssa): Skip array_set pass for Brillig functions (noir-lang/noir#6513)
chore: Reverse ssa parser diff order (noir-lang/noir#6511)
chore: Parse negatives in SSA parser (noir-lang/noir#6510)
feat: avoid unnecessary ssa passes while loop unrolling (noir-lang/noir#6509)
fix(tests): Use a file lock as well as a mutex to isolate tests cases (noir-lang/noir#6508)
fix: set local_module before elaborating each trait (noir-lang/noir#6506)
fix: parse Slice type in SSa (noir-lang/noir#6507)
fix: perform arithmetic simplification through `CheckedCast` (noir-lang/noir#6502)
feat: SSA parser (noir-lang/noir#6489)
chore(test): Run test matrix on test_programs (noir-lang/noir#6429)
chore(ci): fix cargo deny (noir-lang/noir#6501)
feat: Deduplicate instructions across blocks (noir-lang/noir#6499)
chore: move tests for arithmetic generics closer to the code (noir-lang/noir#6497)
fix(docs): Fix broken links in oracles doc (noir-lang/noir#6488)
fix: Treat all parameters as possible aliases of each other (noir-lang/noir#6477)
chore: bump rust dependencies (noir-lang/noir#6482)
feat: use a full `BlackBoxFunctionSolver` implementation when execution brillig during acirgen (noir-lang/noir#6481)
chore(docs): Update How to Oracles (noir-lang/noir#5675)
chore: Release Noir(0.38.0) (noir-lang/noir#6422)
fix(ssa): Change array_set to not mutate slices coming from function inputs (noir-lang/noir#6463)
chore: update example to show how to split public inputs in bash (noir-lang/noir#6472)
fix: Discard optimisation that would change execution ordering or that is related to call outputs (noir-lang/noir#6461)
chore: proptest for `canonicalize` on infix type expressions (noir-lang/noir#6269)
fix: let formatter respect newlines between comments (noir-lang/noir#6458)
fix: check infix expression is valid in program input (noir-lang/noir#6450)
fix: don't crash on AsTraitPath with empty path (noir-lang/noir#6454)
fix(tests): Prevent EOF error while running test programs (noir-lang/noir#6455)
fix(sea): mem2reg to treat block input references as alias (noir-lang/noir#6452)
chore: revamp attributes (noir-lang/noir#6424)
feat!: Always Check Arithmetic Generics at Monomorphization (noir-lang/noir#6329)
chore: split path and import lookups (noir-lang/noir#6430)
fix(ssa): Resolve value IDs in terminator before comparing to array (noir-lang/noir#6448)
fix: right shift is not a regular division (noir-lang/noir#6400)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Run entire test suite with multiple values for inliner_aggressiveness
2 participants