Skip to content

v0.0.19
Compare
Choose a tag to compare
@maoueh maoueh released this 23 Aug 16:02
· 2672 commits to develop since this release
v0.0.19
df5a068

Changelog

New! A updatePolicy of type append has been added, it allows one to build a store that concatenates values and supports parallelism. This affects the server, the manifest format (additive only), the substreams crate and the generated code therein.

Rust API

  • Store APIs methods now accept key of type AsRef<str> which means for example that both String an &str are accepted as inputs in:

    • StoreSet::set
    • StoreSet::set_many
    • StoreSet::set_if_not_exists
    • StoreSet::set_if_not_exists_many
    • StoreAddInt64::add
    • StoreAddInt64::add_many
    • StoreAddFloat64::add
    • StoreAddFloat64::add_many
    • StoreAddBigFloat::add
    • StoreAddBigFloat::add_many
    • StoreAddBigInt::add
    • StoreAddBigInt::add_many
    • StoreMaxInt64::max
    • StoreMaxFloat64::max
    • StoreMaxBigInt::max
    • StoreMaxBigFloat::max
    • StoreMinInt64::min
    • StoreMinFloat64::min
    • StoreMinBigInt::min
    • StoreMinBigFloat::min
    • StoreAppend::append
    • StoreAppend::append_bytes
    • StoreGet::get_at
    • StoreGet::get_last
    • StoreGet::get_first

  • Low-level state methods now accept key of type AsRef<str> which means for example that both String an &str are accepted as inputs in:

    • state::get_at
    • state::get_last
    • state::get_first
    • state::set
    • state::set_if_not_exists
    • state::append
    • state::delete_prefix
    • state::add_bigint
    • state::add_int64
    • state::add_float64
    • state::add_bigfloat
    • state::set_min_int64
    • state::set_min_bigint
    • state::set_min_float64
    • state::set_min_bigfloat
    • state::set_max_int64
    • state::set_max_bigint
    • state::set_max_float64
    • state::set_max_bigfloat

  • Bumped prost (and related dependencies) to ^0.11.0

CLI

  • Environment variables are now accepted in manifest's imports list.

  • Environment variables are now accepted in manifest's protobuf.importPaths list.

  • Fixed relative path not resolved correctly against manifest's location in imports list.

  • Changed the output modes: module-* modes are gone and become the
    format for jsonl and json. This means all printed outputs are
    wrapped to provide the module name, and other metadata.

  • Added --initial-snapshots (or -i) to the run command, which
    will dump the stores specified as output modules.

  • Added color for ui output mode under a tty.

  • Added some request validation on both client and server (validate
    that output modules are present in the modules graph)

Service

  • Added support to serve the initial snapshot

Commits

  • 9360b95 Builder renamed to Store everywhere
  • a21fe56 Builder renamed to Store everywhere
  • 469ca52 Bumped prost to ^0.11.0
  • 0431365 Fix error handling
  • 5e2c29c Improvements to Manifest imports and protobuf.importPaths
  • fd92720 Merge branch 'develop' into feature/wasmtime
  • c05f9b2 Merge branch 'substreams-workable' into develop
  • f68b9d4 Merge pull request #33 from streamingfast/feature/wasmtime
  • 71d2318 Merge pull request #45 from streamingfast/feature/stack_trace
  • b152c35 Merge remote-tracking branch 'origin/develop' into feature/wasmtime
  • 8cda6d9 Prepare release v0.0.16
  • df5a068 Preparing release
  • 9952d2e Preparing release for version 0.0.17
  • 69dbbd9 Remove noisy log
  • 460cd0b Removed worker statistics as fmt.Println and turn it into a proper log line
  • ac3d554 Replacing derr.RetryContext with for lopp to retry the job
  • 88927e5 add cache entry for empty output
  • 8d2051c added memory size log
  • 58265e5 added memory size log (fixed)
  • 3131b33 added todo
  • 67c096a adding RetryableErr concept if the error is considered a RetryableErr and to retry on the other side of the scheduler.go and returning a normal error when it is considered an unRetryableErr
  • 248f89a adding cacheEnabled attribute to pipeline and adding if statements to use and not use the cache in places where the output cache is loaded, get, save and update
  • 5855ebe adding decode command for store and map
  • a770d75 adding decoder for decoding bytes to a protobuf definition
  • 8c609fb adding execution stack on executor and returning ErrorExecutor when getting an instance.Execute()
  • 2bbda15 adding fix me for jobStat update method
  • 2c09764 adding job planner test for dependency priority bug
  • 44c8805 adding pipeline options to pipeline
  • 603bb38 adding the execution stack when returning the logs if a substreams failed
  • 90dce93 adding the logs to the execution stack when calling info,.bu debug! and println! in the substreams code
  • d4b39ae all test passed
  • a35b9b5 augmenting worker stat
  • 2526acc better exec error handling
  • 6ed2840 bring back job stats update
  • bb33a01 bummped firehose@develop
  • cba7427 bump bstream/firehose, don,t pre-decode block
  • 08d1f84 bump crate version to 0.0.18
  • feff865 bump crate version to 0.0.18
  • 4559084 bumping substreams, substreams-macro and test cargo toml versions to 0.0.15 for release
  • 013d83f change some log level
  • a59364e create a workerPool by service instead of by request and add job stats
  • d36f035 fix all tests
  • e7132b7 fix all tests
  • 7b33054 fix cache memory issue
  • 494ba42 fix getLast
  • 83626f1 fix index out of range
  • 0fac649 fix log
  • 3a73f56 fix memory leak
  • e7a44cd fix panic when request does not contain any module
  • 4d9b76c fix remote host
  • c90238b fix some tests
  • 2f8fe9b fix stats update
  • 35e91f6 fix step to proto matching in response
  • f3e318b fix step to proto matching in response
  • 83b0a3e fix work progress handling
  • 7f889d6 fix work progress handling again
  • bd391c9 fix worker not returned to pool on error
  • d21299f fixing cache no-output flag for backprocessing store
  • c69345b fuel integration, wip
  • 62b4d54 merge workable-substreams
  • c099c2a merged develop
  • 215313c more cleanup of builder keyword
  • 84f788d more cleanup of builder keyword
  • 8207556 moving functionnal options pattern for Service struct to options file and adding cacheEnabled attribute
  • 040e0ff now backend by wasmtime. All tests pass
  • 15378bb only send hostname in grpc header if SUBSTREAMS_SEND_HOSTNAME is set to 'true'
  • 0e7f087 only send hostname in grpc header if SUBSTREAMS_SEND_HOSTNAME is set to 'true'
  • ca19bdc remove way to verbose logs
  • c71225f removing dependencies for wazero
  • a890810 removing error returned value on cache get as there is no error that can occur and adding GetAtBlock method
  • b8bb54a removing flip from stringMap and byteMap
  • bc853af removing storeOptions as it is unused
  • 2692aa0 resetting kv on store when unmarshalling
  • 33a9600 resetting the kv once it was unmarshalled
  • d335118 revert fuel integration. Merged develop
  • 14c30da setting 8076 iterations for deterministic value where any other iteration wasmtime will fail deterministically
  • fff9722 signaling completion in the outter loop after the write go routines have been launched
  • 15577d8 skipping recursion test
  • 6f56d1d skipping recursion test
  • 43599e7 spacing, removing wazero sys dependency, no more check on the ExitError like previously done on wazero as the return value is handled differently and changing some Module struct value attribute names
  • 05f20da state: remove unused file
  • 62dfed6 state: remove unused file
  • 5f35b85 switching back to github.com/tetratelabs/wazero
  • d46e304 temporary commentating the update for jobStat
  • 7017af4 update executionStack
  • 67f0119 wasm module and instance refactoring
  • 2974147 wip
Assets 7
Loading