Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

new trie #210

Open
wants to merge 95 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
95 commits
Select commit Hold shift + click to select a range
91681c8
Adopt the new HashDB API
arkpar Jul 25, 2023
6f097f6
Fixed a test
arkpar Aug 10, 2023
d9ac5aa
Fixes
arkpar Aug 12, 2023
00b4fa9
fmt
arkpar Aug 12, 2023
719aa79
fixing triedbmut lookup, added some testing in test. (#198)
cheme Aug 13, 2023
bd9e3d1
Fixed fuzzing tests
arkpar Aug 13, 2023
1cf9d43
Removed missing_location
arkpar Sep 6, 2023
bac0cbc
Comments
arkpar Sep 20, 2023
ada1da6
Merge branch 'master' of github.com:paritytech/trie into arkpar/new-trie
arkpar Sep 20, 2023
5a9c3b4
Merge branch 'master' of github.com:paritytech/trie into arkpar/new-trie
arkpar Oct 3, 2023
0ff5025
Tests and fixes
arkpar Nov 26, 2023
d00240e
fmt
arkpar Nov 26, 2023
81446ab
Test fixes
arkpar Nov 27, 2023
599b0f1
api for child changeset
cheme Jan 15, 2024
314179d
attach in triedbmut node
cheme Jan 15, 2024
269ef62
push changeset (warn need to init key individually for compat)
cheme Jan 15, 2024
675ae01
fix test and assert
cheme Jan 15, 2024
87d60c8
push attached changeset at last.
cheme Jan 16, 2024
e031dbf
init a test
cheme Jan 16, 2024
7faf394
meh, removed key pass through nodemaybe
cheme Jan 16, 2024
557b52d
better
cheme Jan 16, 2024
ed42db6
memtrie issue
cheme Jan 16, 2024
8d334d2
change children location
cheme Jan 16, 2024
a244144
access to child trie through location.
cheme Jan 17, 2024
f5bc93c
allow access to prefixed child trie
cheme Jan 17, 2024
caf67f3
share more code in test
cheme Jan 17, 2024
de3bc22
extract in fn
cheme Jan 17, 2024
8edcc92
need support of location to init triedbmut
cheme Jan 17, 2024
1f9b8ed
ok
cheme Jan 17, 2024
270aa81
default location instead of none
cheme Jan 17, 2024
d858765
remove unused location parameters from lookup (root location is a loo…
cheme Jan 17, 2024
b5076f8
apply_with_prefix should not be needed.
cheme Jan 18, 2024
39e81cb
attach changeset in explicit function
cheme Jan 18, 2024
04e7a18
no_std
cheme Jan 18, 2024
49bc5a7
Merge branch 'master' into cheme/new-trie
cheme Jan 22, 2024
956ed99
fix
cheme Jan 22, 2024
37a0555
crumb on layout
cheme Jan 22, 2024
ac8a09f
less node decoding, but would need to put location directly in node plan
cheme Jan 22, 2024
bc76a21
minor changes
cheme Jan 23, 2024
5df1cd8
fix
cheme Jan 23, 2024
aeec4fa
avoid using child trie name
cheme Jan 23, 2024
f0d3662
more renamings
cheme Jan 23, 2024
e83c8f0
rename
cheme Jan 23, 2024
367c0e7
HashDB to NodeDB
cheme Jan 23, 2024
c7547c6
cleaning more references to child trie
cheme Jan 23, 2024
1a969ad
use removed key from node only
cheme Jan 23, 2024
6cc8363
remove ChangesetNodeRef
cheme Jan 23, 2024
28a5476
move memorydb hashdb and mem-tree-db into triedb, except Hasher trait.
cheme Jan 23, 2024
b6e4393
Merge branch 'master' into cheme/new-trie
cheme Jan 23, 2024
93433ca
fix
cheme Jan 23, 2024
16f1cd5
no_std
cheme Jan 23, 2024
04a1ee4
missing file write
cheme Jan 23, 2024
5e26628
don't check memorydb
cheme Jan 23, 2024
554187b
flatten crate, only keep h256-std-hasher.
cheme Jan 24, 2024
10cc948
fix
cheme Jan 24, 2024
024e964
fmt
cheme Jan 24, 2024
d176e42
fix
cheme Jan 24, 2024
7a575ef
ci
cheme Jan 24, 2024
fca9577
add testutils when using bench
cheme Jan 24, 2024
26e462d
restore empty function, call it unchanged
cheme Jan 24, 2024
8801370
store child index in node plan
cheme Feb 14, 2024
bb561a0
NodeDBMut trait for testing in polkadot sdk
cheme Feb 15, 2024
e911ebc
Allow non send sync db
cheme Feb 16, 2024
bef6e0f
Revert "Allow non send sync db"
cheme Feb 16, 2024
04ac7a0
rem ref to TrieMut
cheme Feb 16, 2024
e4b29e4
return root from apply changeset
cheme Feb 19, 2024
1b8cced
method to visit changeset to avoid apply_to cost in sc-client-db
cheme Feb 21, 2024
8ad4ee8
Revert "method to visit changeset to avoid apply_to cost in sc-client…
cheme Feb 21, 2024
d352340
empty tree change set
cheme Feb 29, 2024
8f33c2c
better naming
cheme Feb 29, 2024
8e2852f
Switch from trie-db crate to subtrie.
cheme Mar 27, 2024
71bcaf8
fix doctests
cheme Mar 27, 2024
af1b48c
ci
cheme Mar 27, 2024
61c6261
switch to single subtrie crate dep (#1)
cheme Apr 2, 2024
2699e69
possibly repro issue on ksm
cheme Apr 9, 2024
e00e121
fix to bring (warn change test need to be restored and with removal h…
cheme Apr 10, 2024
96867d4
switch to tree ref changeset enum
cheme Apr 15, 2024
f46295a
change types to contain attached location
cheme Apr 15, 2024
52c9f52
rem unuse prefix
cheme Apr 15, 2024
8a63867
direction TODO
cheme Apr 15, 2024
95c5917
refact payload
cheme Apr 16, 2024
3d6b21f
insert only looks ok (except key collision but out of scope).
cheme Apr 16, 2024
3698096
rem print
cheme Apr 16, 2024
f240f4a
default to old_root on empty
cheme Apr 16, 2024
95b443b
fix
cheme Apr 17, 2024
62cef48
use death_row_child
cheme Apr 17, 2024
a6916c0
fix test
cheme Apr 19, 2024
2f13772
removing half attached
cheme Apr 19, 2024
4bb8717
check iter
cheme Apr 19, 2024
51e97f4
Found issue
cheme Apr 19, 2024
ecc52d3
test it
cheme Apr 19, 2024
ed7c5f0
export
cheme May 6, 2024
2c06f2e
clean unused and clone on changeset
cheme May 6, 2024
7533461
Merge branch 'submaster' into cheme/new-trie
cheme May 7, 2024
620d676
fmt
cheme May 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,12 @@ members = [
"memory-db",
"hash256-std-hasher",
"test-support/keccak-hasher",
"test-support/mem-tree-db",
"test-support/reference-trie",
"test-support/trie-standardmap",
"test-support/trie-bench",
"trie-db",
"trie-db/test",
"trie-eip1186",
"trie-eip1186/test",
"trie-root",
"trie-root/test"
]
152 changes: 16 additions & 136 deletions hash-db/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@

#![cfg_attr(not(feature = "std"), no_std)]

#[cfg(not(feature = "std"))]
extern crate alloc;

#[cfg(not(feature = "std"))]
use alloc::vec::Vec;
#[cfg(not(feature = "std"))]
use core::hash;
#[cfg(feature = "std")]
Expand Down Expand Up @@ -73,144 +78,19 @@ pub trait Hasher: Sync + Send {
fn hash(x: &[u8]) -> Self::Out;
}

/// Trait modelling a plain datastore whose key is a fixed type.
cheme marked this conversation as resolved.
Show resolved Hide resolved
/// The caller should ensure that a key only corresponds to
/// one value.
pub trait PlainDB<K, V>: Send + Sync + AsPlainDB<K, V> {
/// Look up a given hash into the bytes that hash to it, returning None if the
/// hash is not known.
fn get(&self, key: &K) -> Option<V>;

/// Check for the existence of a hash-key.
fn contains(&self, key: &K) -> bool;

/// Insert a datum item into the DB. Insertions are counted and the equivalent
/// number of `remove()`s must be performed before the data is considered dead.
/// The caller should ensure that a key only corresponds to one value.
fn emplace(&mut self, key: K, value: V);

/// Remove a datum previously inserted. Insertions can be "owed" such that the
/// same number of `insert()`s may happen without the data being eventually
/// being inserted into the DB. It can be "owed" more than once.
/// The caller should ensure that a key only corresponds to one value.
fn remove(&mut self, key: &K);
}

/// Trait for immutable reference of PlainDB.
pub trait PlainDBRef<K, V> {
/// Look up a given hash into the bytes that hash to it, returning None if the
/// hash is not known.
fn get(&self, key: &K) -> Option<V>;

/// Check for the existance of a hash-key.
fn contains(&self, key: &K) -> bool;
}

impl<'a, K, V> PlainDBRef<K, V> for &'a dyn PlainDB<K, V> {
fn get(&self, key: &K) -> Option<V> {
PlainDB::get(*self, key)
}
fn contains(&self, key: &K) -> bool {
PlainDB::contains(*self, key)
}
}
/// Trait modelling datastore keyed by a hash defined by the `Hasher` and optional location tag.
pub trait HashDB<H: Hasher, T, L>: Send + Sync {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be a good idea to rename this to NodeDB or NodeStore at this point

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also change traits to return errors ? I mean if we return error I would go for NodeDB, if not NodeStore.

(also wondering about renaming crate here)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also change traits to return errors ?

Probably, but in a separate PR.

/// Look up a trie node by hash and location.
/// Returns the node bytes and the list of children node locations if any.
fn get(&self, key: &H::Out, prefix: Prefix, location: L) -> Option<(T, Vec<L>)>;

impl<'a, K, V> PlainDBRef<K, V> for &'a mut dyn PlainDB<K, V> {
fn get(&self, key: &K) -> Option<V> {
PlainDB::get(*self, key)
/// Check for the existence of a hash-key at the location.
fn contains(&self, key: &H::Out, prefix: Prefix, location: L) -> bool {
self.get(key, prefix, location).is_some()
}
fn contains(&self, key: &K) -> bool {
PlainDB::contains(*self, key)
}
}

/// Trait modelling datastore keyed by a hash defined by the `Hasher`.
pub trait HashDB<H: Hasher, T>: Send + Sync + AsHashDB<H, T> {
/// Look up a given hash into the bytes that hash to it, returning None if the
/// hash is not known.
fn get(&self, key: &H::Out, prefix: Prefix) -> Option<T>;

/// Check for the existence of a hash-key.
fn contains(&self, key: &H::Out, prefix: Prefix) -> bool;

/// Insert a datum item into the DB and return the datum's hash for a later lookup. Insertions
/// are counted and the equivalent number of `remove()`s must be performed before the data
/// is considered dead.
fn insert(&mut self, prefix: Prefix, value: &[u8]) -> H::Out;

/// Like `insert()`, except you provide the key and the data is all moved.
fn emplace(&mut self, key: H::Out, prefix: Prefix, value: T);

/// Remove a datum previously inserted. Insertions can be "owed" such that the same number of
/// `insert()`s may happen without the data being eventually being inserted into the DB.
/// It can be "owed" more than once.
fn remove(&mut self, key: &H::Out, prefix: Prefix);
}

/// Trait for immutable reference of HashDB.
pub trait HashDBRef<H: Hasher, T> {
/// Look up a given hash into the bytes that hash to it, returning None if the
/// hash is not known.
fn get(&self, key: &H::Out, prefix: Prefix) -> Option<T>;

/// Check for the existance of a hash-key.
fn contains(&self, key: &H::Out, prefix: Prefix) -> bool;
}

impl<'a, H: Hasher, T> HashDBRef<H, T> for &'a dyn HashDB<H, T> {
fn get(&self, key: &H::Out, prefix: Prefix) -> Option<T> {
HashDB::get(*self, key, prefix)
}
fn contains(&self, key: &H::Out, prefix: Prefix) -> bool {
HashDB::contains(*self, key, prefix)
}
}

impl<'a, H: Hasher, T> HashDBRef<H, T> for &'a mut dyn HashDB<H, T> {
fn get(&self, key: &H::Out, prefix: Prefix) -> Option<T> {
HashDB::get(*self, key, prefix)
}
fn contains(&self, key: &H::Out, prefix: Prefix) -> bool {
HashDB::contains(*self, key, prefix)
}
}

/// Upcast trait for HashDB.
pub trait AsHashDB<H: Hasher, T> {
/// Perform upcast to HashDB for anything that derives from HashDB.
fn as_hash_db(&self) -> &dyn HashDB<H, T>;
/// Perform mutable upcast to HashDB for anything that derives from HashDB.
fn as_hash_db_mut<'a>(&'a mut self) -> &'a mut (dyn HashDB<H, T> + 'a);
}

/// Upcast trait for PlainDB.
pub trait AsPlainDB<K, V> {
/// Perform upcast to PlainDB for anything that derives from PlainDB.
fn as_plain_db(&self) -> &dyn PlainDB<K, V>;
/// Perform mutable upcast to PlainDB for anything that derives from PlainDB.
fn as_plain_db_mut<'a>(&'a mut self) -> &'a mut (dyn PlainDB<K, V> + 'a);
}

// NOTE: There used to be a `impl<T> AsHashDB for T` but that does not work with generics.
// See https://stackoverflow.com/questions/48432842/
// implementing-a-trait-for-reference-and-non-reference-types-causes-conflicting-im
// This means we need concrete impls of AsHashDB in several places, which somewhat defeats
// the point of the trait.
impl<'a, H: Hasher, T> AsHashDB<H, T> for &'a mut dyn HashDB<H, T> {
fn as_hash_db(&self) -> &dyn HashDB<H, T> {
&**self
}
fn as_hash_db_mut<'b>(&'b mut self) -> &'b mut (dyn HashDB<H, T> + 'b) {
&mut **self
}
}

#[cfg(feature = "std")]
impl<'a, K, V> AsPlainDB<K, V> for &'a mut dyn PlainDB<K, V> {
fn as_plain_db(&self) -> &dyn PlainDB<K, V> {
&**self
}
fn as_plain_db_mut<'b>(&'b mut self) -> &'b mut (dyn PlainDB<K, V> + 'b) {
&mut **self
/// Compute value hash.
fn hash(&self, value: &[u8]) -> H::Out {
H::hash(value)
}
}
2 changes: 1 addition & 1 deletion memory-db/benches/bench.rs
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
// limitations under the License.

use criterion::{black_box, criterion_group, criterion_main, Criterion};
use hash_db::{HashDB, Hasher, EMPTY_PREFIX};
use hash_db::{Hasher, EMPTY_PREFIX};
use keccak_hasher::KeccakHasher;
use memory_db::{HashKey, MemoryDB};

Expand Down
Loading