feat: stub for instant resharding state root creation #12083
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #12083 +/- ##
==========================================
- Coverage 71.59% 71.52% -0.07%
==========================================
Files 818 819 +1
Lines 164532 164694 +162
Branches 164532 164694 +162
==========================================
+ Hits 117792 117796 +4
- Misses 41597 41753 +156
- Partials 5143 5145 +2
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Longarithm
force-pushed
the
r-new-state-root
branch
from
e47af53
to
71b6358
Compare
Longarithm
changed the title
Longarithm
requested review from
wacban,
shreyan-gupta,
marcelo-gonzalez and
Trisfald
| /// Returns id of the node after deletion applied. | ||
| fn delete_multi_range_recursive<'a, M: ArenaMemory>( | ||
| root: MemTrieNodePtr<'a, M>, | ||
| key_nibbles: Vec<u8>, |
There was a problem hiding this comment.
this feels like it could be some more efficient representation like &[u8] but it's hard to say if it matters and how exactly to do it
There was a problem hiding this comment.
Another option is to cut nibbles from the front of intervals_nibbles. But as the keys are short and this happens only once, I want to be simple wherever it's possible.
|
|
||
| match node_view { | ||
| MemTrieNodeView::Leaf { extension, .. } => { | ||
| let extension = NibbleSlice::from_encoded(extension).0; |
There was a problem hiding this comment.
nit: Would be nice to implement a getter method instead of using .0 (I know it's outside of this PR but it's a small change so might as well do it).
There was a problem hiding this comment.
There are a lot of places using NibbleSlice::from_encoded(...).0 and I wouldn't touch them for now, seems better to fix them all at once.
Also not sure if a getter method is applicable because from_encoded returns (Self, bool).
|
@wacban thanks for the great feedback! applied a lot of changes |
|
nice start! 🚀 |
|
@shreyan-gupta answering globally - yeah, reusing even more from MemTrieUpdate works more nicely. |
| } | ||
|
|
||
| /// Based on the key and the intervals, makes decision on the subtree exploration. | ||
| fn retain_decision(key: &[u8], intervals: &[Range<Vec<u8>>]) -> RetainDecision { |
Create the code flow intended for resharding state root creation.
The ultimate goal will be the temporary memtries created, which will be used until new memtries are created from flat storage.
This will be a multi-step work. To complete it, all mentioned TODOs must be resolved.
Overview
The core is a
MemTrieUpdate::retain_multi_rangefunction and a simpletest_retain_multi_rangetest. It follows the logic in #12074. I plan to enhance logic sequentially until it will be fit for the actual resharding.Drawback is that code is not used on chain, but setting up full resharding test requires separate work. I suggest to start from the the incomplete flow; hopefully I will need ~5 PRs to set up the full test.
It follows the "top-down" logic as in insert/delete functions: first we pull root node to in-memory update list, then we descend into children. In the end, we call
to_trie_changeswhich callspost_order_traverse_updated_nodesto create full set of changes to be applied to trie.Alternative
For the multi-range deletion, better logic is "bottom-up": we first understand the result of children subtrees, and then mark root node as updated if needed.
It's not clear whether everything must be "bottom-up". It probably has its own challenges. But here it is strictly better because it automatically gives the post traversal order of updated nodes.
We could compute hashes on the fly but it's less LoC to postpone it to reuse existing logic, so I just call
compute_hashes_and_serialized_nodesin the end.