[ReshardingV3] State #12050
Comments
71 tasks
github-merge-queue bot
pushed a commit
that referenced
this issue
Oct 11, 2024
Tracking issue: #12050 #### Summary Currently the changes should be almost no-op, as we do not explicitly save anything to `DBCol::ShardUIdMapping`. The only difference is that we make an additional read from `DBCol::ShardUIdMapping` column every time we access `State` column. The main logic is in `Store::get_impl_state()`. These changes implement mapping for reads, writes will be handled in the next PR. #### Changes: - Added `DBCol::ShardUIdMapping` that is initially empty and will be populated on future resharding events. - Slight refactor: only allow `Store` to create `StoreUpdate`. - `Store::get_impl_state()` - special `get()` implementation for the State column. #### Next steps (see tracking issue #12050): - Use mapping for writes to db. - Handle `copy_state_from_store` in `cold_storage.rs`. - Integration. - State clean up (e.g. gc parent state when it is no longer referenced by any child). - Tests.
github-merge-queue bot
pushed a commit
that referenced
this issue
Oct 29, 2024
Tracking issue: #12050 Summary: - Refactor `ReshardingManager::process_memtrie_resharding_storage_update()` into `start_resharding()` as it is an entry point to start resharding. - Add `set_state_shard_uid_mapping()` and call it from `start_resharding()`. - Test that State mapping integrates with resharding flow in `resharding_v3.rs` testloop. Test would fail with `MissingTrieValue` for children shards if disabled `set_state_shard_uid_mapping`. Perhaps `test_resharding_v3_base` should be as minimal as possible and I should not test State mapping there. But for now I do not see much value in doing it differently.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
We agreed to use the mapping strategy (design doc).
Implementation plan:
Store::get()for State column.copy_state_from_storeincold_storage.rs(see comment).load_one_subtreeinparallel_loader.rs.The text was updated successfully, but these errors were encountered: