From e488308046352d6451f926ee8fe0bb6076979484 Mon Sep 17 00:00:00 2001 From: Trisfald Date: Fri, 15 Nov 2024 16:08:16 +0100 Subject: [PATCH] fix lint --- neps/nep-0568.md | 56 +++++++++++++++++++++++++++--------------------- 1 file changed, 32 insertions(+), 24 deletions(-) diff --git a/neps/nep-0568.md b/neps/nep-0568.md index 720c165c9..04aacf072 100644 --- a/neps/nep-0568.md +++ b/neps/nep-0568.md @@ -229,11 +229,12 @@ During a resharding event, at the boundary of the epoch, when we need to split t Resharding Flat State is a time consuming operation and it runs in parallel with block processing for several block heights. Thus, there are a few important aspects to consider during implementation: -- Flat State's own status should be resilient to application crashes. -- The parent shard's Flat State should be split at the correct block height. -- New shards' Flat States should eventually converge to same representation the chain is using to process blocks (MemTries). -- Resharding should work correctly in the presence of chain forks. -- Retired shards are cleaned up. + +* Flat State's own status should be resilient to application crashes. +* The parent shard's Flat State should be split at the correct block height. +* New shards' Flat States should eventually converge to same representation the chain is using to process blocks (MemTries). +* Resharding should work correctly in the presence of chain forks. +* Retired shards are cleaned up. Note that the Flat States of the newly created shards won't be available until resharding is completed. This is fine because the temporary MemTries are built instantly and they can satisfy all block processing needs. @@ -241,16 +242,18 @@ built instantly and they can satisfy all block processing needs. The main component responsible to carry out resharding on Flat State is the [FlatStorageResharder](https://github.com/near/nearcore/blob/f4e9dd5d6e07089dfc789221ded8ec83bfe5f6e8/chain/chain/src/flat_storage_resharder.rs#L68). #### Flat State's status persistence + Every shard Flat State has a status associated to it and stored in the database, called `FlatStorageStatus`. We propose to extend the existing object by adding the new enum variant named `FlatStorageStatus::Resharding`. This approach has two benefits. First, the progress of any Flat State resharding is persisted to disk, which makes the operation resilient to a node crash or restart. Second, resuming resharding on node restart shares the same code path as Flat State creation (see `FlatStorageShardCreator`), reducing the code duplication factor. `FlatStorageStatus` is updated at every committable step of resharding. The commit points are the following: -- Beginning of resharding or, in other words, the last block of the old shard layout. -- Scheduling of the _"split parent shard"_ task. -- Execution, cancellation or failure of the _"split parent shard"_ task. -- Execution or failure of any _"child catchup"_ task. + +* Beginning of resharding or, in other words, the last block of the old shard layout. +* Scheduling of the _"split parent shard"_ task. +* Execution, cancellation or failure of the _"split parent shard"_ task. +* Execution or failure of any _"child catchup"_ task. #### Splitting a shard Flat State @@ -269,30 +272,35 @@ One current technical limitation is that, upon a node crash or restart, the _"sp A reference implementation of splitting a Flat State can be found in [FlatStorageResharder::split_shard_task](https://github.com/near/nearcore/blob/fecce019f0355cf89b63b066ca206a3cdbbdffff/chain/chain/src/flat_storage_resharder.rs#L295). #### Values assignment from parent to child shards + Key-value pairs in the parent shard Flat State are inherited by children according to the rules stated below. Elements inherited by the child shard which tracks the `account_id` contained in the key: -- `ACCOUNT` -- `CONTRACT_DATA` -- `CONTRACT_CODE` -- `ACCESS_KEY` -- `RECEIVED_DATA` -- `POSTPONED_RECEIPT_ID` -- `PENDING_DATA_COUNT` -- `POSTPONED_RECEIPT` -- `PROMISE_YIELD_RECEIPT` + +* `ACCOUNT` +* `CONTRACT_DATA` +* `CONTRACT_CODE` +* `ACCESS_KEY` +* `RECEIVED_DATA` +* `POSTPONED_RECEIPT_ID` +* `PENDING_DATA_COUNT` +* `POSTPONED_RECEIPT` +* `PROMISE_YIELD_RECEIPT` Elements inherited by both children: -- `DELAYED_RECEIPT_OR_INDICES` -- `PROMISE_YIELD_INDICES` -- `PROMISE_YIELD_TIMEOUT` -- `BANDWIDTH_SCHEDULER_STATE` + +* `DELAYED_RECEIPT_OR_INDICES` +* `PROMISE_YIELD_INDICES` +* `PROMISE_YIELD_TIMEOUT` +* `BANDWIDTH_SCHEDULER_STATE` Elements inherited only be the lowest index child: -- `BUFFERED_RECEIPT_INDICES ` -- `BUFFERED_RECEIPT` + +* `BUFFERED_RECEIPT_INDICES ` +* `BUFFERED_RECEIPT` #### Bring children shards up to date with the chain's head + Children shards' Flat States build a complete view of their content at the height of the `resharding block` sometime during the new epoch after resharding. At that point in time many new blocks have been processed already, and these will most likely contain updates for the new shards. A catch-up step is necessary to apply all Flat State deltas accumulated until now.