Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Builder Efficient reverts #90

Merged
merged 57 commits into from
Sep 27, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
1593828
First attempt
Wazzymandias Jul 13, 2023
39e25e5
Try granular snaps
Wazzymandias Jul 13, 2023
d323b21
Try appending to access list and reverting on bundle failure
Wazzymandias Jul 19, 2023
d51683a
Revert to beginning using extra access list when available
Wazzymandias Jul 19, 2023
2445edb
Add access list and revision after apply to avoid index out of range …
Wazzymandias Jul 19, 2023
de80c00
Try copy before revert and avoid initial environment copy
Wazzymandias Jul 19, 2023
95b97ea
Update revert logic
Wazzymandias Jul 19, 2023
4b77fbe
Add access list parity
Wazzymandias Jul 19, 2023
e369cf6
Initial commit for efficient revert experimentation
Wazzymandias Jul 19, 2023
c2a1a95
Add additional unit tests and fix unit test errors, reduce duplicate …
Wazzymandias Jul 26, 2023
4908ea8
Init BuilderRateLimitResubmitInterval builder config param from comma…
vazha Jul 19, 2023
dc5cc49
Update readme to include greedy-buckets for miner.algotype (#87)
Wazzymandias Jul 24, 2023
4a9e785
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Jul 26, 2023
5ddc19b
Update godoc, remove unused getter for access list, add CLI flag to b…
Wazzymandias Jul 26, 2023
6ccec93
Fix linter errors
Wazzymandias Jul 26, 2023
91b5283
Add retry logic for multi-tx-snapshot block build algorithm
Wazzymandias Jul 28, 2023
da04d6f
Update unit tests to test EnableMultiTxSnap
Wazzymandias Jul 28, 2023
b04fd4b
Rollback retry logic
Wazzymandias Jul 28, 2023
597534f
Fix linter error
Wazzymandias Jul 28, 2023
e1207eb
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Jul 28, 2023
f282589
Change account touch tracer to access list tracer for env changes
Wazzymandias Jul 31, 2023
bacd7e4
Update greedy builder to use passed in algorithm configuration rather…
Wazzymandias Jul 31, 2023
49ba79b
Rollback profit for env changes
Wazzymandias Jul 31, 2023
d8cf812
Fix bugs
Wazzymandias Jul 31, 2023
bf94410
Merge branch 'quickfix-cli' into efficient-reverts
Wazzymandias Aug 1, 2023
e227c37
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Aug 1, 2023
4317b0a
Add new multi-transaction snapshot stack to support more than one act…
Wazzymandias Aug 2, 2023
072890f
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Aug 3, 2023
0e3057b
Clean up code, add comprehensive stack tests with fuzzing, fix edge c…
Wazzymandias Aug 4, 2023
a16791d
Fix linter error
Wazzymandias Aug 4, 2023
d6d5cae
Add refund support to efficient revert so state returns to correct re…
Wazzymandias Aug 7, 2023
52ddc82
Revert some refactor changes
Wazzymandias Aug 7, 2023
f3911d2
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Aug 9, 2023
0d612c6
Add unit tests for state comparison, potential fix for gas and root m…
Wazzymandias Aug 9, 2023
fc32a84
Fix linter error
Wazzymandias Aug 9, 2023
bdc7422
Try using top level changes to avoid env mutations
Wazzymandias Aug 9, 2023
e526d0e
Debug commit
Wazzymandias Aug 9, 2023
da5c54e
DRY profit logic
Wazzymandias Aug 10, 2023
ab3aaaf
Revert commit
Wazzymandias Aug 15, 2023
b116abd
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Aug 15, 2023
4e3bc45
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Aug 16, 2023
1823163
small changes
Wazzymandias Aug 19, 2023
72f182f
Add fuzz state using state smart contract, add Copy method for multi-…
Wazzymandias Aug 21, 2023
0259e06
fix a bug with state reverts of accounts that are not touched accordi…
dvush Aug 21, 2023
3045b14
Lint
Wazzymandias Aug 21, 2023
c1ae873
Simplify test
Wazzymandias Aug 21, 2023
1062c82
Fix linter error
Wazzymandias Aug 21, 2023
c14c299
Use different builders instead of configuration switch since major re…
Wazzymandias Aug 23, 2023
ca7e7a7
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Aug 23, 2023
811b294
Update env changes to reduce redundancy and make control flow easier …
Wazzymandias Aug 23, 2023
49157e8
Fix linter error
Wazzymandias Aug 23, 2023
03dd29c
Remove debug validation
Wazzymandias Aug 23, 2023
f2c5320
Update comments, add touch change to state fuzz test smart contract
Wazzymandias Aug 25, 2023
3695a2b
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Sep 1, 2023
ef00a1f
Add fuzz tests for transient storage and account touch operations
Wazzymandias Sep 8, 2023
2d3c57a
Remove unused code
Wazzymandias Sep 27, 2023
5f07c8c
Merge remote-tracking branch 'origin/main' into efficient-reverts
Wazzymandias Sep 27, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
692 changes: 692 additions & 0 deletions core/state/multi_tx_snapshot.go

Large diffs are not rendered by default.

929 changes: 929 additions & 0 deletions core/state/multi_tx_snapshot_test.go

Large diffs are not rendered by default.

3 changes: 3 additions & 0 deletions core/state/state_object.go
Original file line number Diff line number Diff line change
Expand Up @@ -247,6 +247,9 @@ func (s *stateObject) setState(key, value common.Hash) {
func (s *stateObject) finalise(prefetch bool) {
slotsToPrefetch := make([][]byte, 0, len(s.dirtyStorage))
for key, value := range s.dirtyStorage {
prev, ok := s.pendingStorage[key]
s.db.multiTxSnapshotStack.UpdatePendingStorage(s.address, key, prev, ok)

s.pendingStorage[key] = value
if value != s.originStorage[key] {
slotsToPrefetch = append(slotsToPrefetch, common.CopyBytes(key[:])) // Copy needed for closure
Expand Down
40 changes: 40 additions & 0 deletions core/state/statedb.go
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,9 @@ type StateDB struct {
validRevisions []revision
nextRevisionId int

// Multi-Transaction Snapshot Stack
multiTxSnapshotStack *MultiTxSnapshotStack

// Measurements gathered during execution for debugging purposes
AccountReads time.Duration
AccountHashes time.Duration
Expand Down Expand Up @@ -151,6 +154,8 @@ func New(root common.Hash, db Database, snaps *snapshot.Tree) (*StateDB, error)
transientStorage: newTransientStorage(),
hasher: crypto.NewKeccakState(),
}

sdb.multiTxSnapshotStack = NewMultiTxSnapshotStack(sdb)
if sdb.snaps != nil {
if sdb.snap = sdb.snaps.Snapshot(root); sdb.snap != nil {
sdb.snapAccounts = make(map[common.Hash][]byte)
Expand Down Expand Up @@ -712,6 +717,8 @@ func (s *StateDB) Copy() *StateDB {
journal: newJournal(),
hasher: crypto.NewKeccakState(),
}
// Initialize copy of multi-transaction snapshot stack for the copied state
state.multiTxSnapshotStack = s.multiTxSnapshotStack.Copy(state)
// Copy the dirty states, logs, and preimages
for addr := range s.journal.dirties {
// As documented [here](https://github.com/ethereum/go-ethereum/pull/16485#issuecomment-380438527),
Expand Down Expand Up @@ -842,6 +849,8 @@ func (s *StateDB) GetRefund() uint64 {
// the journal as well as the refunds. Finalise, however, will not push any updates
// into the tries just yet. Only IntermediateRoot or Commit will do that.
func (s *StateDB) Finalise(deleteEmptyObjects bool) {
s.multiTxSnapshotStack.UpdateFromJournal(s.journal)

addressesToPrefetch := make([][]byte, 0, len(s.journal.dirties))
for addr := range s.journal.dirties {
obj, exist := s.stateObjects[addr]
Expand All @@ -855,6 +864,8 @@ func (s *StateDB) Finalise(deleteEmptyObjects bool) {
continue
}
if obj.suicided || (deleteEmptyObjects && obj.empty()) {
s.multiTxSnapshotStack.UpdateObjectDeleted(obj.address, obj.deleted)

obj.deleted = true

// We need to maintain account deletions explicitly (will remain
Expand All @@ -872,6 +883,12 @@ func (s *StateDB) Finalise(deleteEmptyObjects bool) {
} else {
obj.finalise(true) // Prefetch slots in the background
}

if s.multiTxSnapshotStack.Size() > 0 {
_, wasPending := s.stateObjectsPending[addr]
_, wasDirty := s.stateObjectsDirty[addr]
s.multiTxSnapshotStack.UpdatePendingStatus(addr, wasPending, wasDirty)
}
s.stateObjectsPending[addr] = struct{}{}
s.stateObjectsDirty[addr] = struct{}{}

Expand All @@ -894,6 +911,10 @@ func (s *StateDB) IntermediateRoot(deleteEmptyObjects bool) common.Hash {
// Finalise all the dirty storage states and write them into the tries
s.Finalise(deleteEmptyObjects)

// Intermediate root writes updates to the trie, which will cause
// in memory multi-transaction snapshot to be incompatible with the committed state, so we invalidate.
s.multiTxSnapshotStack.Invalidate()

// If there was a trie prefetcher operating, it gets aborted and irrevocably
// modified after we start retrieving tries. Remove it from the statedb after
// this round of use.
Expand Down Expand Up @@ -1181,3 +1202,22 @@ func (s *StateDB) convertAccountSet(set map[common.Address]struct{}) map[common.
}
return ret
}

func (s *StateDB) NewMultiTxSnapshot() (err error) {
_, err = s.multiTxSnapshotStack.NewSnapshot()
return
}

func (s *StateDB) MultiTxSnapshotRevert() (err error) {
_, err = s.multiTxSnapshotStack.Revert()
return
}

func (s *StateDB) MultiTxSnapshotCommit() (err error) {
_, err = s.multiTxSnapshotStack.Commit()
return
}

func (s *StateDB) MultiTxSnapshotStackSize() int {
return s.multiTxSnapshotStack.Size()
}
Loading
Loading