-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat(stf/branch): simplify merged iterator #22131
base: main
Are you sure you want to change the base?
Conversation
Warning Rate limit exceeded@testinginprod has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 16 minutes and 10 seconds before requesting another review. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. 📝 Walkthrough📝 Walkthrough📝 WalkthroughWalkthroughThe changes introduce a generic Changes
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very nice refactoring! The code looks cleaner and easier to read.
I have added some questions to better understand the motivation for generics and the different behaviour on invalid iterator
// The cache iterator may contain items that shadow or override items in the parent iterator. | ||
// If the cache iterator has the same key as the parent, the cache's value takes precedence. | ||
// Deleted items in the cache (indicated by nil values) are skipped. | ||
type mergedIterator[Parent, Cache corestore.Iterator] struct { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do you use generics here? mergedIterator
is a package private type and not likely to be extended. It should be fine to stick with the corestore.Iterator
type.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in theory, by fact i am not sure, they should provide a performance benefit since it does not do dynamic dispatch on every call
return true | ||
// Skip deleted items (value is nil) | ||
if value == nil { | ||
continue |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
personal preference: a recursion could be used instead of the for loop. But this may be easier to read
tested with #22141 |
@testinginprod your pull request is missing a changelog! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 1
📜 Review details
Configuration used: .coderabbit.yml
Review profile: CHILL
📒 Files selected for processing (1)
- server/v2/stf/branch/mergeiter.go (1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
server/v2/stf/branch/mergeiter.go (1)
Pattern
**/*.go
: Review the Golang code for conformity with the Uber Golang style guide, highlighting any deviations.
🔇 Additional comments (1)
server/v2/stf/branch/mergeiter.go (1)
68-71
: Verify the expected behavior ofKey()
andValue()
when the iterator is invalidThe
Key()
andValue()
methods returnnil
when the iterator is invalid (!i.valid
). According to thecorestore.Iterator
interface, it's important to confirm whether returningnil
is appropriate in this scenario or if these methods should panic or handle errors differently. This ensures consistency and prevents unexpected behavior for users of the iterator.Please verify the interface's requirements:
Also applies to: 76-80
// advance moves the iterator to the next valid (non-deleted) item. | ||
// It handles merging logic between the parent and cache iterators. | ||
func (i *mergedIterator[Parent, Cache]) advance() { | ||
for { | ||
// If parent is invalid, fast-forward cache. | ||
if !iter.parent.Valid() { | ||
iter.skipCacheDeletes(nil) | ||
return iter.cache.Valid() | ||
// Check if both iterators have reached the end | ||
if !i.parent.Valid() && !i.cache.Valid() { | ||
i.valid = false | ||
return | ||
} | ||
// Parent is valid. | ||
|
||
if !iter.cache.Valid() { | ||
return true | ||
var key, value []byte | ||
|
||
// If parent iterator is exhausted, use the cache iterator | ||
if !i.parent.Valid() { | ||
key = i.cache.Key() | ||
value = i.cache.Value() | ||
i.cache.Next() | ||
} else if !i.cache.Valid() { | ||
// If cache iterator is exhausted, use the parent iterator | ||
key = i.parent.Key() | ||
value = i.parent.Value() | ||
i.parent.Next() | ||
} else { | ||
// Both iterators are valid; compare keys | ||
keyP, keyC := i.parent.Key(), i.cache.Key() | ||
switch cmp := i.compare(keyP, keyC); { | ||
case cmp < 0: | ||
// Parent key is less than cache key | ||
key = keyP | ||
value = i.parent.Value() | ||
i.parent.Next() | ||
case cmp == 0: | ||
// Keys are equal; cache overrides parent | ||
key = keyC | ||
value = i.cache.Value() | ||
i.parent.Next() | ||
i.cache.Next() | ||
case cmp > 0: | ||
// Cache key is less than parent key | ||
key = keyC | ||
value = i.cache.Value() | ||
i.cache.Next() | ||
} | ||
} | ||
// Parent is valid, cache is valid. | ||
|
||
// Compare parent and cache. | ||
keyP := iter.parent.Key() | ||
keyC := iter.cache.Key() | ||
|
||
switch iter.compare(keyP, keyC) { | ||
case -1: // parent < cache. | ||
return true | ||
// Skip deleted items (value is nil) | ||
if value == nil { | ||
continue | ||
} | ||
|
||
case 0: // parent == cache. | ||
// Skip over if cache item is a delete. | ||
valueC := iter.cache.Value() | ||
if valueC == nil { | ||
iter.parent.Next() | ||
iter.cache.Next() | ||
// Update the current key and value, and mark iterator as valid | ||
i.currKey = key | ||
i.currValue = value | ||
i.valid = true | ||
return | ||
} | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reset i.err
in the advance
method to avoid stale errors
In the advance()
method, when advancing the iterator, i.err
is not reset. This could result in the Error()
method returning an outdated error even after the iterator has recovered from a previous invalid state. To ensure that i.err
accurately reflects the current state of the iterator, consider resetting it at the beginning of the advance()
method.
Apply this diff to reset i.err
appropriately:
func (i *mergedIterator[Parent, Cache]) advance() {
+ i.err = nil
for {
if !i.parent.Valid() && !i.cache.Valid() {
i.valid = false
return
}
// Rest of the code...
}
}
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
// advance moves the iterator to the next valid (non-deleted) item. | |
// It handles merging logic between the parent and cache iterators. | |
func (i *mergedIterator[Parent, Cache]) advance() { | |
for { | |
// If parent is invalid, fast-forward cache. | |
if !iter.parent.Valid() { | |
iter.skipCacheDeletes(nil) | |
return iter.cache.Valid() | |
// Check if both iterators have reached the end | |
if !i.parent.Valid() && !i.cache.Valid() { | |
i.valid = false | |
return | |
} | |
// Parent is valid. | |
if !iter.cache.Valid() { | |
return true | |
var key, value []byte | |
// If parent iterator is exhausted, use the cache iterator | |
if !i.parent.Valid() { | |
key = i.cache.Key() | |
value = i.cache.Value() | |
i.cache.Next() | |
} else if !i.cache.Valid() { | |
// If cache iterator is exhausted, use the parent iterator | |
key = i.parent.Key() | |
value = i.parent.Value() | |
i.parent.Next() | |
} else { | |
// Both iterators are valid; compare keys | |
keyP, keyC := i.parent.Key(), i.cache.Key() | |
switch cmp := i.compare(keyP, keyC); { | |
case cmp < 0: | |
// Parent key is less than cache key | |
key = keyP | |
value = i.parent.Value() | |
i.parent.Next() | |
case cmp == 0: | |
// Keys are equal; cache overrides parent | |
key = keyC | |
value = i.cache.Value() | |
i.parent.Next() | |
i.cache.Next() | |
case cmp > 0: | |
// Cache key is less than parent key | |
key = keyC | |
value = i.cache.Value() | |
i.cache.Next() | |
} | |
} | |
// Parent is valid, cache is valid. | |
// Compare parent and cache. | |
keyP := iter.parent.Key() | |
keyC := iter.cache.Key() | |
switch iter.compare(keyP, keyC) { | |
case -1: // parent < cache. | |
return true | |
// Skip deleted items (value is nil) | |
if value == nil { | |
continue | |
} | |
case 0: // parent == cache. | |
// Skip over if cache item is a delete. | |
valueC := iter.cache.Value() | |
if valueC == nil { | |
iter.parent.Next() | |
iter.cache.Next() | |
// Update the current key and value, and mark iterator as valid | |
i.currKey = key | |
i.currValue = value | |
i.valid = true | |
return | |
} | |
} | |
// advance moves the iterator to the next valid (non-deleted) item. | |
// It handles merging logic between the parent and cache iterators. | |
func (i *mergedIterator[Parent, Cache]) advance() { | |
i.err = nil | |
for { | |
// Check if both iterators have reached the end | |
if !i.parent.Valid() && !i.cache.Valid() { | |
i.valid = false | |
return | |
} | |
var key, value []byte | |
// If parent iterator is exhausted, use the cache iterator | |
if !i.parent.Valid() { | |
key = i.cache.Key() | |
value = i.cache.Value() | |
i.cache.Next() | |
} else if !i.cache.Valid() { | |
// If cache iterator is exhausted, use the parent iterator | |
key = i.parent.Key() | |
value = i.parent.Value() | |
i.parent.Next() | |
} else { | |
// Both iterators are valid; compare keys | |
keyP, keyC := i.parent.Key(), i.cache.Key() | |
switch cmp := i.compare(keyP, keyC); { | |
case cmp < 0: | |
// Parent key is less than cache key | |
key = keyP | |
value = i.parent.Value() | |
i.parent.Next() | |
case cmp == 0: | |
// Keys are equal; cache overrides parent | |
key = keyC | |
value = i.cache.Value() | |
i.parent.Next() | |
i.cache.Next() | |
case cmp > 0: | |
// Cache key is less than parent key | |
key = keyC | |
value = i.cache.Value() | |
i.cache.Next() | |
} | |
} | |
// Skip deleted items (value is nil) | |
if value == nil { | |
continue | |
} | |
// Update the current key and value, and mark iterator as valid | |
i.currKey = key | |
i.currValue = value | |
i.valid = true | |
return | |
} | |
} |
// If the iterator is invalid, it returns nil. | ||
func (i *mergedIterator[Parent, Cache]) Key() []byte { | ||
if !i.valid { | ||
panic("called key on invalid iterator") |
Check warning
Code scanning / CodeQL
Panic in BeginBock or EndBlock consensus methods Warning
// If the iterator is invalid, it returns nil. | ||
func (i *mergedIterator[Parent, Cache]) Value() []byte { | ||
if !i.valid { | ||
panic("called value on invalid iterator") |
Check warning
Code scanning / CodeQL
Panic in BeginBock or EndBlock consensus methods Warning
Description
Closes: #XXXX
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!
in the type prefix if API or client breaking changeCHANGELOG.md
Reviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
Please see Pull Request Reviewer section in the contributing guide for more information on how to review a pull request.
I have...
Summary by CodeRabbit
New Features
Improvements