You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Segment refs are kept in the ra_log_reader in a list. To find the first index in the log the entire list needs to be scanned. For some uses, such as quorum queues, messages that are consumed are always near the beginning of the list of segments thus it makes sense to try to optimise this part.
Describe the solution you'd like
Use a skiplist. Care needs to be taken such that we don't accidently read overwritten entries that have the same index but a lower term.
Skiplists in erlang are very memory efficient due to structural sharing of lists and are relatively simple to implement.
a new module: ra_log_segrefs should be introduced to hold the logic for the list of segment references.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Segment refs are kept in the
ra_log_reader
in a list. To find the first index in the log the entire list needs to be scanned. For some uses, such as quorum queues, messages that are consumed are always near the beginning of the list of segments thus it makes sense to try to optimise this part.Describe the solution you'd like
Use a skiplist. Care needs to be taken such that we don't accidently read overwritten entries that have the same index but a lower term.
Skiplists in erlang are very memory efficient due to structural sharing of lists and are relatively simple to implement.
a new module:
ra_log_segrefs
should be introduced to hold the logic for the list of segment references.The text was updated successfully, but these errors were encountered: