-
Notifications
You must be signed in to change notification settings - Fork 595
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix(memtables): track memtables with a weakset to allow overwriting tables with the same name but different data #10133
Conversation
0ede913
to
7c69b36
Compare
I know that previously we decided against using |
7c69b36
to
c6c4465
Compare
That said, I think we'll want to adjust this test to use |
Clouds passing:
|
3b1fe38
to
aac07cf
Compare
…ables with the same name but different data
aac07cf
to
3b38e15
Compare
# mapping of memtable names to their finalizers | ||
self._finalizers = {} | ||
self._memtables = weakref.WeakSet() | ||
self._current_memtables = weakref.WeakValueDictionary() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can reduce this state to just a single weakset and dict, mind if I push up a commit?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Go for it!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jcrist I am still open to you pushing up a commit if you've already got it locally, otherwise, no big deal.
Description of changes
Fixes an issue with memtable name reuse where we were never overwriting
memtable data in the backend, even if the data changed and the name remained
the same.
The solution I came up with here was to add a
weakref.WeakSet
to trackmemtable ops, which are hashed based on the data-containing object.
The reason a
WeakSet
is necessary is to preserve the drop-on-unreachablesemantics of memtable.
Issues closed
Closes #10131.