[Enhancement] release metadata memory after rowsets have been compacted in pk table (backport #46067) #46304
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why I'm doing:
After
Rowset
generated, SR will load metadata aboutRowset
,Segment
andColumnReader
, which will cause lots of memory, and it's good for query performance.And in DUP/AGG/UNIQ table, SR will release these memory after
Rowset
has been compacted, but in PK table, it won't.So now, PK table will only release metadata memory after
Rowset
has been deleted, even after thisRowset
is too old to be queried anymore. It will waste memory here.What I'm doing:
release metadata memory after rowsets have been compacted in PK table.
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist:
Bugfix cherry-pick branch check:
This is an automatic backport of pull request #46067 done by [Mergify](https://mergify.com). ## Why I'm doing: After `Rowset` generated, SR will load metadata about `Rowset`, `Segment` and `ColumnReader`, which will cause lots of memory, and it's good for query performance. And in DUP/AGG/UNIQ table, SR will release these memory after `Rowset` has been compacted, but in PK table, it won't.
So now, PK table will only release metadata memory after
Rowset
has been deleted, even after thisRowset
is too old to be queried anymore. It will waste memory here.What I'm doing:
release metadata memory after rowsets have been compacted in PK table.
What type of PR is this:
Does this PR entail a change in behavior?
If yes, please specify the type of change:
Checklist: