Skip to content

Commit

Permalink
Merge directly without vector in Filter class (#1633)
Browse files Browse the repository at this point in the history
This commit affects the case when the input to a FILTER is lazy, but the result is fully materialized. Now the local vocabs of the lazy input blocks are directly merged into the local vocab of the result. Previously they were first stored in a vector, which had a higher RAM footprint for the case of many redundant and typically empty local vocabs in the input.
  • Loading branch information
RobinTF authored Nov 25, 2024
1 parent 5f28e83 commit afd35ec
Showing 1 changed file with 4 additions and 6 deletions.
10 changes: 4 additions & 6 deletions src/engine/Filter.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -71,19 +71,17 @@ ProtoResult Filter::computeResult(bool requestLaziness) {
// single IdTable.
size_t width = getSubtree().get()->getResultWidth();
IdTable result{width, getExecutionContext()->getAllocator()};
std::vector<LocalVocab> localVocabs;

LocalVocab resultLocalVocab{};
ad_utility::callFixedSize(
width, [this, &subRes, &result, &localVocabs]<int WIDTH>() {
width, [this, &subRes, &result, &resultLocalVocab]<int WIDTH>() {
for (Result::IdTableVocabPair& pair : subRes->idTables()) {
computeFilterImpl<WIDTH>(result, pair.idTable_, pair.localVocab_,
subRes->sortedBy());
localVocabs.emplace_back(std::move(pair.localVocab_));
resultLocalVocab.mergeWith(std::span{&pair.localVocab_, 1});
}
});

LocalVocab resultLocalVocab{};
resultLocalVocab.mergeWith(localVocabs);

LOG(DEBUG) << "Filter result computation done." << endl;

return {std::move(result), resultSortedOn(), std::move(resultLocalVocab)};
Expand Down

0 comments on commit afd35ec

Please sign in to comment.