Skip to content

Commit

Permalink
## Problem
Browse files Browse the repository at this point in the history
The current implementation of compaction task submission reserves permits before task execution, which can lead to inefficient resource utilization. Tasks waiting in the thread pool queue may hold permits, potentially blocking other tasks from being executed.

## Solution

Modify the `_submit_compaction_task` method to request permits only when the compaction task is actually about to be executed, rather than at the time of submission to the thread pool.

## Changes

- Moved permit request from task submission to task execution stage
- Ensures permits are only held during actual task processing
- Maintains the original blocking behavior for permit acquisition
- Improves overall resource utilization across different compaction thread pools

## Benefits

- More efficient use of system resources
- Prevents tasks from blocking permits while waiting in the queue
- Preserves the existing task scheduling logic
- No additional complexity in error handling
  • Loading branch information
Yukang-Lian committed Dec 9, 2024
1 parent aec14e6 commit 159599a
Showing 1 changed file with 3 additions and 3 deletions.
6 changes: 3 additions & 3 deletions be/src/olap/olap_server.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1059,9 +1059,6 @@ Status StorageEngine::_submit_compaction_task(TabletSharedPtr tablet,
Status st = Tablet::prepare_compaction_and_calculate_permits(compaction_type, tablet,
compaction, permits);
if (st.ok() && permits > 0) {
if (!force) {
_permit_limiter.request(permits);
}
std::unique_ptr<ThreadPool>& thread_pool =
(compaction_type == CompactionType::CUMULATIVE_COMPACTION)
? _cumu_compaction_thread_pool
Expand All @@ -1074,6 +1071,9 @@ Status StorageEngine::_submit_compaction_task(TabletSharedPtr tablet,
<< tablet->tablet_id() << "tablet_state=" << tablet->tablet_state();
return;
}
if (!force) {
_permit_limiter.request(permits);
}
tablet->compaction_stage = CompactionStage::EXECUTING;
TEST_SYNC_POINT_RETURN_WITH_VOID("olap_server::execute_compaction");
tablet->execute_compaction(*compaction);
Expand Down

0 comments on commit 159599a

Please sign in to comment.