Skip to content

Commit

Permalink
[improvement](compaction) reduce tablet skip compaction time (#44273)
Browse files Browse the repository at this point in the history
The time for tablet skip compaction is 120 seconds, which is too long.
In the scenario of high-frequency import (mow), it leads to a high
compaction score. Therefore, reducing the skip time to 10 seconds is
necessary.
  • Loading branch information
luwei16 committed Nov 29, 2024
1 parent 9893fd7 commit 10c9ecc
Show file tree
Hide file tree
Showing 3 changed files with 6 additions and 3 deletions.
4 changes: 3 additions & 1 deletion be/src/common/config.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -413,6 +413,8 @@ DEFINE_mDouble(base_compaction_min_data_ratio, "0.3");
DEFINE_mInt64(base_compaction_dup_key_max_file_size_mbytes, "1024");

DEFINE_Bool(enable_skip_tablet_compaction, "true");
DEFINE_mInt32(skip_tablet_compaction_second, "10");

// output rowset of cumulative compaction total disk size exceed this config size,
// this rowset will be given to base compaction, unit is m byte.
DEFINE_mInt64(compaction_promotion_size_mbytes, "1024");
Expand Down Expand Up @@ -448,7 +450,7 @@ DEFINE_mInt32(multi_get_max_threads, "10");
DEFINE_mInt64(total_permits_for_compaction_score, "10000");

// sleep interval in ms after generated compaction tasks
DEFINE_mInt32(generate_compaction_tasks_interval_ms, "10");
DEFINE_mInt32(generate_compaction_tasks_interval_ms, "100");

// sleep interval in second after update replica infos
DEFINE_mInt32(update_replica_infos_interval_seconds, "60");
Expand Down
1 change: 1 addition & 0 deletions be/src/common/config.h
Original file line number Diff line number Diff line change
Expand Up @@ -465,6 +465,7 @@ DECLARE_mDouble(base_compaction_min_data_ratio);
DECLARE_mInt64(base_compaction_dup_key_max_file_size_mbytes);

DECLARE_Bool(enable_skip_tablet_compaction);
DECLARE_mInt32(skip_tablet_compaction_second);
// output rowset of cumulative compaction total disk size exceed this config size,
// this rowset will be given to base compaction, unit is m byte.
DECLARE_mInt64(compaction_promotion_size_mbytes);
Expand Down
4 changes: 2 additions & 2 deletions be/src/olap/tablet.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -2503,10 +2503,10 @@ void Tablet::set_skip_compaction(bool skip, CompactionType compaction_type, int6

bool Tablet::should_skip_compaction(CompactionType compaction_type, int64_t now) {
if (compaction_type == CompactionType::CUMULATIVE_COMPACTION && _skip_cumu_compaction &&
now < _skip_cumu_compaction_ts + 120) {
now < _skip_cumu_compaction_ts + config::skip_tablet_compaction_second) {
return true;
} else if (compaction_type == CompactionType::BASE_COMPACTION && _skip_base_compaction &&
now < _skip_base_compaction_ts + 120) {
now < _skip_base_compaction_ts + config::skip_tablet_compaction_second) {
return true;
}
return false;
Expand Down

0 comments on commit 10c9ecc

Please sign in to comment.