Skip to content

Commit

Permalink
fix: compaction support pick by max_seq (#1041)
Browse files Browse the repository at this point in the history
## Rationale
Part of #987.

Current implementation will compact by file size, max_seq is not
considered, this may cause data corruption
in corner case, eg:
- sst1, max_seq:10, PK1=10
- sst2, max_seq:11, PK1=9
- sst3, max_seq:12, no PK1

If compact pick sst1 and sst3, and output sst4, its max_seq will be 12,
now PK1 exists in two files:
- sst2, max_seq:11, PK1=9
- sst4, max_seq:12, PK1=10

That's to say, PK1's value is 10 now, which is wrong value(9 is right).

## Detailed Changes
When do compaction, first sort sst by max_seq desc, then only pick
adjacent ssts, the original issue is fixed in this way.
At the same time picked ssts are ensured to meet other requirements such
as `min_threshold`, `max_threshold`, `max_input_size`.

## Test Plan
UT and manually.
  • Loading branch information
jiacai2050 authored Jul 5, 2023
1 parent 4a7c6e4 commit ab1b509
Showing 1 changed file with 259 additions and 71 deletions.
Loading

0 comments on commit ab1b509

Please sign in to comment.