Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Compatibility] Add hash expiration - HEXPIRE, HPEXPIRE, HEXPIREAT, HPEXPIREAT, HTTL, HPTTL, HEXPIRETIME, HPEXPIRETIME, HPERSIST and HCOLLECT #864

Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
4c955b3
WIP of HEXPIRE
Vijay-Nirmal Dec 9, 2024
e7695e2
WIP of EXPIRE
Vijay-Nirmal Dec 9, 2024
06e2d4c
Initial impelementation of HEXIRE and family
Vijay-Nirmal Dec 9, 2024
722a134
Merged latest
Vijay-Nirmal Dec 9, 2024
35e385f
Added more test cases
Vijay-Nirmal Dec 11, 2024
c27de0d
Handling serialization and Deserialization
Vijay-Nirmal Dec 11, 2024
a650ba9
Added LTM test cases
Vijay-Nirmal Dec 11, 2024
052fc06
Merged latest
Vijay-Nirmal Dec 11, 2024
968e5a3
Added docs
Vijay-Nirmal Dec 12, 2024
4a7a711
Added HCOLLECT and ACL, Slot verification tests
Vijay-Nirmal Dec 12, 2024
f31633c
Add a background task to call HCOLLECT to delete expired items
Vijay-Nirmal Dec 12, 2024
006a202
Code format fix
Vijay-Nirmal Dec 12, 2024
a1c5641
Merge branch 'main' into new/HEXPIRE-command
Vijay-Nirmal Dec 12, 2024
f8628f1
Fixed creation of empty object
Vijay-Nirmal Dec 12, 2024
85527f2
Finial commit before review comment fixes, hopefully
Vijay-Nirmal Dec 12, 2024
14fa5ba
This time for sure, this is the last commit before review comments
Vijay-Nirmal Dec 12, 2024
f0ef454
Fixed code format issue
Vijay-Nirmal Dec 12, 2024
7d7965b
Trying to fix test failure in pipeline (not happening in local)
Vijay-Nirmal Dec 12, 2024
29bfc24
Merge branch 'main' into new/HEXPIRE-command
Vijay-Nirmal Dec 13, 2024
fa20261
Merged latest
Vijay-Nirmal Dec 13, 2024
0b96546
Removed AsEnumerable to remove the allocation
Vijay-Nirmal Dec 13, 2024
23f5ea2
Fixed missed TODO
Vijay-Nirmal Dec 13, 2024
84973ff
Merge branch 'main' into new/HEXPIRE-command
Vijay-Nirmal Dec 14, 2024
7e71c9b
Review comment fixes
Vijay-Nirmal Dec 14, 2024
6d01ad8
Merge branch 'new/HEXPIRE-command' of https://github.com/Vijay-Nirmal…
Vijay-Nirmal Dec 14, 2024
764a7b4
Review comment fixes
Vijay-Nirmal Dec 14, 2024
dc69d98
Test fix, maybe
Vijay-Nirmal Dec 14, 2024
1cd8c28
Merged latest
Vijay-Nirmal Dec 17, 2024
444fdf6
Review command fix
Vijay-Nirmal Dec 17, 2024
e99e19d
Fixed test failure
Vijay-Nirmal Dec 17, 2024
569bc45
Merged latest
Vijay-Nirmal Dec 18, 2024
8576434
Merge branch 'main' into new/HEXPIRE-command
Vijay-Nirmal Dec 19, 2024
2f07f2d
Merge branch 'main' into new/HEXPIRE-command
Vijay-Nirmal Dec 19, 2024
6c08adb
Test fix
Vijay-Nirmal Dec 19, 2024
b1c4bbd
Test fix
Vijay-Nirmal Dec 19, 2024
b900b5d
Merge branch 'main' into new/HEXPIRE-command
Vijay-Nirmal Dec 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions libs/host/Configuration/Options.cs
Original file line number Diff line number Diff line change
Expand Up @@ -233,6 +233,9 @@ internal sealed class Options
[IntRangeValidation(0, int.MaxValue)]
[Option("compaction-freq", Required = false, HelpText = "Background hybrid log compaction frequency in seconds. 0 = disabled (compaction performed before checkpointing instead)")]
public int CompactionFrequencySecs { get; set; }
[IntRangeValidation(0, int.MaxValue)]
[Option("hcollect-freq", Required = false, HelpText = "Frequency in seconds for the background task to perform Hash collection. 0 = disabled. Hash collect is used to delete expired fields from hash without waiting for a write operation.")]
badrishc marked this conversation as resolved.
Show resolved Hide resolved
public int HashCollectFrequencySecs { get; set; }

[Option("compaction-type", Required = false, HelpText = "Hybrid log compaction type. Value options: None - no compaction, Shift - shift begin address without compaction (data loss), Scan - scan old pages and move live records to tail (no data loss), Lookup - lookup each record in compaction range, for record liveness checking using hash chain (no data loss)")]
public LogCompactionType CompactionType { get; set; }
Expand Down Expand Up @@ -652,6 +655,7 @@ public GarnetServerOptions GetServerOptions(ILogger logger = null)
WaitForCommit = WaitForCommit.GetValueOrDefault(),
AofSizeLimit = AofSizeLimit,
CompactionFrequencySecs = CompactionFrequencySecs,
HashCollectFrequencySecs = HashCollectFrequencySecs,
CompactionType = CompactionType,
CompactionForceDelete = CompactionForceDelete.GetValueOrDefault(),
CompactionMaxSegments = CompactionMaxSegments,
Expand Down
3 changes: 3 additions & 0 deletions libs/host/defaults.conf
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,9 @@
/* Background hybrid log compaction frequency in seconds. 0 = disabled (compaction performed before checkpointing instead) */
"CompactionFrequencySecs" : 0,

/* Frequency in seconds for the background task to perform Hash collection. 0 = disabled. Hash collect is used to delete expired fields from hash without waiting for a write operation. */
"HashCollectFrequencySecs" : 0,

/* Hybrid log compaction type. Value options: */
/* None - no compaction */
/* Shift - shift begin address without compaction (data loss) */
Expand Down
Loading
Loading