Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: ledger checkpoint, rewinding, and height action #104

Merged
merged 32 commits into from
Apr 9, 2024

Conversation

Meshiest
Copy link
Contributor

@Meshiest Meshiest commented Apr 5, 2024

automatically create checkpoints given retention policies in a format similar to duplicati's

this mechanism works by providing Rules formatted like 1W:1D, aka for a 1 week time period, keep backups 1 day apart.

The following is the default policy

"4h:1h", // for 4 hours, keep a checkpoint every hour
"1D:8h", // for 1 day, keep a checkpoint every 8 hours
"1W:1D", // for 1 week, keep a checkpoint every day
"4W:1W", // for 4 weeks, keep a checkpoint every week
"4M:1M", // for 4 months, keep a checkpoint every month
"U:1Y",  // for all time, keep a checkpoint every year

This policy will be applied to the replay command when enabled, and the running node.

Adds aot ledger checkpoint commands:

Usage: snarkos-aot ledger --genesis <GENESIS> --ledger <LEDGER> checkpoint <COMMAND>

Commands:
  create  Create a checkpoint for the given ledger
  apply   Apply a checkpoint to the given ledger
  view    View the available checkpoints
  clean   Cleanup old checkpoints

Changes truncate --replay into aot ledger replay with an optional --checkpoint flag to generate checkpoints, and aot ledger rewind <checkpoint> to rewind a ledger given a checkpoint file

crates/aot/src/checkpoint/retention.rs Outdated Show resolved Hide resolved
@Meshiest Meshiest changed the title WIP feat(checkpoints): retention policy feat(checkpoints): retention policy Apr 6, 2024
@Meshiest Meshiest marked this pull request as ready for review April 6, 2024 02:08
crates/aot/src/checkpoint/retention.rs Outdated Show resolved Hide resolved
crates/aot/src/checkpoint/retention.rs Outdated Show resolved Hide resolved
@Meshiest Meshiest changed the title feat(checkpoints): retention policy WIP feat(checkpoints): retention policy Apr 6, 2024
@Meshiest Meshiest changed the title WIP feat(checkpoints): retention policy WIP feat: ledger checkpoint, rewinding, and height action Apr 7, 2024
@Meshiest Meshiest changed the title WIP feat: ledger checkpoint, rewinding, and height action feat: ledger checkpoint, rewinding, and height action Apr 8, 2024
@Meshiest Meshiest requested a review from voximity April 8, 2024 04:55
@Meshiest Meshiest requested a review from gluax April 8, 2024 04:55
#[cfg(feature = "write")]
pub enum CheckpointContentError {
#[error("error opening storage: {0}")]
OpenLedger(#[source] anyhow::Error),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💀

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to be fair the write feature is only available when running snarkos nodes which uses anyhow anyway

crates/checkpoint/src/header.rs Show resolved Hide resolved
crates/checkpoint/src/retention.rs Outdated Show resolved Hide resolved
// step 3 - if the last time is outside the duration of the current rule, reject it
match curr_rule.duration.as_delta() {
Some(duration) if last_delta > duration => {
/* println!(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need these printlns?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

they'll probably be useful for debugging in a few days

@voximity voximity merged commit 43017ff into main Apr 9, 2024
2 checks passed
@Meshiest Meshiest deleted the feat-retention-policy branch April 9, 2024 11:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants