Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

publish: build checkpoint files during ledger close #4446

Open
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

marta-lokhova
Copy link
Contributor

@marta-lokhova marta-lokhova commented Aug 29, 2024

Resolves #4212

The change removes publish dependency on txhistory and ledgerheaders SQL tables, and instead constructs ledger headers/tx sets/tx results checkpoints at ledger close. This allows us to deprecate and drop support for txhistory entirely, which is beneficial for performance. Additionally, the code became simpler, and the xdr->sql->xdr roundtrip processing has been removed (as part of this, txsethistory table was dropped as well). The tradeoff in this approach is that we lose atomicity we had with SQL transactions. As a result, core now manually has to handle crashes. This is done by always constructing "tmp" checkpoints first, and only promoting them after a checkpoint ledger has been committed to the database. In the event that tmp file has been written, but core crashed in apply and rolled back the transaction, there's automatic file cleanup on startup, that trims any uncommitted ledgers from the temp file.

@marta-lokhova
Copy link
Contributor Author

Note: the current version of this PR drops txhistory table for simplicity, however depending on who depends on it, we might need to still support it, while also building checkpoint files directly.

@marta-lokhova marta-lokhova changed the title Build checkpoint files during ledger close publish: build checkpoint files during ledger close Sep 5, 2024
@marta-lokhova marta-lokhova force-pushed the publishSQlPath branch 3 times, most recently from 6151b9f to 560d267 Compare September 12, 2024 01:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

consider moving as much of the checkpoint building logic inline with closing ledgers
1 participant