Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(continuations): Have bounded number of segment data while proving #403

Open
Nashtare opened this issue Jul 16, 2024 · 2 comments · May be fixed by #793
Open

perf(continuations): Have bounded number of segment data while proving #403

Nashtare opened this issue Jul 16, 2024 · 2 comments · May be fixed by #793
Assignees
Labels
crate: evm_arithmetization Anything related to the evm_arithmetization crate. performance Performance improvement related changes

Comments

@Nashtare
Copy link
Collaborator

Proving with continuations requires to first generate segment data for all segment payloads within a block. This has a negative effect on memory as these segments are persisted until consumed, which happens at a much slower rate.

We should consider having some buffered approach to not hold in memory more than N segment data ahead (where N most likely would be the runtime number of workers to make sure that they can all pick up a next job if they happened to finish their current one at the same time).

@Nashtare Nashtare added performance Performance improvement related changes crate: evm_arithmetization Anything related to the evm_arithmetization crate. labels Jul 16, 2024
@Nashtare Nashtare added this to the Performance Tuning milestone Jul 16, 2024
@github-project-automation github-project-automation bot moved this to Backlog in Zero EVM Jul 16, 2024
@atanmarko atanmarko self-assigned this Jul 22, 2024
@atanmarko
Copy link
Member

Related, may be solved by #627

@atanmarko atanmarko linked a pull request Nov 12, 2024 that will close this issue
@atanmarko atanmarko moved this from Backlog to Ready to Review in Zero EVM Nov 13, 2024
@atanmarko
Copy link
Member

atanmarko commented Nov 13, 2024

In the PR #793, segment generation is performed separately from the proving, and new segments are communicated to the proving task with the channel. It is easy to limit their number by adding new semaphore to task generation of the number of segments in the batch that will be proven in parallel. I have decided not to do that (not to introduce new potential choking point), as we have already implemented block_pool_size limitation that manages the number of blocks proven in parallel.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
crate: evm_arithmetization Anything related to the evm_arithmetization crate. performance Performance improvement related changes
Projects
Status: Ready to Review
Development

Successfully merging a pull request may close this issue.

2 participants