Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encode backward and use C stubs #223

Merged
merged 30 commits into from
Nov 29, 2023
Merged

Conversation

c-cube
Copy link
Collaborator

@c-cube c-cube commented Nov 27, 2023

rationale: encoding backward means we can use a single buffer even in the presence of nested messages. We encode from the end, in reverse order, which means we encode the sub-message before having to write its size; so when it's time to write the size we know it and we can just write the varint. If we write forward we cannot do that because we don't know how much space to leave in front to encode the length as a varint.

The C stubs are there to make varint operations faster: computing how many bytes a varint takes (so we can allocate a little slice for it in the buffer); and writing the varint into this slice.

Note that we also require OCaml >= 4.08 so we can rely on Bytes' operations for fixed-size bits32/bits64.

also update code-generation so it writes values backward.
run exactly one encoding per iteration, do better checks
we preallocate the encoder. Actual encoding time is more interesting
because in high perf situations we'll reuse allocators.
it's no alloc, so as @Armael pointed out we don't need the CAML***
macros
@c-cube c-cube merged commit bbe7094 into master Nov 29, 2023
2 checks passed
@c-cube c-cube deleted the wip-encode-backward-with-c-stubs branch November 29, 2023 02:38
@c-cube c-cube mentioned this pull request Dec 5, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant