Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize protobuf serialization #253

Open
seanlinsley opened this issue Jul 18, 2024 · 0 comments
Open

Optimize protobuf serialization #253

seanlinsley opened this issue Jul 18, 2024 · 0 comments

Comments

@seanlinsley
Copy link
Member

seanlinsley commented Jul 18, 2024

Per this flamegraph (from an internal benchmark for the pg_query Rust crate), most of the time is spent serializing and deserializing the protobuf structure.

The straightforward options to improve this are:

Breaking changes might be:

  • remove extra nesting in the protobuf definition, like with the abstract Node containing a Option<node::Node> enum
  • remove nesting altogether, returning a flat array of nodes that include index references to their children
  • have the Rust crate directly manipulate the C data structures instead of using protobuf

And non-breaking (but not straightforward) changes might be:

  • move the Rust logic to C so there's no need to access the protobuf in our hot paths

Screenshot 2024-07-18 at 9 12 03 AM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant