Protobuf performance improvements

[ennerf]

@PeterJohnson I'm not sure whether this is the right place for this discussion, but I came across some of your performance numbers here and had a quick look at the ProtobufTranslation2d definition.

Protobuf is unfortunately quite bad at encoding nested messages because of the additional size overhead and cost to pre-compute the header. Messages also can't be streamed in a single pass. (group is better for streaming, but the syntax is awful and I think they were removed from proto3).

message Translation2d {
  double x = 1;
  double y = 2;
}

message Translation2dList {
  repeated Translation2d translations = 3;
}

The above definition would encode an array of translations as a series of [id3][varint-length][id1][float][id2][float]...[id3][varint-length][id1][float][id2][float]. That's 4 bytes overhead and 3x varint encoded values for 8 bytes payload. The varint encodings and switch statements (required for parsing) also add a lot of branching that can quickly become expensive on embedded CPUs that don't have a good branch predictor.

message Translation2dList {
  repeated double x = 1 [packed=true];
  repeated double y = 2 [packed=true];
}

The flatter definition would be encoded as two flat double arrays and (on little endian systems) can be de-/serialized using a simple memcpy. It'd take a bit more input validation, but you'd probably get an order of magnitude less serialization overhead.

I don't know whether you can break backwards compatibility though.

[PeterJohnson]

Thanks for the suggestion! For the simple use cases like a simple array of fixed size structures, this is why we’ve developed Struct (basically C-style packed structures with a simple description language) as a higher speed and more byte efficient alternative to protobuf.

While going to decomposed parallel arrays would be more storage efficient and decoding performant, it’s much less user-friendly for our dynamic use cases, where we want generic dashboards to be able to blow out data structures as the user would expect to see them.

Your suggested approach might make sense to explore for the PhotonVision project, as they largely are using protobuf just for transport and not really intending it for interactive user inspection in the same way? @mcm001 @srimanachanta

[ennerf]

That makes sense. For image data it might be best to just copy the entire data as is and let the other side know to decode it, e.g.,

message Image {

  enum DataFormat {
    UNKNOWN = 0;
    ARGB = 1;
    RGBA = 2;
    JPEG = 3;
    etc.
  }

  DataFormat format = 1;
  bytes data = 2;

}

[PeterJohnson]

We don’t actually transport image data currently; what PhotonVision is transporting is results of image processing, eg arrays of AprilTags detected poses.

[mcm001]

@ennerf here's what we (photon) want to notionally do for our protobufs to send targeting data around between coprocessors. I'll poke at your suggestion above and get some benchmarks. We also have a diy hand rolled serde thing in Packet.java we have been using for the last couple years

https://github.com/mcm001/photonvision/blob/2023-10-20_protobuf/photon-targeting/src/main/proto/photon.proto

[ennerf]

I'm not sure how much you can improve on the proto, but I'd probably replace Packet.java with ProtoSource and ProtoSink. Those work with different sources (arrays, ByteBuffer, InputStream) and should be quite a bit more efficient (e.g. writing a double array).

You're also serializing in big endian while almost all platforms nowadays are little endian. That prohibits a lot of optimizations.