Protobuf Serialization Layer

[clux]

We hope to release a new serialization layer for kube that supports protobuf; k8s-pb.

This is an edited umbrella issue tracking progress and major outstanding work. It pertains to the ProtoGen category on the roadmap.

Background

This is a gold client requirement, that will help reduce the IO overhead of large watches, but also has the potential to bring many other benefits as we bring better integration and delineation between the serialization and api layers.

It would serve as a replacement for k8s-openapi if it proves viable - and it's looking like it can be. But it's still pretty early stage. See #371 and issues linked herein for further background.

Tasks

Initial POC:

• Protobuf Encoding protobuf encoding exploration #371 - exploration / downloads of swagger protobuf / repo creation 🎉
• Create a codegen crate - Move codegen to a separate crate and use improved transform k8s-pb#5
• Make library compilable - Make k8s-pb compilable k8s-pb#13
• Add minimum required traits - Add Traits k8s-pb#14
• HasSpec and HasObject traits Utilize HasSpec and HasObject traits for api methods #295 (no parallel in k8s-openapi)
• version after kubernetes version + release process Crate versioning k8s-pb#10
• handle irregular casings issue via prost - Handle irregular camelCase to snake_case conversion k8s-pb#2
• necessary wrappers:
  • envelope - Handling Envelope Wrapper k8s-pb#6
  • IntOrString - IntOrString needs a wrapper k8s-pb#12
  • Time / Microtime - Time/MicroTime needs a wrapper k8s-pb#7
• initial PR against kube using kube-pb - Integrate with kube under a feature k8s-pb#27

Release Readiness:

• release strategy for migrating k8s-openapi for kube-pb
• codegen'd traits - Define and implement equivalent k8s-openapi traits k8s-pb#4
• re-export types from kube to prevent lock-step dependency issues -> no, avoid mutually exclusive features instead: Crate versioning k8s-pb#10

Quality of life:

• case standardisation and disambiguation - PascalCase conversion on structs in codegen k8s-pb#15
• optional builder patterns - optional builder patterns k8s-pb#9
• support selecting api groups to build - Support selecting API groups to build k8s-pb#11

Potential future steps:

• consider donating parts of the protobuf repo to kubernetes-client org for better relations with sigapimachinery
• think about how to tackle apply configurations - shortcuts for Api::apply and Api::apply_status #649
• find a solution for metrics API - Add support for metrics API #492