Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better support for x-kubernetes properties in schemas #1367

Open
clux opened this issue Dec 7, 2023 · 3 comments
Open

Better support for x-kubernetes properties in schemas #1367

clux opened this issue Dec 7, 2023 · 3 comments
Labels
derive kube-derive proc_macro related discussions possibly more of a discussion piece than an issue question Direction unclear; possibly a bug, possibly could be improved.

Comments

@clux
Copy link
Member

clux commented Dec 7, 2023

There is a slew of x-kubernetes extension properties that is usable inside CRD schemas to improve the validation and merge behaviour of structures of members.

From Kubernetes 1.25 this is being leaned into even more heavily with x-kubernetes-validations rules on CRD schemas:

  openAPIV3Schema:
    type: object
    properties:
      spec:
        type: object
        x-kubernetes-validations:
          - rule: "self.minReplicas <= self.replicas"
            message: "replicas should be greater than or equal to minReplicas."
          - rule: "self.replicas <= self.maxReplicas"
            message: "replicas should be smaller than or equal to maxReplicas."

This type of validation makes it possible to write controllers with much richer type of native validation without having to lean on the clunkier admission controllers / admission controller frameworks to manage extraneous policy rules - and has a lot of potential use. There's a KubeConNA'23 talk called Declarative Everything that explores this and the direction of the feature from sig apimachinery.

Current Usage

Such properties can be specified via schemars attributes to override the schema generation for a type as per

// Listable field with specified 'set' merge strategy
#[serde(default)]
#[schemars(schema_with = "set_listable_schema")]
set_listable: Vec<u32>,
}
// https://kubernetes.io/docs/reference/using-api/server-side-apply/#merge-strategy
fn set_listable_schema(_: &mut schemars::gen::SchemaGenerator) -> schemars::schema::Schema {
serde_json::from_value(serde_json::json!({
"type": "array",
"items": {
"format": "u32",
"minium": 0,
"type": "integer"
},
"x-kubernetes-list-type": "set"
}))
.unwrap()
}

This is not ideal because you then have to manage all the other schema properties for that type yourself.

Ideas

Schemars Extensions

To properly support adding properties like this, it would be interesting to see if schemas which has support for a large amount of attributes already, could be enhanced with a kubernetes attribute feature set to allow stuff like e.g.:

struct Spec {
    #[schemars(x-kubernetes-validations(
        rule = "self.minReplicas <= self.replicas",
        message = "replicas should be greater than or equal to minReplicas."
    ))]
    spec: usize,
}

Have asked upstream about what they think about this in GREsau/schemars#258

CEL Crate?

Maybe it's better to start some of this from a CEL POV because maybe we want to be able to run validations we put into CEL rules locally. In this case a crate that allows us to do something like

struct Spec {
    #[cel_validation(
        rule = "self.minReplicas <= self.replicas",
        message = "replicas should be greater than or equal to minReplicas."
    )]
    spec: usize,
}

in either case we probably still need something in schemars so that the rules are forwarded into the schema, but it means we will have our own properties (like serde/validator) that schemars can pickup on later.

And we likely would need a new crate or majorly extend the cel-parser / cel-interpreter crate (both of which live in cel-rust).

@clux clux added discussions possibly more of a discussion piece than an issue derive kube-derive proc_macro related question Direction unclear; possibly a bug, possibly could be improved. labels Dec 7, 2023
clux added a commit that referenced this issue Dec 14, 2023
clux added a commit that referenced this issue Dec 15, 2023
* Naive CEL validation example

for #1367 and kube-rs/website#53

Signed-off-by: clux <[email protected]>

* fmt

Signed-off-by: clux <[email protected]>

---------

Signed-off-by: clux <[email protected]>
@benkeil
Copy link

benkeil commented Feb 14, 2024

This would be a great feature. Looking for how to enforce immutability

@azat
Copy link

azat commented Jul 30, 2024

FWIW here is a better example, that allows to preserve existing schema - https://github.com/s3rius/kuo/blob/4c83b4033093e84035cefb0f5a22c46c7b9d471c/src/crds/managed_user.rs#L70-L84

@clux
Copy link
Member Author

clux commented Jul 30, 2024

That example is really useful @azat, thank you.

We could maybe create a higher level version of that that allows generating schema modifying functions like that, if we can curry the "rule" and "message" as initial arguments, and return the mutating fn from this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
derive kube-derive proc_macro related discussions possibly more of a discussion piece than an issue question Direction unclear; possibly a bug, possibly could be improved.
Projects
None yet
Development

No branches or pull requests

3 participants