Fix ingress annotation warnings

[karlaspuldaro]

Summary

• Update deprecated annotation (Fixes annotation "kubernetes.io/ingress.class" is deprecated, please use 'spec.ingressClassName' instead #1158)
• Remove deprecated annotation support for older kubernetes versions (<1.18)
• Add tests to cover the changes Test changes locally

Details and comments

Since Kubernetes only supports their few latest minor versions, this PR removes support to the deprecated way of setting annotations.

[CLAassistant]

All committers have signed the CLA.

[karlaspuldaro]

@psschwei @akihikokuroda Can you help with some guidance of how I can add tests to this fix?
For a test case, I imagine checking the className value in the gateway config, perhaps adding to this test file?

[akihikokuroda]

@karlaspuldaro It is difficult to add the test verifying the changes. We don't have the kubernetes level end to end testing now. We have the notebook tests that verify most of the middleware functionalities but it's based on the docker compose that doesn't use the kubernetes ingress. I believe that this PR can be accepted without adding the test. @psschwei WDYT?

[psschwei]

Yeah, I think we can accept this without tests in the CI, just run it locally to verify that it does what we expect it to.

[karlaspuldaro]

@psschwei Local testing update: I'm going through some build issues related to kind and docker. I'll come back with another update once I confirm the warning is gone.

[karlaspuldaro]

Update: still having issues completing my build and deployment.

So far, I was able to build gateway and ray node, but couldn't load the ray image into my kind cluster.
For context, I am using containerd (the only way I could successfully build from source) and it is apparently not supported by kind (even though I could load the gateway image just fine, just not ray).
Once I get the ray image push done, I can install quantum-serverless helm chart - and test if the warning messages are really gone.

[Tansito]

Let us know if we can help you with something @karlaspuldaro 👍

[karlaspuldaro]

@Tansito @akihikokuroda I think I need help reproducing the warnings locally from the main branch, then checking out to this PRs branch and comparing logs to see if they're gone.
I thought the warnings would be logged when installing helm charts, after building gateway and ray node, but I was wrong.
If someone could point me to the exact deployment step to recreate the issue, it would be very helpful. Thx!

[psschwei]

If someone could point me to the exact deployment step to recreate the issue, it would be very helpful. Thx!

You can modify the ingress section of values.yaml to include host/tls info... something like:

ingress:
  annotations:
    # For IBM Cloud the valid ingress class values are: public-iks-k8s-nginx and private-iks-k8s-nginx
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/proxy-buffers-number: "4"
    nginx.ingress.kubernetes.io/proxy-buffer-size: "512k"
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
#  tls: []
  tls:
    - hosts:
      - "quantum-serverless-url.cloud"
      secretName: "tls-secret-name"
#  hosts: []
  hosts:
    - host: "quantum-serverless-url.cloud"
      paths:
      - path: /
        pathType: Prefix
        serviceName: "gateway"
        servicePort: 8000

[karlaspuldaro]

Thank you @psschwei for helping test this PR locally and confirm the changes fix the issue.
For reviewers, to test locally:

1. Reproduce ingress warnings on main branch: Deploy serverless helm chart on k8s 1.28+ (make sure ingress config is also deployed). helm install step should output ingress warnings.
2. Check out to this branch and repeat steps above. helm install completes with no ingress warnings.

[Tansito]

Thank you so much @karlaspuldaro ! 😄