Helm Load Balancer Discussion

[jackyzha0]

Just finished up the first draft implementation for using Helm charts to install the YataiService (#945) and I've implemented the basic YataiService as well as the Postgres Persistent Volume. I'm currently wondering what the best way to handle load balancing is.

In the issue #716, NGINX is suggested as per this article: https://www.nginx.com/blog/nginx-1-13-10-grpc/, but this definitely seems like a really complex way to do it. Another option I've heard of in the past for something similar is to use Linkerd which seems to be more up to date with current DevOps best practices and supports automatic gRPC load balancing (https://linkerd.io/2/features/load-balancing/)

Thoughts?

[flosincapite]

Definitely something for the maintainers to decide, but I have an opinion.

I find NGINX pretty easy to reason about. I didn't realize NGINX already had support for different load-balancing algorithms, so it seems like it would be pretty straightforward.

The thing I don't know is how well NGINX plays with other services in a Kubernetes cluster. Is it relatively straightforward to specify other nodes in a cluster as upstream.server in NGINX? I would assume so, but if not (e.g., nontrivial firewall or permissions stuff between nodes), Linkerd might be better--it's built for exactly this use case, so there's no doubt it will play nicely with Kubernetes.

[jackyzha0]

I think K8s has it's own DNS resolver so we can refer to the service by name so upstream.server could be yatai-service:50051 or something

[flosincapite]

Oh, nice! Yeah, that sounds really straightforward, then. I guess it's kind of a strategic question for the maintainers at this point, since either option increases reliance on something that's currently supported but not essential (i.e., either NGINX or K8s).

[yubozhao]

@jackyzha0 Have you check out Ingress/Ingress controller? Ingress is a Kubernetes resource that contains a collection of rules and configuration for routing external HTTP(S) traffic to internal services. It has some functionality for load balancing.

A lot of network-related services support Ingress, such as Istio. Istio is another popular service mesh service similar to Linkerd. It is a project backed by Google and Lyft.

[jackyzha0]

@yubozhao Yeah I've looked at the default Ingress resource but it doesn't seem to support gRPC out of the box. I did find this official K8s doc that might be helpful though: https://github.com/kubernetes/ingress-nginx/tree/master/docs/examples/grpc

[parano]

@jackyzha0 looking at some of the other Helm chart implementations and the ingress documentation @yubozhao shared, I think the YataiService chart only needs to create a Cluster IP type Service and an Ingress. The user can choose to install their own ingress-controller which will handle the actual ingress resource creation. The ingress controller could be nginx based ingress-nginx, but it can be other ingress controllers as well: https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/#additional-controllers

Ideally we would want to expose the Ingress resource definition's annotation field to the user via a helm chart parameter so that user can configure the ingress by setting annotation such as nginx.ingress.kubernetes.io/backend-protocol: "GRPCS"

[parano]

On a related note, currently, there is no support for horizontally scaling YataiService(running multiple YataiService instances at the same time, pointing to the same database). It may get into some unexpected state or error if the user tries to do that. We will need to add application level lock(e.g. lock deployment operations based on the deployment ID) in order to do that.