-
Notifications
You must be signed in to change notification settings - Fork 639
Kubernetes Autoscaling
Kubernetes lets you scale the pods automatically to optimize resource usage and make the backend ready according to the load in your service. Horizontal Pod Autoscaler which is a built-in component can scale your pods automatically.
Firstly, we need to have a Metrics Server to collect the metrics of the pods. To provide metric via the Metrics API, a metric server monitoring must be deployed on the cluster. Horizontal Pod Autoscaler uses this API to collect metrics.
Metric Server is usually deployed by the cloud providers. If you are using a custom Kubernetes or the Metric Server is not deployed by your cloud provider you should deploy it manually as explained below. Firstly, Check if metrics-server is installed using the command below.
kubectl get pods --all-namespaces | grep -i "metric"
You are going to see an output exactly like the below.
kube-system metrics-server-5bb577dbd8-7f58c 1/1 Running 7 23h`
- Download the components.yaml file on the master.
wget https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
- Add the following line to line 132 of the file.
--kubelet-insecure-tls
. The lines are going to seem exactly as below.spec: containers: - args: - --kubelet-insecure-tls - --cert-dir=/tmp - --secure-port=4443 - --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname - --kubelet-use-node-status-port image: k8s.gcr.io/metrics-server/metrics-server:v0.4.2
- Deploy the yaml file that we have made changes.
kubectl apply -f components.yaml
- Check whether everything is working properly.
The output of the command should be as follows.
kubectl get apiservices |grep "v1beta1.metrics.k8s.io"
v1beta1.metrics.k8s.io kube-system/metrics-server True 21h
-
Make a small change in our yaml file in Ant Media Server.
kubectl edit deployment ant-media-server
. Edit and save the following lines under the container according to yourself. Before proceeding let us tell about Millicores. Millicores is a metric which is used to measure CPU usage. It is a CPU core divided into 1000 units (milli = 1000). 1000 = 1 core. So the below configuration uses 1 cores.resources: requests: cpu: 1000m
Check the accuracy of the value we entered using the command below.
kubectl describe deployment/ant-media-server
Now that the deployment is running, we're going to create a Horizontal Pod Autoscaler for it.
-
Create Horizontal Pod AutoScaling
kubectl autoscale deployment ant-media-server --cpu-percent=60 --min=1 --max=10
or you can use the following yaml file.
kubectl create -f https://raw.githubusercontent.com/ant-media/Scripts/master/kubernetes/ams-k8s-hpa.yaml
In the above configuration, we set the CPU average as 60% and we set the pods as min 1 and maximum 10. A new pod will be created every time the CPU average passes 60%.
-
You can monitor the situation in the following output.
root@k8s-master:~# kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE ant-media-server Deployment/ant-media-server 3%/60% 1 10 1 20h
New pods are going to be created when we start loading and the cpu exceeds 60%. When the cpu average value decreases below 60%, then the pods are going to be terminated.
root@k8s-master:~# kubectl get hpa NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE ant-media-server Deployment/ant-media-server 52%/60% 1 10 4 20h
-
Check the number of pods running using the following command.
root@k8s-master:~# kubectl get pods NAME READY STATUS RESTARTS AGE ant-media-server-7b9c6844b9-4dtwj 1/1 Running 0 42m ant-media-server-7b9c6844b9-7b8hp 1/1 Running 0 19h ant-media-server-7b9c6844b9-9rrwf 1/1 Running 0 18m ant-media-server-7b9c6844b9-tdxhl 1/1 Running 0 47m mongodb-9b99f5c-x8j5x 1/1 Running 0 20h
- Following command gives information about AutoScale.
kubectl get hpa
- Check the load of pods running using the command below.
prints out something below
kubectl top nodes
root@k8s-master:~# kubectl top node NAME CPU(cores) CPU% MEMORY(bytes) MEMORY% k8s-node 111m 5% 717Mi 38% k8s-node-2 114m 5% 1265Mi 68% k8s-node-3 98m 4% 663Mi 35% k8s-node-4 102m 5% 666Mi 35% n8s-master 236m 11% 1091Mi 58%
- Introduction
- Quick Start
- Installation
- Publishing Live Streams
- Playing Live Streams
- Conference Call
- Peer to Peer Call
- Adaptive Bitrate(Multi-Bitrate) Streaming
- Data Channel
- Video on Demand Streaming
- Simulcasting to Social Media Channels
- Clustering & Scaling
- Monitor Ant Media Servers with Apache Kafka and Grafana
- WebRTC SDKs
- Security
- Integration with your Project
- Advanced
- WebRTC Load Testing
- TURN Servers
- AWS Wavelength Deployment
- Multi-Tenancy Support
- Monitor Ant Media Server with Datadog
- Clustering in Alibaba
- Playlist
- Kubernetes
- Time based One Time Password
- Kubernetes Autoscaling
- Kubernetes Ingress
- How to Install Ant Media Server on EKS
- Release Tests
- Spaceport Volumetric Video
- WebRTC Viewers Info
- Webhook Authentication for Publishing Streams
- Recording Streams
- How to Update Ant Media Server with Cloudformation
- How to Install Ant Media Server on GKE
- Ant Media Server on Docker Swarm
- Developer Quick Start
- Recording HLS, MP4 and how to recover
- Re-streaming update
- Git Branching
- UML Diagrams