draft v 1
If you are using Meteor for real world projects sooner or later you will face that:
- Your app doesn't handle well the increasing the load. Meteor and primarily Node.js doesn't support multi-core parallelism so there is little use of vertical scaling. For horizontal scaling you need to split your users between separate instances of the identical app.
- Deploying new versions becomes more of a struggle.
- Your server becomes complicated and it will be a difficult task to create a copy of it for a similar project or for a horizontal scaling.
- Some parts of your monolithic project should be divided into smaller pieces (services) perhaps even rewritten using other technological stack.
- You need to operate everything by hands.
Kubernetes is a great tool in dealing with all of these things and more.
This guide will give a basic knowledge of how to deploy a Meteor app with Kubernetes.
Kubernetes is the well know tool, eco-system and platform created by Google (https://kubernetes.io/). Its open-source and you can run it anywhere you want or use existing cloud services like Google Cloud.
Kubernetes is used for deploying and orchestrating applications. In heart of it lies a containerized applications mainly [docker containers](https://en.wikipedia.org/wiki/Docker_(software).
Containers killing feature is the ability to encapsulate your application and the required environment (OS, other applications, system settings, etc) into a relatively small isolated chunks called images. Think of it as an operating-system-level virtualization.
Kubernetes operates the containers and allows you to use different stack of technologies for your project (Meteor, plain Node.js, Java, Python and everything else).
Kubernetes consists of tools and a cluster.
The cluster is where your apps are going to run. The cluster works on a number of physical or cloud machines bound with network. These machines are called Nodes and Mater Nodes. Your apps and other customs run on Nodes while Master Nodes are the Kubernetes cores.
The tools are used to manipulate the cluster and everything inside of it. Most of the time you will create YAML files (declarative instructions) and pass them (apply) to the cluster.
Here are the main Kubernetes features mentioned on the official website:
- Service discovery and load balancing. Means that you can balance the load (traffic, for example) across your apps, give routes to your apps, etc.
- Horizontal scaling. You can create as many active instances of your app (or even use your own script for smart scaling) as you want and your computational power allows.
- Storage orchestration. Mount any storage system your app needs (local storage, cloud storage, network storage).
- Automated rollouts and rollbacks. Allows you to rollout new versions of your app without any downtime and rollback to previous versions in case of an error.
- Batch execution. Kubernetes can manage your batch and CI workloads, replacing containers that fail, if desired.
- Automatic bin packing. Optimal usage of all the computational resources.
- Self-healing. If a container fails Kubernetes is smart enough to create a new one which replaces the failed.
- Secret and configuration management. You can store environment variables, sensitive information, passwords or configurations for your app separately from it. Changes in secrets doesn't require rebuilding the image.
There are a lot of information about Kubernetes on the Internet, if you are not familiar with it you should start with the basics https://kubernetes.io/docs/tutorials/kubernetes-basics/ but it is not mandatory. If you are going to use cloud Kubernetes providers to deploy your Meteor project this guide is enough for you to start.
- Image. Your app inside build Docker container.
- Pod. Instance of an Image inside Kubernetes Cluster.
- Replica set. Controls the number of the identical Pods.
- Secret. Encrypted information inside the Kubernetes Cluster.
- Deployment. Your app, its version, its settings, Docker Image, required Secrets, Replica policy and more.
- Service. Groups Pods of your app.
- Load balancer. Splits the incoming load between Services and their Pods.
First of all you need to install Docker on the machine where you are going to build your Meteor project. You can read about installation here https://docs.docker.com/install/.
You can create a Docker image from scratch but we are going to use abernix/meteord:onbuild
image as a the base image. This image was specially created for Meteor, uses Ubuntu and builds your Meteor project into Node.js automatically.
You can find more Docker images for Meteor at Abernix hub.
Make a text file with name Dockerfile
in the root of your project.
The file contents should be:
FROM abernix/meteord:onbuild
TheDockerfile
can contain other useful references and commands beside the base image.
Read more.
For example, you can include graphicsmagick
into your Docker image.
The Dockerfile
should look like this:
FROM abernix/meteord:onbuild
RUN apt-get update && apt-get install graphicsmagick -y
After you build your Image you have to put it somewhere so Kubernetes Cluster can use the Image to create the Pods. For this purpose exists Docker Registry. You can run your local Docker Registry, run Docker Registry on a dedicated server or even inside the Kubernetes Cluster.
In this guide we are going to use existing solution Docker Hub.
Docker Hub is like npm but for containers. It has free pricing plan for individuals which allows you to have unlimited public repositories, 1 private repository and 1 parallel build (May, 2019).
Create an account on Docker Hub. You will need username, email and password.
Now you can authenticate on your building machine with this command:
docker login --username your_username --password your_password
To build a Meteor project run the following command in the terminal from your project's root directory:
docker build -t your_username/project_name:some_tag .
.
means that Dockerfile is in the same directory.-t
means that we want to tag the built Image to distinguish it among different projects and builds. You can use a projects build version in place ofsome_tag
like1.0.0
.
After the build you should upload the Image to Docker Registry:
docker push your_username/project_name
WIP https://cloud.google.com/kubernetes-engine/docs/quickstart.
- Read Before you begin.
- Choose a local shell
kubectl
. - Configuring default settings for gcloud.
- Creating a GKE cluster.
After that you can move on reading this guide.
As we can find in the official docs:
Kubernetes secret objects let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. Putting this information in a secret is safer and more flexible than putting it verbatim in a Pod Lifecycle definition or in a container image .
Read more about Kubernetes secrets.
To create a Secret for pulling Images from Docker Registry run:
kubectl create secret docker-registry regsec \
--docker-server=https://index.docker.io/v1/ \
--docker-username=your_username \
--docker-password=your_password \
--docker-email=your_email
where:
- regsec is secret's name
- docker-server is your Private Docker Registry. (https://index.docker.io/v1/ for DockerHub)
- docker-username is your Docker username.
- docker-password is your Docker password.
- docker-email is your Docker email.
Secrets are good for storing environment variables for further use inside the containers. We going to create secrets both for METEOR_SETTINGS and MONGO_URL environment variables.
Let's assume you have a file with Meteor settings called meteor_settings.json.
{
"public": {},
"someThingCustom": {
"first": 0,
"last": 10
}
}
To create a Secret from this file run:
kubectl create secret generic meteor-settings --from-file=./meteor_settings.json
This will create a Secret with name my-meteor-settings
which includes only one key meteor_settings.json
.
The same way you can create a mongo_url.txt with the desired url and use the same command to create the Secret.
Create a file _mongo_url.txt:
mongodb://login:password@ip:port/db
Replace login, password, ip, port and db with your data.
Run:
kubectl create secret generic mongo-url --from-file=./mongo_url.txt
You will see how to use created ENVs in deployment section of this guide.
If you need to update a Secret the easiest way is to delete the Secret and create it again.
To delete a Secret run:
kubectl delete secret secret_name
If you want to update a Secret created from a file you can use this command:
kubectl create secret generic secret_name \
--from-file=./file.name --dry-run -o yaml |
kubectl apply -f -
Its the same command you use to create a Secret plus you add --dry-run -o yaml | kubectl apply -f -
in the end.
As it was mentioned everything you pass to Kubernetes is a YAML instruction (even if you write this instruction in JSON or use commands from terminal it is compiled to YAML).
So if you need to apply the instruction to Kubernetes from a file or update a previously applied instruction you can use one simple command for both tasks:
kubectl apply -f your_file
I suggest to create a special folder for yml files and don't mess them with your project sources.
Read more about YAML. Read more about different API versions of YAML instructions.
To deploy your app you should create a Deployment.
Create the following YAML file (deployment.yml). Everything you need is commented:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: my-project # your project name
name: my-project-deployment # your project deployment name
spec:
progressDeadlineSeconds: 600
replicas: 1 # number of Pods your deployment creates
revisionHistoryLimit: 10 # number of old ReplicaSets to retain to allow rollback
selector:
matchLabels:
app: my-project # your project name
strategy:
rollingUpdate:
maxSurge: 25% # maximum number of Pods that can be created over the desired number of Pods
maxUnavailable: 25% # maximum number of Pods that can be unavailable during the update process
type: RollingUpdate
template:
metadata:
labels:
app: my-project # your project name
spec:
containers:
- env: # ENV variables for your project
- name: ROOT_URL
value: https://my.domain/ # string value
- name: MONGO_URL
valueFrom: # value from previosly created Secret
secretKeyRef:
key: mongo_url.txt
name: mongo-url
- name: METEOR_SETTINGS
valueFrom: # value from previosly created Secret
secretKeyRef:
key: meteor_settings.json
name: meteor-settings
image: your_username/project_name:some_tag # image of your app
imagePullPolicy: Always
name: my-project # your project name
ports:
- containerPort: 80 # which port should be open on Pods
name: http-server
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /userimages
name: images
- mountPath: /userdocuments
name: documents
dnsPolicy: ClusterFirst
imagePullSecrets:
- name: regsec # secret with Docker Hub credentials
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
terminationGracePeriodSeconds: 30
volumes:
- emptyDir: {}
name: images
- emptyDir: {}
name: documents
status: {}
In this example we assume that the app uses folders /userimages
and /userdocuments
for temp files.
For this we describe volumes
type of emptyDir
(you can check other types of volumes
here):
volumes:
- emptyDir: {}
name: images
- emptyDir: {}
name: documents
And mount them with corresponding dir names:
volumeMounts:
- mountPath: /userimages
name: images
- mountPath: /userdocuments
name: documents
To create a Deployment run:
kubectl apply -f deployment.yml
To update your app in Kubernetes cluster simply build new image with new tag and push it to Docker Hub.
Then update deployment.yml with new image name and tag image: your_username/project_name:some_tag
.
Run to apply changes:
kubectl apply -f deployment.yml
Kubernetes allows you to auto scale your deployment based on CPU usage.
For this create the following YAML file (autoscaler.yml):
apiVersion: autoscaling/v1
kind: HorizontalPodAutoscaler
metadata:
name: my-project
namespace: default
spec:
scaleTargetRef:
apiVersion: extensions/v1beta1
kind: Deployment
name: my-project
minReplicas: 1
maxReplicas: 10
targetCPUUtilizationPercentage: 50
where:
minReplicas
is a minimum number of PodsmaxReplicas
is a maximum number of PodstargetCPUUtilizationPercentage
is a CPU utilization limit in percents when a new Pod should be created.
Don't forget to replace my-project
with the name of deployed app.
To apply this YAML instruction run:
kubectl apply -f autoscaler.yml
Your app is deployed and functioning but it is invisible to outside world.
To make it visible first of all you should create a corresponding Service for your app.
Service is an abstraction which defines a logical set of Pods and a policy by which to access them.
Create the following YAML file (service.yml):
kind: Service
apiVersion: v1
metadata:
name: my-project-service
spec:
ports:
- port: 80
targetPort: http-server
selector:
app: my-project
This service provides 80 port on your pods as http-server.
Don't forget to replace my-project
with the name of deployed app.
To apply this YAML instruction run:
kubectl apply -f service.yml
Now you app Pods are targeted by a Service. Lets make the Service accessible from outside by creating a corresponding Ingress.
Ingress is an API object that manages external access to the services in a cluster, typically HTTP. Ingress can provide load balancing, SSL termination and name-based virtual hosting.
As it was mentioned Ingress also provides load balancing between Service's Pods and SSL/TLS (HTTPS). You can achieve this by creating Ingress Controller. There are a lot of different controllers, you can find more here.
In this guide we use nginx based controller which is officially supported by Kubernetes team. As you might guess this Ingress Controller will use nginx for load balancing and SSL/TLS termination.
We will use simplest router possible which sends every HTTP request for your domain to one Service.
Create the following YAML file (ingress.yml):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-project-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.org/websocket-services: "my-project"
nginx.ingress.kubernetes.io/proxy-body-size: 10m
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
spec:
rules:
- host: my.domain
http:
paths:
- path: /
backend:
serviceName: my-project-service
servicePort: 80
where:
nginx.ingress.kubernetes.io/proxy-body-size
is a maximum size of incoming requests,10m
means 10 megabytes. For example, if your app allows users to upload their photos, a user won't be able to upload a file with size more than 10 Mb.
Don't forget to replace my.domain
, my-project-service
, nginx.org/websocket-services
,
my-project
with your data.
Apply file with:
kubectl apply -f ingress.yml
Now you have to bind your domain with the load balancer's IP.
To check Ingress Controller IP use command:
kubectl get svc -n ingress-nginx | grep ingress | grep LoadBalancer
The last step is to delegate DNS A record for your domain and the obtained IP.
If you have custom TLS certificate that you want to use for HTTPS you need to create an corresponding Secret
with kubectl
:
kubectl create secret tls test-secret-tls --cert=server.crt --key=server.key
where:
- test-secret-tls is the name of the Secret
- server.crt is the certificate with public key
- server.key is the private key
Now you can modify your ingress.yml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.org/websocket-services: "my-project"
nginx.ingress.kubernetes.io/proxy-body-size: 10m
nginx.ingress.kubernetes.io/affinity: "cookie"
nginx.ingress.kubernetes.io/session-cookie-hash: "sha1"
ingress.kubernetes.io/ssl-redirect: "true"
spec:
tls:
- hosts:
- my.domain
secretName: ingress-tls-secret
rules:
- host: lk.nobi.services
http:
paths:
- path: /
backend:
serviceName: my-project-service
servicePort: 80
As you can see we added tls
to spec
and described there the host we want to accociate with the Ingress and the Secret that holds certificate:
tls:
- hosts:
- my.domain
secretName: ingress-tls-secret
We have also added ingress.kubernetes.io/ssl-redirect: "true"
to annotations section which forces HTTPS usage over HTTP.
WIP https://docs.cert-manager.io/en/latest/tutorials/acme/quick-start/index.html
Usually we do automated testing of our application locally but its also a good idea to check that everything works as expected in real production before it is available to the endpoint user. The suggest solution is to create a test deployment with a separate domain (test.my.domain) and Ingress and deploy your new project version to test zone first.
Here you can find a guide of using basic auth with your Nginx Ingress https://imti.co/kubernetes-ingress-basic-auth/.
WIP
WIP