Skip to content

Latest commit

 

History

History
272 lines (266 loc) · 7.95 KB

index.org

File metadata and controls

272 lines (266 loc) · 7.95 KB

Kubernetes and Weave.net on bare metal

 

Public solutions

Node view

./nodes.png

Components view

./components.png

What’s this all about

  • SSL
  • Etcd
  • Master node
  • Worker node
  • CNI

SSL

SSL: Bundles

1 to 4 bundles per cluster

SSL: single bundle

  • Single CA
  • Keypairs:
    • apiserver
    • kubelets
    • etcd clients
    • etcd peers

SSL: two bundles

  • Kubernetes
    • CA
    • apiserver keypair
    • kubelets keypair
  • Etcd
    • CA
      • peers keypair
      • clients keypair

SSL: full paranoia

  • apiserver
    • CA
    • keypairs
  • kubelets
    • CA
    • keypairs
  • etcd peers
    • CA
    • keypairs
  • etcd clients
    • CA
    • keypairs

SSL: keypairs

  • keypair per host
  • keypair per component

SSL: CA

  • Validity!

Etcd

https://2eof2j3oc7is20vt9q3g7tlo5xe-wpengine.netdna-ssl.com/wp-content/uploads/2015/01/Screen-Shot-2015-01-28-at-1.46.38-PM-370x219.png Distributed reliable key-value store

Etcd: Intercommunications

./etcd-schema.png

Etcd: Apiserver communications

./etcd-apiserver.png

Etcd: initial args

--initial-advertise-peer-urls=https://etcd0:2380 \
--initial-cluster-state=new \
--initial-cluster-token=RfDz6BPYvQWSshe8J0cEhUoAGbnm1LfgS0A77EsjCa \
--initial-cluster=etcd0=https://etcd0:2380,etcd1=https://etcd1:2380,etcd2=https://etcd2:2380 \

Etcd: add node

  • on any old member: 
    $ etcdctl member add name peerURL
  • on a new member start etcd changing following opts: 
    --initial-cluster-state=existing
--initial-cluster=all-old-members,https://new-member:2380

Etcd: remove node

  • on any live member: 
    $ etcdctl member list
$ etcdctl member remove ID

Etcd: fault tolerance

Tolerance table

CLUSTER SIZEMAJORITYFAILURE TOLERANCE
110
220
321
431
532

Majority = floor(Size/2) + 1

Tolerance = Size - Majority

Tolerance = Size - floor(Size/2) - 1

Etcd: proxy mode

$ etcd grpc-proxy start --endpoints=...

Master node

./master_node.png

Master node: multimaster

./multimaster.png Problems:

  • load balancing
  • leases

Master node: custom schedulers

./custom-schedulers.png

spec:
  template:
    spec:
      schedulerName: default-scheduler

Master node: addon manager

  • simple shell script
  • /etc/kubernetes/addons
  • labels 
       metadata:
     labels:
       addonmanager.kubernetes.io/mode: Reconcile
	 kubernetes.io/cluster-service: "true"

Worker node

./worker.png

CNI

./cni_logo.png Container Network Interface

CNI: versions

  • Specification: 0.3.1
  • Tool: 0.6.0
  • Plugins: 0.7.0

https://vignette.wikia.nocookie.net/whatever-you-want/images/d/d1/3d75c86f47.jpg

CNI: portmap

  • forward traffic from one or more ports on the host to the container
  • chained

CNI: configuration

  • /etc/cni/net.d/10-weave.conflist 
    {
    "cniVersion": "0.3.1",
    "name": "weave",
    "plugins": [
	{
	    "name": "weave",
	    "type": "weave-net",
	    "hairpinMode": true
	},
	{
	    "type": "portmap",
	    "capabilities": {
		"portMappings": true
	    },
	    "snat": true
	}
    ]
}

Weave

Weave: Node view

./weave-node.png

Weave: Topology

./weave-topology.png

Weave: FDP

https://www.weave.works/docs/net/latest/concepts/weave-net-fdp1-1024x454.png

Weave: Multi-hop routing

./weave-multihop.png

Weave: installation

$ kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=\
$(kubectl version | base64 | tr -d '\n')"

Weave: be aware

Always remove /etc/cni/net.d/10-weave.conf

Tips and tricks

http://vodvore.net/prikols/prikol01727.jpg

Networks

  • Host network
  • Pod network (CNI)
  • Service network (Netfilter)

Don’t forget

ip route add service_network dev internal_interface

Encryption config

kind: EncryptionConfig
apiVersion: v1
resources:
- resources:
  - secrets
  - configmaps
  providers:
  - aescbc:
      keys:
        - name: key1
          secret: RnVjayB0aGlzIHNoaXQhCg==
  - identity: {}

Encryption setup

# kube-apiserver --experimental-encryption-provider-config=/etc/kubernetes/encryption-config.yaml ...

Actually encrypting the data

$ kubectl get secrets --all-namespaces -o json | kubectl replace -f -

Endpoint reconciler

  • Good 
    # kube-apiserver --endpoint-reconciler-type=lease ...
  • Bad 
    # kube-apiserver --endpoint-reconciler-type=master-count ...

Authentication

Authentication: WARNING

There is no authentication inside kubernetes! AT ALL!

Authentication: strategies

Bootstrap tokens

  • Apiserver: 
    --enable-bootstrap-token-auth
  • Kubelet: 
    --bootstrap-kubeconfig=/etc/kubernetes/kubeconfig-bootstrap.yaml

Upgrade

  • Patch versions: smooth and simple
  • Minor versions: all pods restart

Questions?