This repository has been archived by the owner on Aug 25, 2023. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adopt Fluentd chart and update image (#43)
* adopt fluentd chart * add SA annotations and fix resource labelling * fix selector labels
- Loading branch information
Showing
19 changed files
with
758 additions
and
3 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
# Patterns to ignore when building packages. | ||
# This supports shell glob matching, relative path matching, and | ||
# negation (prefixed with !). Only one pattern per line. | ||
.DS_Store | ||
# Common VCS dirs | ||
.git/ | ||
.gitignore | ||
.bzr/ | ||
.bzrignore | ||
.hg/ | ||
.hgignore | ||
.svn/ | ||
# Common backup files | ||
*.swp | ||
*.bak | ||
*.tmp | ||
*~ | ||
# Various IDEs | ||
.project | ||
.idea/ | ||
*.tmproj |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
apiVersion: v1 | ||
description: A Fluentd Helm chart for Kubernetes. | ||
icon: https://raw.githubusercontent.com/fluent/fluentd-docs/master/public/logo/Fluentd_square.png | ||
name: fluentd | ||
version: 2.6.0 | ||
appVersion: v3.2.0 | ||
home: https://www.fluentd.org/ | ||
sources: | ||
- https://github.com/helm/charts/tree/269d7791a5ddb8622e66e02e3fff756a4cdbf4f9/stable/fluentd | ||
- https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-elasticsearch/fluentd-es-image |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,58 @@ | ||
# fluentd | ||
|
||
[Fluentd](https://www.fluentd.org/) collects events from various data sources and writes them to files, RDBMS, NoSQL, IaaS, SaaS, Hadoop and so on. Fluentd helps you unify your logging infrastructure (Learn more about the Unified Logging Layer). | ||
|
||
This chart originates from https://github.com/helm/charts/tree/269d7791a5ddb8622e66e02e3fff756a4cdbf4f9/stable/fluentd | ||
|
||
## Configuration | ||
|
||
The following table lists the configurable parameters of the fluentd chart and their default values. | ||
|
||
Parameter | Description | Default | ||
--- | --- | --- | ||
`useStatefulSet` | Deploy as a StatefulSet regardless of whether autoscaling is enabled | `nil` | ||
`affinity` | node/pod affinities | `{}` | ||
`configMaps` | Fluentd configuration | See [values.yaml](values.yaml) | ||
`image.pullPolicy` | Image pull policy | `IfNotPresent` | ||
`image.repository` | Image repository | `gcr.io/google-containers/fluentd-elasticsearch` | ||
`image.tag` | Image tag | `v2.4.0` | ||
`imagePullSecrets` | Specify image pull secrets | `nil` (does not add image pull secrets to deployed pods) | ||
`extraEnvVars` | Adds additional environment variables to the deployment (in yaml syntax) | `{}` See [values.yaml](values.yaml) | ||
`extraVolumeMounts` | Mount extra volumes (in yaml syntax) | `` See [values.yaml](values.yaml) | ||
`extraVolumes` | Extra volumes (in yaml syntax) | `` See [values.yaml](values.yaml) | ||
`ingress.enabled` | enable ingress | `false` | ||
`ingress.labels` | list of labels for the ingress rule | See [values.yaml](values.yaml) | ||
`ingress.annotations` | list of annotations for the ingress rule | `kubernetes.io/ingress.class: nginx` See [values.yaml](values.yaml) | ||
`ingress.hosts` | host definition for ingress | See [values.yaml](values.yaml) | ||
`ingress.tls` | tls rules for ingress | See [values.yaml](values.yaml) | ||
`nodeSelector` | node labels for pod assignment | `{}` | ||
`replicaCount` | desired number of pods | `1` ??? | ||
`resources` | pod resource requests & limits | `{}` | ||
`plugins.enabled` | Enable Plugins Installation | `false` | ||
`plugins.pluginsList` | List of plugins to install | `[]` | ||
`rbac.create` | Specifies whether RBAC resources should be created | `true` | ||
`serviceAccount.create` | Specifies whether a service account should be created. | `true` | ||
`serviceAccount.name` | Name of the service account. | ||
`priorityClassName` | priorityClassName | `nil` | ||
`service.loadBalancerIP` | If `service.type` is `LoadBalancer` set custom IP load balancer IP address | `nil` | ||
`service.ports` | port definition for the service | See [values.yaml](values.yaml) | ||
`service.type` | type of service | `ClusterIP` | ||
`service.annotations` | list of annotations for the service | `{}` | ||
`service.sessionAffinity` | Session affinity type for the service | `None` | ||
`service.sessionAffinityConfig.timeoutSeconds` | ClientIP type session affinity timeout duration in seconds. | `nil` | ||
`tolerations` | List of node taints to tolerate | `[]` | ||
`persistence.enabled` | Enable buffer persistence | `false` | ||
`persistence.accessMode` | Access mode for buffer persistence | `ReadWriteOnce` | ||
`persistence.size` | Volume size for buffer persistence | `10Gi` | ||
`autoscaling.enabled` | Set this to `true` to enable autoscaling | `false` | ||
`autoscaling.minReplicas` | Set minimum number of replicas | `2` | ||
`autoscaling.maxReplicas` | Set maximum number of replicas | `5` | ||
`autoscaling.metrics` | metrics used for autoscaling | See [values.yaml](values.yaml) | ||
`terminationGracePeriodSeconds` | Optional duration in seconds the pod needs to terminate gracefully | `30` | ||
`metrics.enabled` | Set this to `true` to enable Prometheus metrics HTTP endpoint | `false` | ||
`metrics.service.port` | Prometheus metrics HTTP endpoint port | `24231` | ||
`metrics.serviceMonitor.enabled` | Set this to `true` to create ServiceMonitor for Prometheus operator | `false` | ||
`metrics.serviceMonitor.additionalLabels` | Additional labels that can be used so ServiceMonitor will be discovered by Prometheus | `{}` | ||
`metrics.serviceMonitor.namespace` | Optional namespace in which to create ServiceMonitor | `nil` | ||
`metrics.serviceMonitor.interval` | Scrape interval. If not set, the Prometheus default scrape interval is used | `nil` | ||
`metrics.serviceMonitor.scrapeTimeout` | Scrape timeout. If not set, the Prometheus default scrape timeout is used | `nil` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
To verify that Fluentd Elasticsearch has started, run: | ||
|
||
kubectl --namespace={{ .Release.Namespace }} get all -l "app={{ template "fluentd.name" . }},release={{ .Release.Name }}" | ||
|
||
THIS APPLICATION CAPTURES ALL CONSOLE OUTPUT AND FORWARDS IT TO Elasticsearch. Anything that might be identifying, | ||
including things like IP addresses, container images, and object names will NOT be anonymized. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,64 @@ | ||
{{/* vim: set filetype=mustache: */}} | ||
{{/* | ||
Expand the name of the chart. | ||
*/}} | ||
{{- define "fluentd.name" -}} | ||
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" -}} | ||
{{- end -}} | ||
|
||
{{/* | ||
Create a default fully qualified app name. | ||
We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec). | ||
If release name contains chart name it will be used as a full name. | ||
*/}} | ||
{{- define "fluentd.fullname" -}} | ||
{{- if .Values.fullnameOverride -}} | ||
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}} | ||
{{- else -}} | ||
{{- $name := default .Chart.Name .Values.nameOverride -}} | ||
{{- if contains $name .Release.Name -}} | ||
{{- .Release.Name | trunc 63 | trimSuffix "-" -}} | ||
{{- else -}} | ||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} | ||
{{- end -}} | ||
{{- end -}} | ||
{{- end -}} | ||
|
||
{{/* | ||
Create chart name and version as used by the chart label. | ||
*/}} | ||
{{- define "fluentd.chart" -}} | ||
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" -}} | ||
{{- end -}} | ||
|
||
{{/* | ||
Create the name of the service account to use | ||
*/}} | ||
{{- define "fluentd.serviceAccountName" -}} | ||
{{- if .Values.serviceAccount.create -}} | ||
{{ default (include "fluentd.fullname" .) .Values.serviceAccount.name }} | ||
{{- else -}} | ||
{{ default "default" .Values.serviceAccount.name }} | ||
{{- end -}} | ||
{{- end -}} | ||
|
||
{{/* | ||
Common labels | ||
*/}} | ||
{{- define "fluentd.labels" -}} | ||
app.kubernetes.io/name: {{ include "fluentd.name" . }} | ||
helm.sh/chart: {{ include "fluentd.chart" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
{{- if .Chart.AppVersion }} | ||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} | ||
{{- end }} | ||
app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
{{- end -}} | ||
|
||
{{/* | ||
Selector labels | ||
*/}} | ||
{{- define "fluentd.selectorLabels" -}} | ||
app.kubernetes.io/name: {{ include "fluentd.name" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
{{- end -}} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
{{- if .Values.rbac.create -}} | ||
kind: ClusterRole | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: {{ include "fluentd.fullname" . }} | ||
labels: {{- include "fluentd.labels" . | nindent 4 }} | ||
rules: | ||
- apiGroups: | ||
- "" | ||
resources: | ||
- "namespaces" | ||
- "pods" | ||
verbs: | ||
- "get" | ||
- "watch" | ||
- "list" | ||
{{- end -}} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
{{- if .Values.rbac.create -}} | ||
kind: ClusterRoleBinding | ||
apiVersion: rbac.authorization.k8s.io/v1 | ||
metadata: | ||
name: {{ template "fluentd.fullname" . }} | ||
labels: {{- include "fluentd.labels" . | nindent 4 }} | ||
subjects: | ||
- kind: ServiceAccount | ||
name: {{ template "fluentd.fullname" . }} | ||
namespace: {{ .Release.Namespace }} | ||
roleRef: | ||
kind: ClusterRole | ||
name: {{ template "fluentd.fullname" . }} | ||
apiGroup: rbac.authorization.k8s.io | ||
{{- end -}} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
apiVersion: v1 | ||
kind: ConfigMap | ||
metadata: | ||
name: {{ template "fluentd.fullname" . }} | ||
labels: {{- include "fluentd.labels" . | nindent 4 }} | ||
data: | ||
{{- range $key, $value := .Values.configMaps }} | ||
{{ $key }}: |- | ||
{{ $value | indent 4 }} | ||
{{- end }} | ||
{{- if .Values.metrics.enabled }} | ||
metrics.conf: | | ||
<source> | ||
@type prometheus | ||
port {{ .Values.metrics.service.port }} | ||
</source> | ||
<source> | ||
@type prometheus_monitor | ||
</source> | ||
<source> | ||
@type prometheus_output_monitor | ||
</source> | ||
{{- end }} | ||
{{- if and (.Values.plugins.enabled) (gt (len .Values.plugins.pluginsList) 0) }} | ||
install-plugins.sh: |- | ||
#!/bin/sh | ||
{{- range $plugin := .Values.plugins.pluginsList }} | ||
fluent-gem install {{ $plugin }} | ||
{{- end }} | ||
exec /run.sh | ||
{{- end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,133 @@ | ||
{{- $statefulSet := or (.Values.autoscaling.enabled) (.Values.useStatefulSet) -}} | ||
apiVersion: apps/v1 | ||
{{- if $statefulSet }} | ||
kind: StatefulSet | ||
{{- else}} | ||
kind: Deployment | ||
{{- end}} | ||
metadata: | ||
name: {{ template "fluentd.fullname" . }} | ||
labels: {{- include "fluentd.labels" . | nindent 4 }} | ||
spec: | ||
{{- if not .Values.autoscaling.enabled }} | ||
replicas: {{ .Values.replicaCount }} | ||
{{- end }} | ||
{{- if .Values.autoscaling.enabled }} | ||
serviceName: {{ template "fluentd.name" . }} | ||
{{- end }} | ||
selector: | ||
matchLabels: | ||
{{- include "fluentd.selectorLabels" . | nindent 6 }} | ||
{{- if and .Values.persistence.enabled (not .Values.autoscaling.enabled) }} | ||
strategy: | ||
type: Recreate | ||
{{- end }} | ||
template: | ||
metadata: | ||
labels: | ||
{{- include "fluentd.selectorLabels" . | nindent 8 }} | ||
{{- with .Values.deployment.labels }} | ||
{{ toYaml . | indent 8 }} | ||
{{- end }} | ||
annotations: | ||
checksum/configmap: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} | ||
{{- if .Values.annotations }} | ||
{{- toYaml .Values.annotations | nindent 8 }} | ||
{{- end }} | ||
spec: | ||
{{- if .Values.image.pullSecrets }} | ||
imagePullSecrets: | ||
{{- range $pullSecret := .Values.image.pullSecrets }} | ||
- name: {{ $pullSecret }} | ||
{{- end }} | ||
{{- end }} | ||
terminationGracePeriodSeconds: {{ .Values.terminationGracePeriodSeconds }} | ||
containers: | ||
- name: {{ .Chart.Name }} | ||
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}" | ||
imagePullPolicy: {{ .Values.image.pullPolicy }} | ||
{{- if and (.Values.plugins.enabled) (gt (len .Values.plugins.pluginsList) 0) }} | ||
command: ["/bin/sh", "-c", "/etc/fluent/config.d/install-plugins.sh"] | ||
{{- end }} | ||
env: | ||
{{- range $key, $value := .Values.env }} | ||
- name: {{ $key | quote }} | ||
value: {{ $value | quote }} | ||
{{- end }} | ||
{{- if .Values.extraEnvVars }} | ||
{{ toYaml .Values.extraEnvVars | indent 10 }} | ||
{{- end }} | ||
resources: | ||
{{ toYaml .Values.resources | indent 12 }} | ||
ports: | ||
{{- range $port := .Values.service.ports }} | ||
- name: {{ $port.name }} | ||
containerPort: {{ $port.containerPort }} | ||
protocol: {{ $port.protocol }} | ||
{{- end }} | ||
{{- if .Values.metrics.enabled }} | ||
- name: metrics | ||
containerPort: {{ .Values.metrics.service.port }} | ||
protocol: TCP | ||
{{- end }} | ||
- name: http-input | ||
containerPort: 9880 | ||
protocol: TCP | ||
livenessProbe: | ||
httpGet: | ||
# Use percent encoding for query param. | ||
# The value is {"log": "health check"}. | ||
# the endpoint itself results in a new fluentd | ||
# tag 'fluentd.pod-healthcheck' | ||
path: /fluentd.pod.healthcheck?json=%7B%22log%22%3A+%22health+check%22%7D | ||
port: 9880 | ||
initialDelaySeconds: 5 | ||
timeoutSeconds: 1 | ||
volumeMounts: | ||
- name: config-volume-{{ template "fluentd.fullname" . }} | ||
mountPath: /etc/fluent/config.d | ||
- name: buffer | ||
mountPath: "/var/log/fluentd-buffers" | ||
{{- if .Values.extraVolumeMounts }} | ||
{{ toYaml .Values.extraVolumeMounts | indent 8 }} | ||
{{- end }} | ||
serviceAccountName: {{ template "fluentd.fullname" . }} | ||
volumes: | ||
- name: config-volume-{{ template "fluentd.fullname" . }} | ||
configMap: | ||
name: {{ template "fluentd.fullname" . }} | ||
defaultMode: 0777 | ||
{{- if and .Values.persistence.enabled (not $statefulSet) }} | ||
- name: buffer | ||
persistentVolumeClaim: | ||
claimName: {{ template "fluentd.fullname" . }} | ||
{{- else if (not .Values.persistence.enabled) }} | ||
- name: buffer | ||
emptyDir: {} | ||
{{- end }} | ||
{{- if .Values.extraVolumes }} | ||
{{ toYaml .Values.extraVolumes | indent 8 }} | ||
{{- end }} | ||
{{- with .Values.nodeSelector }} | ||
nodeSelector: | ||
{{ toYaml . | indent 8 }} | ||
{{- end }} | ||
{{- with .Values.affinity }} | ||
affinity: | ||
{{ toYaml . | indent 8 }} | ||
{{- end }} | ||
{{- with .Values.tolerations }} | ||
tolerations: | ||
{{ toYaml . | indent 8 }} | ||
{{- end }} | ||
{{- if and .Values.persistence.enabled ($statefulSet) }} | ||
volumeClaimTemplates: | ||
- metadata: | ||
name: buffer | ||
spec: | ||
accessModes: [{{ .Values.persistence.accessMode }}] | ||
storageClassName: {{ .Values.persistence.storageClass }} | ||
resources: | ||
requests: | ||
storage: {{ .Values.persistence.size }} | ||
{{- end }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{{- if and .Values.autoscaling.enabled}} | ||
apiVersion: autoscaling/v2beta2 | ||
kind: HorizontalPodAutoscaler | ||
metadata: | ||
name: {{ include "fluentd.fullname" . }} | ||
labels: {{- include "fluentd.labels" . | nindent 4 }} | ||
spec: | ||
scaleTargetRef: | ||
apiVersion: apps/v1 | ||
kind: StatefulSet | ||
name: {{ include "fluentd.fullname" . }} | ||
minReplicas: {{ .Values.autoscaling.minReplicas }} | ||
maxReplicas: {{ .Values.autoscaling.maxReplicas }} | ||
metrics: | ||
{{- toYaml .Values.autoscaling.metrics | nindent 4 }} | ||
{{- end }} |
Oops, something went wrong.