Skip to content

Commit

Permalink
Merge pull request #289 from moleskin-smile/upgrade-ruby-to-2.6-and-f…
Browse files Browse the repository at this point in the history
…luentd-to-1.4

Upgrade Fluentd to 1.4 and Ruby to 2.6
  • Loading branch information
repeatedly authored May 8, 2019
2 parents 0be55b4 + 2bbc20e commit 8c76f51
Show file tree
Hide file tree
Showing 202 changed files with 10,084 additions and 23 deletions.
28 changes: 14 additions & 14 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,20 +12,20 @@

IMAGE_NAME := fluent/fluentd-kubernetes
ALL_IMAGES := \
v1.3/debian-elasticsearch:v1.3.3-debian-elasticsearch-1.8,v1.3-debian-elasticsearch-1 \
v1.3/debian-loggly:v1.3.3-debian-loggly-1.3,v1.3-debian-loggly-1 \
v1.3/debian-logentries:v1.3.3-debian-logentries-1.3,v1.3-debian-logentries-1 \
v1.3/debian-cloudwatch:v1.3.3-debian-cloudwatch-1.4,v1.3-debian-cloudwatch-1 \
v1.3/debian-stackdriver:v1.3.3-debian-stackdriver-1.3,v1.3-debian-stackdriver-1 \
v1.3/debian-s3:v1.3.3-debian-s3-1.3,v1.3-debian-s3-1 \
v1.3/debian-syslog:v1.3.3-debian-syslog-2.0,v1.3-debian-syslog-2 \
v1.3/debian-forward:v1.3.3-debian-forward-1.3,v1.3-debian-forward-1 \
v1.3/debian-gcs:v1.3.3-debian-gcs-1.3,v1.3-debian-gcs-1 \
v1.3/debian-graylog:v1.3.3-debian-graylog-1.4,v1.3-debian-graylog-1 \
v1.3/debian-papertrail:v1.3.3-debian-papertrail-1.3,v1.3-debian-papertrail-1 \
v1.3/debian-logzio:v1.3.3-debian-logzio-1.3,v1.3-debian-logzio-1 \
v1.3/debian-kafka:v1.3.3-debian-kafka-1.4,v1.3-debian-kafka-1 \
v1.3/debian-kinesis:v1.3.3-debian-kinesis-1.3,v1.3-debian-kinesis-1
v1.4/debian-elasticsearch:v1.4.2-debian-elasticsearch-1.0,v1.4-debian-elasticsearch-1 \
v1.4/debian-loggly:v1.4.2-debian-loggly-1.0,v1.4-debian-loggly-1 \
v1.4/debian-logentries:v1.4.2-debian-logentries-1.0,v1.4-debian-logentries-1 \
v1.4/debian-cloudwatch:v1.4.2-debian-cloudwatch-1.0,v1.4-debian-cloudwatch-1 \
v1.4/debian-stackdriver:v1.4.2-debian-stackdriver-1.0,v1.4-debian-stackdriver-1 \
v1.4/debian-s3:v1.4.2-debian-s3-1.0,v1.4-debian-s3-1 \
v1.4/debian-syslog:v1.4.2-debian-syslog-1.0,v1.4-debian-syslog-1 \
v1.4/debian-forward:v1.4.2-debian-forward-1.0,v1.4-debian-forward-1 \
v1.4/debian-gcs:v1.4.2-debian-gcs-1.0,v1.4-debian-gcs-1 \
v1.4/debian-graylog:v1.4.2-debian-graylog-1.0,v1.4-debian-graylog-1 \
v1.4/debian-papertrail:v1.4.2-debian-papertrail-1.0,v1.4-debian-papertrail-1 \
v1.4/debian-logzio:v1.4.2-debian-logzio-1.0,v1.4-debian-logzio-1 \
v1.4/debian-kafka:v1.4.2-debian-kafka-1.0,v1.4-debian-kafka-1 \
v1.4/debian-kinesis:v1.4.2-debian-kinesis-1.0,v1.4-debian-kinesis-1

# <Dockerfile>:<version>,<tag1>,<tag2>,...

Expand Down
14 changes: 14 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,20 @@ See also dockerhub tags page: https://hub.docker.com/r/fluent/fluentd-kubernetes

### Debian

- `v1.4.2-debian-elasticsearch-1.0,v1.4-debian-elasticsearch-1` [docker-image/v1.4/debian-elasticsearch/Dockerfile](docker-image/v1.4/debian-elasticsearch/Dockerfile)
- `v1.4.2-debian-loggly-1.0,v1.4-debian-loggly-1` [docker-image/v1.4/debian-loggly/Dockerfile](docker-image/v1.4/debian-loggly/Dockerfile)
- `v1.4.2-debian-logentries-1.0,v1.4-debian-logentries-1` [docker-image/v1.4/debian-logentries/Dockerfile](docker-image/v1.4/debian-logentries/Dockerfile)
- `v1.4.2-debian-cloudwatch-1.0,v1.4-debian-cloudwatch-1` [docker-image/v1.4/debian-cloudwatch/Dockerfile](docker-image/v1.4/debian-cloudwatch/Dockerfile)
- `v1.4.2-debian-stackdriver-1.0,v1.4-debian-stackdriver-1` [docker-image/v1.4/debian-stackdriver/Dockerfile](docker-image/v1.4/debian-stackdriver/Dockerfile)
- `v1.4.2-debian-s3-1.0,v1.4-debian-s3-1` [docker-image/v1.4/debian-s3/Dockerfile](docker-image/v1.4/debian-s3/Dockerfile)
- `v1.4.2-debian-syslog-1.0,v1.4-debian-syslog-1` [docker-image/v1.4/debian-syslog/Dockerfile](docker-image/v1.4/debian-syslog/Dockerfile)
- `v1.4.2-debian-forward-1.0,v1.4-debian-forward-1` [docker-image/v1.4/debian-forward/Dockerfile](docker-image/v1.4/debian-forward/Dockerfile)
- `v1.4.2-debian-gcs-1.0,v1.4-debian-gcs-1` [docker-image/v1.4/debian-gcs/Dockerfile](docker-image/v1.4/debian-gcs/Dockerfile)
- `v1.4.2-debian-graylog-1.0,v1.4-debian-graylog-1` [docker-image/v1.4/debian-graylog/Dockerfile](docker-image/v1.4/debian-graylog/Dockerfile)
- `v1.4.2-debian-papertrail-1.0,v1.4-debian-papertrail-1` [docker-image/v1.4/debian-papertrail/Dockerfile](docker-image/v1.4/debian-papertrail/Dockerfile)
- `v1.4.2-debian-logzio-1.0,v1.4-debian-logzio-1` [docker-image/v1.4/debian-logzio/Dockerfile](docker-image/v1.4/debian-logzio/Dockerfile)
- `v1.4.2-debian-kafka-1.0,v1.4-debian-kafka-1` [docker-image/v1.4/debian-kafka/Dockerfile](docker-image/v1.4/debian-kafka/Dockerfile)
- `v1.4.2-debian-kinesis-1.0,v1.4-debian-kinesis-1` [docker-image/v1.4/debian-kinesis/Dockerfile](docker-image/v1.4/debian-kinesis/Dockerfile)
- `v1.3.3-debian-elasticsearch-1.8,v1.3-debian-elasticsearch-1` [docker-image/v1.3/debian-elasticsearch/Dockerfile](docker-image/v1.3/debian-elasticsearch/Dockerfile)
- `v1.3.3-debian-loggly-1.3,v1.3-debian-loggly-1` [docker-image/v1.3/debian-loggly/Dockerfile](docker-image/v1.3/debian-loggly/Dockerfile)
- `v1.3.3-debian-logentries-1.3,v1.3-debian-logentries-1` [docker-image/v1.3/debian-logentries/Dockerfile](docker-image/v1.3/debian-logentries/Dockerfile)
Expand Down
1 change: 1 addition & 0 deletions docker-image/v1.4/debian-cloudwatch/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
**/*.gitkeep
49 changes: 49 additions & 0 deletions docker-image/v1.4/debian-cloudwatch/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# AUTOMATICALLY GENERATED
# DO NOT EDIT THIS FILE DIRECTLY, USE /templates/Dockerfile.erb

FROM fluent/fluentd:v1.4.2-debian-2.0

LABEL maintainer="Eduardo Silva <[email protected]>"
USER root
WORKDIR /home/fluent
ENV PATH /fluentd/vendor/bundle/ruby/2.6.0/bin:$PATH
ENV GEM_PATH /fluentd/vendor/bundle/ruby/2.6.0
ENV GEM_HOME /fluentd/vendor/bundle/ruby/2.6.0
# skip runtime bundler installation
ENV FLUENTD_DISABLE_BUNDLER_INJECTION 1

COPY Gemfile* /fluentd/
RUN buildDeps="sudo make gcc g++ libc-dev libffi-dev" \
&& apt-get update \
&& apt-get upgrade -y \
&& apt-get install \
-y --no-install-recommends \
$buildDeps net-tools \
&& gem install bundler --version 1.16.2 \
&& bundle config silence_root_warning true \
&& bundle install --gemfile=/fluentd/Gemfile --path=/fluentd/vendor/bundle \
&& SUDO_FORCE_REMOVE=yes \
apt-get purge -y --auto-remove \
-o APT::AutoRemove::RecommendsImportant=false \
$buildDeps \
&& rm -rf /var/lib/apt/lists/* \
&& gem sources --clear-all \
&& rm -rf /tmp/* /var/tmp/* /usr/lib/ruby/gems/*/cache/*.gem

# Copy configuration files
COPY ./conf/fluent.conf /fluentd/etc/
COPY ./conf/systemd.conf /fluentd/etc/
COPY ./conf/kubernetes.conf /fluentd/etc/
COPY ./conf/prometheus.conf /fluentd/etc/
RUN touch /fluentd/etc/disable.conf

# Copy plugins
COPY plugins /fluentd/plugins/
COPY entrypoint.sh /fluentd/entrypoint.sh

# Environment variables
ENV FLUENTD_OPT=""
ENV FLUENTD_CONF="fluent.conf"

# Overwrite ENTRYPOINT to run fluentd as root for /var/log / /var/lib
ENTRYPOINT ["tini", "--", "/fluentd/entrypoint.sh"]
17 changes: 17 additions & 0 deletions docker-image/v1.4/debian-cloudwatch/Gemfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# AUTOMATICALLY GENERATED
# DO NOT EDIT THIS FILE DIRECTLY, USE /templates/Gemfile.erb

source "https://rubygems.org"

gem "fluentd", "1.4.2"
gem "oj", "3.5.1"
gem "fluent-plugin-multi-format-parser", "~> 1.0.0"
gem "fluent-plugin-concat", "~> 2.3.0"
gem "fluent-plugin-grok-parser", "~> 2.5.0"
gem "fluent-plugin-prometheus", "~> 1.3.0"
gem "fluent-plugin-rewrite-tag-filter", "~> 2.1.0"
gem "aws-sdk-cloudwatchlogs", "~> 1.0"
gem "fluent-plugin-cloudwatch-logs", "~> 0.7.3"
gem "fluent-plugin-kubernetes_metadata_filter", "~> 2.1.4"
gem "ffi"
gem "fluent-plugin-systemd", "~> 1.0.1"
132 changes: 132 additions & 0 deletions docker-image/v1.4/debian-cloudwatch/Gemfile.lock
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
GEM
remote: https://rubygems.org/
specs:
activesupport (5.2.3)
concurrent-ruby (~> 1.0, >= 1.0.2)
i18n (>= 0.7, < 2)
minitest (~> 5.1)
tzinfo (~> 1.1)
addressable (2.6.0)
public_suffix (>= 2.0.2, < 4.0)
aws-eventstream (1.0.3)
aws-partitions (1.155.0)
aws-sdk-cloudwatchlogs (1.17.0)
aws-sdk-core (~> 3, >= 3.48.2)
aws-sigv4 (~> 1.1)
aws-sdk-core (3.48.6)
aws-eventstream (~> 1.0, >= 1.0.2)
aws-partitions (~> 1.0)
aws-sigv4 (~> 1.1)
jmespath (~> 1.0)
aws-sigv4 (1.1.0)
aws-eventstream (~> 1.0, >= 1.0.2)
concurrent-ruby (1.1.5)
cool.io (1.5.4)
dig_rb (1.0.1)
domain_name (0.5.20180417)
unf (>= 0.0.5, < 1.0.0)
ffi (1.10.0)
fluent-config-regexp-type (1.0.0)
fluentd (> 1.0.0, < 2)
fluent-plugin-cloudwatch-logs (0.7.3)
aws-sdk-cloudwatchlogs (~> 1.0)
fluentd (>= 0.14.15)
fluent-plugin-concat (2.3.0)
fluentd (>= 0.14.0, < 2)
fluent-plugin-grok-parser (2.5.1)
fluentd (>= 0.14.6, < 2)
fluent-plugin-kubernetes_metadata_filter (2.1.6)
fluentd (>= 0.14.0, < 2)
kubeclient (~> 1.1.4)
lru_redux
fluent-plugin-multi-format-parser (1.0.0)
fluentd (>= 0.14.0, < 2)
fluent-plugin-prometheus (1.3.0)
fluentd (>= 0.14.20, < 2)
prometheus-client
fluent-plugin-rewrite-tag-filter (2.1.1)
fluent-config-regexp-type
fluentd (>= 0.14.2, < 2)
fluent-plugin-systemd (1.0.2)
fluentd (>= 0.14.11, < 2)
systemd-journal (~> 1.3.2)
fluentd (1.4.2)
cool.io (>= 1.4.5, < 2.0.0)
dig_rb (~> 1.0.0)
http_parser.rb (>= 0.5.1, < 0.7.0)
msgpack (>= 0.7.0, < 2.0.0)
serverengine (>= 2.0.4, < 3.0.0)
sigdump (~> 0.2.2)
strptime (>= 0.2.2, < 1.0.0)
tzinfo (~> 1.0)
tzinfo-data (~> 1.0)
yajl-ruby (~> 1.0)
http (0.9.8)
addressable (~> 2.3)
http-cookie (~> 1.0)
http-form_data (~> 1.0.1)
http_parser.rb (~> 0.6.0)
http-cookie (1.0.3)
domain_name (~> 0.5)
http-form_data (1.0.3)
http_parser.rb (0.6.0)
i18n (1.6.0)
concurrent-ruby (~> 1.0)
jmespath (1.4.0)
kubeclient (1.1.4)
activesupport
http (= 0.9.8)
recursive-open-struct (= 1.0.0)
rest-client
lru_redux (1.1.0)
mime-types (3.2.2)
mime-types-data (~> 3.2015)
mime-types-data (3.2019.0331)
minitest (5.11.3)
msgpack (1.2.10)
netrc (0.11.0)
oj (3.5.1)
prometheus-client (0.9.0)
quantile (~> 0.2.1)
public_suffix (3.0.3)
quantile (0.2.1)
recursive-open-struct (1.0.0)
rest-client (2.0.2)
http-cookie (>= 1.0.2, < 2.0)
mime-types (>= 1.16, < 4.0)
netrc (~> 0.8)
serverengine (2.1.1)
sigdump (~> 0.2.2)
sigdump (0.2.4)
strptime (0.2.3)
systemd-journal (1.3.3)
ffi (~> 1.9)
thread_safe (0.3.6)
tzinfo (1.2.5)
thread_safe (~> 0.1)
tzinfo-data (1.2019.1)
tzinfo (>= 1.0.0)
unf (0.1.4)
unf_ext
unf_ext (0.0.7.6)
yajl-ruby (1.4.1)

PLATFORMS
ruby

DEPENDENCIES
aws-sdk-cloudwatchlogs (~> 1.0)
ffi
fluent-plugin-cloudwatch-logs (~> 0.7.3)
fluent-plugin-concat (~> 2.3.0)
fluent-plugin-grok-parser (~> 2.5.0)
fluent-plugin-kubernetes_metadata_filter (~> 2.1.4)
fluent-plugin-multi-format-parser (~> 1.0.0)
fluent-plugin-prometheus (~> 1.3.0)
fluent-plugin-rewrite-tag-filter (~> 2.1.0)
fluent-plugin-systemd (~> 1.0.1)
fluentd (= 1.4.2)
oj (= 3.5.1)

BUNDLED WITH
1.17.2
18 changes: 18 additions & 0 deletions docker-image/v1.4/debian-cloudwatch/conf/fluent.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@

# AUTOMATICALLY GENERATED
# DO NOT EDIT THIS FILE DIRECTLY, USE /templates/conf/fluent.conf.erb

@include "#{ENV['FLUENTD_SYSTEMD_CONF'] || 'systemd'}.conf"
@include "#{ENV['FLUENTD_PROMETHEUS_CONF'] || 'prometheus'}.conf"
@include kubernetes.conf
@include conf.d/*.conf

<match **>
@type cloudwatch_logs
@id out_cloudwatch_logs
log_group_name "#{ENV['LOG_GROUP_NAME']}"
auto_create_stream true
use_tag_as_stream true
json_handler yajl # To avoid UndefinedConversionError
log_rejected_request "#{ENV['LOG_REJECTED_REQUEST']}" # Log rejected request for missing parts
</match>
Loading

0 comments on commit 8c76f51

Please sign in to comment.