transport-security.md

Transport Security

Fluent Bit provides integrated support for Transport Layer Security (TLS) and its predecessor Secure Sockets Layer (SSL). This section refers only to TLS for both implementations.

Both input and output plugins that perform Network I/O can optionally enable TLS and configure the behavior. The following table describes the properties available:

Property	Description	Default
tls	Enable or disable TLS support.	Off
tls.verify	Force certificate validation.	On
tls.verify_hostname	Force TLS verification of host names.	Off
tls.debug	Set TLS debug verbosity level. Accepted values: 0 (No debug), 1 (Error), 2 (State change), 3 (Informational) and 4. (Verbose)	1
tls.ca_file	Absolute path to CA certificate file.	none
tls.ca_path	Absolute path to scan for certificate files.	none
tls.crt_file	Absolute path to Certificate file.	none
tls.key_file	Absolute path to private Key file.	none
tls.key_passwd	Optional password for tls.key_file file.	none
tls.vhost	Hostname to be used for TLS SNI extension.	none

To use TLS on input plugins, you must provide both a certificate and a private key.

The listed properties can be enabled in the configuration file, specifically in each output plugin section or directly through the command line.

The following output plugins can take advantage of the TLS feature:

• Amazon S3
• Apache SkyWalking
• Azure
• Azure Blob
• Azure Data Explorer (Kusto)
• Azure Logs Ingestion API
• BigQuery
• Dash0
• Datadog
• Elasticsearch
• Forward
• GELF
• Google Chronicle
• HTTP
• InfluxDB
• Kafka REST Proxy
• LogDNA
• Loki
• New Relic
• OpenSearch
• OpenTelemetry
• Oracle Cloud Infrastructure Logging Analytics
• Prometheus Remote Write
• Slack
• Splunk
• Stackdriver
• Syslog
• TCP & TLS
• Treasure Data
• WebSocket

The following input plugins can take advantage of the TLS feature:

• Docker Events
• Elasticsearch (Bulk API)
• Forward
• Health
• HTTP
• Kubernetes Events
• MQTT
• NGINX Exporter Metrics
• OpenTelemetry
• Prometheus Scrape Metrics
• Prometheus Remote Write
• Splunk (HTTP HEC)
• Syslog
• TCP

In addition, other plugins implement a subset of TLS support, with restricted configuration:

• Kubernetes Filter

Example: enable TLS on HTTP input

By default, the HTTP input plugin uses plain TCP. Run the following command to enable TLS:

./bin/fluent-bit -i http \
           -p port=9999 \
           -p tls=on \
           -p tls.verify=off \
           -p tls.crt_file=self_signed.crt \
           -p tls.key_file=self_signed.key \
           -o stdout \
           -m '*'

In the previous command, the two properties tls and tls.verify are set for demonstration purposes. Always enable verification in production environments.

The same behavior can be accomplished using a configuration file:

[INPUT]
    name http
    port 9999
    tls on
    tls.verify off
    tls.crt_file self_signed.crt
    tls.key_file self_signed.key

[OUTPUT]
    Name       stdout
    Match      *

Example: enable TLS on HTTP output

By default, the HTTP output plugin uses plain TCP. Run the following command to enable TLS:

fluent-bit -i cpu -t cpu -o http://192.168.2.3:80/something \
    -p tls=on         \
    -p tls.verify=off \
    -m '*'

In the previous command, the properties tls and tls.verify are enabled for demonstration purposes. Always enable verification in production environments.

The same behavior can be accomplished using a configuration file:

[INPUT]
    Name  cpu
    Tag   cpu

[OUTPUT]
    Name       http
    Match      *
    Host       192.168.2.3
    Port       80
    URI        /something
    tls        On
    tls.verify Off

Tips and Tricks

Generate a self signed certificates for testing purposes

The following command generates a 4096 bit RSA key pair and a certificate that's signed using SHA-256 with the expiration date set to 30 days in the future. In this example, test.host.net is set as the common name. This example opts out of DES, so the private key is stored in plain text.

openssl req -x509 \
            -newkey rsa:4096 \
            -sha256 \
            -nodes \
            -keyout self_signed.key \
            -out self_signed.crt \
            -subj "/CN=test.host.net"

Connect to virtual servers using TLS

Fluent Bit supports TLS server name indication. If you are serving multiple host names on a single IP address (for example, using virtual hosting), you can make use of tls.vhost to connect to a specific hostname.

[INPUT]
    Name  cpu
    Tag   cpu

[OUTPUT]
    Name        forward
    Match       *
    Host        192.168.10.100
    Port        24224
    tls         On
    tls.verify  On
    tls.ca_file /etc/certs/fluent.crt
    tls.vhost   fluent.example.com

Verify subjectAltName

By default, TLS verification of host names isn't done automatically. As an example, you can extract the X509v3 Subject Alternative Name from a certificate:

X509v3 Subject Alternative Name:
    DNS:my.fluent-aggregator.net

This certificate covers only my.fluent-aggregator.net so if you use a different hostname it should fail.

To fully verify the alternative name and demonstrate the failure, enable tls.verify_hostname:

[INPUT]
    Name  cpu
    Tag   cpu

[OUTPUT]
    Name                forward
    Match               *
    Host                other.fluent-aggregator.net
    Port                24224
    tls                 On
    tls.verify          On
    tls.verify_hostname on
    tls.ca_file         /path/to/fluent-x509v3-alt-name.crt

This outgoing connect will fail and disconnect:

[2024/06/17 16:51:31] [error] [tls] error: unexpected EOF with reason: certificate verify failed
[2024/06/17 16:51:31] [debug] [upstream] connection #50 failed to other.fluent-aggregator.net:24224
[2024/06/17 16:51:31] [error] [output:forward:forward.0] no upstream connections available