Skip to content

Commit

Permalink
fix: add tail parser type to support containerd
Browse files Browse the repository at this point in the history
The `containerd` runtime generate logs as a non-JSON string.  When switched to `containerd` runtime, `fluentd` will fail to parse any non-JSON log message and produce a large amount of parse error messages in its container logs.

Here is an open issue at `fluentd` repo:  fluent/fluentd-kubernetes-daemonset#412

**docker** runtime (a valid JSON string)
`{"log":"2023-05-02 20:17:16 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0\n","stream":"stdout","time":"2023-05-02T20:17:16.666667387Z"}`

**containerd** runtime (just a string)
`2023-05-02T20:17:28.143532061Z stdout F 2023-05-02 20:17:28 +0000 [info]: #0 [filter_kube_metadata_host] stats - namespace_cache_size: 0, pod_cache_size: 0`

Here is an example of a short entry from a `fluentd` container log.
```
2023-05-02 19:51:40 +0000 [warn]: #0 [in_tail_fluentd_logs] pattern not matched: "2023-05-02T19:51:17.411234908Z stdout F \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\""
```
  • Loading branch information
htquach authored May 3, 2023
1 parent 0e3a868 commit 3b3657a
Showing 1 changed file with 4 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ data:
tag *
read_from_head true
<parse>
@type json
time_format %Y-%m-%dT%H:%M:%S.%NZ
@type "#{ENV['FLUENT_CONTAINER_TAIL_PARSER_TYPE'] || 'json'}"
time_format %Y-%m-%dT%H:%M:%S.%N%:z
</parse>
</source>
Expand All @@ -86,8 +86,8 @@ data:
tag *
read_from_head true
<parse>
@type json
time_format %Y-%m-%dT%H:%M:%S.%NZ
@type "#{ENV['FLUENT_CONTAINER_TAIL_PARSER_TYPE'] || 'json'}"
time_format %Y-%m-%dT%H:%M:%S.%N%:z
</parse>
</source>
Expand Down

0 comments on commit 3b3657a

Please sign in to comment.