Skip to content

Commit

Permalink
remove references to mutate filter (#12084) (#12105)
Browse files Browse the repository at this point in the history
(cherry picked from commit 8b0f906)

Co-authored-by: Colleen McGinnis <[email protected]>
  • Loading branch information
mergify[bot] and colleenmcginnis authored Nov 24, 2023
1 parent d0d2909 commit e6d492b
Showing 1 changed file with 1 addition and 59 deletions.
60 changes: 1 addition & 59 deletions docs/configure/outputs/logstash.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ include::../../shared-kibana-endpoint.asciidoc[tag=shared-kibana-config]
=== {ls} configuration pipeline

Finally, you must create a {ls} configuration pipeline that listens for incoming
APM Server connections, dedots the `data_stream.*` fields, and indexes received events into {es}.
APM Server connections and indexes received events into {es}.

. Use the {logstash-ref}/plugins-inputs-elastic_agent.html[Elastic Agent input plugin] to configure
{ls} to receive events from the APM Server. A minimal `input` config might look like this:
Expand All @@ -68,54 +68,6 @@ input {
}
----

. Use the {logstash-ref}/plugins-filters-mutate.html[Mutate filter plugin] to set up <<apm-data-streams,data streams>>.
Because the {ls} {es} output doesn't understand dotted field notation, you must use this filter to
dedot the default `data_stream.*` fields sent from APM Server to {ls}.
+
[source,conf]
----
filter {
mutate {
rename => {
"[data_stream.type]" => "[data_stream][type]"
"[data_stream.dataset]" => "[data_stream][dataset]"
"[data_stream.namespace]" => "[data_stream][namespace]"
}
}
}
----
+
.Expand to learn more
[%collapsible]
====
****
APM Server sends data stream information to {ls} in the following format:
[source,json]
----
{
"data_stream.dataset": "apm",
"data_stream.type": "traces",
"data_stream.namespace": "default"
}
----
{es} expects to receive data stream information in the following format:
[source,json]
----
"data_stream" {
"dataset": "apm",
"type": "traces",
"dataset": "default"
}
----
The mutation defined above transforms what APM Server sends to {ls} into a data format that {es} understands.
This allows you to automatically route APM data to the appropriate data streams.
****
====

. Use the {logstash-ref}/plugins-outputs-elasticsearch.html[{es} output plugin] to send
events to {es} for indexing. A minimal `output` config might look like this:
+
Expand All @@ -142,16 +94,6 @@ input {
}
}
filter {
mutate {
rename => {
"[data_stream.type]" => "[data_stream][type]"
"[data_stream.dataset]" => "[data_stream][dataset]"
"[data_stream.namespace]" => "[data_stream][namespace]"
}
}
}
output {
elasticsearch {
data_stream => "true"
Expand Down

0 comments on commit e6d492b

Please sign in to comment.