From 6be8489a7b46ce600c4e551ab8a1138bc664210a Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Thu, 30 May 2024 20:08:00 -0400 Subject: [PATCH 1/5] Initial docs for agent OTel mode --- .../running-otel-agent-configuration.asciidoc | 51 +++++++++++++++++++ .../running-otel-agent-quickstart.asciidoc | 6 +++ .../elastic-agent/running-otel-agent.asciidoc | 20 ++++++++ docs/en/ingest-management/index.asciidoc | 6 +++ 4 files changed, 83 insertions(+) create mode 100644 docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc create mode 100644 docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc create mode 100644 docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc diff --git a/docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc b/docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc new file mode 100644 index 000000000..968625c5f --- /dev/null +++ b/docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc @@ -0,0 +1,51 @@ +[[running-otel-agent-configuration]] += {agent} OTel configuration + +preview::[] + +When running in `otel` mode, {agent} supports a standard OTel Collector configuration. + +You can provide the OTel configuration to {agent} using any of the following four schemes: + +* `file` - The configuration is read from a file. +* `env` - The configuration is read from an environment variable. +* `yaml` - The configuration is read from a standard YAML-formatted file. +* `http` - The configuration is read from an HTTP URL. + +To help you get started, when you download an {agent} install package, the package comes with a sample OTel configuration file, `otel.yml` located in the package base directory. + +[source,shell] +---- +receivers: <1> + filelog: + include: [ /var/log/system.log ] + start_at: beginning + +processors: <2> + resource: + attributes: + - key: service.name + action: insert + value: elastic-otel-test + +exporters: <3> + debug: + verbosity: detailed + sampling_initial: 10000 + sampling_thereafter: 10000 + +service: <4> + pipelines: + logs: + receivers: [filelog] + processors: [resource] + exporters: + - debug +---- + +<1> link:https://opentelemetry.io/docs/collector/configuration/#receivers[Receivers] collect telemetry from one or more sources. +<2> link:https://opentelemetry.io/docs/collector/configuration/#processors[Processors] take the data collected by receivers and modify or transform it before sending it to the exporters. +<3> link:https://opentelemetry.io/docs/collector/configuration/#exporters[Exporters] send data to one or more backends or destinations. +<4> The link:https://opentelemetry.io/docs/collector/configuration/#connectors[Service] The section is used to configure what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it's not enabled. + +For full details about setting up this file, refer to link:https://opentelemetry.io/docs/collector/configuration/[Configuration] in the OpenTelemetry documentation. \ No newline at end of file diff --git a/docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc b/docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc new file mode 100644 index 000000000..9a700aba7 --- /dev/null +++ b/docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc @@ -0,0 +1,6 @@ +[[running-otel-agent-quickstart]] += {agent} OTel configuration + +preview::[] + +quick start \ No newline at end of file diff --git a/docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc b/docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc new file mode 100644 index 000000000..914dfb37a --- /dev/null +++ b/docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc @@ -0,0 +1,20 @@ +[[running-otel-agent]] += Running {agent} as an OTel Collector + +preview::[] + +The link:https://opentelemetry.io/docs/collector/[OpenTelemetry Collector] is a vendor-neutral way to receive, process, and export telemetry data. {agent} includes an embedded OTel Collector, enabling you to instrument your applications and infrastructure once, and send data to multiple vendors and backends. + +When you run {agent} in `otel` mode it supports the standard OTel Collector configuration format that defines a set of receivers, processors, exporters, and connectors. Logs, metrics, and traces can be ingested using OpenTelemetry data formats. + +[discrete] +== Run {agent} in OTel mode + +To run the agent in OTel mode, specify the `otel` parameter and + + +To learn more refer to: + +* <> +* <> + diff --git a/docs/en/ingest-management/index.asciidoc b/docs/en/ingest-management/index.asciidoc index 8adafbfcc..ab0f2e078 100644 --- a/docs/en/ingest-management/index.asciidoc +++ b/docs/en/ingest-management/index.asciidoc @@ -65,6 +65,12 @@ include::elastic-agent/configuration/env/container-envs.asciidoc[leveloffset=+3] include::elastic-agent/install-agent-msi.asciidoc[leveloffset=+2] +include::elastic-agent/running-otel-agent.asciidoc[leveloffset=+2] + +include::elastic-agent/running-otel-agent-quickstart.asciidoc[leveloffset=+3] + +include::elastic-agent/running-otel-agent-configuration.asciidoc[leveloffset=+3] + include::elastic-agent/installation-layout.asciidoc[leveloffset=+2] include::fleet/air-gapped.asciidoc[leveloffset=+2] From 08980dc053efbbeff8a72f981034eb43eb1eb1c6 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Fri, 31 May 2024 14:55:34 -0400 Subject: [PATCH 2/5] Add steps; add command syntax --- docs/en/ingest-management/commands.asciidoc | 58 +++++++ ...idoc => otel-agent-configuration.asciidoc} | 2 +- .../elastic-agent/otel-agent.asciidoc | 145 ++++++++++++++++++ .../running-otel-agent-quickstart.asciidoc | 6 - .../elastic-agent/running-otel-agent.asciidoc | 20 --- docs/en/ingest-management/index.asciidoc | 8 +- 6 files changed, 207 insertions(+), 32 deletions(-) rename docs/en/ingest-management/elastic-agent/{running-otel-agent-configuration.asciidoc => otel-agent-configuration.asciidoc} (98%) create mode 100644 docs/en/ingest-management/elastic-agent/otel-agent.asciidoc delete mode 100644 docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc delete mode 100644 docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc diff --git a/docs/en/ingest-management/commands.asciidoc b/docs/en/ingest-management/commands.asciidoc index d5e829cb1..6a6a00ab7 100644 --- a/docs/en/ingest-management/commands.asciidoc +++ b/docs/en/ingest-management/commands.asciidoc @@ -28,6 +28,7 @@ invoking the wrong binary. * <> * <> * <> +* <> [technical preview] * <> * <> * <> @@ -834,6 +835,63 @@ elastic-agent install --url=https://fleet-server:8220 \ --certificate-authorities=/path/to/ca.crt ---- +++++ +
+++++ + +[discrete] +[[elastic-agent-otel-command]] +== elastic-agent otel + +preview::[] + +Run {agent} as an <>. + +[discrete] +=== Synopsis + +[source,shell] +---- +elastic-agent otel [flags] +elastic-agent otel [command] +---- + +[discrete] +=== Available commands + +`--validate`:: +Validates the OpenTelemetry collector configuration without running the collector. + +[discrete] +=== Flags + +`--config --config=file:/path/to/first --config=file:path/to/second`:: +Locations to the config file(s). Note that only a single location can be set per flag entry, for example `--config=file:/path/to/first --config=file:path/to/second`. + +`--feature-gates flag`:: +Comma-delimited list of feature gate identifiers. Prefix with `-` to disable the feature. Prefixing with `+` or no prefix will enable the feature. + +`-h, --help`:: +Get help for the `otel` sub-command. Use `elastic-agent otel [command] --help` for more information about a command. + +`--set stringArray`:: +Set an arbitrary component config property. The component has to be defined in the configuration file and the flag has a higher precedence. Array configuration properties are overridden and maps are joined. For example, `--set=processors.batch.timeout=2s`. + +[discrete] +=== Examples + +Run {agent} as on OTel Collector using the supplied `otel.yml` configuration file. +[source,shell] +---- +./elastic-agent otel --config otel.yml +---- + +Change the default verbosity setting in the {agent} OTel configuration from `detailed` to `normal`. + +[source,shell] +---- +./elastic-agent otel --config otel.yml --set "exporters::debug::verbosity=normal" +---- ++++
diff --git a/docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc similarity index 98% rename from docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc rename to docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc index 968625c5f..cac7daaac 100644 --- a/docs/en/ingest-management/elastic-agent/running-otel-agent-configuration.asciidoc +++ b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc @@ -1,4 +1,4 @@ -[[running-otel-agent-configuration]] +[[otel-agent-configuration]] = {agent} OTel configuration preview::[] diff --git a/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc new file mode 100644 index 000000000..925fe54b1 --- /dev/null +++ b/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc @@ -0,0 +1,145 @@ +[[otel-agent]] += Run {agent} as an OTel Collector + +preview::[] + +The link:https://opentelemetry.io/docs/collector/[OpenTelemetry Collector] is a vendor-neutral way to receive, process, and export telemetry data. {agent} includes an embedded OTel Collector, enabling you to instrument your applications and infrastructure once, and send data to multiple vendors and backends. + +When you run {agent} in `otel` mode it supports the standard OTel Collector configuration format that defines a set of receivers, processors, exporters, and connectors. Logs, metrics, and traces can be ingested using OpenTelemetry data formats. + +The following steps will help you to get set up quickly using {agent} as an OTel Collector. Note that all steps are run with `sudo` as it's generally recommended to run {agent} with administrative privileges. + +[discrete] +== Run {agent} in OTel mode + +For simplicity, these steps demonstrate OTel mode using standalone {agent}. The install steps for {fleet}-managed {agent} are very similar. Refer to the <> section for the full set of requirements and steps for any install scenario. + +. On your host, download and extract the {agent} installation package. ++ +-- +// tag::install-elastic-agent[] + +include::{ingest-docs-root}/docs/en/ingest-management/tab-widgets/download-widget.asciidoc[] + +// end::install-elastic-agent[] +-- ++ +The commands shown are for AMD platforms, but ARM packages are also available. +Refer to the {agent} https://www.elastic.co/downloads/elastic-agent[downloads page] +for the full list of available packages. + +. Change into the agent package directory and run {agent} with the `otel` subcommand and the `-h` option to view the command syntax and available parameters: ++ +[source,shell] +-- +cd +sudo ./elastic-agent otel -h +-- ++ +TIP: You can also refer to the <> command documentation for the full command syntax. + +. View or update the OTel configuration file. ++ +A basic OTel configuration file named `otel.yml` is included in the {agent} package. You can view or update it using any text editor. ++ +[source,yaml] +-- +receivers: + filelog: + include: [ /var/log/system.log ] + start_at: beginning + +processors: + resource: + attributes: + - key: service.name + action: insert + value: elastic-otel-test + +exporters: + debug: + verbosity: detailed + sampling_initial: 10000 + sampling_thereafter: 10000 + +service: + pipelines: + logs: + receivers: [filelog] + processors: [resource] + exporters: + - debug% +-- ++ +By default, the default OTel configuration sets {agent} to: + +* Read the contents of `/var/log/system.log`. +* Run a processor to insert `elastic-otel-test` into the output. +* Send the data with a `detailed` level of verbosity. ++ +Refer to <> to learn more about setting up this file. + +. [Optional] Override a setting in the OTel configuration ++ +You can use the `set` flag to manually override any setting defined in the OTel configuration file. For example, run the following command to change the default verbosity setting from `detailed` to `normal`. ++ +[source,shell] +---- +sudo ./elastic-agent otel --config otel.yml --set "exporters::debug::verbosity=normal" +---- + +. Validate the configuration ++ +To validate the OTel configuration for your {agent}, run the `otel` command with the `validate` option and the path to the config file: ++ +[source,shell] +-- +sudo ./elastic-agent otel validate --config otel.yml +-- ++ +If no error is returned, the configuration file is determined to be valid. + +. Run {agent} ++ +Now, run {agent} with OTel mode configured: ++ +[source,shell] +-- +sudo ./elastic-agent otel --config otel.yml +-- ++ +You should see a response indicating that {agent} has started successfully in OTel mode: ++ +[source,shell] +-- +Starting in otel mode +2024-05-31T12:39:43.047-0400 info service@v0.94.1/telemetry.go:59 Setting up own telemetry... +2024-05-31T12:39:43.049-0400 info service@v0.94.1/telemetry.go:104 Serving metrics {"address": ":8888", "level": "Basic"} +2024-05-31T12:39:43.052-0400 info exporter@v0.94.1/exporter.go:275 Development component. May change in the future. {"kind": "exporter", "data_type": "logs", "name": "debug"} +2024-05-31T12:39:43.055-0400 info service@v0.94.1/service.go:140 Starting ./elastic-agent... {"Version": "8.13.4", "NumCPU": 10} +2024-05-31T12:39:43.055-0400 info extensions/extensions.go:34 Starting extensions... +2024-05-31T12:39:43.056-0400 info adapter/receiver.go:45 Starting stanza receiver {"kind": "receiver", "name": "filelog", "data_type": "logs"} +2024-05-31T12:39:43.057-0400 info service@v0.94.1/service.go:166 Everything is ready. Begin running and processing data. +2024-05-31T12:39:43.057-0400 warn localhostgate/featuregate.go:63 The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new default. {"feature gate ID": "component.UseLocalHostAsDefaultHost"} +2024-05-31T12:39:43.261-0400 info fileconsumer/file.go:261 Started watching file {"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "/var/log/system.log"} +-- + +. Check the {agent} status ++ +Finally, confirm that {agent} is running as expected. Open a second terminal shell and run the `status` command. ++ +[source,shell] +-- +sudo ./elastic-agent status +-- ++ +The returned status indicates that {agent} is healthy and not enrolled in {fleet} (which is expected because in this example we're using standalone {agent}). ++ +[source,shell] +-- +┌─ fleet +│ └─ status: (STOPPED) Not enrolled into Fleet +└─ elastic-agent + └─ status: (HEALTHY) Running +-- + diff --git a/docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc b/docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc deleted file mode 100644 index 9a700aba7..000000000 --- a/docs/en/ingest-management/elastic-agent/running-otel-agent-quickstart.asciidoc +++ /dev/null @@ -1,6 +0,0 @@ -[[running-otel-agent-quickstart]] -= {agent} OTel configuration - -preview::[] - -quick start \ No newline at end of file diff --git a/docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc b/docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc deleted file mode 100644 index 914dfb37a..000000000 --- a/docs/en/ingest-management/elastic-agent/running-otel-agent.asciidoc +++ /dev/null @@ -1,20 +0,0 @@ -[[running-otel-agent]] -= Running {agent} as an OTel Collector - -preview::[] - -The link:https://opentelemetry.io/docs/collector/[OpenTelemetry Collector] is a vendor-neutral way to receive, process, and export telemetry data. {agent} includes an embedded OTel Collector, enabling you to instrument your applications and infrastructure once, and send data to multiple vendors and backends. - -When you run {agent} in `otel` mode it supports the standard OTel Collector configuration format that defines a set of receivers, processors, exporters, and connectors. Logs, metrics, and traces can be ingested using OpenTelemetry data formats. - -[discrete] -== Run {agent} in OTel mode - -To run the agent in OTel mode, specify the `otel` parameter and - - -To learn more refer to: - -* <> -* <> - diff --git a/docs/en/ingest-management/index.asciidoc b/docs/en/ingest-management/index.asciidoc index ab0f2e078..d668308fc 100644 --- a/docs/en/ingest-management/index.asciidoc +++ b/docs/en/ingest-management/index.asciidoc @@ -63,13 +63,11 @@ include::elastic-agent/ingest-pipeline-kubernetes.asciidoc[leveloffset=+3] include::elastic-agent/configuration/env/container-envs.asciidoc[leveloffset=+3] -include::elastic-agent/install-agent-msi.asciidoc[leveloffset=+2] - -include::elastic-agent/running-otel-agent.asciidoc[leveloffset=+2] +include::elastic-agent/otel-agent.asciidoc[leveloffset=+2] -include::elastic-agent/running-otel-agent-quickstart.asciidoc[leveloffset=+3] +include::elastic-agent/otel-agent-configuration.asciidoc[leveloffset=+3] -include::elastic-agent/running-otel-agent-configuration.asciidoc[leveloffset=+3] +include::elastic-agent/install-agent-msi.asciidoc[leveloffset=+2] include::elastic-agent/installation-layout.asciidoc[leveloffset=+2] From bd9d71d8ec1852f1acdebd867de45793e9017111 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Mon, 3 Jun 2024 15:55:43 -0400 Subject: [PATCH 3/5] touchup --- .../elastic-agent/otel-agent-configuration.asciidoc | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc index cac7daaac..76355bd4c 100644 --- a/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc +++ b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc @@ -7,10 +7,11 @@ When running in `otel` mode, {agent} supports a standard OTel Collector configur You can provide the OTel configuration to {agent} using any of the following four schemes: -* `file` - The configuration is read from a file. -* `env` - The configuration is read from an environment variable. -* `yaml` - The configuration is read from a standard YAML-formatted file. -* `http` - The configuration is read from an HTTP URL. +* `file` - The configuration is read from a file. For example, `file:path/to/config.yaml`. +* `env` - The configuration is read from an environment variable. For example, `env:MY_CONFIG_IN_AN_ENVVAR`. +* `yaml` - The configuration is read from a standard YAML-formatted file. For example, `yaml:exporters::debug::verbosity: detailed`. +* `http` - The configuration is read from an HTTP URL. For example, `http://www.example.com`. +// Perhaps we can improve these examples to make them more clear. I'm not sure I understand what the configuration would look like when passed as a URL, for example. To help you get started, when you download an {agent} install package, the package comes with a sample OTel configuration file, `otel.yml` located in the package base directory. From f13617a05d34d769c9c9d2209a35e5b0f3e36654 Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Wed, 12 Jun 2024 16:04:25 -0400 Subject: [PATCH 4/5] Address feedback --- docs/en/ingest-management/commands.asciidoc | 10 +-- .../otel-agent-components.asciidoc | 64 +++++++++++++++++++ .../otel-agent-configuration.asciidoc | 12 +++- .../elastic-agent/otel-agent.asciidoc | 4 ++ docs/en/ingest-management/index.asciidoc | 3 + 5 files changed, 86 insertions(+), 7 deletions(-) create mode 100644 docs/en/ingest-management/elastic-agent/otel-agent-components.asciidoc diff --git a/docs/en/ingest-management/commands.asciidoc b/docs/en/ingest-management/commands.asciidoc index 6a6a00ab7..583c0341c 100644 --- a/docs/en/ingest-management/commands.asciidoc +++ b/docs/en/ingest-management/commands.asciidoc @@ -856,16 +856,18 @@ elastic-agent otel [flags] elastic-agent otel [command] ---- +NOTE: You can also run the `otelcol` command, which calls `elastic-agent otel` and passes any arguments to it. + [discrete] === Available commands -`--validate`:: +`validate`:: Validates the OpenTelemetry collector configuration without running the collector. [discrete] === Flags -`--config --config=file:/path/to/first --config=file:path/to/second`:: +`--config=file:/path/to/first --config=file:path/to/second`:: Locations to the config file(s). Note that only a single location can be set per flag entry, for example `--config=file:/path/to/first --config=file:path/to/second`. `--feature-gates flag`:: @@ -874,8 +876,8 @@ Comma-delimited list of feature gate identifiers. Prefix with `-` to disable the `-h, --help`:: Get help for the `otel` sub-command. Use `elastic-agent otel [command] --help` for more information about a command. -`--set stringArray`:: -Set an arbitrary component config property. The component has to be defined in the configuration file and the flag has a higher precedence. Array configuration properties are overridden and maps are joined. For example, `--set=processors.batch.timeout=2s`. +`--set string`:: +Set an arbitrary component config property. The component has to be defined in the configuration file and the flag has a higher precedence. Array configuration properties are overridden and maps are joined. For example, `--set=processors::batch::timeout=2s`. [discrete] === Examples diff --git a/docs/en/ingest-management/elastic-agent/otel-agent-components.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent-components.asciidoc new file mode 100644 index 000000000..b6ca85245 --- /dev/null +++ b/docs/en/ingest-management/elastic-agent/otel-agent-components.asciidoc @@ -0,0 +1,64 @@ +[[otel-agent-components]] += {agent} OTel components + +preview::[] + +The following components are included in the Elastic Distribution for OpenTelemetry Collector. + +[discrete] + + +.Receivers +|=== +|Component |Version + +|`filelogreceiver` +|v0.97.0 + +|`otlpreceiver` +|v0.97.0 + +|=== + +.Exporters +|=== +|Component |Version + +|`elasticsearchexporter` +|v0.97.0 + +|`fileexporter` +|v0.97.0 + +|`debugexporter` +|v0.97.0 + +|`otlpexporter` +|v0.97.0 + +|=== + + +.Processors +|=== +|Component |Version + +|`attributesprocessor` +|v0.97.0 + +|`filterprocessor` +|v0.97.0 + +|`resourceprocessor` +|v0.97.0 + +|`transformprocessor` +|v0.97.0 + +|`batchprocessor` +|v0.97.0 + +|`memorylimiterprocessor` +|v0.97.0 + +|=== \ No newline at end of file diff --git a/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc index 76355bd4c..c7f2d8488 100644 --- a/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc +++ b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc @@ -10,8 +10,14 @@ You can provide the OTel configuration to {agent} using any of the following fou * `file` - The configuration is read from a file. For example, `file:path/to/config.yaml`. * `env` - The configuration is read from an environment variable. For example, `env:MY_CONFIG_IN_AN_ENVVAR`. * `yaml` - The configuration is read from a standard YAML-formatted file. For example, `yaml:exporters::debug::verbosity: detailed`. -* `http` - The configuration is read from an HTTP URL. For example, `http://www.example.com`. -// Perhaps we can improve these examples to make them more clear. I'm not sure I understand what the configuration would look like when passed as a URL, for example. +* `http` - The configuration is read from an HTTP URL. For example, `http://www.example.com/config.yaml`. + +For example, the following runs {agent} in `otel` mode, passing the OTel configuration as an HTTP URL: + +[source,shell] +---- +./elastic-agent otel --config=http://localhost:8000/config.yaml +---- To help you get started, when you download an {agent} install package, the package comes with a sample OTel configuration file, `otel.yml` located in the package base directory. @@ -47,6 +53,6 @@ service: <4> <1> link:https://opentelemetry.io/docs/collector/configuration/#receivers[Receivers] collect telemetry from one or more sources. <2> link:https://opentelemetry.io/docs/collector/configuration/#processors[Processors] take the data collected by receivers and modify or transform it before sending it to the exporters. <3> link:https://opentelemetry.io/docs/collector/configuration/#exporters[Exporters] send data to one or more backends or destinations. -<4> The link:https://opentelemetry.io/docs/collector/configuration/#connectors[Service] The section is used to configure what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it's not enabled. +<4> The link:https://opentelemetry.io/docs/collector/configuration/#connectors[Service] section is used to configure what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it's not enabled. For full details about setting up this file, refer to link:https://opentelemetry.io/docs/collector/configuration/[Configuration] in the OpenTelemetry documentation. \ No newline at end of file diff --git a/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc index 925fe54b1..e87deb1c1 100644 --- a/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc +++ b/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc @@ -9,6 +9,10 @@ When you run {agent} in `otel` mode it supports the standard OTel Collector conf The following steps will help you to get set up quickly using {agent} as an OTel Collector. Note that all steps are run with `sudo` as it's generally recommended to run {agent} with administrative privileges. +For the full command syntax for running {agent} in `otel` mode, refer to <> in the command reference. + +For a list of components included in the Elastic Distribution for OpenTelemetry Collector, refer to <>. + [discrete] == Run {agent} in OTel mode diff --git a/docs/en/ingest-management/index.asciidoc b/docs/en/ingest-management/index.asciidoc index d668308fc..601a7c589 100644 --- a/docs/en/ingest-management/index.asciidoc +++ b/docs/en/ingest-management/index.asciidoc @@ -3,6 +3,7 @@ include::{docs-root}/shared/attributes.asciidoc[] :doctype: book + = {fleet} and {agent} Guide include::overview.asciidoc[leveloffset=+1] @@ -67,6 +68,8 @@ include::elastic-agent/otel-agent.asciidoc[leveloffset=+2] include::elastic-agent/otel-agent-configuration.asciidoc[leveloffset=+3] +include::elastic-agent/otel-agent-components.asciidoc[leveloffset=+3] + include::elastic-agent/install-agent-msi.asciidoc[leveloffset=+2] include::elastic-agent/installation-layout.asciidoc[leveloffset=+2] From 70ef0de8e5801084c26be63215a4eb30e759a57a Mon Sep 17 00:00:00 2001 From: David Kilfoyle Date: Tue, 18 Jun 2024 11:45:46 -0400 Subject: [PATCH 5/5] Updates for additional comments --- docs/en/ingest-management/commands.asciidoc | 2 +- .../otel-agent-configuration.asciidoc | 11 ++++++++-- .../elastic-agent/otel-agent.asciidoc | 20 ++++++++++++------- 3 files changed, 23 insertions(+), 10 deletions(-) diff --git a/docs/en/ingest-management/commands.asciidoc b/docs/en/ingest-management/commands.asciidoc index 583c0341c..14c01759c 100644 --- a/docs/en/ingest-management/commands.asciidoc +++ b/docs/en/ingest-management/commands.asciidoc @@ -856,7 +856,7 @@ elastic-agent otel [flags] elastic-agent otel [command] ---- -NOTE: You can also run the `otelcol` command, which calls `elastic-agent otel` and passes any arguments to it. +NOTE: You can also run the `./otelcol` command, which calls `./elastic-agent otel` and passes any arguments to it. [discrete] === Available commands diff --git a/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc index c7f2d8488..922aea0e4 100644 --- a/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc +++ b/docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc @@ -41,7 +41,13 @@ exporters: <3> sampling_initial: 10000 sampling_thereafter: 10000 -service: <4> +extensions: <4> + memory_limiter: + check_interval: 1s + limit_mib: 700 + spike_limit_mib: 180 + +service: <5> pipelines: logs: receivers: [filelog] @@ -53,6 +59,7 @@ service: <4> <1> link:https://opentelemetry.io/docs/collector/configuration/#receivers[Receivers] collect telemetry from one or more sources. <2> link:https://opentelemetry.io/docs/collector/configuration/#processors[Processors] take the data collected by receivers and modify or transform it before sending it to the exporters. <3> link:https://opentelemetry.io/docs/collector/configuration/#exporters[Exporters] send data to one or more backends or destinations. -<4> The link:https://opentelemetry.io/docs/collector/configuration/#connectors[Service] section is used to configure what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it's not enabled. +<4> link:https://opentelemetry.io/docs/collector/configuration/#extensions[Extensions] are optional components that expand the capabilities of the Collector to accomplish tasks not directly involved with processing telemetry data. +<5> The link:https://opentelemetry.io/docs/collector/configuration/#connectors[Service] section is used to configure what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it's not enabled. For full details about setting up this file, refer to link:https://opentelemetry.io/docs/collector/configuration/[Configuration] in the OpenTelemetry documentation. \ No newline at end of file diff --git a/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc b/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc index e87deb1c1..5ec187fc5 100644 --- a/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc +++ b/docs/en/ingest-management/elastic-agent/otel-agent.asciidoc @@ -7,7 +7,7 @@ The link:https://opentelemetry.io/docs/collector/[OpenTelemetry Collector] is a When you run {agent} in `otel` mode it supports the standard OTel Collector configuration format that defines a set of receivers, processors, exporters, and connectors. Logs, metrics, and traces can be ingested using OpenTelemetry data formats. -The following steps will help you to get set up quickly using {agent} as an OTel Collector. Note that all steps are run with `sudo` as it's generally recommended to run {agent} with administrative privileges. +The following steps will help you to get set up quickly using {agent} as an OTel Collector. For the full command syntax for running {agent} in `otel` mode, refer to <> in the command reference. @@ -37,7 +37,7 @@ for the full list of available packages. [source,shell] -- cd -sudo ./elastic-agent otel -h +./elastic-agent otel -h -- + TIP: You can also refer to the <> command documentation for the full command syntax. @@ -89,16 +89,18 @@ You can use the `set` flag to manually override any setting defined in the OTel + [source,shell] ---- -sudo ./elastic-agent otel --config otel.yml --set "exporters::debug::verbosity=normal" +./elastic-agent otel --config otel.yml --set "exporters::debug::verbosity=normal" ---- +NOTE: You can also run the `./otelcol` command, which calls `./elastic-agent otel` and passes any arguments to it. + . Validate the configuration + To validate the OTel configuration for your {agent}, run the `otel` command with the `validate` option and the path to the config file: + [source,shell] -- -sudo ./elastic-agent otel validate --config otel.yml +./elastic-agent otel validate --config otel.yml -- + If no error is returned, the configuration file is determined to be valid. @@ -109,7 +111,7 @@ Now, run {agent} with OTel mode configured: + [source,shell] -- -sudo ./elastic-agent otel --config otel.yml +./elastic-agent otel --config otel.yml -- + You should see a response indicating that {agent} has started successfully in OTel mode: @@ -134,7 +136,7 @@ Finally, confirm that {agent} is running as expected. Open a second terminal she + [source,shell] -- -sudo ./elastic-agent status +./elastic-agent status --output full -- + The returned status indicates that {agent} is healthy and not enrolled in {fleet} (which is expected because in this example we're using standalone {agent}). @@ -144,6 +146,10 @@ The returned status indicates that {agent} is healthy and not enrolled in {fleet ┌─ fleet │ └─ status: (STOPPED) Not enrolled into Fleet └─ elastic-agent - └─ status: (HEALTHY) Running + ├─ status: (HEALTHY) Running + └─ info + ├─ id: + ├─ version: 8.14.0 + └─ commit: d0bce03b68012224ac1bab4ca20ba36060d135a3 --