Initial docs for agent OTel mode

docs/en/ingest-management/commands.asciidoc

@@ -28,6 +28,7 @@ invoking the wrong binary.
 * <<elastic-agent-help-command,help>>
 * <<elastic-agent-inspect-command,inspect>>
 * <<elastic-agent-install-command,install>>
+* <<elastic-agent-otel-command,otel>> [technical preview]
 * <<elastic-agent-privileged-command,privileged>>
 * <<elastic-agent-restart-command,restart>>
 * <<elastic-agent-run-command,run>>
@@ -884,6 +885,68 @@ elastic-agent install --url=https://fleet-server:8220 \
 <hr>
 ++++
 
+[discrete]
+[[elastic-agent-otel-command]]
+== elastic-agent otel
+
+preview::[]
+
+Run {agent} as an <<otel-agent,OpenTelemetry Collector>>. 
+
+[discrete]
+=== Synopsis
+
+[source,shell]
+----
+elastic-agent otel [flags]
+elastic-agent otel [command]
+----
+
+NOTE: You can also run the `./otelcol` command, which calls `./elastic-agent otel` and passes any arguments to it.
+
+[discrete]
+=== Available commands
+
+`validate`::
+Validates the OpenTelemetry collector configuration without running the collector.
+
+[discrete]
+=== Flags
+
+`--config=file:/path/to/first --config=file:path/to/second`::
+Locations to the config file(s). Note that only a single location can be set per flag entry, for example `--config=file:/path/to/first --config=file:path/to/second`.
+
+`--feature-gates flag`::
+Comma-delimited list of feature gate identifiers. Prefix with `-` to disable the feature. Prefixing with `+` or no prefix will enable the feature.
+
+`-h, --help`::
+Get help for the `otel` sub-command. Use `elastic-agent otel [command] --help` for more information about a command.
+
+`--set string`::
+Set an arbitrary component config property. The component has to be defined in the configuration file and the flag has a higher precedence. Array configuration properties are overridden and maps are joined. For example, `--set=processors::batch::timeout=2s`.
+
+[discrete]
+=== Examples
+
+Run {agent} as on OTel Collector using the supplied `otel.yml` configuration file.
+[source,shell]
+----
+./elastic-agent otel --config otel.yml
+----
+
+Change the default verbosity setting in the {agent} OTel configuration from `detailed` to `normal`.
+
+[source,shell]
+----
+./elastic-agent otel --config otel.yml --set "exporters::debug::verbosity=normal"
+----
+
+=======
+
+++++
+<hr>
+++++
+
 [discrete]
 [[elastic-agent-restart-command]]
 == elastic-agent restart

docs/en/ingest-management/elastic-agent/otel-agent-components.asciidoc

@@ -0,0 +1,64 @@
+[[otel-agent-components]]
+= {agent} OTel components
+
+preview::[]
+
+The following components are included in the Elastic Distribution for OpenTelemetry Collector.
+
+[discrete]
+
+
+.Receivers
+|===
+|Component |Version
+
+|`filelogreceiver`
+|v0.97.0
+
+|`otlpreceiver`
+|v0.97.0
+
+|===
+
+.Exporters
+|===
+|Component |Version
+
+|`elasticsearchexporter`
+|v0.97.0
+
+|`fileexporter`
+|v0.97.0
+
+|`debugexporter`
+|v0.97.0
+
+|`otlpexporter`
+|v0.97.0
+
+|===
+
+
+.Processors
+|===
+|Component |Version
+
+|`attributesprocessor`
+|v0.97.0
+
+|`filterprocessor`
+|v0.97.0
+
+|`resourceprocessor`
+|v0.97.0
+
+|`transformprocessor`
+|v0.97.0
+
+|`batchprocessor`
+|v0.97.0
+
+|`memorylimiterprocessor`
+|v0.97.0
+
+|===

docs/en/ingest-management/elastic-agent/otel-agent-configuration.asciidoc

@@ -0,0 +1,65 @@
+[[otel-agent-configuration]]
+= {agent} OTel configuration
+
+preview::[]
+
+When running in `otel` mode, {agent} supports a standard OTel Collector configuration. 
+
+You can provide the OTel configuration to {agent} using any of the following four schemes:
+
+* `file` - The configuration is read from a file. For example, `file:path/to/config.yaml`.
+* `env` - The configuration is read from an environment variable. For example, `env:MY_CONFIG_IN_AN_ENVVAR`.
+* `yaml` - The configuration is read from a standard YAML-formatted file. For example, `yaml:exporters::debug::verbosity: detailed`.
+* `http` - The configuration is read from an HTTP URL. For example, `http://www.example.com/config.yaml`.
+
+For example, the following runs {agent} in `otel` mode, passing the OTel configuration as an HTTP URL:
+
+[source,shell]
+----
+./elastic-agent otel --config=http://localhost:8000/config.yaml
+----
+
+To help you get started, when you download an {agent} install package, the package comes with a sample OTel configuration file, `otel.yml` located in the package base directory.
+
+[source,shell]
+----
+receivers: <1>
+  filelog:
+    include: [ /var/log/system.log ]
+    start_at: beginning
+
+processors: <2>
+  resource:
+    attributes:
+    - key: service.name
+      action: insert
+      value: elastic-otel-test
+
+exporters: <3>
+  debug:
+    verbosity: detailed
+    sampling_initial: 10000
+    sampling_thereafter: 10000
+
+extensions: <4>
+  memory_limiter:
+    check_interval: 1s
+    limit_mib: 700
+    spike_limit_mib: 180
+
+service: <5>
+  pipelines:
+    logs:
+      receivers: [filelog]
+      processors: [resource]
+      exporters:
+        - debug
+----
+
+<1> link:https://opentelemetry.io/docs/collector/configuration/#receivers[Receivers] collect telemetry from one or more sources.
+<2> link:https://opentelemetry.io/docs/collector/configuration/#processors[Processors] take the data collected by receivers and modify or transform it before sending it to the exporters.
+<3> link:https://opentelemetry.io/docs/collector/configuration/#exporters[Exporters] send data to one or more backends or destinations.
+<4> link:https://opentelemetry.io/docs/collector/configuration/#extensions[Extensions] are optional components that expand the capabilities of the Collector to accomplish tasks not directly involved with processing telemetry data. 
+<5> The link:https://opentelemetry.io/docs/collector/configuration/#connectors[Service] section is used to configure what components are enabled in the Collector based on the configuration found in the receivers, processors, exporters, and extensions sections. If a component is configured, but not defined within the service section, then it's not enabled.
+
+For full details about setting up this file, refer to link:https://opentelemetry.io/docs/collector/configuration/[Configuration] in the OpenTelemetry documentation.

docs/en/ingest-management/elastic-agent/otel-agent.asciidoc

@@ -0,0 +1,155 @@
+[[otel-agent]]
+= Run {agent} as an OTel Collector
+
+preview::[]
+
+The link:https://opentelemetry.io/docs/collector/[OpenTelemetry Collector] is a vendor-neutral way to receive, process, and export telemetry data. {agent} includes an embedded OTel Collector, enabling you to instrument your applications and infrastructure once, and send data to multiple vendors and backends. 
+
+When you run {agent} in `otel` mode it supports the standard OTel Collector configuration format that defines a set of receivers, processors, exporters, and connectors. Logs, metrics, and traces can be ingested using OpenTelemetry data formats.
+
+The following steps will help you to get set up quickly using {agent} as an OTel Collector.
+
+For the full command syntax for running {agent} in `otel` mode, refer to <<elastic-agent-otel-command>> in the command reference.
+
+For a list of components included in the Elastic Distribution for OpenTelemetry Collector, refer to <<otel-agent-components>>.
+
+[discrete]
+== Run {agent} in OTel mode
+
+For simplicity, these steps demonstrate OTel mode using standalone {agent}. The install steps for {fleet}-managed {agent} are very similar. Refer to the <<elastic-agent-installation,Install {agents}>> section for the full set of requirements and steps for any install scenario.
+
+. On your host, download and extract the {agent} installation package. 
++
+--
+// tag::install-elastic-agent[]
+
+include::{ingest-docs-root}/docs/en/ingest-management/tab-widgets/download-widget.asciidoc[]
+
+// end::install-elastic-agent[]
+--
++
+The commands shown are for AMD platforms, but ARM packages are also available.
+Refer to the {agent} https://www.elastic.co/downloads/elastic-agent[downloads page]
+for the full list of available packages.
+
+. Change into the agent package directory and run {agent} with the `otel` subcommand and the `-h` option to view the command syntax and available parameters:
++
+[source,shell]
+--
+cd <agent-package-name>
+./elastic-agent otel -h
+--
++
+TIP: You can also refer to the <<elastic-agent-otel-command,{agent} OTel>> command documentation for the full command syntax.
+
+. View or update the OTel configuration file.
++
+A basic OTel configuration file named `otel.yml` is included in the {agent} package. You can view or update it using any text editor.
++
+[source,yaml]
+--
+receivers:
+  filelog:
+    include: [ /var/log/system.log ]
+    start_at: beginning
+
+processors:
+  resource:
+    attributes:
+    - key: service.name
+      action: insert
+      value: elastic-otel-test
+
+exporters:
+  debug:
+    verbosity: detailed
+    sampling_initial: 10000
+    sampling_thereafter: 10000
+
+service:
+  pipelines:
+    logs:
+      receivers: [filelog]
+      processors: [resource]
+      exporters:
+        - debug%
+--
++
+By default, the default OTel configuration sets {agent} to:
+
+* Read the contents of `/var/log/system.log`.
+* Run a processor to insert `elastic-otel-test` into the output.
+* Send the data with a `detailed` level of verbosity.
++
+Refer to <<otel-agent-configuration>> to learn more about setting up this file.
+
+. [Optional] Override a setting in the OTel configuration
++
+You can use the `set` flag to manually override any setting defined in the OTel configuration file. For example, run the following command to change the default verbosity setting from `detailed` to `normal`.
++
+[source,shell]
+----
+./elastic-agent otel --config otel.yml --set "exporters::debug::verbosity=normal"
+----
+
+NOTE: You can also run the `./otelcol` command, which calls `./elastic-agent otel` and passes any arguments to it.
+
+. Validate the configuration
++
+To validate the OTel configuration for your {agent}, run the `otel` command with the `validate` option and the path to the config file:
++
+[source,shell]
+--
+./elastic-agent otel validate --config otel.yml
+--
++
+If no error is returned, the configuration file is determined to be valid.
+
+. Run {agent}
++
+Now, run {agent} with OTel mode configured:
++
+[source,shell]
+--
+./elastic-agent otel --config otel.yml
+--
++
+You should see a response indicating that {agent} has started successfully in OTel mode:
++
+[source,shell]
+--
+Starting in otel mode
+2024-05-31T12:39:43.047-0400	info	[email protected]/telemetry.go:59	Setting up own telemetry...
+2024-05-31T12:39:43.049-0400	info	[email protected]/telemetry.go:104	Serving metrics	{"address": ":8888", "level": "Basic"}
+2024-05-31T12:39:43.052-0400	info	[email protected]/exporter.go:275	Development component. May change in the future.	{"kind": "exporter", "data_type": "logs", "name": "debug"}
+2024-05-31T12:39:43.055-0400	info	[email protected]/service.go:140	Starting ./elastic-agent...	{"Version": "8.13.4", "NumCPU": 10}
+2024-05-31T12:39:43.055-0400	info	extensions/extensions.go:34	Starting extensions...
+2024-05-31T12:39:43.056-0400	info	adapter/receiver.go:45	Starting stanza receiver	{"kind": "receiver", "name": "filelog", "data_type": "logs"}
+2024-05-31T12:39:43.057-0400	info	[email protected]/service.go:166	Everything is ready. Begin running and processing data.
+2024-05-31T12:39:43.057-0400	warn	localhostgate/featuregate.go:63	The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new default.	{"feature gate ID": "component.UseLocalHostAsDefaultHost"}
+2024-05-31T12:39:43.261-0400	info	fileconsumer/file.go:261	Started watching file	{"kind": "receiver", "name": "filelog", "data_type": "logs", "component": "fileconsumer", "path": "/var/log/system.log"}
+--
+
+. Check the {agent} status
++
+Finally, confirm that {agent} is running as expected. Open a second terminal shell and run the `status` command.
++
+[source,shell]
+--
+./elastic-agent status --output full
+--
++
+The returned status indicates that {agent} is healthy and not enrolled in {fleet} (which is expected because in this example we're using standalone {agent}).
++
+[source,shell]
+--
+┌─ fleet
+│  └─ status: (STOPPED) Not enrolled into Fleet
+└─ elastic-agent
+   ├─ status: (HEALTHY) Running
+   └─ info
+      ├─ id:
+      ├─ version: 8.14.0
+      └─ commit: d0bce03b68012224ac1bab4ca20ba36060d135a3
+--
+

docs/en/ingest-management/index.asciidoc

@@ -3,6 +3,7 @@ include::{docs-root}/shared/attributes.asciidoc[]
 
 :doctype: book
 
+
 = {fleet} and {agent} Guide
 
 include::overview.asciidoc[leveloffset=+1]
@@ -63,6 +64,12 @@ include::elastic-agent/ingest-pipeline-kubernetes.asciidoc[leveloffset=+3]
 
 include::elastic-agent/configuration/env/container-envs.asciidoc[leveloffset=+3]
 
+include::elastic-agent/otel-agent.asciidoc[leveloffset=+2]
+
+include::elastic-agent/otel-agent-configuration.asciidoc[leveloffset=+3]
+
+include::elastic-agent/otel-agent-components.asciidoc[leveloffset=+3]
+
 include::elastic-agent/elastic-agent-unprivileged-mode.asciidoc[leveloffset=+2]
 
 include::elastic-agent/install-agent-msi.asciidoc[leveloffset=+2]