Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Elastic OpenTelemetry Collector docs #1

Merged
merged 17 commits into from
Aug 6, 2024

Conversation

mdbirnstiehl
Copy link
Contributor

@mdbirnstiehl mdbirnstiehl commented Jul 30, 2024

This PR creates the docs for the Elastic distribution of the OpenTelemetry Collector.
I've created some draft diagrams just to get conversation going, but once they are finalized we can make them look better.

I still need to create documentation on configuring a custom collector (aka the brownfield issue) as well.

Please feel free to add any additional reviewers.

@mdbirnstiehl mdbirnstiehl self-assigned this Jul 30, 2024
README.md Outdated Show resolved Hide resolved
Copy link
Contributor

@colleenmcginnis colleenmcginnis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdbirnstiehl I'm leaving a preliminary review of the README with some questions/suggestions around terminology. Maybe we can get input from @mlunadia on terminology here and apply it throughout the files in this PR.

I'll move on to review the other docs now, but I won't clutter those with any more terminology suggestions at this point.

README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
mdbirnstiehl and others added 2 commits July 31, 2024 16:45
Co-authored-by: Colleen McGinnis <[email protected]>
Co-authored-by: Colleen McGinnis <[email protected]>
Copy link
Contributor

@colleenmcginnis colleenmcginnis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some more questions/suggestions for ya. Feel free to use or disregard any suggestions. 🙃

docs/collector-components.md Outdated Show resolved Hide resolved
docs/collector-components.md Outdated Show resolved Hide resolved
docs/collector-components.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Outdated Show resolved Hide resolved
docs/guided-onboarding.md Outdated Show resolved Hide resolved
docs/manual-configuration.md Show resolved Hide resolved
docs/manual-configuration.md Outdated Show resolved Hide resolved
docs/manual-configuration.md Outdated Show resolved Hide resolved
docs/manual-configuration.md Outdated Show resolved Hide resolved

You can configure an upstream collector, like a [custom collector](https://opentelemetry.io/docs/collector/custom-collector/) or [contrib](https://github.com/open-telemetry/opentelemetry-collector-contrib) distribution for the collector, to collect logs and metrics and send them to Elastic Observability.

<!-- We might want to add some pros and cons of using an upstream collector -->
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mlunadia would it be valuable to layout some pros and cons of using an upstream collector vs. the elastic distro?

docs/configure-upstream-collector.md Outdated Show resolved Hide resolved
@mdbirnstiehl mdbirnstiehl marked this pull request as ready for review August 2, 2024 13:52
Copy link
Contributor

@mlunadia mlunadia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will follow up on the rest of files next week

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Outdated Show resolved Hide resolved
The Elastic Distribution for the OpenTelemetry Collector has the following limitations:

- `host.network.*` metrics aren't present from the OpenTelemetry side.
- `process.state` isn't present in the OpenTelemetry host metric. It's set to a dummy value of **Unknown** in the **State** column of the host processes table.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ishleenk17 is this an upstream limitation?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The process.state is not present in the Hostmetrics receiver. So, yes its an upstream limitation.
At Elastic, we show the state as Unknown for all processes because of this limitation.

docs/configure-upstream-collector.md Outdated Show resolved Hide resolved
docs/configure-upstream-collector.md Outdated Show resolved Hide resolved
docs/collector-components.md Show resolved Hide resolved
Refer to the [guided onboarding](guided-onboarding.md) docs or the [manual configuration](manual-configuration.md) docs for more on configuring the Elastic distribution.

## Upstream collector configuration examples
Refer to the OpenTelemetry documentation on [building a custom collector](https://opentelemetry.io/docs/collector/custom-collector/) for more on creating an upstream collector.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we need more concrete steps on how exactly to build a custom collector that contains all the components required. Could we maybe even provide the config file for ocb? @ishleenk17 could you help with the concrete steps here?

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Outdated Show resolved Hide resolved
The Elastic Distribution for the OpenTelemetry Collector has the following limitations:

- `host.network.*` metrics aren't present from the OpenTelemetry side.
- `process.state` isn't present in the OpenTelemetry host metric. It's set to a dummy value of **Unknown** in the **State** column of the host processes table.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The process.state is not present in the Hostmetrics receiver. So, yes its an upstream limitation.
At Elastic, we show the state as Unknown for all processes because of this limitation.

docs/collector-limitations.md Outdated Show resolved Hide resolved
docs/collector-limitations.md Show resolved Hide resolved
Copy link
Contributor

@mlunadia mlunadia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mdbirnstiehl I'm good for this to be merged so we can start sharing it with people who can provide feedback when trying it out.

@mlunadia mlunadia merged commit f250533 into elastic:main Aug 6, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants