Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce global logging environment variables #869

Open
Mpdreamz opened this issue Jun 4, 2024 · 1 comment
Open

Introduce global logging environment variables #869

Mpdreamz opened this issue Jun 4, 2024 · 1 comment

Comments

@Mpdreamz
Copy link
Member

Mpdreamz commented Jun 4, 2024

In the .NET agent and OpenTelemetry distribution we are introducing new OTEL variables to enable global file logging:

Overview

  • ELASTIC_OTEL_LOG_LEVEL
  • ELASTIC_OTEL_LOG_DIRECTORY
  • ELASTIC_OTEL_LOG_TARGETS

If any of these are provided the agent or distribution will start logging, regardless of how the application is set up to log.

ELASTIC_OTEL_LOG_DIRECTORY defaults to:

  • %PROGRAMDATA%\elastic\apm-agent-dotnet (on Windows)
  • /var/log/elastic/apm-agent-dotnet (on Linux)
  • ~/Library/Application_Support/elastic/apm-agent-dotnet (on OSX)

We don't differentiate between agent and distro for the application moniker apm-agent-dotnet, should we?

ELASTIC_OTEL_LOG_TARGETS

Semicolon separated list of options.

  • file, enable file logging
  • stdout, enable logging to stdout
  • none , explicitly disables global logging.

The default is none unless the other two variables are set in which case its file

ELASTIC_OTEL_LOG_LEVEL
The name of the log level.

  • trace => Trace,
  • debug => Debug,
  • info, information => Information,
  • warn, warning => Warning,
  • error => Error,
  • none => None, explicitly disables global logging.

Benefit

Troubleshooting the agent becomes rather easy for our users. We only have to instruct them to set ELASTIC_OTEL_LOG_LEVEL=Trace to get logs.

Comparing that to what we point users to today: https://www.elastic.co/guide/en/apm/agent/dotnet/current/troubleshooting.html#collect-agent-logs where we first need to find out how the users is running the agent and depending on the actual stack have various different ways to turn on logging. This might be very .NET specific though.

Discussion

We expect our users to set these variables system wide, even if they don't technically have to.

Should we standardize on these variables to enable logging for our new distributions?
Have one common way to start debugging and troubleshooting them?

Cc @elastic/apm-agent-devs

@trentm
Copy link
Member

trentm commented Jun 4, 2024

I would worry about the confusion of ELASTIC_OTEL_LOG_LEVEL with the OTel-defined OTEL_LOG_LEVEL var:
https://opentelemetry.io/docs/specs/otel/configuration/sdk-environment-variables/#general-sdk-configuration

(Note that the level names for OTEL_LOG_LEVEL aren't current well-defined. Issue 2039 in https://github.com/open-telemetry/opentelemetry-specification (I didn't want a link back to this from there) is discussing defining those names.)

The Node.js OTel distro is currently using OTEL_LOG_LEVEL (defaulting to info) to determine its log level. It logs to stdout, with no option to target anything else. Yes, logging to stdout means mixing output in with the application's output, but I've never seen that being considered a problem with the classic agent.

.NET-land may have different expectations for wanting APM agent logs going somewhere else.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants