Including trace ID in log messages

[blindmonkey]

Hi team!

We're currently using RUM, the network interceptor, and logging, and are trying to make these play nice together. The use case that we're trying to figure out right now is that we use the logger to notify of network request failures, which may include a failure to parse the response. When such an error occurs, it's properly interleaved in the RUM view and that's super helpful. However, the log message doesn't get associated with the network trace, and it's not clear how to make this happen.

I've also tried creating a span and using span.log, but that just doesn't show anywhere -- not in RUM, and not in APM, and certainly not in the trace view for the relevant request. There doesn't really seem to be any good way to get a reference to either the span for the actual network request, or the trace ID; so my current plan is to propagate the URLRequest to our outer network layer, extract the trace ID from the headers, send it as a custom attribute, and configure Datadog to use that as per this documentation I found.

However, this seems like a fairly common use case, so I find myself wondering.. are we missing something obvious?

[maxep]

Hi @blindmonkey 👋

This has never been requested tho it's a valid use case. We currently don't support correlation between distributed tracing and logs on the SDK, and I'm afraid that manually injecting the dd.trace_id as a log attribute won't work: It is considered as a reserved attribute and it will be filtered out during sanitisation.

With the current state of the SDK I don't see any workaround to offer sadly, I will bring this topic to the team as a feature request.

[blindmonkey]

@maxep that's surprising, and I guess I'm confused about why manually specifying the trace ID in our logs won't work. The documentation I linked to earlier appears to imply that we can specify a custom attribute in our logs and configure Datadog to interpret it as a trace ID, which would connect the logs to the trace. Am I misunderstanding the documentation?

[maxep]

Hi @blindmonkey !

Sorry for the confusion, Yes you are right: You can manually inject the Trace ID as a Log custom attribute and configure Trace Id attributes section of your Log preprocessing. You should then be able to see your logs associated with that Trace. But don't use dd.trace_id as your custom attribute name as it is reserved.

We will explore ways to provide the Trace ID automatically when using Distributed Tracing, so I will keep this ticket open 👍

[sschizas]

Any update on this one?

[mariedm]

Hi @sschizas, this hasn't been addressed yet. I'm creating a ticket so that our team can look into it.