Exported OTLP data being parsed incorrectly (data point labels being parsed as unknown fields)

[hxliu33]

I am currently using a Collector with an otlphttp exporter to send metrics to a custom backend I am writing in Java, but when I process the request body using ExportMetricsServiceRequest.parseFrom() method, it parses data point attributes into the data point's unknownFields instead of its labels_. Logging the output to stdout shows everything correctly (in the right places, as attributes under each data point). Here is the structure of the parsed request:

And here is what I am (correctly) getting through the logging exporter:

Through attaching a debugger, I have pinpointed the issue to the constructor methods of the IntDataPoint and DoubleDataPoint classes. In the switch-case statements, the labels are associated with tags of 58 (putting them in the default case that moves it to the unknownFields attribute) rather than tags of 10 (which would've put them in the correct labels attribute).

Please let me know if this is not actually a bug, and I am doing something incorrectly.

[tigrannajaryan]

This repository is not the right place to ask for help implementing your own backends. A better place may be Slack channel otel-java

[hxliu33]

This repository is not the right place to ask for help implementing your own backends. A better place may be Slack channel otel-java

@tigrannajaryan I did actually ask in that Slack channel a week ago now with no response, but I suspect it could be a bug within the OpenTelemetry Java implementation of OTLP (see my last screenshot) in the IntDataPoint and DoubleDataPoint classes. Is this not the repository for that? I couldn't find another repository containing the implementation, so this seemed like the closest one.

[tigrannajaryan]

Moving to https://github.com/open-telemetry/opentelemetry-java

[tigrannajaryan]

@open-telemetry/java-maintainers please take a look.

[jack-berg]

@hxliu33 it looks like you may be trying to parse the message with classes generated from an older version of the protos. What version of opentelemetry-proto is used to generate the ExportMetricsServiceRequest?

[hxliu33]

@hxliu33 it looks like you may be trying to parse the message with classes generated from an older version of the protos. What version of opentelemetry-proto is used to generate the ExportMetricsServiceRequest?

@jack-berg The dependency version that I am calling ExportMetricsServiceRequest.parseFrom() from is 1.0.0-alpha (I was using 0.17.0 at first but changed it to 1.0.0-alpha and continued to see the issue), but I'm using an OTel Collector to send the requests (the collector is scraping the data from a prometheus endpoint and exporting it to both console and my custom endpoint). I'm assuming then that the Collector is generating the ExportMetricsServiceRequest, but how can I tell which version of opentelemetry-proto the Collector is using for that? The version of the Collector itself was 0.83.0.

[jack-berg]

I'm seeing indications in your screenshot that you're not using 1.0.0-alpha - your screenshot shows a field called instrumentationLibraryMetrics, but this was renamed to scopeMetrics prior to 1.0.0-alpha. Also, the idea of labels on metrics was replaced quite a while ago with attributes. Can you double check that you are using io.opentelemetry.proto:opentelemetry-proto:1.0.0-alpha as published from opentelemetry-proto-java? The source code I see in that artifact appears to be different than what you show.

[hxliu33]

@jack-berg Ohh interesting, maybe the package never updated to the version I was declaring. I will try ensuring that I am using 1.0.0-alpha (by making sure I see scopeMetrics/attributes instead of instrumentationLibraryMetrics and labels) and see if that works, thanks!

[jack-berg]

I'm going to close this because there are numerous tests which show this working. Also, parsing OTLP isn't technically part of the scope of this project. 🙂 If you have future issues with io.opentelemetry.proto:opentelemetry-proto, please open an issue in opentelemetry-proto-java.