You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
varactivity= Source.StartActivity(activityName, ActivityKind.Client);if(activity is not { IsAllDataRequested:true})returnactivity;
activity.SetTag("db.statement", commandText);// This tag will contain FULL command textif(dbOperation!=null)
activity.SetTag("db.operation", dbOperation);if(dbSqlTable!=null)
activity.SetTag("db.sql.table", dbSqlTable);returnactivity;
APM Agent gets reacts to Activity creation/stop events in the ElasticActivityListener class and retrieves OTEL-attributes from the tags of the Activity instance as they are written there, without any modifications.
Then when in PayloadSenderV2 class the event is serialized into JSON for transmission to APM Server, the OTEL attribute values also remain unchanged, no matter how long they are (below, the db.statement value has been manually trimmed to avoid display issues).
Obviously, the data size of this event can easily exceed the allowed value, which is set on the APM Server side in the max_event_size parameter (defaults to 307200 bytes). When trying to transfer such event data to APM Server - the error event exceeded the allowed size will occur.
A long query text can be generated by EF Core during bulk insert/update operations, which is what we observe in our application.
Error event exceeded the allowed size will appear in the log
using System.Diagnostics;using Microsoft.AspNetCore.Mvc;namespace WebApiExample.Controllers;[ApiController][Route("[controller]")]publicclassWeatherForecastController:ControllerBase{privatestaticreadonlystring[]Summaries=["Freezing","Bracing","Chilly","Cool","Mild","Warm","Balmy","Hot","Sweltering","Scorching"];[HttpGet(Name ="GetWeatherForecast")]publicIEnumerable<WeatherForecast>Get(){// Constuct Activity the same way:// https://github.com/npgsql/npgsql/blob/6990cceffbca2d2de4c5f12df32729bc78bbeafb/src/Npgsql/NpgsqlActivitySource.cs#L50C57-L50C58usingvaractivity=new ActivitySource("Npgsql","0.1.0").StartActivity("test", ActivityKind.Client);
activity?.SetTag("db.statement",newstring('1',400_000));return Enumerable.Range(1,5).Select(index =>new WeatherForecast{Date= DateOnly.FromDateTime(DateTime.Now.AddDays(index)),TemperatureC= Random.Shared.Next(-20,55),Summary= Summaries[Random.Shared.Next(Summaries.Length)]}).ToArray();}}
Expected behavior
Values of OTEL-attributes that by specification contain SQL command text (db.query.text, db.statement) should be truncated by the same rules, as it is done now for the Statement property in the Database class. Maybe you should apply such a policy to all OTEL-attributes.
The SQL query text that comes in the tags of the Activity instance is passed in OTEL attributes to APM Server as is, which results in an error event exceeded the allowed size when the query text is huge.
The text was updated successfully, but these errors were encountered:
OlegUfaev
changed the title
[BUG] The APM Agent does not control the length of the SQL queries it receives in OTEL attributes via ActivitySource
[BUG] The APM Agent does not control the length of the SQL query text it receives in OTEL attributes via ActivitySource
Sep 19, 2024
APM Agent version
1.29.0
Environment
Operating system and version: Linux, Windows
.NET Framework/Core name and version : .NET 8
Application Target Framework(s): .NET 8
Describe the bug
In our application we are using
Npgsql EF Core Provider
and when we execute commands to the database - it creates anActivity
viaActivitySource
and add OTEL-attributes to it via tags:https://github.com/npgsql/npgsql/blob/6990cceffbca2d2de4c5f12df32729bc78bbeafb/src/Npgsql/NpgsqlActivitySource.cs#L50
APM Agent gets reacts to
Activity
creation/stop events in theElasticActivityListener
class and retrieves OTEL-attributes from the tags of theActivity
instance as they are written there, without any modifications.apm-agent-dotnet/src/Elastic.Apm/OpenTelemetry/ElasticActivityListener.cs
Lines 193 to 200 in 31cb8cb
Then when in
PayloadSenderV2
class the event is serialized into JSON for transmission to APM Server, the OTEL attribute values also remain unchanged, no matter how long they are (below, thedb.statement
value has been manually trimmed to avoid display issues).Obviously, the data size of this event can easily exceed the allowed value, which is set on the APM Server side in the
max_event_size
parameter (defaults to 307200 bytes). When trying to transfer such event data to APM Server - the errorevent exceeded the allowed size
will occur.A long query text can be generated by EF Core during bulk insert/update operations, which is what we observe in our application.
To Reproduce
Steps to reproduce the behavior:
WebApiExample
appsettings.json
add required ElasticApm configuration for your APM Server instance (ServerUrl
,SecretToken
, etc. )WeatherForecastController
implementation (see code snippet below)event exceeded the allowed size
will appear in the logExpected behavior
Values of OTEL-attributes that by specification contain SQL command text (
db.query.text
,db.statement
) should be truncated by the same rules, as it is done now for theStatement
property in theDatabase
class. Maybe you should apply such a policy to all OTEL-attributes.apm-agent-dotnet/src/Elastic.Apm/Api/Database.cs
Lines 13 to 24 in 31cb8cb
Actual behavior
The SQL query text that comes in the tags of the
Activity
instance is passed in OTEL attributes to APM Server as is, which results in an errorevent exceeded the allowed size
when the query text is huge.The text was updated successfully, but these errors were encountered: