Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Query] Adds public backend metrics property to Diagnostics #4001

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
f440f1b
initial commit
Maya-Painter Jul 21, 2023
b287e27
some pr comments, WIP
Maya-Painter Jul 21, 2023
82b9871
Refactor
Maya-Painter Jul 24, 2023
1770560
more
Maya-Painter Jul 24, 2023
6fa6e2a
Public constructors and modify accumulators
Maya-Painter Jul 26, 2023
451763b
accumulator updates and undo test changes
Maya-Painter Jul 27, 2023
d4f6c07
add test
Maya-Painter Jul 27, 2023
de863d9
PR comments
Maya-Painter Jul 27, 2023
b1a7823
bug fix
Maya-Painter Jul 27, 2023
b96203a
ToString() refactor
Maya-Painter Jul 28, 2023
f84e698
contract updates
Maya-Painter Jul 28, 2023
e2137d2
test updates
Maya-Painter Jul 28, 2023
e3f1876
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Jul 28, 2023
4c5152d
small fixes
Maya-Painter Jul 28, 2023
f18b585
text fix
Maya-Painter Jul 28, 2023
b25f872
Update accumulators
Maya-Painter Jul 31, 2023
7f8aa59
fix
Maya-Painter Aug 1, 2023
0ebb267
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Aug 1, 2023
f9af873
PR comments
Maya-Painter Aug 1, 2023
d320017
small fix
Maya-Painter Aug 1, 2023
edc5aab
Rename BE -> ServerSide
Maya-Painter Aug 2, 2023
b64d837
more renaming
Maya-Painter Aug 2, 2023
8d71294
Update API and tests
Maya-Painter Aug 2, 2023
95f8efb
separate public and internal classes
Maya-Painter Aug 3, 2023
4f9dd80
API update
Maya-Painter Aug 3, 2023
a1a3915
change namespace
Maya-Painter Aug 4, 2023
eccf865
Pr comments
Maya-Painter Aug 4, 2023
120bccf
public constructors and bug fix
Maya-Painter Aug 7, 2023
94f5612
API updates
Maya-Painter Aug 8, 2023
065d323
renaming and test updates
Maya-Painter Aug 11, 2023
99679c0
PR comments
Maya-Painter Aug 11, 2023
41af90a
more PR comments
Maya-Painter Aug 11, 2023
ba80d30
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Aug 11, 2023
b31e382
PR comments, test additions
Maya-Painter Aug 14, 2023
e6825c3
API updates and more tests
Maya-Painter Aug 14, 2023
2bb36c8
tests and pkrangeid update
Maya-Painter Aug 15, 2023
466be56
PR comments
Maya-Painter Aug 15, 2023
6abd122
more PR comments
Maya-Painter Aug 16, 2023
0b4fcba
smol test fix
Maya-Painter Aug 16, 2023
eca1292
PR comments - renaming properties and constructor rehash
Maya-Painter Aug 16, 2023
c273463
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Aug 16, 2023
f19bafc
contract update
Maya-Painter Aug 16, 2023
e08aff7
seal classes and private fields.
Maya-Painter Aug 23, 2023
2f0cf75
update indexHitRatio calc
Maya-Painter Aug 23, 2023
4a0b17f
mocking refactor to abstract classes
Maya-Painter Aug 25, 2023
6ecdd0c
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Aug 25, 2023
e30b8bc
contract updates
Maya-Painter Sep 5, 2023
3d9fc86
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Sep 5, 2023
806a160
PR comments - Update documentation
Maya-Painter Sep 8, 2023
255b10d
Merge branch 'master' into users/mayapainter/querymetrics
Maya-Painter Sep 8, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions Microsoft.Azure.Cosmos/src/Diagnostics/CosmosDiagnostics.cs
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,19 @@ public virtual int GetFailedRequestCount()
throw new NotImplementedException($"{nameof(CosmosDiagnostics)}.{nameof(GetFailedRequestCount)}");
}

/// <summary>
/// This represents the backend query metrics for the request.
/// </summary>
/// <remarks>
/// This is only applicable for query operations. For all other operations this will return null.
/// </remarks>
/// <returns>The accumulated backend metrics for the request.</returns>
public virtual ServerSideCumulativeMetrics GetQueryMetrics()
{
// Default implementation avoids breaking change for users upgrading.
throw new NotImplementedException($"{nameof(CosmosDiagnostics)}.{nameof(GetQueryMetrics)}");
}

/// <summary>
/// Gets the string field <see cref="CosmosDiagnostics"/> instance in the Azure Cosmos DB database service.
/// </summary>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,15 @@ namespace Microsoft.Azure.Cosmos.Diagnostics
using System.Linq;
using System.Text;
using Microsoft.Azure.Cosmos.Json;
using Microsoft.Azure.Cosmos.Query.Core.Metrics;
using Microsoft.Azure.Cosmos.Tracing;
using Microsoft.Azure.Cosmos.Tracing.TraceData;
using static Microsoft.Azure.Cosmos.Tracing.TraceData.ClientSideRequestStatisticsTraceDatum;

internal sealed class CosmosTraceDiagnostics : CosmosDiagnostics
{
private readonly Lazy<ServerSideCumulativeMetrics> accumulatedMetrics;

public CosmosTraceDiagnostics(ITrace trace)
{
if (trace == null)
Expand All @@ -30,6 +33,7 @@ public CosmosTraceDiagnostics(ITrace trace)
}

this.Value = rootTrace;
this.accumulatedMetrics = new Lazy<ServerSideCumulativeMetrics>(() => PopulateServerSideCumulativeMetrics(this.Value));
}

public ITrace Value { get; }
Expand All @@ -49,6 +53,11 @@ public override TimeSpan GetClientElapsedTime()
return this.Value?.Summary?.RegionsContacted;
}

public override ServerSideCumulativeMetrics GetQueryMetrics()
Maya-Painter marked this conversation as resolved.
Show resolved Hide resolved
adityasa marked this conversation as resolved.
Show resolved Hide resolved
{
return this.accumulatedMetrics.Value;
}

internal bool IsGoneExceptionHit()
{
return this.WalkTraceTreeForGoneException(this.Value);
Expand All @@ -61,9 +70,9 @@ private bool WalkTraceTreeForGoneException(ITrace currentTrace)
return false;
}

foreach (object datums in currentTrace.Data.Values)
foreach (object datum in currentTrace.Data.Values)
{
if (datums is ClientSideRequestStatisticsTraceDatum clientSideRequestStatisticsTraceDatum)
if (datum is ClientSideRequestStatisticsTraceDatum clientSideRequestStatisticsTraceDatum)
{
foreach (StoreResponseStatistics responseStatistics in clientSideRequestStatisticsTraceDatum.StoreResponseStatisticsList)
{
Expand Down Expand Up @@ -99,6 +108,17 @@ private ReadOnlyMemory<byte> WriteTraceToJsonWriter(JsonSerializationFormat json
return jsonTextWriter.GetResult();
}

private static ServerSideCumulativeMetrics PopulateServerSideCumulativeMetrics(ITrace trace)
{
ServerSideMetricsInternalAccumulator accumulator = new ServerSideMetricsInternalAccumulator();
ServerSideMetricsInternalAccumulator.WalkTraceTreeForQueryMetrics(trace, accumulator);

IReadOnlyList<ServerSidePartitionedMetricsInternal> serverSideMetricsList = accumulator.GetPartitionedServerSideMetrics().Select(metrics => new ServerSidePartitionedMetricsInternal(metrics)).ToList();

ServerSideCumulativeMetrics accumulatedMetrics = new ServerSideCumulativeMetricsInternal(serverSideMetricsList);
return accumulatedMetrics.PartitionedMetrics.Count != 0 ? accumulatedMetrics : null;
}

public override DateTime? GetStartTimeUtc()
{
if (this.Value == null || this.Value.StartTime == null)
Expand Down
237 changes: 0 additions & 237 deletions Microsoft.Azure.Cosmos/src/Query/Core/Metrics/BackendMetrics.cs

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,6 @@ namespace Microsoft.Azure.Cosmos.Query.Core.Metrics
{
using System;
using System.Collections.Generic;
using System.Linq;

/// <summary>
/// Stores client side QueryMetrics.
Expand Down Expand Up @@ -54,42 +53,5 @@ public ClientSideMetrics(
/// Gets the Fetch Execution Ranges for this continuation of the query.
/// </summary>
public IEnumerable<FetchExecutionRange> FetchExecutionRanges { get; }

public ref struct Accumulator
{
public Accumulator(long retries, double requestCharge, IEnumerable<FetchExecutionRange> fetchExecutionRanges)
{
this.Retries = retries;
this.RequestCharge = requestCharge;
this.FetchExecutionRanges = fetchExecutionRanges;
}

public long Retries { get; }

public double RequestCharge { get; }

public IEnumerable<FetchExecutionRange> FetchExecutionRanges { get; }

public Accumulator Accumulate(ClientSideMetrics clientSideMetrics)
{
if (clientSideMetrics == null)
{
throw new ArgumentNullException(nameof(clientSideMetrics));
}

return new Accumulator(
retries: this.Retries + clientSideMetrics.Retries,
requestCharge: this.RequestCharge + clientSideMetrics.RequestCharge,
fetchExecutionRanges: (this.FetchExecutionRanges ?? Enumerable.Empty<FetchExecutionRange>()).Concat(clientSideMetrics.FetchExecutionRanges));
}

public static ClientSideMetrics ToClientSideMetrics(Accumulator accumulator)
{
return new ClientSideMetrics(
retries: accumulator.Retries,
requestCharge: accumulator.RequestCharge,
fetchExecutionRanges: accumulator.FetchExecutionRanges);
}
}
}
}
Loading
Loading