Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Add Adaptive Sampling Support for gRPC Remote Storage #6308

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

akstron
Copy link
Contributor

@akstron akstron commented Dec 4, 2024

Which problem is this PR solving?

Description of the changes

How was this change tested?

  • Not tested yet

Checklist

Signed-off-by: Alok Kumar Singh <[email protected]>
@akstron
Copy link
Contributor Author

akstron commented Dec 4, 2024

Creating a draft PR to get reviews if I am on the right track or probably missing something. Thanks

Copy link

codecov bot commented Dec 4, 2024

Codecov Report

Attention: Patch coverage is 70.96774% with 45 lines in your changes missing coverage. Please review.

Project coverage is 49.06%. Comparing base (bcccf81) to head (2e23786).
Report is 19 commits behind head on main.

Files with missing lines Patch % Lines
plugin/storage/grpc/shared/grpc_client.go 69.38% 10 Missing and 5 partials ⚠️
plugin/storage/grpc/shared/grpc_handler.go 58.33% 10 Missing and 5 partials ⚠️
plugin/storage/grpc/shared/sampling_store.go 79.66% 9 Missing and 3 partials ⚠️
cmd/remote-storage/app/server.go 62.50% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #6308      +/-   ##
==========================================
+ Coverage   48.78%   49.06%   +0.27%     
==========================================
  Files         181      183       +2     
  Lines       11111    11352     +241     
==========================================
+ Hits         5421     5570     +149     
- Misses       5241     5317      +76     
- Partials      449      465      +16     
Flag Coverage Δ
badger_v1 8.65% <0.00%> (-0.25%) ⬇️
badger_v2 1.58% <0.00%> (-0.04%) ⬇️
cassandra-4.x-v1-manual 14.42% <0.00%> (-0.42%) ⬇️
cassandra-4.x-v2-auto 1.53% <0.00%> (-0.04%) ⬇️
cassandra-4.x-v2-manual 1.53% <0.00%> (-0.04%) ⬇️
cassandra-5.x-v1-manual 14.42% <0.00%> (-0.42%) ⬇️
cassandra-5.x-v2-auto 1.53% <0.00%> (-0.04%) ⬇️
cassandra-5.x-v2-manual 1.53% <0.00%> (-0.04%) ⬇️
elasticsearch-6.x-v1 18.08% <0.00%> (-0.52%) ⬇️
elasticsearch-7.x-v1 18.16% <0.00%> (-0.52%) ⬇️
elasticsearch-8.x-v1 18.32% <0.00%> (-0.52%) ⬇️
elasticsearch-8.x-v2 1.57% <0.00%> (-0.05%) ⬇️
grpc_v1 11.46% <70.96%> (+1.09%) ⬆️
grpc_v2 7.85% <3.87%> (-0.02%) ⬇️
kafka-v1 8.34% <0.00%> (-0.25%) ⬇️
kafka-v2 1.58% <0.00%> (-0.04%) ⬇️
memory_v2 1.58% <0.00%> (-0.04%) ⬇️
opensearch-1.x-v1 18.20% <0.00%> (-0.53%) ⬇️
opensearch-2.x-v1 18.20% <0.00%> (-0.52%) ⬇️
opensearch-2.x-v2 1.58% <0.00%> (-0.04%) ⬇️
tailsampling-processor 0.44% <0.00%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@yurishkuro
Copy link
Member

you are on the right track, but before you continue, what is your plan for e2e testing?

Signed-off-by: Alok Kumar Singh <[email protected]>
@akstron
Copy link
Contributor Author

akstron commented Dec 5, 2024

you are on the right track, but before you continue, what is your plan for e2e testing?

I was thinking of using one of the already created backend implementation with a gRPC connection.

@yurishkuro
Copy link
Member

ok. Maybe start with that.

@akstron akstron marked this pull request as ready for review December 8, 2024 11:33
@akstron akstron requested a review from a team as a code owner December 8, 2024 11:33
@akstron akstron requested a review from mahadzaryab1 December 8, 2024 11:33
@akstron akstron changed the title [WIP] Add Adaptive Sampling Support for gRPC Remote Storage Add Adaptive Sampling Support for gRPC Remote Storage Dec 8, 2024
@@ -67,12 +68,17 @@ func createGRPCHandler(f storage.BaseFactory, logger *zap.Logger) (*shared.GRPCH
if err != nil {
return nil, err
}
samplingStore, err := samplingStoreFactory.CreateSamplingStore(1)
Copy link
Member

@yurishkuro yurishkuro Dec 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for readability, define named constant instead of passing unnamed 1

@@ -374,6 +377,7 @@ func TestServerHandlesPortZero(t *testing.T) {
storageMocks.factory,
tenancy.NewManager(&tenancy.Options{}),
telset,
nil,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a test where something real is passed

@@ -77,7 +77,7 @@ func main() {
tm := tenancy.NewManager(&opts.Tenancy)
telset := baseTelset // copy
telset.Metrics = metricsFactory
server, err := app.NewServer(opts, storageFactory, tm, telset)
server, err := app.NewServer(opts, storageFactory, tm, telset, nil)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how will it work if you pass nil?

@@ -148,6 +148,63 @@ message FindTraceIDsResponse {
];
}

message Throughput {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema should be documented. Wherever you are copying this from, we have comments explaining all types.

@@ -112,6 +112,7 @@ func TestNewGRPCClient(t *testing.T) {
assert.Implements(t, (*storage_v1.PluginCapabilitiesClient)(nil), client.capabilitiesClient)
assert.Implements(t, (*storage_v1.DependenciesReaderPluginClient)(nil), client.depsReaderClient)
assert.Implements(t, (*storage_v1.StreamingSpanWriterPluginClient)(nil), client.streamWriterClient)
assert.Implements(t, (*storage_v1.SamplingStorePluginClient)(nil), client.samplingStoreClient)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no unit tests for new functions?

Signed-off-by: Alok Kumar Singh <[email protected]>
@akstron akstron changed the title Add Adaptive Sampling Support for gRPC Remote Storage [WIP] Add Adaptive Sampling Support for gRPC Remote Storage Dec 16, 2024
@akstron akstron marked this pull request as draft December 18, 2024 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants