Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add telemetrygen-based OTLP bench to apmbench #55

Closed
wants to merge 1 commit into from
Closed

Add telemetrygen-based OTLP bench to apmbench #55

wants to merge 1 commit into from

Conversation

carsonip
Copy link
Member

@carsonip carsonip commented Feb 15, 2024 •
edited
Loading

Use a otel collector contrib fork for a patch such that telemetrygen internal packages can be used: carsonip/opentelemetry-collector-contrib@b3fab3f

Output of apmbench -benchtime=1m -count=2 -run=^BenchmarkTelemetrygen

--- SKIP: Benchmark1000Transactions
--- SKIP: BenchmarkOTLPTraces
--- SKIP: BenchmarkAgentAll
--- SKIP: BenchmarkAgentGo
--- SKIP: BenchmarkAgentNodeJS
--- SKIP: BenchmarkAgentPython
--- SKIP: BenchmarkAgentRuby
--- SKIP: Benchmark10000AggregationGroups
2024-02-15T14:11:54.032Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:11:54.034Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:11:54.034Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 1}
2024-02-15T14:11:54.034Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:11:54.035Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:11:54.037Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:11:54.038Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:11:54.039Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 100}
2024-02-15T14:11:54.039Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:11:54.045Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:11:54.046Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:11:54.047Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:11:54.129Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 10000}
2024-02-15T14:11:54.129Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:11:54.200Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:11:54.203Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:11:54.207Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:12:03.325Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 1000000}
2024-02-15T14:12:03.325Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:12:03.399Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:12:03.401Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:12:03.403Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:13:02.513Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 7829168}
2024-02-15T14:13:02.513Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:13:02.579Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:13:02.581Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:13:02.583Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:14:16.016Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 9525536}
2024-02-15T14:14:16.016Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:14:16.088Z        INFO    traces/traces.go:64     stopping the exporter
BenchmarkTelemetrygenOTLPTraces          9525536              7717 ns/op
2024-02-15T14:15:16.147Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:15:16.147Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:15:16.147Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 1}
2024-02-15T14:15:16.147Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:15:16.148Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:15:16.150Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:15:16.151Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:15:16.151Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 100}
2024-02-15T14:15:16.151Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:15:16.156Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:15:16.158Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:15:16.158Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:15:16.231Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 10000}
2024-02-15T14:15:16.232Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:15:16.289Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:15:16.291Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:15:16.292Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:15:23.986Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 1000000}
2024-02-15T14:15:23.987Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:15:24.058Z        INFO    traces/traces.go:64     stopping the exporter
2024-02-15T14:15:24.060Z        INFO    traces/traces.go:52     starting gRPC exporter
2024-02-15T14:15:24.061Z        INFO    traces/traces.go:114    generation of traces isn't being throttled
2024-02-15T14:16:34.369Z        INFO    traces/worker.go:108    traces generated        {"worker": 0, "traces": 9269620}
2024-02-15T14:16:34.369Z        INFO    traces/traces.go:74     stop the batch span processor
2024-02-15T14:16:34.429Z        INFO    traces/traces.go:64     stopping the exporter
BenchmarkTelemetrygenOTLPTraces          9269620              7591 ns/op
2024-02-15T14:17:34.486Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:17:34.488Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 1}
2024-02-15T14:17:34.490Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:17:34.514Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 100}
2024-02-15T14:17:34.515Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:17:36.915Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 10000}
2024-02-15T14:17:36.917Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:23:15.751Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 300008}
BenchmarkTelemetrygenOTLPLogs             300008           1129418 ns/op
2024-02-15T14:24:15.756Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:24:15.758Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 1}
2024-02-15T14:24:15.760Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:24:15.785Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 100}
2024-02-15T14:24:15.786Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:24:18.184Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 10000}
2024-02-15T14:24:18.186Z        INFO    logs/logs.go:52 generation of logs isn't being throttled
2024-02-15T14:29:59.194Z        INFO    logs/worker.go:69       logs generated  {"worker": 0, "logs": 300280}
BenchmarkTelemetrygenOTLPLogs             300280           1135635 ns/op
2024-02-15T14:30:59.254Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":1,"MetricType":"Sum"}}
2024-02-15T14:30:59.254Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:30:59.254Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:30:59.256Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 1}
2024-02-15T14:30:59.256Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
2024-02-15T14:30:59.258Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":100,"MetricType":"Sum"}}
2024-02-15T14:30:59.258Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:30:59.258Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:30:59.284Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 100}
2024-02-15T14:30:59.284Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
2024-02-15T14:30:59.287Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":10000,"MetricType":"Sum"}}
2024-02-15T14:30:59.287Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:30:59.287Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:31:01.778Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 10000}
2024-02-15T14:31:01.778Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
2024-02-15T14:31:01.780Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":288992,"MetricType":"Sum"}}
2024-02-15T14:31:01.781Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:31:01.781Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:36:32.283Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 288992}
2024-02-15T14:36:32.283Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
BenchmarkTelemetrygenOTLPMetrics          288992           1143639 ns/op
2024-02-15T14:37:32.334Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":1,"MetricType":"Sum"}}
2024-02-15T14:37:32.334Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:37:32.334Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:37:32.336Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 1}
2024-02-15T14:37:32.336Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
2024-02-15T14:37:32.338Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":100,"MetricType":"Sum"}}
2024-02-15T14:37:32.338Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:37:32.338Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:37:32.363Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 100}
2024-02-15T14:37:32.363Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
2024-02-15T14:37:32.365Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":10000,"MetricType":"Sum"}}
2024-02-15T14:37:32.365Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:37:32.365Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:37:34.825Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 10000}
2024-02-15T14:37:34.825Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
2024-02-15T14:37:34.827Z        INFO    metrics/metrics.go:30   starting the metrics generator with configuration       {"config": {"WorkerCount":1,"Rate":0,"TotalDuration":0,"ReportingInterval":0,"SkipSettingGRPCLogger":true,"CustomEndpoint":"127.0.0.1:8200","Insecure":true,"UseHTTP":false,"HTTPPath":"/v1/metrics","Headers":{},"ResourceAttributes":null,"TelemetryAttributes":null,"CaFile":"","ClientAuth":{"Enabled":false,"ClientCertFile":"","ClientKeyFile":""},"NumMetrics":292704,"MetricType":"Sum"}}
2024-02-15T14:37:34.827Z        INFO    metrics/metrics.go:81   generation of metrics isn't being throttled
2024-02-15T14:37:34.827Z        INFO    metrics/metrics.go:49   starting gRPC exporter
2024-02-15T14:43:09.328Z        INFO    metrics/worker.go:103   metrics generated       {"worker": 0, "metrics": 292704}
2024-02-15T14:43:09.328Z        INFO    metrics/worker.go:41    stopping the exporter   {"worker": 0}
BenchmarkTelemetrygenOTLPMetrics          292704           1142798 ns/op
{"level":"info","ts":1708008249.376862,"caller":"apmbench/main.go:61","msg":"finished running benchmarks"}

@carsonip
Add telemetry OTLP bench to apmbench
f9702fd 
Use a otel collector contrib fork for a patch
@carsonip carsonip mentioned this pull request Feb 20, 2024
Merged
@carsonip
Copy link
Member Author

Close as alternative #58 is merged

@carsonip carsonip closed this Mar 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@carsonip