Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[extension/encoding] support skywalkingencoding extension #36968

Open
wants to merge 33 commits into
base: main
Choose a base branch
from

Conversation

JaredTan95
Copy link
Member

@JaredTan95 JaredTan95 commented Dec 27, 2024

Description

The kafkareceiver supports jaeger and zipkin protocol traces, and can also receive skywalkng traces via skywalkingreceiver.

From the practical use of skywalking, we often use kafka to host large-scale traces data. However, kafka receiver does not support skywalking encoding currently.

with this PR, we can fectch skywalking traces and parse it into otlp traces:

receivers:
  otlp:
    protocols:
      grpc:
        endpoint: 0.0.0.0:4317
  kafka:
    encoding: skywalking_encoding
    topic: skywalking-segments
    brokers:
      - "localhost:9092"
    resolve_canonical_bootstrap_servers_only: true
    client_id: otel-collector
    group_id: otel-collector
    metadata:
      retry:
        max: 10
        backoff: 5s

exporters:
  debug:

processors:
  batch:

extensions:
  health_check:
  pprof:
    endpoint: :1888
  zpages:
    endpoint: :55679
  skywalking_encoding:

service:
  extensions: [pprof, zpages, skywalking_encoding, health_check]
  pipelines:
    traces:
      receivers: [kafka]
      processors: [batch]
      exporters: [debug]

Link to tracking issue

Fixes #28693

Testing

Documentation

@JaredTan95 JaredTan95 changed the title [extension/encoding] suuport Skywalkingencoding extension [extension/encoding] support skywalkingencoding extension Dec 27, 2024
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
@JaredTan95 JaredTan95 force-pushed the skywalkingencoding_extension branch from 8d1b0e5 to b01b7a1 Compare December 27, 2024 06:28
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
.github/CODEOWNERS Outdated Show resolved Hide resolved
@JaredTan95
Copy link
Member Author

Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Signed-off-by: Jared Tan <[email protected]>

Signed-off-by: Jared Tan <[email protected]>
Copy link
Contributor

@fatsheep9146 fatsheep9146 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you know how to fix the check-owners failed lint?

@JaredTan95
Copy link
Member Author

Do you know how to fix the check-owners failed lint?

logs show that I need to do some changes:

- extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana
+ extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana @JaredTan95 

in .github/CODEOWNERS file, but @atoulme told me do it in separate PR, so, I'm confused~

@crobert-1
Copy link
Member

Do you know how to fix the check-owners failed lint?

logs show that I need to do some changes:

- extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana
+ extension/encoding/                               @open-telemetry/collector-contrib-approvers @atoulme @dao-jun @dmitryax @MovieStoreGuy @VihasMakwana @JaredTan95 

in .github/CODEOWNERS file, but @atoulme told me do it in separate PR, so, I'm confused~

This is the current action failure output:

Generated code is out of date, please apply this diff and commit the changes in this PR.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
index 004df01..3baf60[9](https://github.com/open-telemetry/opentelemetry-collector-contrib/actions/runs/12665959423/job/35296498228?pr=36968#step:5:10) 100644
--- a/.github/CODEOWNERS
+++ b/.github/CODEOWNERS
@@ -95,6 +95,7 @@ extension/encoding/avrologencodingextension/      @open-telemetry/collector-cont
 extension/encoding/jaegerencodingextension/       @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/encoding/jsonlogencodingextension/      @open-telemetry/collector-contrib-approvers @VihasMakwana @atoulme
 extension/encoding/otlpencodingextension/         @open-telemetry/collector-contrib-approvers @dao-jun @VihasMakwana
+extension/encoding/skywalkingencodingextension/   @open-telemetry/collector-contrib-approvers @JaredTan95
 extension/encoding/textencodingextension/         @open-telemetry/collector-contrib-approvers @MovieStoreGuy @atoulme
 extension/encoding/zipkinencodingextension/       @open-telemetry/collector-contrib-approvers @MovieStoreGuy @dao-jun
 extension/googleclientauthextension/              @open-telemetry/collector-contrib-approvers @dashpole @aabmass @jsuereth @punya @psx95
Error: Process completed with exit code 1.

It's simply missing a line for the new extension you're adding, it's no longer complaining about the extension/encoding entry 👍

@JaredTan95 JaredTan95 added the ready to merge Code review completed; ready to merge by maintainers label Jan 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cmd/otelcontribcol otelcontribcol command extension/encoding ready to merge Code review completed; ready to merge by maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[receiver/kafka] support receive skywalking traces from kafka
5 participants