Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MeshTrace not working on deployment without kube service #12045

Closed
slavogiez opened this issue Nov 14, 2024 · 2 comments · Fixed by #12173
Closed

MeshTrace not working on deployment without kube service #12045

slavogiez opened this issue Nov 14, 2024 · 2 comments · Fixed by #12173
Labels
kind/bug A bug triage/accepted The issue was reviewed and is complete enough to start working on it

Comments

@slavogiez
Copy link
Contributor

What happened?

We configured a MeshTrace policy with a Datadog backend.
This works fine and we can see traces in Datadog, but we have one case where it's not working.

We run a curl command from the source service container to another mesh service. We can see the traces from the destination sidecar, but not from the source sidecar.

It looks like it doesn't work when the source service doesn't have a kube service (no server listening).

We configured the following MeshTrace policy :

apiVersion: kuma.io/v1alpha1
kind: MeshTrace
metadata:
  labels:
    k8s.kuma.io/namespace: kong-mesh
    kuma.io/mesh: mesh01
    kuma.io/policy-role: system
  name: trace-default
  namespace: kong-mesh
spec:
  default:
    backends:
      - datadog:
          splitService: true
          url: http://trace-svc.datadog-agent.svc.cluster.local:8126
        type: Datadog
    sampling:
      client: 100
      overall: 100
      random: 100
    tags:
      - literal: xxx
        name: org_name
      - literal: yyy
        name: mesh_name
  targetRef:
    kind: Mesh

And we have the following deployment :

apiVersion: "apps/v1"
kind: "Deployment"
metadata:
  name: curl
spec:
  replicas: 1
  selector:
    matchLabels:
      app: curl
  template:
    metadata:
      annotations:
        kuma.io/sidecar-env-vars: "DD_ENV=dev;DD_SERVICE=curl-mesh-sidecar"
      labels:
        app: curl
        kuma.io/sidecar-injection: "enabled"
        kuma.io/mesh: mesh01
    spec:
      terminationGracePeriodSeconds: 5
      containers:
      - name: curl
        image: alpine/curl
        command: [ "/bin/sh", "-c", "--" ]
        args: [ "while true; do sleep 30; done;" ]

In the XDS config from the source service, we can see the datadog extension, its cluster but not the tracing configuration.
In the manager ui, we can see the MeshTrace policy applied on the DP proxy.
@slavogiez slavogiez added kind/bug A bug triage/pending This issue will be looked at on the next triage meeting labels Nov 14, 2024
@lobkovilya lobkovilya added triage/needs-reproducing Someone else should try to reproduce this and removed triage/pending This issue will be looked at on the next triage meeting labels Nov 18, 2024
@slavogiez
Copy link
Contributor Author

Not sure what changed but now we can see expected spans from the source service. The only remaining issue is that they don't appear in the whole trace, so we're not able to find the source service in the trace.

@lukidzi lukidzi added triage/accepted The issue was reviewed and is complete enough to start working on it and removed triage/needs-reproducing Someone else should try to reproduce this labels Dec 4, 2024
@lukidzi
Copy link
Contributor

lukidzi commented Dec 4, 2024

It seems we don't configure listeners for MeshService.

@lukidzi lukidzi mentioned this issue Dec 4, 2024
Merged
lukidzi added a commit that referenced this issue Jan 7, 2025
@lukidzi @lahabana
fix(meshtrace): add support for real resources (#12173)
1d419f6 
## Motivation

Issue #12045 showed that we didn't
configure real resources

## Implementation information

e1d1ed7 extracted test string to golden
files
f6ae3d1 added unit test showing an
issue
e7d8c6c implementation of real resource
and generated file

Added a function that loops over real resources to configure listeners
for MeshService and MeshMultiZoneService. Since we don't have a
`kuma.io/service` as a destination, we set the output of
`DestinationName(port)`, which is used for clusters.

## Supporting documentation

<!-- Is there a MADR? An Issue? A related PR? -->

Fix #12045

<!--
> Changelog: skip
-->
<!--
Uncomment the above section to explicitly set a [`> Changelog:` entry
here](https://github.com/kumahq/kuma/blob/master/CONTRIBUTING.md#submitting-a-patch)?
-->

---------

Signed-off-by: Lukasz Dziedziak <[email protected]>
Co-authored-by: Charly Molter <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug A bug triage/accepted The issue was reviewed and is complete enough to start working on it
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix(meshtrace): add support for real resources lukidzi/kuma
3 participants
@slavogiez @lobkovilya @lukidzi