Skip to content

Commit

Permalink
symbolication: remove noisy error log
Browse files Browse the repository at this point in the history
when we only filter out fs.ErrNotExist, we get frequent failures
in the local development environment with the following error:
/proc/<pid>/root// is a directory

we seem to think that /proc/<pid>/root// is a executable path,
this happens for short-lived runc processes where /proc/<pid>/maps
has "/" as the pathname for the runc executable itself.

so when we try to open the elf file, we first try to read it from
/proc/<pid>/map_files and (as the process is short lived) when
this fails, we try to open /proc/<pid>/root// which fails with
a "is a directory" error instead of fs.ErrNotExist

to fix this, we just ignore all errors returned by GetElf() and
only log them for debug purposes.

https://github.com/elastic/otel-profiling-agent/blob/0945fe6/libpf/process/process.go#L221-L239
  • Loading branch information
Gandem committed Jun 21, 2024
1 parent 036a0ba commit 967d1d5
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 9 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ For this to work you need to run a Datadog agent that listens for APM traffic at

### Local symbol upload (Experimental)

For compiled languages (C/C++/Rust/Go), the profiling-agent can upload local symbols (when available) to Datadog for symbolication. Symbols need to be available locally (unstripped binairies).
For compiled languages (C/C++/Rust/Go), the profiling-agent can upload local symbols (when available) to Datadog for symbolication. Symbols need to be available locally (unstripped binaries).

To enable local symbol upload:
1. Set the `DD_EXPERIMENTAL_LOCAL_SYMBOL_UPLOAD` environment variable to `true`.
Expand Down
13 changes: 5 additions & 8 deletions symbolication/datadog_uploader.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,8 @@ import (
"compress/gzip"
"context"
"encoding/json"
"errors"
"fmt"
"io"
"io/fs"
"mime/multipart"
"net/http"
"net/textproto"
Expand Down Expand Up @@ -68,17 +66,16 @@ func (d *DatadogUploader) HandleExecutable(ctx context.Context, elfRef *pfelf.Re
// If the ELF file is not found, we ignore it
// This can happen for short-lived processes that are already gone by the time
// we try to upload symbols
if errors.Is(err, fs.ErrNotExist) {
log.Debugf("Skipping executable %s as it does not exist", fileName)
return nil
}
if err != nil {
return fmt.Errorf("could not get ELF: %w", err)
log.Debugf("Skipping symbol upload for executable %s: %v",
fileName, err)
return nil
}

// We only upload symbols for executables that have DWARF data
if !ef.HasDWARFData() {
log.Debugf("Skipping executable %s as it does not have DWARF data", fileName)
log.Debugf("Skipping symbol upload for executable %s as it does not have DWARF data",
fileName)
return nil
}

Expand Down

0 comments on commit 967d1d5

Please sign in to comment.