Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stripe/veneur docker images fail using default configuration with obtuse error #970

Open
ramfjord opened this issue Aug 4, 2022 · 3 comments

Comments

@ramfjord
Copy link

ramfjord commented Aug 4, 2022

I've attempted to use the official seeming images tag versions 14.2.0 (then noticed the releases section doesn't say this is in released status), 14.1.0, 13.2.0 so far. These are failing with an error like this:

$ docker run -t stripe/veneur:14.1.0 ./veneur -f config.yaml
WARNING: Package "github.com/golang/protobuf/protoc-gen-go/generator" is deprecated.
	A future release of golang/protobuf will delete this package,
	which has long been excluded from the compatibility promise.

INFO[0000] Set mutex profile fraction                    MutexProfileFraction=0 previousMutexProfileFraction=0
INFO[0000] Set block profile rate (nanoseconds)          BlockProfileRate=0
INFO[0000] Preparing workers                             number=96
INFO[0000] Creating X-Ray client                         Address="localhost:2000"
INFO[0000] Configured X-Ray span sink                    num_annotation_tags="[]" sample_percentage=100
INFO[0000] Configured Falconer trace sink
INFO[0000] Configured Prometheus metric sink.
INFO[0000] Creating SignalFx sink signalfx               endpoint_base="https://ingest.signalfx.com" metric_sink=signalfx
WARN[0000] Unknown sink kind signalfx; skipping.
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x118d5fc]

goroutine 1 [running]:
github.com/stripe/veneur/v14.(*Server).createSpanSinks(0x166b140, 0xc000208d20, 0xc00041d278, 0xc000592730)
	/veneur/server.go:324 +0x1bc
github.com/stripe/veneur/v14.NewFromConfig({{{0xc00044ced0, 0x3, 0x3}, {{0x0, 0x0}}, {0x0, 0x0}, {0x0, 0x0}, {{0x0, ...}}, ...}, ...})
	/veneur/server.go:716 +0x2025
main.main()
	/veneur/cmd/veneur/main.go:75 +0x14fc

I was assuming this would have been built using the 14.1.0 release tag in the example above, but when I look at server.go:324 on that release tag, it's an empty line, which seems to me to prove that this binary was built... I have no idea where, but not using the 14.1.0 release tag! I'm aware this probably should fail in some way without supplying my datadog key, but it would be much easier for me to figure out what is wrong with my supplied configuration if there were at least some kind of error message related to what's wrong with the config here. I'm thinking my next step here should be to compile from source so I can actually know where my errors are coming from at least, but is this an officially maintained image that I'm trying to use? Also, is the current release 14.2.0 or 14.1.0? The release notes for 14.2.0 don't have a release date like the others.

@ramfjord
Copy link
Author

ramfjord commented Aug 4, 2022

Well, the specific issue on 14.1.0's default config is something I've fixed by going through all of the various sink config options (grepping for SpanSinks in code), and I have currently got my instance working by setting a bunch of options to empty string by default. The last one that got it finally working was disabling the various splunk options referenced here (still using stripe/veneur:14.1.0 image)

@ramfjord
Copy link
Author

ramfjord commented Aug 5, 2022

I suppose I'm asking for a few things here, and feel free to work on or deny any of them:

  • Please update the tag for 14.1.0 to reference the commit used to generate the stripe/veneur:14.1.0 docker image
  • Either: note that 14.2.0 is not in a "released" state in the dockerhub page, or say that it's been released in the CHANGELOG.md
  • The default config file should load without errors, or the errors should explain what fields need to be configured
  • When the config file contains invalid config, it would be awesome if it said what was invalid/missing. At least once, in my case, that error should have been: "Can't load splunk config because..."

@ramfjord
Copy link
Author

ramfjord commented Aug 18, 2022

BTW it appears the 14.1.0 tag should probably be referencing the b73d186 commit, which I found by checking the update time of the docker image, and seems to correspond with file/line numbers to the various error messages I was getting

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant