[bug] consul-template does not fail on missing key despite configuration

[akamensky]

Version

consul-template v0.35.0 (2d2654f)

Steps to reproduce

1. Use CLI options list -once and -template-error-fatal=<bool>
2. OR use configuration file as below on screenshot
3. Use template that references key that does not exist
4. Run consul-template with either CLI or config

Expected outcome

consul-template fails to process template on a key lookup

Actual outcome

consul-template waits indefinitely for key to appear in KV

Screenshot

References

Are there any other GitHub issues (open or closed) that should
be linked here? For example:

• Fetaure request: a way for consul-template to exit upon a failed key lookup #1637 was closed as done, but it doesn't work (writing template condition to check whether key exists is quite silly approach in my opinion)

Suggested solutions (not exclusive)

1. Have a CLI flag that will cause consul-template to exit with error code when a key referenced in template does not exist in KV (or not set via envvars). (generally would be nice to have parity between config file options and CLI arguments).
2. Make a configuration file option err_on_failed_lookup = true actually work as advertised
3. Add a default configuration file location (now there seems to be none, judging by the code), for example /etc/consul-template.[hcl|json]

[brettcave]

err_on_failed_lookup was added in 0.30.0, but is only available as a config file option (#1637)

For this request, would it be possible for 2 enhancements:

1. Add as a CLI flag - e.g. -err-on-failed-lookup
2. Allow to control this for specific keys - similar to how mustEnv works for environmentVariables: add mustEnv function #1657 - where certain keys which must come from consul can be defined with mustKey - this will give flexibility to have a mix of keyOrDefault for a fallback, key for polling / waits and mustKey for failfast mode.

@akamensky : the failed key behaviour definitely works in version 0.30 or later.

Set up a config file - in the file below, i allow it to wait for 2 seconds to try resolve the key - the first 2 options are needed for the behaviour to work:

$ cat path/to/config.hcl
err_on_failed_lookup = true
template_error_fatal = true
block_query_wait = "2s"

Run consul-template with -config path/to/config.hcl argument.

2024-03-18T23:09:53.025Z [DEBUG] (watcher) adding kv.block(noSuchKeyInConsulForTesting)
2024-03-18T23:09:53.025Z [DEBUG] (runner) diffing and updating dependencies
2024-03-18T23:09:53.025Z [DEBUG] (runner) watching 1 dependencies
2024-03-18T23:09:55.120Z [ERR] (runner) watcher reported error: lookup error
2024-03-18T23:09:55.120Z [ERR] (cli) lookup error

2s delay with exit failure can be seen.

[akamensky]

@brettcave the config option didn't work for me using the version provided in top post. OTOH without CLI option and needing to always provide location of configuration every time makes it very hard to use configuration file in environments that are managed programmatically.

[brettcave]

@brettcave the config option didn't work for me using the version provided in top post. OTOH without CLI option and needing to always provide location of configuration every time makes it very hard to use configuration file in environments that are managed programmatically.

@akamensky - yep - i think a CLI option + ability to control "must have key" vs "wait and poll" or "fall back to default" gives a lot more flexibility for this