http-get with --url returns an error, but exits 0

[asachs01]

When running the following check:

type: CheckConfig
api_version: core/v2
metadata:
  annotations:
    runbook: ""
    team: sre
  created_by: admin
  labels:
    sensu.io/managed_by: sensuctl
  name: sensu-backend-metrics
  namespace: monitoring
spec:
  check_hooks: null
  command: >-
    http-get --timeout 10 --redirect-ok --url https://{{ .system.hostname }}:8080/metrics
  env_vars: null
  handlers: []
  high_flap_threshold: 0
  interval: 10
  low_flap_threshold: 0
  output_metric_format: prometheus_text
  output_metric_handlers:
  - influxdb
  pipelines: []
  proxy_entity_name: ""
  publish: true
  round_robin: false
  runtime_assets:
  - sensu/http-checks
  secrets: null
  stdin: false
  subdue: null
  subscriptions:
  - sensu-backend
  timeout: 5
  ttl: 0

What's more, the check exits with a 0 rather than an error:

Event info

{
  "type": "Event",
  "api_version": "core/v2",
  "metadata": {
    "namespace": "monitoring"
  },
  "spec": {
    "check": {
      "check_hooks": null,
      "command": "http-get --timeout 10 --redirect-ok --url https://{{ .system.hostname }}:8080/metrics",
      "duration": 0.008565643,
      "env_vars": null,
      "executed": 1653677971,
      "handlers": [],
      "high_flap_threshold": 0,
      "history": [
        {
          "executed": 1653677771,
          "status": 0
        },
        {
          "executed": 1653677781,
          "status": 0
        },
        {
          "executed": 1653677791,
          "status": 0
        },
        {
          "executed": 1653677801,
          "status": 0
        },
        {
          "executed": 1653677811,
          "status": 0
        },
        {
          "executed": 1653677821,
          "status": 0
        },
        {
          "executed": 1653677831,
          "status": 0
        },
        {
          "executed": 1653677841,
          "status": 0
        },
        {
          "executed": 1653677851,
          "status": 0
        },
        {
          "executed": 1653677861,
          "status": 0
        },
        {
          "executed": 1653677871,
          "status": 0
        },
        {
          "executed": 1653677881,
          "status": 0
        },
        {
          "executed": 1653677891,
          "status": 0
        },
        {
          "executed": 1653677901,
          "status": 0
        },
        {
          "executed": 1653677911,
          "status": 0
        },
        {
          "executed": 1653677921,
          "status": 0
        },
        {
          "executed": 1653677931,
          "status": 0
        },
        {
          "executed": 1653677941,
          "status": 0
        },
        {
          "executed": 1653677951,
          "status": 0
        },
        {
          "executed": 1653677961,
          "status": 0
        },
        {
          "executed": 1653677971,
          "status": 0
        }
      ],
      "interval": 10,
      "is_silenced": false,
      "issued": 1653677971,
      "last_ok": 1653677971,
      "low_flap_threshold": 0,
      "metadata": {
        "annotations": {
          "runbook": "",
          "team": "sre"
        },
        "labels": {
          "sensu.io/managed_by": "sensuctl"
        },
        "name": "sensu-backend-metrics",
        "namespace": "monitoring"
      },
      "occurrences": 245,
      "occurrences_watermark": 245,
      "output": "Error executing http-get: unknown command \"https://sensu02.sachshaus.net:8080/metrics\" for \"http-get\"\n",
      "output_metric_format": "prometheus_text",
      "output_metric_handlers": [
        "influxdb"
      ],
      "pipelines": [],
      "processed_by": "sensu02.sachshaus.net",
      "proxy_entity_name": "",
      "publish": true,
      "round_robin": false,
      "runtime_assets": [
        "sensu/http-checks"
      ],
      "scheduler": "memory",
      "secrets": null,
      "state": "passing",
      "status": 0,
      "stdin": false,
      "subdue": null,
      "subscriptions": [
        "sensu-backend"
      ],
      "timeout": 5,
      "total_state_change": 0,
      "ttl": 0
    },
    "entity": {
      "deregister": false,
      "deregistration": {},
      "entity_class": "agent",
      "last_seen": 1653677971,
      "metadata": {
        "labels": {
          "sensu.io/managed_by": "sensu-agent"
        },
        "name": "sensu02.sachshaus.net",
        "namespace": "monitoring"
      },
      "redact": [
        "password",
        "passwd",
        "pass",
        "api_key",
        "api_token",
        "access_key",
        "secret_key",
        "private_key",
        "secret"
      ],
      "sensu_agent_version": "6.7.1",
      "subscriptions": [
        "linux",
        "system",
        "monitoring",
        "sensu-backend",
        "ssl",
        "entity:sensu02.sachshaus.net"
      ],
      "system": {
        "arch": "amd64",
        "cloud_provider": "",
        "hostname": "sensu02.sachshaus.net",
        "libc_type": "glibc",
        "network": {
          "interfaces": [
            {
              "addresses": [
                "127.0.0.1/8",
                "::1/128"
              ],
              "name": "lo"
            },
            {
              "addresses": [
                "",
                ""
              ],
              "mac": "",
              "name": ""
            },
            {
              "addresses": [
                "redacted"
              ],
              "name": "tailscale0"
            }
          ]
        },
        "os": "linux",
        "platform": "ubuntu",
        "platform_family": "debian",
        "platform_version": "20.04",
        "processes": null,
        "vm_role": "guest",
        "vm_system": "kvm"
      },
      "user": "agent"
    },
    "id": "6b67e77f-d0c0-4bb5-b376-691ab463b9dd",
    "pipelines": null,
    "sequence": 15650,
    "timestamp": 1653677971
  }
}

Steps to replicate

1. Create a check using the example above
2. Manually execute check (or wait)
3. Observe the error

Environment

• OS: Ubuntu 20.04
• Deployment: Sensu Go .deb package
• Sensu Go version: 6.7.1

[jspaleta]

I think this partly an sdk bug, a catalog integration bug and possibly something lurking in this plugin specifically.

Findings

• --redirect-ok is not supported by http-get as documented in http-get --help This is a catalog integration readme bug, being fix asap.
• you get different error messages depending on where in the argument list an unknown argument is placed. This is possibly an sdk bug (needs to be investigated)
• this command is not throwing an error return status in some cases when an unknown argument is used, possibly sdk bug or a bug specific to this plugin (needs to be investigated)