Explore alternatives to Pingdom

[digitalronin]

Virtually all teams in the MoJ use Pingdom for uptime monitoring, and alerting via Slack and/or pagerduty.

Pingdom have no official terraform provider, so part of the process of setting up monitoring for services relies on clickops.

Explore alternatives such as StatusCake to try and find something with better support for our preferred infrastructure as code approach.

related to #57

[SteveMarshall]

I'd be a little hesitant to move to another supplier just because of the lack of Terraform provider: this one for example, seems like a fairly well maintained, featureful community one.

That said, if they're as good but maintain their own provider, and we can see a relatively easy migration path, I'd be game.

[digitalronin]

That's the provider we're using. It's OK, but has a couple of issues:

• It only allows you to manage "checks", but each service also needs an "integration" - to get that, we have to create it via clickops and then get its numeric ID by interpreting the URL or inspecting the HTML of the pingdom web UI (seriously). The problem with that is a) clickops, and b) nothing ever gets cleaned up, so for example a lot of the integrations that exist are for hipchat.
• There's a long-standing issue with the list of tags being returned in an arbitrary order, so when you do a terraform plan, and nothing has changed, there's a good chance the plan will tell you it needs to make a change: tags: foo,bar -> tags: bar,foo That sounds trivial, but it means you can't have a reliable regular task to compare your code to the reality and report whenever they drift out of sync.

To be fair, that provider is being updated, and maybe our best strategy would be to raise PRs for the fixes we'd like to see.

I think it's only worth considering moving to a different supplier if we can properly manage everything in code. So, I think it's worth exploring if such a service exists.

[digitalronin]

This open-source, github actions-based monitoring system looks interesting, although it lacks the pingdom feature of multiple different points of origin for service checks:
https://upptime.js.org/

[AntonyBishop]

Requires User Research

[AntonyBishop]

Stale.