Skip to content

Options for Micrometer in Helidon

Jump to bottom
Tim Quinn edited this page Nov 18, 2020 · 11 revisions

Intro

What is Micrometer

Micrometer [1] is

a simple facade over the instrumentation clients for the most popular monitoring systems

with the aim of insulating application developers from particular metrics implementations and their APIs.

Micrometer elsewhere (very briefly)

MicroProfile

Recent discussions on the MicroProfile mailing list suggest that in an upcoming major release (likely MP Metrics 4.0), MP will adopt Micrometer as the (preferred) API. There has been conversation about perhaps dropping JSON output support. In the Micrometer world, different output formats take the form of different MeterRegistry instances which is a bit different from the MP approach and our implementation in which the knowledge of how to format a metric is embedded in each metric's implementation class.

Quarkus

Quarkus supports MicroProfile and Micrometer but prefers the Micrometer API, and it provides a Quarkus extension which maps the MP metrics API to Micrometer.

Terminology

Meters and metrics

A Micrometer meter refers to what MP refers to as a metric: the prescription to collect data of a given type (counter, timing, etc.) from a given point in the app. Micrometer identifies each meter with a name and zero or more tags (as with MP metrics).

In Micrometer, a metric is a single data observation of a meter.

Registry

In Micrometer, a registry is a collection of meters (and their metrics) to be output in the same, particular way. Micrometer's web page currently lists 18 different implementations (including Prometheus). Differences in output among various registry implementations can be push vs. pull as well as different formats of output. Individual meters are registered with a registry.

Micrometer provides a CompositeRegistry which is a collection of registries and meters. Adding a meter to a composite registry adds it to all the collected registries, and a registry added to a composite registry includes all the meters previously registered with the composite registry.

For example, here is one way to approach this for something like Helidon:

  • the app (or a framework) could use and expose to developers a CompositeRegistry
  • Helidon (through configuration) or an app (via a hypothetical Helidon MicrometerSupport class) could add specific registries for specific formats or output techniques
  • the app registers meters with the exposed CompositeRegistry
  • when metrics are pulled, choose which specific registry's format is requested and get the output from that registry for return to the client.

By contrast, in Helidon a registry typically contains a given category of metrics and there are three built-in ones: base, vendor, and application. Each metric implementation knows how to express itself in both Prometheus and JSON formats.

Main types of meters exposed

  • counter - monotonically increasing value
  • gauge - current value of some varying quantity that typically has an upper bound
  • timer - total time and count of a short-duration activity
  • distribution summary - tracks distribution of values over a range (somewhat similar to MP histogram)
  • long-task timer - timer with added functionality for long-running actions (e.g., while the action is in progress, the value is available and alerts can be delivered at reporting intervals)

Micrometer provides various specialized variants of some of these with added behavior.

Simplest example: adding Micrometer/Prometheus to an SE app (no changes to Helidon)

Using the Helidon SE quickstart:

  1. Add the Micrometer dependency: 
    <dependency>
    <groupId>io.micrometer</groupId>
    <artifactId>micrometer-registry-prometheus</artifactId>
    <version>1.6.1</version>
</dependency>
  2. Create an instance of PrometheusMeterRegistry during server start-up and register an endpoint at /micrometer: Main#createRouting 
        private static Routing createRouting(Config config) {
        MetricsSupport metrics = MetricsSupport.create();
        prometheusMeterRegistry = new PrometheusMeterRegistry(PrometheusConfig.DEFAULT); // [1]
        GreetService greetService = new GreetService(config, prometheusMeterRegistry);   // [2]
        HealthSupport health = HealthSupport.builder()
                .addLiveness(HealthChecks.healthChecks())   // Adds a convenient set of checks
                .build();

        return Routing.builder()
                .register(health)                   // Health at "/health"
                .register(metrics)                  // Metrics at "/metrics"
                .get("/micrometer", (req, resp) -> resp.send(prometheusMeterRegistry.scrape())) // [3]
                .register("/greet", greetService)
                .build();
    }
    [1] Creates and saves the Prometheus Micrometer registry instance.
    [2] Passes the registry to the service so it can create a meter (a counter of all get invocations).
    [3] Sets up the /micrometer endpoint which reports the contents of the Micrometer registry.
  3. Change GreetService.java:
    1. Change constructor: 
          GreetService(Config config, MeterRegistry meterRegistry) {
        this.meterRegistry = meterRegistry;                          // [1]
        getCounter = meterRegistry.counter("greeting.get");          // [2]
        greeting.set(config.get("app.greeting").asString().orElse("Ciao"));
    }
    2. Change routing: 
          public void update(Routing.Rules rules) {
        rules
            .get((ServerRequest req, ServerResponse resp) -> { // [3]
                getCounter.increment();
                req.next();
            })
            .get("/", this::getDefaultMessageHandler)
            .get("/{name}", this::getMessageHandler)
            .put("/greeting", this::updateGreetingHandler);
    }
    [1] Saves the meter registry.
    [2] Creates the counter in the meter registry to count gets.
    [3] Adds a handler for all gets which updates the get counter.

Proposed strategy for Micrometer and Helidon MP (and SE)

Short-term

  1. Provide a simple io.helidon.metrics.micrometer:helidon-micrometer module containing a MicrometerSupport class -- and document it -- to make it easier than the example above for developers to use Micrometer from their apps.

    This would not change any of the existing functionality around metrics in Helidon. It would purely be a way developers could add Micrometer support to their SE or MP app. There is one Micrometer annotation -- @Timed -- which we could support in a Helidon MP Micrometer module.

    Key features:

    1. /micrometer endpoint (changeable by configuration)
    2. By default, support the Prometheus Micrometer registry, configurable from Helidon config.
    3. Possibly add our own registry that provides JSON output. (Quarkus has this feature for their Micrometer support.)
    4. Allow the app to add other Micrometer registries to the Helidon-managed Micrometer support.
      1. We would need to decide how to tell at runtime which Micrometer registry (that is, what output format) to use in response to a given request to the endpoint. Today we use the media type: text/plain means Prometheus, application/json means JSON. There are many Micrometer registry implementations out there which a developer might pull into an app. We probably need to allow a query parameter which specifies which Micrometer registry to override what we infer from the media type.
      2. There is no abstract method defined by MeterRegistry for producing its output; each implementation has its own way. For a simple Helidon Micrometer module we do not want to be responsible for knowing how to do this for each implementation. We will need a way -- either through configuration or by the application passing us a lambda, for example -- to let the developer tell us how to retrieve output from a particular MeterRegistry implementation. Our module would know how to do this for the PrometheusMeterRegistry.

  2. Write a short blog about the new Micrometer support and how to use it.

Medium-term

Investigate how the Quarkus extension maps MP metrics to Micrometer, in particular how they handle some of the misalignments that have been discussed in the MP Google group. Providing a similar adapter in Helidon could be useful to our developers and could be a good step for us toward eventual full migration to Micrometer in Helidon (if that's what MP decides to do).

Long-term

The current MP metrics model permeates the Helidon SE and MP metrics implementation. Basically, except for annotations and interceptors, Helidon SE metrics is Helidon MP metrics.

There has been extensive discussion in the MP Google group [2] about MP metrics.next and Micrometer. It looks as if MP will adopt Micrometer as the preferred metrics API. Some existing MP metrics do not align perfectly with their Micrometer counterpart meters (e.g., timers, histograms) and those incompatibilities will need to be sorted out by MP.

Plan: Wait for the dust to settle with MP metrics.next, see how they plan to handle incompatibilities, what backward compatibility support they plan to offer, then do the same in our MP and SE implementations.

[1] https://micrometer.io/docs/concepts
[2] https://groups.google.com/g/microprofile/c/E-kWz47VDVg/m/acLqUxlEAgAJ

Clone this wiki locally