Spring Boot Metrics 3.0

Micrometer 1.10 Observation

Micrometer 1.10 ships with a new Observation concept that enables both Metrics and Traces. With Spring Boot 3.0, we are starting to use the new infrastructure and will continue to work on this theme in the 3.x generation.

So far, Metrics were supported in Spring Boot in the "spring-boot-actuator" module (and configured by the "spring-boot-actuator-autoconfigure" one). Metrics support offers a way to create timers, gauges or counters for collecting information about your application at runtime. For example, Spring Boot 2.x instruments Spring MVC to time how long it takes to process HTTP requests, with a "http.server.requests" timer.

Applications previously required the "spring-cloud-sleuth" project to instrument their code and produce Traces. Traces provide a holisitic view of an entire system, crossing application boundaries. This project will be now replaced by Observation instrumentation, as traces can be produced from observations.

Individual Spring projects and 3rd party libraries can now instrument their codebase directly using the Observation API, effectively replacing the instrumentation shipped in "spring-boot-actuator" and "spring-cloud-sleuth".

In Spring Boot 3.0, we are now auto-configuring the instrumentation for Spring for GraphQL, Spring MVC, Spring WebFlux, RestTemplate and WebClient. In most cases, no action will be required on your side, as your application will continue sending metrics and now will also record traces if tracing is configured. The Spring team works on making the upgrade as smooth as possible, so please report any problem (unexpected change of metrics behavior, tags, broken dashboards) in our issue tracker.

Managing Spring Boot Metrics deprecations

As a result of this new integration with the Observation support, we are now deprecating the previous instrumentation. The filters, interceptors performing the actual instrumentation have been removed entirely, as entire classes of bugs could not be resolved and the risk of duplicate instrumentation was too high. For example, the WebMvcMetricsFilter has been deleted entirely and is effectively replaced by Spring Framework’s ServerHttpObservationFilter. The corresponding *TagProvider *TagContributor and *Tags classes have been deprecated. They are not used by default anymore by the observation instrumentation. We are keeping them around during the deprecation phase so that developers can migrate their existing infrastructure to the new one.

Micrometer Observation concepts

If you are seeing new deprecations in your codebase regarding metrics, here’s a quick summary of the new concepts you should know.

• Observation is the actual recording of something happening in your application. This is processed by ObservationHandler implementations to produce metrics or traces.

• Each observation has a corresponding ObservationContext implementation; this class holds all the relevant information for extracting metadata for it. In the case of an HTTP server observation, the context implementation could hold the HTTP request, the HTTP response, any Exception thrown during processing…​

• Each Observation holds KeyValues metadata. In the case of an server HTTP observation, this could be the HTTP request method, the HTTP response status…​ This metadata is contributed by ObservationConvention implementations which should declare the type of ObservationContext they support.

• KeyValues are said to be "low cardinality" if there is a low, bounded number of possible values for the KeyValue tuple (HTTP methods is a good example). Low cardinality values are contributed to metrics only. "High cardinality" are on the other hand unbounded (for example, HTTP request URIs) and are only contributed to Traces.

• An ObservationDocumentation documents all observations in a particular domain, listing the expected key names and their meaning. Check out Spring Framework’s ServerHttpObservationDocumentation as an example.

Migrating Tag providers and contributors

If your application is customizing metrics, you might see new deprecations in your codebase. In our new model, both tag providers and contributors are replaced by observation conventions.

Let’s take the example of the Spring MVC "http.server.requests" metrics instrumentation support in Spring Boot 2.x.

If you are contributing additional Tags with TagContributor or only partially overriding a TagProvider, you should probably extend the DefaultServerRequestObservationConvention for your requirements:

public class ExtendedServerRequestObservationConvention extends DefaultServerRequestObservationConvention {

	@Override
	public KeyValues getLowCardinalityKeyValues(ServerRequestObservationContext context) {
        // here, we just want to have an additional KeyValue to the observation, keeping the default values
		return super.getLowCardinalityKeyValues(context).and(custom(context));
	}

	protected KeyValue custom(ServerRequestObservationContext context) {
		return KeyValue.of("custom.method", context.getCarrier().getMethod());
	}

}

If you are significantly changing metrics Tags, you are probably replacing the WebMvcTagsProvider with a custom implementation and contributing it as a bean. In this case, you should probably implement the convention for the observation you’re interested in. Here, we’ll implement ServerRequestObservationConvention - it’s using ServerRequestObservationContext to extract information about the current request. You can then implement methods with your requirements in mind:

public class CustomServerRequestObservationConvention implements ServerRequestObservationContext {

	@Override
	public String getName() {
        // will be used for the metric name
		return "http.server.requests";
	}

	@Override
	public String getContextualName(ServerRequestObservationContext context) {
        // will be used for the trace name
		return "http " + context.getCarrier().getMethod().toLowerCase();
	}

	@Override
	public KeyValues getLowCardinalityKeyValues(ServerRequestObservationContext context) {
		return KeyValues.of(method(context), status(context), exception(context));
	}

	@Override
	public KeyValues getHighCardinalityKeyValues(ServerRequestObservationContext context) {
		return KeyValues.of(httpUrl(context));
	}

	protected KeyValue method(ServerRequestObservationContext context) {
        // You should reuse as much as possible the corresponding ObservationDocumentation for key names
		return KeyValue.of(ServerHttpObservationDocumentation.LowCardinalityKeyNames.METHOD, context.getCarrier().getMethod());
	}

    //...
}

In both cases, you can contribute those as beans to the application context and they will be picked up by the auto-configuration, effectively replacing the default ones.

@Configuration
public class CustomMvcObservationConfiguration {

    @Bean
    public ExtendedServerRequestObservationConvention extendedServerRequestObservationConvention() {
        return new ExtendedServerRequestObservationConvention();
    }

}

You can also similar goals using a custom ObservationFilter - adding or removing key values for an observation. Filters do not replace the default convention and are used as a post-processing component.

public class ServerRequestObservationFilter implements ObservationFilter {

	@Override
	public Observation.Context map(Observation.Context context) {
		if (context instanceof ServerRequestObservationContext serverContext) {
			context.addLowCardinalityKeyValue(KeyValue.of("project", "spring"));
			String customAttribute = (String) serverContext.getCarrier().getAttribute("customAttribute");
			context.addLowCardinalityKeyValue(KeyValue.of("custom.attribute", customAttribute));
		}
		return context;
	}
}

You can contribute ObservationFilter beans to your application and Spring Boot will auto-configure them with the ObservationRegistry.

Disabling specific instrumentations

With the new Observation API, many Spring projects now ship observations that were not available before. You can selectively disable observations by contributing ObservationPredicate beans that return false when the name or the context match the observation we’re trying to ignore:

// ignoring observations by matching their name
@Bean
ObservationPredicate disableHttpServerObservationsFromName() {
	return (name, context) -> !name.startsWith("http.server.");
}

// ignoring observations by matching their context
@Bean
ObservationPredicate disableHttpServerObservationsFromContext() {
	return (name, context) -> !(context instanceof ServerRequestObservationContext);
}

You can also group several customizations within a single ObservationRegistryCustomizer:

@Bean
ObservationRegistryCustomizer<ObservationRegistry> ignoreSecurityAndHttpServerObservations() {
	ObservationPredicate ignoreSecurity = (name, context) -> !name.startsWith("spring.security.");
	ObservationPredicate ignoreHttpServer = (name, context) -> !(context instanceof ServerRequestObservationContext);
	return (registry) -> registry.observationConfig().observationPredicate(ignoreSecurity)
			.observationPredicate(ignoreHttpServer);
}