Merge pull request #41244 from brunobat/lgtm-properties

Set extension properties directly
quarkusio · Jul 12, 2024 · 13ba085 · 13ba085
2 parents 4f29520 + 10c335d
commit 13ba085
Show file tree

Hide file tree

Showing 12 changed files with 185 additions and 130 deletions.
diff --git a/docs/src/main/asciidoc/observability-devservices-lgtm.adoc b/docs/src/main/asciidoc/observability-devservices-lgtm.adoc
@@ -35,32 +35,28 @@ implementation("quarkus-observability-devservices-lgtm")
 
 === Metrics
 
-If you're using https://micrometer.io/[MicroMeter's] Quarkiverse OTLP registry to push metrics to Grafana OTel LGTM, this is how you would define the export endpoint url; where `quarkus.otel-collector.url` is provided by the Observability Dev Services extension.
+If you need metrics, add the Micrometer OTLP registry to your build file:
 
-[source,properties]
+[source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
+.pom.xml
 ----
-# Micrometer OTLP registry
-%test.quarkus.micrometer.export.otlp.url=http://${quarkus.otel-collector.url}/v1/metrics
-%dev.quarkus.micrometer.export.otlp.url=http://${quarkus.otel-collector.url}/v1/metrics
-%prod.quarkus.micrometer.export.otlp.url=http://localhost:4318/v1/metrics
+<dependency>
+    <groupId>io.quarkiverse.micrometer.registry</groupId>
+    <artifactId>quarkus-micrometer-registry-otlp</artifactId>
+</dependency>
 ----
-Please note that the `${quarkus.otel-collector.url}` value is generated by quarkus when it starts the Grafana OTel LGTM Dev Resource.
-
-Along OTel collector enpoint url, LGTM Dev Resource also provides a Grafana endpoint url - under `quarkus.grafana.url` property.
-
-In this case LGTM Dev Resource would be automatically started and used by Observability Dev Services.
-
-If you don't want all the hassle with Dev Services (e.g. lookup and re-use of existing running containers, etc) you can simply disable Dev Services and enable just Dev Resource usage:
 
-[source,properties]
+[source,gradle,role="secondary asciidoc-tabs-target-sync-gradle"]
+.build.gradle
 ----
-quarkus.observability.enabled=false
-quarkus.observability.dev-resources=true
+implementation("io.quarkiverse.micrometer.registry:quarkus-micrometer-registry-otlp")
 ----
 
+When using the https://micrometer.io/[MicroMeter's] Quarkiverse OTLP registry to push metrics to Grafana OTel LGTM, the `quarkus.micrometer.export.otlp.url` property is automatically set to OTel collector endpoint as seen from the outside of the docker container.
+
 === Tracing
 
-Just add the quarkus-opentelemetry extension to your build file:
+For Tracing add the `quarkus-opentelemetry` extension to your build file:
 [source,xml,role="primary asciidoc-tabs-target-sync-cli asciidoc-tabs-target-sync-maven"]
 .pom.xml
 ----
@@ -76,34 +72,43 @@ Just add the quarkus-opentelemetry extension to your build file:
 implementation("io.quarkus:quarkus-opentelemetry")
 ----
 
-On the `application.properties` file, you can define:
-[source,properties]
-----
-# OpenTelemetry
-quarkus.otel.exporter.otlp.traces.protocol=http/protobuf
-%test.quarkus.otel.exporter.otlp.traces.endpoint=http://${quarkus.otel-collector.url}
-%dev.quarkus.otel.exporter.otlp.traces.endpoint=http://${quarkus.otel-collector.url}
-%prod.quarkus.otel.exporter.otlp.traces.endpoint=http://localhost:4318
-----
+The `quarkus.otel.exporter.otlp.traces.endpoint` property is automatically set to OTel collector endpoint as seen from the outside of the docker container.
+
+The `quarkus.otel.exporter.otlp.traces.protocol` is set to `http/protobuf`.
+
 === Access Grafana
 
 Once you start your app in dev mode:
 
 include::{includes}/devtools/dev.adoc[]
 
-You will see a message like this:
+You will see a log entry like this:
 
 [source, log]
 ----
-Lgtm Dev Services Starting: 2024-02-20 11:15:24,540 INFO  [org.tes.con.wai.str.HttpWaitStrategy] (build-32) /loving_chatelet: Waiting for 60 seconds for URL: http://localhost:61907/ (where port 61907 maps to container port 3000)
+[io.qu.ob.de.ObservabilityDevServiceProcessor] (build-35) Dev Service Lgtm started, config: {grafana.endpoint=http://localhost:42797, quarkus.otel.exporter.otlp.traces.endpoint=http://localhost:34711, otel-collector.url=localhost:34711, quarkus.micrometer.export.otlp.url=http://localhost:34711/v1/metrics, quarkus.otel.exporter.otlp.traces.protocol=http/protobuf}
+
 ----
-Remember that Grafana is accessible in an ephemeral port, so you need to check the logs to see which port is being used. In this example, it's port 61907.
+Remember that Grafana is accessible in an ephemeral port, so you need to check the logs to see which port is being used. In this example, the grafana endpoint is `grafana.endpoint=http://localhost:42797`.
 
 If you miss the message you can always check the port with this Docker command:
 [source, bash]
 ----
 docker ps | grep grafana
 ----
+
+=== Additional configuration
+
+This extension will configure your `quarkus-opentelemetry` and `quarkus-micrometer-registry-otlp` extensions to send data to the OTel Collector bundled with the Grafana OTel LGTM image.
+
+If you don't want all the hassle with Dev Services (e.g. lookup and re-use of existing running containers, etc) you can simply disable Dev Services and enable just Dev Resource usage:
+
+[source,properties]
+----
+quarkus.observability.enabled=false
+quarkus.observability.dev-resources=true
+----
+
 === Tests
 
 And for the least 'auto-magical' usage in the tests, simply disable both (Dev Resources are already disabled by default):
@@ -142,16 +147,6 @@ Use existing Quarkus MicroMeter OTLP registry
 implementation("io.quarkiverse.micrometer.registry:quarkus-micrometer-registry-otlp")
 ----
 
-On the test  `application.properties` file, you need to define:
-[source,properties]
-----
-# Micrometer OTLP registry
-quarkus.micrometer.export.otlp.url=http://${quarkus.otel-collector.url}/v1/metrics
-# OpenTelemetry
-quarkus.otel.exporter.otlp.traces.protocol=http/protobuf
-quarkus.otel.exporter.otlp.traces.endpoint=http://${quarkus.otel-collector.url}
-----
-
 Simply inject the Meter registry into your code -- it will periodically push metrics to Grafana LGTM's OTLP HTTP endpoint.
 
 [source, java]
@@ -182,14 +177,14 @@ Where you can then check Grafana's datasource API for existing metrics data.
 ----
 public class LgtmTestBase {
 
-    @ConfigProperty(name = "quarkus.grafana.url")
-    String url; // NOTE -- injected Grafana endpoint url!
+    @ConfigProperty(name = "grafana.endpoint")
+    String endpoint; // NOTE -- injected Grafana endpoint!
 
     @Test
     public void testTracing() {
         String response = RestAssured.get("/api/poke?f=100").body().asString();
         System.out.println(response);
-        GrafanaClient client = new GrafanaClient("http://" + url, "admin", "admin");
+        GrafanaClient client = new GrafanaClient(endpoint, "admin", "admin");
         Awaitility.await().atMost(61, TimeUnit.SECONDS).until(
                 client::user,
                 u -> "admin".equals(u.login));

diff --git a/...t/src/main/java/io/quarkus/observability/deployment/ObservabilityDevServiceProcessor.java b/...t/src/main/java/io/quarkus/observability/deployment/ObservabilityDevServiceProcessor.java
@@ -12,8 +12,13 @@
 import java.util.stream.Collectors;
 import java.util.stream.Stream;
 
+import org.jboss.jandex.ClassInfo;
+import org.jboss.jandex.DotName;
 import org.jboss.logging.Logger;
 
+import io.quarkus.arc.deployment.BeanArchiveIndexBuildItem;
+import io.quarkus.deployment.Capabilities;
+import io.quarkus.deployment.Capability;
 import io.quarkus.deployment.Feature;
 import io.quarkus.deployment.IsNormal;
 import io.quarkus.deployment.annotations.BuildProducer;
@@ -28,15 +33,18 @@
 import io.quarkus.deployment.console.StartupLogCompressor;
 import io.quarkus.deployment.dev.devservices.GlobalDevServicesConfig;
 import io.quarkus.deployment.logging.LoggingSetupBuildItem;
+import io.quarkus.deployment.metrics.MetricsCapabilityBuildItem;
 import io.quarkus.devservices.common.ContainerLocator;
 import io.quarkus.observability.common.config.ContainerConfig;
 import io.quarkus.observability.common.config.ContainerConfigUtil;
 import io.quarkus.observability.common.config.ModulesConfiguration;
 import io.quarkus.observability.devresource.Container;
 import io.quarkus.observability.devresource.DevResourceLifecycleManager;
 import io.quarkus.observability.devresource.DevResources;
+import io.quarkus.observability.devresource.ExtensionsCatalog;
 import io.quarkus.observability.runtime.config.ObservabilityConfiguration;
 import io.quarkus.runtime.LaunchMode;
+import io.quarkus.runtime.metrics.MetricsFactory;
 
 @BuildSteps(onlyIfNot = IsNormal.class, onlyIf = { GlobalDevServicesConfig.Enabled.class,
         ObservabilityDevServiceProcessor.IsEnabled.class })
@@ -46,6 +54,7 @@ class ObservabilityDevServiceProcessor {
     private static final Map<String, DevServicesResultBuildItem.RunningDevService> devServices = new ConcurrentHashMap<>();
     private static final Map<String, ContainerConfig> capturedDevServicesConfigurations = new ConcurrentHashMap<>();
     private static final Map<String, Boolean> firstStart = new ConcurrentHashMap<>();
+    public static final DotName OTLP_REGISTRY = DotName.createSimple("io.micrometer.registry.otlp.OtlpMeterRegistry");
 
     public static class IsEnabled implements BooleanSupplier {
         ObservabilityConfiguration config;
@@ -74,7 +83,10 @@ public void startContainers(LaunchModeBuildItem launchMode,
             CuratedApplicationShutdownBuildItem closeBuildItem,
             LoggingSetupBuildItem loggingSetupBuildItem,
             GlobalDevServicesConfig devServicesConfig,
-            BuildProducer<DevServicesResultBuildItem> services) {
+            BuildProducer<DevServicesResultBuildItem> services,
+            BeanArchiveIndexBuildItem indexBuildItem,
+            Capabilities capabilities,
+            Optional<MetricsCapabilityBuildItem> metricsConfiguration) {
 
         if (!configuration.enabled()) {
             log.infof("Observability dev services are disabled in config");
@@ -103,7 +115,11 @@ public void startContainers(LaunchModeBuildItem launchMode,
 
             // only do get, not remove, so it can be re-used
             DevServicesResultBuildItem.RunningDevService devService = devServices.get(devId);
-            ContainerConfig currentDevServicesConfiguration = dev.config(configuration);
+            ContainerConfig currentDevServicesConfiguration = dev.config(
+                    configuration,
+                    new ExtensionsCatalog(
+                            capabilities.isPresent(Capability.OPENTELEMETRY_TRACER),
+                            hasMicrometerOtlp(metricsConfiguration, indexBuildItem)));
 
             if (devService != null) {
                 ContainerConfig capturedDevServicesConfiguration = capturedDevServicesConfigurations.get(devId);
@@ -152,19 +168,22 @@ public void startContainers(LaunchModeBuildItem launchMode,
             }
 
             if (firstStart.computeIfAbsent(devId, x -> true)) {
-                Runnable closeTask = () -> {
-                    DevServicesResultBuildItem.RunningDevService current = devServices.get(devId);
-                    if (current != null) {
-                        try {
-                            current.close();
-                        } catch (Throwable t) {
-                            log.errorf("Failed to stop %s container", devId, t);
+                Runnable closeTask = new Runnable() {
+                    @Override
+                    public void run() {
+                        DevServicesResultBuildItem.RunningDevService current = devServices.get(devId);
+                        if (current != null) {
+                            try {
+                                current.close();
+                            } catch (Throwable t) {
+                                log.errorf("Failed to stop %s container", devId, t);
+                            }
                         }
+                        firstStart.remove(devId);
+                        //noinspection resource
+                        devServices.remove(devId);
+                        capturedDevServicesConfigurations.remove(devId);
                     }
-                    firstStart.remove(devId);
-                    //noinspection resource
-                    devServices.remove(devId);
-                    capturedDevServicesConfigurations.remove(devId);
                 };
                 closeBuildItem.addCloseTask(closeTask, true);
             }
@@ -173,6 +192,18 @@ public void startContainers(LaunchModeBuildItem launchMode,
         });
     }
 
+    private static boolean hasMicrometerOtlp(Optional<MetricsCapabilityBuildItem> metricsConfiguration,
+            BeanArchiveIndexBuildItem indexBuildItem) {
+        if (metricsConfiguration.isPresent() &&
+                metricsConfiguration.get().metricsSupported(MetricsFactory.MICROMETER)) {
+            ClassInfo clazz = indexBuildItem.getIndex().getClassByName(OTLP_REGISTRY);
+            if (clazz != null) {
+                return true;
+            }
+        }
+        return false;
+    }
+
     private DevServicesResultBuildItem.RunningDevService startContainer(
             String devId,
             DevResourceLifecycleManager<ContainerConfig> dev,
@@ -190,14 +221,17 @@ private DevServicesResultBuildItem.RunningDevService startContainer(
             return null;
         }
 
-        final Supplier<DevServicesResultBuildItem.RunningDevService> defaultContainerSupplier = () -> {
-            Container<?> container = dev.container(capturedDevServicesConfiguration, root);
-            timeout.ifPresent(container::withStartupTimeout);
-            Map<String, String> config = dev.start();
-            log.infof("Dev Service %s started, config: %s", devId, config);
-            return new DevServicesResultBuildItem.RunningDevService(
-                    Feature.OBSERVABILITY.getName(), container.getContainerId(),
-                    container.closeableCallback(capturedDevServicesConfiguration.serviceName()), config);
+        final Supplier<DevServicesResultBuildItem.RunningDevService> defaultContainerSupplier = new Supplier<DevServicesResultBuildItem.RunningDevService>() {
+            @Override
+            public DevServicesResultBuildItem.RunningDevService get() {
+                Container<?> container = dev.container(capturedDevServicesConfiguration, root);
+                timeout.ifPresent(container::withStartupTimeout);
+                Map<String, String> config = dev.start();
+                log.infof("Dev Service %s started, config: %s", devId, config);
+                return new DevServicesResultBuildItem.RunningDevService(
+                        Feature.OBSERVABILITY.getName(), container.getContainerId(),
+                        container.closeableCallback(capturedDevServicesConfiguration.serviceName()), config);
+            }
         };
 
         Map<String, String> config = new LinkedHashMap<>(); // old config
@@ -206,10 +240,13 @@ private DevServicesResultBuildItem.RunningDevService startContainer(
                 .locateContainer(
                         capturedDevServicesConfiguration.serviceName(), capturedDevServicesConfiguration.shared(),
                         LaunchMode.current(), (p, ca) -> config.putAll(dev.config(p, ca.getHost(), ca.getPort())))
-                .map(cid -> {
-                    log.infof("Dev Service %s re-used, config: %s", devId, config);
-                    return new DevServicesResultBuildItem.RunningDevService(Feature.OBSERVABILITY.getName(), cid,
-                            null, config);
+                .map(new Function<String, DevServicesResultBuildItem.RunningDevService>() {
+                    @Override
+                    public DevServicesResultBuildItem.RunningDevService apply(String cid) {
+                        log.infof("Dev Service %s re-used, config: %s", devId, config);
+                        return new DevServicesResultBuildItem.RunningDevService(Feature.OBSERVABILITY.getName(), cid,
+                                null, config);
+                    }
                 })
                 .orElseGet(defaultContainerSupplier);
     }

diff --git a/...ommon/src/main/java/io/quarkus/observability/devresource/DevResourceLifecycleManager.java b/...ommon/src/main/java/io/quarkus/observability/devresource/DevResourceLifecycleManager.java
@@ -28,8 +28,18 @@ public interface DevResourceLifecycleManager<T extends ContainerConfig> extends
      * @param configuration main observability configuration
      * @return module's config
      */
+    @Deprecated
     T config(ModulesConfiguration configuration);
 
+    /**
+     * Get resource's config from main observability configuration and extension catalog
+     *
+     * @param configuration main observability configuration
+     * @param catalog observability catalog. If OpenTelemetry or Micrometer are enabled.
+     * @return module's config
+     */
+    T config(ModulesConfiguration configuration, ExtensionsCatalog catalog);
+
     /**
      * Should we enable / start this dev resource.
      * e.g. we could already have actual service running

diff --git a/...resource-common/src/main/java/io/quarkus/observability/devresource/ExtensionsCatalog.java b/...resource-common/src/main/java/io/quarkus/observability/devresource/ExtensionsCatalog.java
@@ -0,0 +1,8 @@
+package io.quarkus.observability.devresource;
+
+/**
+ * Relevant Observability extensions present.
+ */
+public record ExtensionsCatalog(boolean hasOpenTelemetry,
+        boolean hasMicrometerOtlp) {
+}