Wrong RestConsumer is used when trying to connect to Elasticsearch

[LostArtist]

Bug description

We want to send a message from paho artemis broker to elasticsearch, but it refuses connection. As was investigated, from elasticsearch side, RestConsumer that accepts host with the default port (http://localhost:9200) is created during initialization and used by application, while we need to connect to the mapped port that is created by container.

StackTrace:

2024-06-06 16:06:12,274 INFO  [org.tes.ima.PullPolicy] (pool-2-thread-1) Image pull policy will be performed by: DefaultPullPolicy()
2024-06-06 16:06:12,278 INFO  [org.tes.uti.ImageNameSubstitutor] (pool-2-thread-1) Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
2024-06-06 16:06:12,424 INFO  [org.tes.doc.DockerClientProviderStrategy] (pool-2-thread-1) Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
2024-06-06 16:06:12,573 INFO  [org.tes.doc.DockerClientProviderStrategy] (pool-2-thread-1) Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
2024-06-06 16:06:12,573 INFO  [org.tes.DockerClientFactory] (pool-2-thread-1) Docker host IP address is localhost
2024-06-06 16:06:12,585 INFO  [org.tes.DockerClientFactory] (pool-2-thread-1) Connected to docker: 
  Server Version: 24.0.5
  API Version: 1.43
  Operating System: Fedora Linux 39 (Workstation Edition)
  Total Memory: 31793 MB
2024-06-06 16:06:12,611 INFO  [tc.tes.6.0] (pool-2-thread-1) Creating container for image: testcontainers/ryuk:0.6.0
2024-06-06 16:06:12,746 INFO  [tc.tes.6.0] (pool-2-thread-1) Container testcontainers/ryuk:0.6.0 is starting: e18e1e211521b0bcfb714f18537dc7c0ce9f52f77b6e807a21dd5c36359f1f10
2024-06-06 16:06:13,081 INFO  [tc.tes.6.0] (pool-2-thread-1) Container testcontainers/ryuk:0.6.0 started in PT0.469818528S
2024-06-06 16:06:13,083 INFO  [org.tes.uti.RyukResourceReaper] (pool-2-thread-1) Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
2024-06-06 16:06:13,083 INFO  [org.tes.DockerClientFactory] (pool-2-thread-1) Checking the system...
2024-06-06 16:06:13,084 INFO  [org.tes.DockerClientFactory] (pool-2-thread-1) ✔︎ Docker server version should be at least 1.6.0
2024-06-06 16:06:13,084 INFO  [tc.qua.io/.0.26] (pool-2-thread-1) Creating container for image: quay.io/artemiscloud/activemq-artemis-broker:1.0.26
2024-06-06 16:06:13,432 INFO  [tc.qua.io/.0.26] (pool-2-thread-1) Container quay.io/artemiscloud/activemq-artemis-broker:1.0.26 is starting: 169b75b8ec35719ceae0364c0f0bcff0dc2286229f2c80079c292f1a63181ac1
2024-06-06 16:06:23,285 INFO  [tc.qua.io/.0.26] (pool-2-thread-1) Container quay.io/artemiscloud/activemq-artemis-broker:1.0.26 started in PT10.200920195S
2024-06-06 16:06:23,289 INFO  [tc.doc.ela.co/.12.0] (pool-2-thread-1) Creating container for image: docker.elastic.co/elasticsearch/elasticsearch:8.12.0
2024-06-06 16:06:23,656 INFO  [tc.doc.ela.co/.12.0] (pool-2-thread-1) Container docker.elastic.co/elasticsearch/elasticsearch:8.12.0 is starting: d6794022a11833b94caa006fcc7ba923e898e7ac22579314fdd18e2066dcfb75
2024-06-06 16:06:35,899 INFO  [tc.doc.ela.co/.12.0] (pool-2-thread-1) Container docker.elastic.co/elasticsearch/elasticsearch:8.12.0 started in PT12.609770495S
2024-06-06 16:06:35,899 DEBUG [org.acm.res.ElasticSearchTestResource] (pool-2-thread-1) Connecting to localhost:32772
2024-06-06 16:06:36,831 INFO  [org.apa.cam.qua.cor.CamelBootstrapRecorder] (main) Bootstrap runtime: org.apache.camel.quarkus.main.CamelMainRuntime
2024-06-06 16:06:36,832 INFO  [org.apa.cam.mai.MainSupport] (main) Apache Camel (Main) 4.5.0 is starting
2024-06-06 16:06:36,872 INFO  [org.apa.cam.mai.BaseMainSupport] (main) Auto-configuration summary
2024-06-06 16:06:36,872 INFO  [org.apa.cam.mai.BaseMainSupport] (main)     [MicroProfilePropertiesSource] camel.component.paho.brokerUrl=tcp://localhost:32771
2024-06-06 16:06:36,964 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 4.5.0 (camel-1) is starting
2024-06-06 16:06:37,771 INFO  [org.apa.cam.mai.BaseMainSupport] (main) Property-placeholders summary
2024-06-06 16:06:37,771 INFO  [org.apa.cam.mai.BaseMainSupport] (main)     [MicroProfilePropertiesSource] elasticsearch.host=localhost:32772
2024-06-06 16:06:37,772 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Routes startup (total:3 rest-dsl:2)
2024-06-06 16:06:37,772 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started paho-devices (paho://devices)
2024-06-06 16:06:37,772 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started direct-devices (rest://post:/devices)
2024-06-06 16:06:37,772 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main)     Started route1 (rest://get:/devices)
2024-06-06 16:06:37,772 INFO  [org.apa.cam.imp.eng.AbstractCamelContext] (main) Apache Camel 4.5.0 (camel-1) started in 807ms (build:0ms init:0ms start:807ms)
2024-06-06 16:06:37,847 INFO  [io.quarkus] (main) camel-quarkus-examples-artemis-elasticsearch 3.10.0 on JVM (powered by Quarkus 3.10.0) started in 26.901s. Listening on: http://localhost:8081
2024-06-06 16:06:37,847 INFO  [io.quarkus] (main) Profile test activated. 
2024-06-06 16:06:37,847 INFO  [io.quarkus] (main) Installed features: [camel-attachments, camel-bean, camel-core, camel-direct, camel-elasticsearch-rest-client, camel-jackson, camel-log, camel-paho, camel-platform-http, camel-rest, camel-timer, cdi, elasticsearch-rest-client, smallrye-context-propagation, vertx]
2024-06-06 16:06:38,517 INFO  [paho-devices] (MQTT Call: camel-paho22720017935435) Message before marshalling is {"devices":"Hey you"}
2024-06-06 16:06:38,534 DEBUG [org.ela.cli.RestClient] (elasticsearch-rest-client-1-thread-1) request [POST http://localhost:9200/devices/_doc] failed: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:946)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
	at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
	at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
	at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
	at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
	at java.base/java.lang.Thread.run(Thread.java:833)

2024-06-06 16:06:38,535 DEBUG [org.ela.cli.RestClient] (elasticsearch-rest-client-1-thread-1) added [[host=http://localhost:9200]] to blacklist
2024-06-06 16:06:38,537 ERROR [org.apa.cam.pro.err.DefaultErrorHandler] (elasticsearch-rest-client-1-thread-1) Failed delivery for (MessageId: C3AA7BF37FCD41D-0000000000000000 on ExchangeId: C3AA7BF37FCD41D-0000000000000000). Exhausted after delivery attempt: 1 caught: java.net.ConnectException: Connection refused

Used dependencies:

        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-rest</artifactId>
        </dependency>
        <dependency>
            <groupId>org.apache.camel.quarkus</groupId>
            <artifactId>camel-quarkus-elasticsearch-rest-client</artifactId>
        </dependency>
        <dependency>
            <groupId>org.testcontainers</groupId>
            <artifactId>elasticsearch</artifactId>
            <scope>test</scope>
        </dependency>

Properties:

        <quarkus.platform.version>3.11.0</quarkus.platform.version>
        <camel-quarkus.platform.version>${quarkus.platform.version}</camel-quarkus.platform.version>

        <quarkus.platform.group-id>io.quarkus.platform</quarkus.platform.group-id>
        <quarkus.platform.artifact-id>quarkus-bom</quarkus.platform.artifact-id>
        <camel-quarkus.platform.group-id>${quarkus.platform.group-id}</camel-quarkus.platform.group-id>
        <camel-quarkus.platform.artifact-id>quarkus-camel-bom</camel-quarkus.platform.artifact-id>

        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <maven.compiler.target>17</maven.compiler.target>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.testTarget>${maven.compiler.target}</maven.compiler.testTarget>
        <maven.compiler.testSource>${maven.compiler.source}</maven.compiler.testSource>

        <formatter-maven-plugin.version>2.23.0</formatter-maven-plugin.version>
        <groovy-maven-plugin.version>2.1.1</groovy-maven-plugin.version>
        <impsort-maven-plugin.version>1.9.0</impsort-maven-plugin.version>
        <license-maven-plugin.version>4.2</license-maven-plugin.version>
        <maven-compiler-plugin.version>3.11.0</maven-compiler-plugin.version>
        <maven-jar-plugin.version>3.3.0</maven-jar-plugin.version>
        <maven-resources-plugin.version>3.3.1</maven-resources-plugin.version>
        <maven-surefire-plugin.version>3.1.2</maven-surefire-plugin.version>

        <quarkus-artemis-jms.version>3.1.4</quarkus-artemis-jms.version>

Basically we need to connect to elasticsearch with the mapped port (not default one), so we need to use RestConsumer that uses this port

Link to the code: https://github.com/LostArtist/camel-quarkus-examples/tree/camel-quarkus-reproducer-1

[orpiske]

Might also want to keep an eye on #6163 ... sounds suspiciously similar.

[jamesnetherton]

Might also want to keep an eye on #6163 ... sounds suspiciously similar.

Yeah I literally just stumbled across this problem. You can probably work around it by adding autowiredEnabled=false to the elasticsearch endpoint URI options.

[jamesnetherton]

The other thing that'd probably work is to add a Maven exclusion for io.quarkus:quarkus-elasticsearch-rest-client on org.apache.camel.quarkus:camel-quarkus-elasticsearch-rest-client.