Sarama TLS not working: client has run out of available brokers to talk to: 3 errors occurred:\n\t* unexpected EOF
#3037
Comments
Larleyt
changed the title
client has run out of available brokers to talk to: 3 errors occurred:\n\t* unexpected EOF
|
More logs with Sarama debug enabled: |
|
@Larleyt hmm, so an "unexpected EOF" at connection time when enabling TLS on the client like this usually suggests that the brokers are expecting plain traffic. If I point a tls-enabled sarama client at a non-TLS broker then I similarly get EOF Can you try running this simple standalone "apiversions" sample? https://gist.github.com/dnwe/8cc61dad3e4fdabf857d03f20368dce5 If you go build and then try (within your aws environment so you have access to the hosts) doing: ./apiversions -broker b-1.t1devkafka.u91w03.c17.kafka.us-east-1.amazonaws.com:9094 and ./apiversions -tls -tls-skip-verify -broker b-1.t1devkafka.u91w03.c17.kafka.us-east-1.amazonaws.com:9094 and share the output from each, that would be useful |
|
Hello @dnwe, thanks for your reply! For first command ( 2024/12/23 14:18:40 unexpected EOFFor ,APIs,
0,ProduceRequest,9
1,FetchRequest,13
2,ListOffsetsRequest,7
3,MetadataRequest,12
4,LeaderAndIsrRequest,7
5,StopReplicaRequest,4
6,UpdateMetadataRequest,8
7,ControlledShutdownRequest,3
8,OffsetCommitRequest,8
9,OffsetFetchRequest,8
10,FindCoordinatorRequest,4
11,JoinGroupRequest,9
12,HeartbeatRequest,4
13,LeaveGroupRequest,5
14,SyncGroupRequest,5
15,DescribeGroupsRequest,5
16,ListGroupsRequest,4
17,SaslHandshakeRequest,1
18,ApiVersionsRequest,3
19,CreateTopicsRequest,7
20,DeleteTopicsRequest,6
21,DeleteRecordsRequest,2
22,InitProducerIdRequest,4
23,OffsetForLeaderEpochRequest,4
24,AddPartitionsToTxnRequest,3
25,AddOffsetsToTxnRequest,3
26,EndTxnRequest,3
27,WriteTxnMarkersRequest,1
28,TxnOffsetCommitRequest,3
29,DescribeAclsRequest,3
30,CreateAclsRequest,3
31,DeleteAclsRequest,3
32,DescribeConfigsRequest,4
33,AlterConfigsRequest,2
34,AlterReplicaLogDirsRequest,2
35,DescribeLogDirsRequest,4
36,SaslAuthenticateRequest,2
37,CreatePartitionsRequest,3
38,CreateDelegationTokenRequest,3
39,RenewDelegationTokenRequest,2
40,ExpireDelegationTokenRequest,2
41,DescribeDelegationTokenRequest,3
42,DeleteGroupsRequest,2
43,ElectLeadersRequest,2
44,IncrementalAlterConfigsRequest,1
45,AlterPartitionReassignmentsRequest,0
46,ListPartitionReassignmentsRequest,0
47,OffsetDeleteRequest,0
48,DescribeClientQuotasRequest,1
49,AlterClientQuotasRequest,1
50,DescribeUserScramCredentialsRequest,0
51,AlterUserScramCredentialsRequest,0
56,AlterPartitionRequest,2
57,UpdateFeaturesRequest,1
58,EnvelopeRequest,0
60,DescribeClusterRequest,0
61,DescribeProducersRequest,0
65,DescribeTransactionsRequest,0
66,ListTransactionsRequest,0
67,AllocateProducerIdsRequest,0As I understand, it doesn't work with plaintext (first cmd), but works with TLS enabled and skip-verify. As expected. But I have |
|
@Larleyt hmm thanks, so that test confirms that generally the brokers are working, they are correctly TLS-enabled (and Sarama can successfully TLS handshake with them) and they have matching support api versions to what you'd see with the official Apache Kafka 3.4.0 release. |
|
Here's what I've got now with LogsConfluent consumers don't work anymore too, when Sarama consumer/producer exist. As soon as I remove Sarama client Confluent work fine (with TLS enabled): Logs
This link is in private repo, no access. |
|
If I leave only Sarama client, it doesn't work: Logs |
|
It looks to me (from last logs) like it can only create one client for Sarama and I need 3 (in the original code in first post you can see I use |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description
I have a service that already had completely working Confluent Kafka Consumers and Producers with SSL enabled in the environment I've got the following bug. Then I've added a goka package which only supports Sarama.
When I'm trying to turn on Sarama's TLS (without cert verifying – the same as Confluent consumer), I get the following error:
Error creating the kafka client: kafka: client has run out of available brokers to talk to: 3 errors occurred:\n\t* unexpected EOF\n\t* unexpected EOF\n\t* unexpected EOF\n"Versions
I also tried different
sarama.V0_11_0_0,V2_8_*, etc. Nothing workedConfiguration
The code that utilizes the Sarama and not starting:
code: CLICK ME
Logs
logs: CLICK ME
Additional Context
As you can see from the logs, Confluent Kafka is starting absolutely fine (first part of the logs), but Sarama does not.
Additionally, I wonder why bootstrap servers' string split by comma (
bootstrap_servers_splitin logs) have this[](Confluent accepts single string with comma, Goka accepts a slice). But that might be just Zerolog representation of slice in logs, cause otherwise this slice would not work locally (without TLS).The text was updated successfully, but these errors were encountered: