KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE contains true, which is an invalid type, it should be a string, number, or a null

[danjrichards]

Hi Sven
Nice tool - thank you.

I ran python3 kafka_docker_composer.py -b 3 -z 3 --racks 3 and it generated 3 errors on starting, each:
services.kafka-?.environment.KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE contains true, which is an invalid type, it should be a string, number, or a null

I changed each instance to KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE="true" and this error went away, but I then get: Error opening zip file or JAR manifest missing : /tmp/jmx_prometheus_javaagent-0.20.0.jar zookeeper-2 | agent library failed to init: instrument

[danjrichards]

instrumenting error was my local issue - for some reason jmx_prometheus_javaagent-0.20.0.jar was an empty directory! 🤷🏼

[sknop]

I am not quite sure how you got these errors, I cannot reproduce any of this.

Which OS are you running, and which docker engine do you use?

[DominiqueLuebben]

Hello,

I am getting the same errors like danjrichards. I am trying to setup "python kafka_docker_composer.py --brokers 1 --controllers 1 --p". I get errors because of "KAFKA_CONFLUENT_CLUSTER_LINK_ENABLE: False" and "KAFKA_CONFLUENT_REPORTERS_TELEMETRY_AUTO_ENABLE: False". Additionally there is no prometheus.yml in the volumes folder

[sknop]

Hi Dominique,

Can you try three brokers and three controllers first to see if that works for you?
There are some extra services enabled in the Confluent server that are set to a replication factor of 3 by default, it is possible I missed a few.

I will investigate.

[sknop]

Also, it is -p, not --p?

Kafka docker-compose Generator

options:
-h, --help show this help message and exit
-r RELEASE, --release RELEASE
Docker images release [7.7.0]
--with-tc Build and use a local image with tc enabled
--shared-mode Enable shared mode for controllers
-b BROKERS, --brokers BROKERS
Number of Brokers [1]
-z ZOOKEEPERS, --zookeepers ZOOKEEPERS
Number of ZooKeepers [0] - mutually exclusive with controllers
-c CONTROLLERS, --controllers CONTROLLERS
Number of Kafka controller instances [0] - mutually exclusive with ZooKeepers
-s SCHEMA_REGISTRIES, --schema-registries SCHEMA_REGISTRIES
Number of Schema Registry instances [0]
-C CONNECT_INSTANCES, --connect-instances CONNECT_INSTANCES
Number of Kafka Connect instances [0]
-k KSQLDB_INSTANCES, --ksqldb-instances KSQLDB_INSTANCES
Number of ksqlDB instances [0]
--control-center Include Confluent Control Center [False]
--uuid UUID UUID of the cluster [Nk018hRAQFytWskYqtQduw]
-p, --prometheus Include Prometheus [False]
--kafka-container KAFKA_CONTAINER
Container used for Kafka, default [cp-server]
--racks RACKS Number of racks among which the brokers will be distributed evenly [1]
--zookeeper-groups ZOOKEEPER_GROUPS
Number of zookeeper groups in a hierarchy [1]
--docker-compose-file DOCKER_COMPOSE_FILE
Output file for docker-compose, default [docker-compose.yml]
--config CONFIG Properties config file, values will be overridden by command line arguments