Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DBZ-2862 Switch tracing to OpenTelemetry #343

Closed
wants to merge 3 commits into from

Conversation

vidourem
Copy link
Contributor

@vidourem vidourem commented Aug 4, 2023

Added OpenTelemetry packages to connect-base image and added interceptor package to connect image

Copy link
Member

@vjuranek vjuranek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the PR @vidourem . It would be good to enabled OTEL libraries only then user wants it, by setting appropriate env. variable. I Left a comment with pointers. Please let me know if anything is unclear.

@@ -42,6 +44,14 @@ RUN mkdir "$KAFKA_CONNECT_PLUGINS_DIR" "$EXTERNAL_LIBS_DIR"
#
RUN docker-maven-download apicurio "$APICURIO_VERSION" ead18a95038adca54e91b7f253717eb7
RUN docker-maven-download central org/jolokia jolokia-jvm "$JOLOKIA_VERSION" d489d62d1143e6a2e85a869a4b824a67
RUN docker-maven-download central io/opentelemetry opentelemetry-api "$OPENTELEMETRY_VERSION" e198a9568ce31a82faaa26f328388e89
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it would be good to download it into external lib directory or newly created dir (e.g. otel) and enable it only on demand, i.e. introduce new env. variable here and later on eventually, if enabled, create the symlink for the jar file by calling set_connector_additonal_resource_availability (see here)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vjuranek
I made a modification, but I don't see how the downloaded file can then be found in the otel subdirectory. I can't test on the PC I'm currently using, I'll check that later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vidourem the bit will be donwloaded into a separated directory. If nothing happens, they won't be on the class path. If the user enables OTEL via env. variable, calling set_connector_additonal_resource_availability with this env. variable enabled will result into a creating a symbolic link from connector directory to OTEL libs and therefore they will be included on the class path.

Copy link
Member

@vjuranek vjuranek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You needs to download the otel bits into the correct directory, other LGMT now.

@@ -42,6 +44,14 @@ RUN mkdir "$KAFKA_CONNECT_PLUGINS_DIR" "$EXTERNAL_LIBS_DIR"
#
RUN docker-maven-download apicurio "$APICURIO_VERSION" ead18a95038adca54e91b7f253717eb7
RUN docker-maven-download central org/jolokia jolokia-jvm "$JOLOKIA_VERSION" d489d62d1143e6a2e85a869a4b824a67
RUN docker-maven-download central io/opentelemetry opentelemetry-api "$OPENTELEMETRY_VERSION" e198a9568ce31a82faaa26f328388e89
RUN docker-maven-download central io/opentelemetry opentelemetry-context "$OPENTELEMETRY_VERSION" 783594a506dbf035e686776d5bcb4bfc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

as you use later on otel directory (when calling set_connector_additonal_resource_availability), the bits needs to be downloaded into otel directory as well, i.e. you need too replace central with otel

@vjuranek
Copy link
Member

vjuranek commented Aug 9, 2023

Merged, thanks @vidourem ! (to make it for the release I did remaing changes myself)

@vjuranek
Copy link
Member

vjuranek commented Aug 9, 2023

tested manually locally with mysql connector, libs are correctly linked:

[kafka@d7b5096f1f2d ~]$ ls -la connect/debezium-connector-mysql/
total 14764
drwxr-xr-x. 1 kafka kafka    4096 Aug  9 10:39 .
drwxr-xr-x. 1 kafka kafka    4096 Aug  9 10:37 ..
-rw-r--r--. 1 kafka kafka  478061 Jul 17 10:40 CHANGELOG.md
-rw-r--r--. 1 kafka kafka   19246 Jul 17 10:40 CONTRIBUTE.md
-rw-r--r--. 1 kafka kafka    7537 Jul 17 10:40 COPYRIGHT.txt
-rw-r--r--. 1 kafka kafka  129157 Jul 17 10:40 LICENSE-3rd-PARTIES.txt
-rw-r--r--. 1 kafka kafka   11357 Jul 17 10:40 LICENSE.txt
-rw-r--r--. 1 kafka kafka   16594 Jul 17 10:40 README.md
-rw-r--r--. 1 kafka kafka   19608 Jul 17 10:40 README_JA.md
-rw-r--r--. 1 kafka kafka   18365 Jul 17 10:40 README_KO.md
-rw-r--r--. 1 kafka kafka   15739 Jul 17 10:40 README_ZH.md
-rw-r--r--. 1 kafka kafka  322137 Jul 17 10:30 antlr4-runtime-4.10.1.jar
-rw-r--r--. 1 kafka kafka   27350 Jul 17 10:41 debezium-api-2.4.0.Alpha1.jar
-rw-r--r--. 1 kafka kafka  273667 Jul 17 10:42 debezium-connector-mysql-2.4.0.Alpha1.jar
-rw-r--r--. 1 kafka kafka 1090318 Jul 17 10:41 debezium-core-2.4.0.Alpha1.jar
-rw-r--r--. 1 kafka kafka 3092738 Jul 17 10:42 debezium-ddl-parser-2.4.0.Alpha1.jar
-rw-r--r--. 1 kafka kafka    6429 Jul 17 10:42 debezium-storage-file-2.4.0.Alpha1.jar
-rw-r--r--. 1 kafka kafka   16170 Jul 17 10:42 debezium-storage-kafka-2.4.0.Alpha1.jar
-rw-r--r--. 1 kafka kafka  209508 Jul 17 10:32 mysql-binlog-connector-java-0.27.2.jar
-rw-r--r--. 1 kafka kafka 2481560 Jul 17 10:32 mysql-connector-j-8.0.33.jar
-rw-r--r--. 1 kafka kafka   45619 Jul 17 10:42 mysql.json
lrwxrwxrwx. 1 kafka kafka      54 Aug  9 10:39 opentelemetry-api-1.23.1.jar -> /kafka/external_libs/otel/opentelemetry-api-1.23.1.jar
lrwxrwxrwx. 1 kafka kafka      58 Aug  9 10:39 opentelemetry-context-1.23.1.jar -> /kafka/external_libs/otel/opentelemetry-context-1.23.1.jar
lrwxrwxrwx. 1 kafka kafka      70 Aug  9 10:39 opentelemetry-instrumentation-api-1.23.0.jar -> /kafka/external_libs/otel/opentelemetry-instrumentation-api-1.23.0.jar
lrwxrwxrwx. 1 kafka kafka      84 Aug  9 10:39 opentelemetry-instrumentation-api-semconv-1.23.0-alpha.jar -> /kafka/external_libs/otel/opentelemetry-instrumentation-api-semconv-1.23.0-alpha.jar
lrwxrwxrwx. 1 kafka kafka      74 Aug  9 10:39 opentelemetry-kafka-clients-2.6-1.23.0-alpha.jar -> /kafka/external_libs/otel/opentelemetry-kafka-clients-2.6-1.23.0-alpha.jar
lrwxrwxrwx. 1 kafka kafka      77 Aug  9 10:39 opentelemetry-kafka-clients-common-1.23.0-alpha.jar -> /kafka/external_libs/otel/opentelemetry-kafka-clients-common-1.23.0-alpha.jar
lrwxrwxrwx. 1 kafka kafka      64 Aug  9 10:39 opentelemetry-semconv-1.23.1-alpha.jar -> /kafka/external_libs/otel/opentelemetry-semconv-1.23.1-alpha.jar
-rw-r--r--. 1 kafka kafka 6762522 Jul 17 10:32 zstd-jni-1.5.0-2.jar

@vjuranek vjuranek closed this Aug 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants