Use keycloak-client libraries instead of keycloak-common, keycloak-core and keycloak-adapter-spi #43260

pskopek · 2024-09-13T07:14:11Z

This PR uses keycloak-client 26.0.0-alpha2. If it passes all checks and reviews I can release final and update the PR.

Closes #43259

extensions/oidc-client/deployment/pom.xml

extensions/oidc/deployment/pom.xml

integration-tests/oidc-client-reactive/pom.xml

rsvoboda · 2024-09-13T09:26:22Z

FYI @maxandersen @gsmet @aloubyansky

sberyozkin · 2024-09-13T09:26:32Z

Thanks @pskopek for the PR, I think I understand the idea, but I'm not sure I appreciate the consequences if any for Quarkus, I'd like to ask @pedroigor and @mposolda to review, thanks

sberyozkin · 2024-09-13T09:30:51Z

@pskopek I can try the devservice myself a little bit later to save you some time

github-actions · 2024-09-13T09:32:39Z

🎊 PR Preview 00852eb has been successfully built and deployed to https://quarkus-pr-main-43260-preview.surge.sh/version/main/guides/

Images of blog posts older than 3 months are not available.
Newsletters older than 3 months are not available.

aloubyansky · 2024-09-13T11:17:08Z

@pskopek afaics, only the dependency on keycloak-adapter-spi is removed, the rest remain to be dependencies. keycloak-common was just a version constraint. Could you clarify the reason to remove Keycloak version constraints from the BOM?

gsmet · 2024-09-13T16:24:09Z

The failures are unrelated.

build-parent/pom.xml

sberyozkin · 2024-09-15T15:13:38Z

@pskopek Keycloak devservice is OK with this update :-), thanks

aloubyansky

Let's add the Keycloak dependencies that are direct dependencies of Quarkus extensions back to the BOM.

pskopek · 2024-10-10T14:21:45Z

@sberyozkin @aloubyansky I fixed the issues with versions you requested. There is a separation between Keycloak Server (25.0.6) for testing purpose and Keycloak Client Libs (26.0.0) used in Quarkus.
Can you review the new changes, please?

build-parent/pom.xml

aloubyansky · 2024-10-10T14:46:32Z

Dependency-wise this looks better @pskopek, thanks!

sberyozkin · 2024-10-18T18:15:27Z

@pskopek I've pushed the latest Quarkus updates to check if the OIDC test failure is repeatable, thanks

extensions/devservices/keycloak/pom.xml

sberyozkin · 2024-10-24T15:47:38Z

@pskopek Hi, the fact that so many dependencies are coming in with the new keycloak client now and we have to exclude them for example in the OIDC deployment, makes me a little bit nervous, any code now which only needs to work, for example, with RealmRepresentation only, will have to exclude all those dependencies manually...
I'd also suggest to keep keycloak.version, someone may be expecting it being set, some integration tests might be picking it up

maxandersen · 2024-10-24T22:39:50Z

If every quarkus user have to exclude those dependencies that is definitely problematic since afaik we cannot even do this in the platform bom.

We need a dependency that does not drag in unnecessary classes.

maxandersen

-1 until this is show to work without users having to exclude dependencies manually.

gsmet · 2024-10-25T09:52:56Z

Hmmm. Wait a bit: it has always been the case that we needed to exclude dependencies: the Keycloak Admin Client comes with a default REST Client implementation (or it wouldn't work out of the box).

In Quarkus, we have 2 extensions for the admin client that comes with either the RESTEasy client or the Quarkus REST client.

In practice, it's not a problem for our users: we are doing the work for them and this PR doesn't change a thing.

Happy to be corrected if I'm wrong.

michalvavrik · 2024-10-25T10:10:50Z

Hmmm. Wait a bit: it has always been the case that we needed to exclude dependencies: the Keycloak Admin Client comes with a default REST Client implementation (or it wouldn't work out of the box).

In Quarkus, we have 2 extensions for the admin client that comes with either the RESTEasy client or the Quarkus REST client.

In practice, it's not a problem for our users: we are doing the work for them and this PR doesn't change a thing.

Happy to be corrected if I'm wrong.

I watched this discussion progress and I think you are right, but there is one point in discussion I am missing. There are exclusions in Keycloak Dev Services for RESTEasy that is not needed. The keycloak-admin-client brings dependencies even though we only use POJOs (DTOs basically). AKA we don't use the REST client in many of these places. Therefore:

a default REST Client implementation (or it wouldn't work out of the box)

I think is mixing points. @pskopek is bringing Keycloak Admin Client instead of Keycloak Core and they don't want to put these DTOs into separate dependency because it would require additional work and that is why exclusions are needed. Do I get situation correctly @pskopek ?

michalvavrik · 2024-10-25T14:17:27Z

@pskopek I think the native failure is related Caused by: java.lang.IllegalStateException: Not able to load any cryptoProvider with the classLoader: jdk.internal.loader.ClassLoaders$AppClassLoader@1134affc.

sberyozkin · 2024-10-31T16:45:31Z

@michalvavrik

@pskopek is bringing Keycloak Admin Client instead of Keycloak Core and they don't want to put these DTOs into separate dependency because it would require additional work and that is why exclusions are needed.

This is likely to be precise... Peter, @pskopek, what do you think, would it be a problem to release keycloak-core on its own ?

I'm pretty sure that Keycloak devservice is not the only piece of code which uses keycloak core classes like RealmRepresentation but does not use any of Keycloak Admin APIs alongside them...

@pskopek Think about it please, check with the team, hopefully it won't be a problem to have keycloak-core separated from the Keycloak Admin Client API...

Let's try to finalize it as we need to to move to Keycloak 26.x

michalvavrik · 2024-11-06T20:07:07Z

...n/java/io/quarkus/keycloak/admin/client/reactive/runtime/ResteasyReactiveClientProvider.java

+                // Same like JSONSerialization class. Makes it possible to use admin-client against older versions of Keycloak server where the properties on representations might be different
+                newObjectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
+                // The client must work with the newer versions of Keycloak server, which might contain the JSON fields not yet known by the client. So unknown fields will be ignored.
+                newObjectMapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);


This change is interesting because it will only impact scenario when user customized objected mapper (or provided his own instance). I kinda expect that most users won't be impacted by this change. @geoand please review this file. Thank you

These changes makes Admin client to not fail when working with older KC versions. I trust that @pskopek determined that KC admin client is always backwards compatible and it is fine that KC Admin client will be used with older versions.

But - these changes will only affect scenarios when user customized ObjectMapper bean or provided his own bean:

https://quarkus.io/guides/all-config#quarkus-jackson_quarkus-jackson-fail-on-unknown-properties is set to false by default

https://quarkus.io/guides/all-config#quarkus-jackson_quarkus-jackson-serialization-inclusion is not set by default

And what I say:

either I read situation wrong

or this needs to be done for all the scenarios, because I expect that affected scenario here is only minor, most users don't need to customize default ObjectMapper?

Please @geoand or @pskopek check again or prove me wrong. Maybe I misread situation. Thank you

This change is interesting because it will only impact scenario when user customized objected mapper (or provided his own instance). I kinda expect that most users won't be impacted by this change. @geoand please review this file. Thank you

Agreed

I don't know what Keycloak's requirements are, but from the looks of this PR, it seems Keycloak needs a completely separate ObjecMapper

Thanks for having a look @geoand

Thanks for bringing it to my attention!

@pskopek Can you please comment here ?

Yes, Keycloak Admin needs differently configured ObjectMapper. I modified the PR, but I am not the Quarkus expert. @geoand , can you review the change, please?

pskopek · 2024-11-06T20:08:11Z

@sberyozkin I have updated keycloak-client to 26.0.2 and fixed all issues in tests.
I changed dependency to pure representations artifact (keycloak-client-common-synced) when possible. The issue with unwanted excludes is resolved now.

@michalvavrik missing CryptoProvider issue is fixed as well.

sberyozkin · 2024-11-07T16:55:38Z

Thanks Peter, @pskopek, looks much better now.

I have a couple of questions:

Does AuthzClientCryptoProvider have to be in the common code ? If that broke Keycloak Authorization native build then it could equally break someone else's extension native build; I propose to push it up to where it is required...
Can you name it keycloak-client-common ? Is -synced suffix important ?

PR is getting much closer to the completion though, thanks

sberyozkin · 2024-11-07T16:59:19Z

@pskopek Oh sorry, ignore please the Does AuthzClientCryptoProvider have to be in the common code ? question, as I think I got it wrong...

So then the only question is, can the common release artifact be named as keycloak-client-common ? It is probably not very important, just curious

pskopek · 2024-11-11T09:29:50Z

So then the only question is, can the common release artifact be named as keycloak-client-common ? It is probably not very important, just curious

We decided to name it with -synced suffix to indicate the fact that it is synced with some Keycloak server classes. It doesn't have to be named like this, but it somehow made sense for us.

sberyozkin · 2024-11-12T14:30:23Z

So then the only question is, can the common release artifact be named as keycloak-client-common ? It is probably not very important, just curious

We decided to name it with -synced suffix to indicate the fact that it is synced with some Keycloak server classes. It doesn't have to be named like this, but it somehow made sense for us.

@pskopek OK, thanks for the explanation, I guess we can work with this artifact name as well.

gsmet

I had another look.

I think we really need to inspect the dependencies of the adjusted modules as I think I have seen the Apache HTTP Client crawling back in the dependencies and I'm not sure that's what we want.

/cc @cescoffier @sberyozkin

gsmet · 2024-11-12T17:50:40Z

extensions/devservices/keycloak/pom.xml

-                    <artifactId>jakarta.activation</artifactId>
-                </exclusion>
-            </exclusions>
+            <artifactId>keycloak-client-common-synced</artifactId>


This now brings the Apache HTTP client as a dependency.

We used to have:

[INFO] +- org.keycloak:keycloak-core:jar:25.0.6:compile [INFO] | +- org.keycloak:keycloak-common:jar:25.0.6:compile [INFO] | | \- jakarta.activation:jakarta.activation-api:jar:2.1.3:compile [INFO] | +- com.fasterxml.jackson.core:jackson-core:jar:2.18.1:compile [INFO] | +- com.fasterxml.jackson.core:jackson-databind:jar:2.18.1:compile [INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.18.1:compile [INFO] | \- org.eclipse.microprofile.openapi:microprofile-openapi-api:jar:4.0.2:compile

we now have:

[INFO] +- org.keycloak:keycloak-client-common-synced:jar:26.0.2:compile [INFO] | +- org.jboss.logging:commons-logging-jboss-logging:jar:1.0.0.Final:runtime [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.14:compile [INFO] | | \- org.apache.httpcomponents:httpcore:jar:4.4.16:compile [INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-jdk8:jar:2.18.1:compile [INFO] | | +- com.fasterxml.jackson.core:jackson-core:jar:2.18.1:compile [INFO] | | \- com.fasterxml.jackson.core:jackson-databind:jar:2.18.1:compile [INFO] | +- com.fasterxml.jackson.datatype:jackson-datatype-jsr310:jar:2.18.1:compile [INFO] | +- org.eclipse.microprofile.openapi:microprofile-openapi-api:jar:4.0.2:compile [INFO] | \- jakarta.activation:jakarta.activation-api:jar:2.1.3:compile

I'm not sure we want that?

Now I don't know if we actually want to use the Admin Client now (or later?) to initialize the container in some way?

Now I don't know if we actually want to use the Admin Client now (or later?) to initialize the container in some way?

We don't use the Admin Client in KC Dev Svc. We use there Vert.x Web Client. We only need DTOs from org.keycloak.representations.

Not sure if it is relevant, but KC Dev Svc is now KC Admin Client dependency, I don't know if above-mentioned tree was done when rebased on current main. It only happened lately.

It's not relevant for the tree of one dependency, but it is relevant for context - if you do exclusions.

gsmet · 2024-11-12T17:53:27Z

extensions/keycloak-admin-rest-client/runtime/pom.xml

-                    <artifactId>jakarta.activation</artifactId>
-                </exclusion>
-            </exclusions>
-        </dependency>


Same here actually. We now have a dependency with the Apache HTTP Client and I don't think we want (except if I'm mistaken) that.

@gsmet This is a mistake at keycloak-client-common-synced side. I will fix it there and update this PR with new version of keycloak-client.

Thanks I will have another look next week.

…re and keycloak-adapter-spi Closes quarkusio#43259 Signed-off-by: Peter Skopek <[email protected]>

quarkus-bot · 2024-11-18T14:12:06Z

Status for workflow `Quarkus Documentation CI`

This is the status report for running Quarkus Documentation CI on commit fbf5c2a.

✅ The latest workflow run for the pull request has completed successfully.

It should be safe to merge provided you have a look at the other checks in the summary.

Warning

There are other workflow runs running, you probably need to wait for their status before merging.

quarkus-bot · 2024-11-18T15:29:41Z

Status for workflow `Quarkus CI`

This is the status report for running Quarkus CI on commit fbf5c2a.

Failing Jobs

Status	Name	Step	Failures	Logs	Raw logs	Build scan
✖	JVM Tests - JDK 17	`Build`	Failures	Logs	Raw logs	🔍
✖	JVM Tests - JDK 21	`Build`	Failures	Logs	Raw logs	🔍

Full information is available in the Build summary check run.
You can consult the Develocity build scans.

Failures

⚙️ JVM Tests - JDK 17 #

- Failing: extensions/keycloak-admin-resteasy-client/deployment 
! Skipped: extensions/keycloak-authorization/deployment integration-tests/keycloak-authorization

📦 extensions/keycloak-admin-resteasy-client/deployment

✖ io.quarkus.keycloak.adminclient.deployment.KeycloakAdminClientMutualTlsDevServicesTest.testCreateRealm line 50 - History - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <204> but was <400>.

	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:500)

⚙️ JVM Tests - JDK 21 #

- Failing: extensions/keycloak-admin-resteasy-client/deployment 
! Skipped: extensions/keycloak-authorization/deployment integration-tests/keycloak-authorization

📦 extensions/keycloak-admin-resteasy-client/deployment

✖ io.quarkus.keycloak.adminclient.deployment.KeycloakAdminClientMutualTlsDevServicesTest.testCreateRealm line 50 - History - More details - Source on GitHub

java.lang.AssertionError: 
1 expectation failed.
Expected status code <204> but was <400>.

	at java.base/jdk.internal.reflect.DirectConstructorHandleAccessor.newInstance(DirectConstructorHandleAccessor.java:62)
	at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:502)
	at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:486)
	at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:73)

quarkus-bot bot added area/dependencies Pull requests that update a dependency file area/keycloak area/oidc area/testing labels Sep 13, 2024

sberyozkin reviewed Sep 13, 2024

View reviewed changes

extensions/oidc-client/deployment/pom.xml Outdated Show resolved Hide resolved

sberyozkin reviewed Sep 13, 2024

View reviewed changes

extensions/oidc/deployment/pom.xml Outdated Show resolved Hide resolved

sberyozkin reviewed Sep 13, 2024

View reviewed changes

integration-tests/oidc-client-reactive/pom.xml Outdated Show resolved Hide resolved

sberyozkin requested a review from pedroigor September 13, 2024 09:27

This comment has been minimized.

Sign in to view

quarkus-bot bot added the triage/flaky-test label Sep 13, 2024

aloubyansky reviewed Sep 13, 2024

View reviewed changes

build-parent/pom.xml Outdated Show resolved Hide resolved

sberyozkin self-requested a review September 16, 2024 13:57

aloubyansky requested changes Sep 16, 2024

View reviewed changes

pskopek force-pushed the kc-client-libs branch from 71f2bbc to 8bd84fc Compare October 10, 2024 14:17

aloubyansky reviewed Oct 10, 2024

View reviewed changes

build-parent/pom.xml Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

pskopek force-pushed the kc-client-libs branch from 8bd84fc to 948263f Compare October 14, 2024 13:27

pskopek requested a review from aloubyansky October 15, 2024 05:52

pskopek force-pushed the kc-client-libs branch from 0f25936 to 3ed1996 Compare October 15, 2024 06:02

sberyozkin force-pushed the kc-client-libs branch from 1fa8ab9 to 9b3fb7f Compare October 18, 2024 18:14

This comment has been minimized.

Sign in to view

gsmet requested changes Oct 22, 2024

View reviewed changes

extensions/devservices/keycloak/pom.xml Outdated Show resolved Hide resolved

sberyozkin self-requested a review October 22, 2024 15:43

pskopek force-pushed the kc-client-libs branch 2 times, most recently from bfe72df to dd260da Compare October 24, 2024 08:24

maxandersen requested changes Oct 24, 2024

View reviewed changes

This comment has been minimized.

Sign in to view

pskopek force-pushed the kc-client-libs branch from dd260da to 2d2e357 Compare November 6, 2024 19:57

michalvavrik reviewed Nov 6, 2024

View reviewed changes

sberyozkin mentioned this pull request Nov 12, 2024

Bump Keycloak version to 26.0.5 #44449

Open

gsmet requested changes Nov 12, 2024

View reviewed changes

Use keycloak-client libraries instead of keycloak-common, keycloak-co…

fbf5c2a

…re and keycloak-adapter-spi Closes quarkusio#43259 Signed-off-by: Peter Skopek <[email protected]>

pskopek force-pushed the kc-client-libs branch from 2d2e357 to fbf5c2a Compare November 14, 2024 16:23

Use keycloak-client libraries instead of keycloak-common, keycloak-core and keycloak-adapter-spi #43260

Are you sure you want to change the base?

Use keycloak-client libraries instead of keycloak-common, keycloak-core and keycloak-adapter-spi #43260

Conversation

pskopek commented Sep 13, 2024

rsvoboda commented Sep 13, 2024

sberyozkin commented Sep 13, 2024

sberyozkin commented Sep 13, 2024

This comment has been minimized.

github-actions bot commented Sep 13, 2024 • edited Loading

aloubyansky commented Sep 13, 2024

This comment has been minimized.

gsmet commented Sep 13, 2024

sberyozkin commented Sep 15, 2024

aloubyansky left a comment

Choose a reason for hiding this comment

pskopek commented Oct 10, 2024

aloubyansky commented Oct 10, 2024

This comment has been minimized.

This comment has been minimized.

sberyozkin commented Oct 18, 2024

This comment has been minimized.

This comment has been minimized.

sberyozkin commented Oct 24, 2024 • edited Loading

maxandersen commented Oct 24, 2024

maxandersen left a comment

Choose a reason for hiding this comment

This comment has been minimized.

gsmet commented Oct 25, 2024

michalvavrik commented Oct 25, 2024 • edited Loading

This comment has been minimized.

michalvavrik commented Oct 25, 2024

sberyozkin commented Oct 31, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

pskopek commented Nov 6, 2024

sberyozkin commented Nov 7, 2024 • edited Loading

sberyozkin commented Nov 7, 2024 • edited Loading

pskopek commented Nov 11, 2024

sberyozkin commented Nov 12, 2024

gsmet left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

quarkus-bot bot commented Nov 18, 2024

Status for workflow Quarkus Documentation CI

quarkus-bot bot commented Nov 18, 2024 • edited by github-actions bot Loading

Status for workflow Quarkus CI

Failing Jobs

Failures

⚙️ JVM Tests - JDK 17 #

📦 extensions/keycloak-admin-resteasy-client/deployment

⚙️ JVM Tests - JDK 21 #

📦 extensions/keycloak-admin-resteasy-client/deployment

github-actions bot commented Sep 13, 2024 •

edited

Loading

sberyozkin commented Oct 24, 2024 •

edited

Loading

michalvavrik commented Oct 25, 2024 •

edited

Loading

sberyozkin commented Nov 7, 2024 •

edited

Loading

sberyozkin commented Nov 7, 2024 •

edited

Loading

Status for workflow `Quarkus Documentation CI`

quarkus-bot bot commented Nov 18, 2024 •

edited by github-actions bot

Loading

Status for workflow `Quarkus CI`