Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SOLR-17541: LBSolrClient implementations should agree on 'getClient()' semantics #2899

Merged
merged 35 commits into from
Dec 23, 2024
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
5261cd3
Initial development
jdyer1 Dec 4, 2024
bc8c888
fix bugs
jdyer1 Dec 5, 2024
a249169
javadoc modifications
jdyer1 Dec 5, 2024
1f8a307
tidy / remove nocommit
jdyer1 Dec 5, 2024
90bf9e2
remove obsolete test
jdyer1 Dec 6, 2024
51676f5
Fix bug: (1) Only generate a client per base url, not base url+core.…
jdyer1 Dec 6, 2024
db76064
tidy
jdyer1 Dec 6, 2024
605bdc7
Only hold the internal builder
jdyer1 Dec 6, 2024
508af0d
remove override base url from jdk client
jdyer1 Dec 6, 2024
39177b4
fix test
jdyer1 Dec 9, 2024
35651b8
Merge branch 'main' into feature/SOLR-17541
jdyer1 Dec 9, 2024
19577dc
add missing @Override
jdyer1 Dec 9, 2024
ca59536
implement code review suggestions
jdyer1 Dec 10, 2024
67c8b8f
fix getClient/buildClient
dsmiley Dec 12, 2024
7b90862
more
dsmiley Dec 12, 2024
1539d97
PKI Authentication Plugin to add the listener factory to subsequently…
jdyer1 Dec 13, 2024
8f17279
@lucene.experimental annotations
jdyer1 Dec 13, 2024
effedb9
tidy
jdyer1 Dec 13, 2024
29a3445
remove system.out.println
jdyer1 Dec 13, 2024
50470cd
httpClientBuilder > httpSolrClientBuilder
jdyer1 Dec 16, 2024
f423bd2
API change to Http2SolrClient.Builder:
jdyer1 Dec 16, 2024
6176a99
Remove unnecessary null check
jdyer1 Dec 16, 2024
783f815
tidy
jdyer1 Dec 16, 2024
e8c998d
small change to doc comment
jdyer1 Dec 16, 2024
e035a0c
fix typo
jdyer1 Dec 17, 2024
99282aa
Better variable names
jdyer1 Dec 17, 2024
45b9e10
remove silly duplicate method
jdyer1 Dec 18, 2024
7df3c3b
variable rename
jdyer1 Dec 19, 2024
90bd2ec
Merge branch 'main' into feature/SOLR-17541
jdyer1 Dec 19, 2024
f984d12
TODO on HttpClientBuilderPlugin
jdyer1 Dec 19, 2024
28aec3b
CHANGES.txt
jdyer1 Dec 19, 2024
30dcdb3
tidy
jdyer1 Dec 19, 2024
730aad0
CHANGES.txt deprecation notice for Solr 9.9
jdyer1 Dec 19, 2024
071fb04
CHANGES.txt: mention rename LBHttp2SolrClient.Builder#withListenerFac…
jdyer1 Dec 19, 2024
05eb1ea
Merge branch 'main' into feature/SOLR-17541
jdyer1 Dec 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
PKI Authentication Plugin to add the listener factory to subsequently…
…-built clients
  • Loading branch information
jdyer1 committed Dec 13, 2024
commit 1539d97ca1fdca235bcb9717853055909de25470
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT @iamsanjay ?

Original file line number Diff line number Diff line change
@@ -37,13 +37,15 @@ final class HttpSolrClientProvider implements AutoCloseable {

private final Http2SolrClient httpSolrClient;

private final Http2SolrClient.Builder httpClientBuilder;

private final InstrumentedHttpListenerFactory trackHttpSolrMetrics;

HttpSolrClientProvider(UpdateShardHandlerConfig cfg, SolrMetricsContext parentContext) {
trackHttpSolrMetrics = new InstrumentedHttpListenerFactory(getNameStrategy(cfg));
initializeMetrics(parentContext);

Http2SolrClient.Builder httpClientBuilder =
this.httpClientBuilder =
new Http2SolrClient.Builder().withListenerFactory(List.of(trackHttpSolrMetrics));

if (cfg != null) {
@@ -76,7 +78,7 @@ Http2SolrClient getSolrClient() {
}

void setSecurityBuilder(HttpClientBuilderPlugin builder) {
builder.setup(httpSolrClient);
builder.setup(httpClientBuilder, httpSolrClient);
}

@Override
Original file line number Diff line number Diff line change
@@ -83,6 +83,7 @@ public class HttpShardHandlerFactory extends ShardHandlerFactory
protected ExecutorService commExecutor;

protected volatile Http2SolrClient defaultClient;
protected Http2SolrClient.Builder httpClientBuilder;
protected InstrumentedHttpListenerFactory httpListenerFactory;
protected LBHttp2SolrClient<Http2SolrClient.Builder> loadbalancer;

@@ -305,16 +306,16 @@ public void init(PluginInfo info) {
sb);
int soTimeout =
getParameter(args, HttpClientUtil.PROP_SO_TIMEOUT, HttpClientUtil.DEFAULT_SO_TIMEOUT, sb);
var defaultClientBuilder =
this.httpClientBuilder =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please name httpSolrClientBuilder -- so as to clarify this isn't an Apache/Jetty HttpClient; it's a SolrClient.

new Http2SolrClient.Builder()
.withConnectionTimeout(connectionTimeout, TimeUnit.MILLISECONDS)
.withIdleTimeout(soTimeout, TimeUnit.MILLISECONDS)
.withExecutor(commExecutor)
.withMaxConnectionsPerHost(maxConnectionsPerHost)
.withListenerFactory(List.of(this.httpListenerFactory));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I noticed an issue. This overwrites the list of listeners that may already be present. Shouldn't we be augmenting, not replacing?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was trying to work within the bounds of the existing API, but I agree this should be an add, not a replace. I modified the API, and broke backwards-compatibility so as to not silently introduce bugs in users' applications.

this.defaultClient = defaultClientBuilder.build();
this.defaultClient = httpClientBuilder.build();

this.loadbalancer = new LBHttp2SolrClient.Builder<>(defaultClientBuilder).build();
this.loadbalancer = new LBHttp2SolrClient.Builder<>(httpClientBuilder).build();

initReplicaListTransformers(getParameter(args, "replicaRouting", null, sb));

@@ -323,8 +324,10 @@ public void init(PluginInfo info) {

@Override
public void setSecurityBuilder(HttpClientBuilderPlugin clientBuilderPlugin) {
System.out.println("IN SET SEC BUILDER");
new Exception().printStackTrace(System.out);
if (clientBuilderPlugin != null) {
clientBuilderPlugin.setup(defaultClient);
clientBuilderPlugin.setup(httpClientBuilder, defaultClient);
}
}

Original file line number Diff line number Diff line change
@@ -34,4 +34,8 @@ public interface HttpClientBuilderPlugin {
public SolrHttpClientBuilder getHttpClientBuilder(SolrHttpClientBuilder builder);

public default void setup(Http2SolrClient client) {}

public default void setup(Http2SolrClient.Builder httpClientBuilder, Http2SolrClient client) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe these should be split up; setup a client and builder separately (2 methods)? Feels weird to take both in setup().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I agree separate methods would be nicer. But the implementing classes do things on setup that should only be done once. I settled on this ugly API with hopes this is the least-trappy thing to leave for future developers.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you point me at such so I can appreciate the problem better?

setup(client);
}
}
Original file line number Diff line number Diff line change
@@ -374,8 +374,17 @@ PublicKey fetchPublicKeyFromRemote(String nodename) {
}
}

@Override
public void setup(Http2SolrClient.Builder httpClientBuilder, Http2SolrClient client) {
setup(client, httpClientBuilder);
}

@Override
public void setup(Http2SolrClient client) {
setup(client, null);
}

private void setup(Http2SolrClient client, Http2SolrClient.Builder builder) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why have this separate and with different parameter order? In other words, lets make this one the @Override with builder then client. Remove the 4 lines earlier on 377.

final HttpListenerFactory.RequestResponseListener listener =
new HttpListenerFactory.RequestResponseListener() {
private static final String CACHED_REQUEST_USER_KEY = "cachedRequestUser";
@@ -431,7 +440,12 @@ private Optional<String> getUserFromJettyRequest(Request request) {
(String) request.getAttributes().get(CACHED_REQUEST_USER_KEY));
}
};
client.addListenerFactory(() -> listener);
if(client != null) {
client.addListenerFactory(() -> listener);
}
if(builder != null) {
builder.withListenerFactory(List.of(() -> listener));
}
}

@Override