Skip to content

Commit

Permalink
HTTP-119 Pickup up public CAs when there are no security options conf…
Browse files Browse the repository at this point in the history 
…igured (#128)

Signed-off-by: davidradl <[email protected]>
  • Loading branch information
@davidradl
davidradl authored Oct 18, 2024
1 parent d712792 commit eac08ed
Show file tree
Hide file tree
Showing 3 changed files with 39 additions and 17 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

### Added

- Added support for built in JVM certificates if no security is configured.
- Added support for OIDC Bearer tokens.

### Fixed
Expand Down
35 changes: 19 additions & 16 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -392,18 +392,21 @@ An example of such a mask would be `3XX, 4XX, 5XX`. In this case, all 300s, 400s
Many status codes can be defined in one value, where each code should be separated with comma, for example:
`401, 402, 403`. In this example, codes 401, 402 and 403 would not be interpreted as error codes.

## TLS and mTLS support
Both Http Sink and Lookup Source connectors supports Https communication using TLS 1.2 and mTLS.
## TLS (more secure replacement for SSL) and mTLS support

Both Http Sink and Lookup Source connectors support HTTPS communication using TLS 1.2 and mTLS.
To enable Https communication simply use `https` protocol in endpoint's URL.
If certificate used by HTTP server is self-signed, or it is signed byt not globally recognize CA
you would have to add this certificate to connector's keystore as trusted certificate.
In order to do so, use `gid.connector.http.security.cert.server` connector property,
which value is a path to the certificate. You can also use your organization's CA Root certificate.
You can specify many certificate, separating each path with `,`.

You can also configure connector to use mTLS. For this simply use `gid.connector.http.security.cert.client`
and `gid.connector.http.security.key.client` connector properties to specify path to certificate and
private key that should be used by connector. Key MUST be in `PCKS8` format. Both PEM and DER keys are

To specify certificate(s) to be used by the server, use `gid.connector.http.security.cert.server` connector property;
the value is a comma separated list of paths to certificate(s), for example you can use your organization's CA
Root certificate, or a self-signed certificate.

Note that if there are no security properties for a `https` url then, the JVMs default certificates are
used - allowing use of globally recognized CAs without the need for configuration.

You can also configure the connector to use mTLS. For this simply use `gid.connector.http.security.cert.client`
and `gid.connector.http.security.key.client` connector properties to specify paths to the certificate and
private key. The key MUST be in `PCKS8` format. Both PEM and DER keys are
allowed.

All properties can be set via Sink's builder `.setProperty(...)` method or through Sink and Source table DDL.
Expand All @@ -415,7 +418,7 @@ To enable this option use `gid.connector.http.security.cert.server.allowSelfSign
## Basic Authentication
The connector supports Basic Authentication using a HTTP `Authorization` header.
The header value can be set via properties, similarly as for other headers. The connector converts the passed value to Base64 and uses it for the request.
If the used value starts with the prefix `Basic `, or `gid.connector.http.source.lookup.use-raw-authorization-header`
If the used value starts with the prefix `Basic`, or `gid.connector.http.source.lookup.use-raw-authorization-header`
is set to `'true'`, it will be used as header value as is, without any extra modification.

## OIDC Bearer Authentication
Expand Down Expand Up @@ -452,13 +455,13 @@ be requested if the current time is later than the cached token expiry time minu
| lookup.max-retries | optional | The max retry times if the lookup failed; default is 3. See the following <a href="#lookup-cache">Lookup Cache</a> section for more details. |
| gid.connector.http.lookup.error.code | optional | List of HTTP status codes that should be treated as errors by HTTP Source, separated with comma. |
| gid.connector.http.lookup.error.code.exclude | optional | List of HTTP status codes that should be excluded from the `gid.connector.http.lookup.error.code` list, separated with comma. |
| gid.connector.http.security.cert.server | optional | Path to trusted HTTP server certificate that should be add to connectors key store. More than one path can be specified using `,` as path delimiter. |
| gid.connector.http.security.cert.server | optional | Comma separated paths to trusted HTTP server certificates that should be added to the connectors trust store. |
| gid.connector.http.security.cert.client | optional | Path to trusted certificate that should be used by connector's HTTP client for mTLS communication. |
| gid.connector.http.security.key.client | optional | Path to trusted private key that should be used by connector's HTTP client for mTLS communication. |
| gid.connector.http.security.cert.server.allowSelfSigned | optional | Accept untrusted certificates for TLS communication. |
| gid.connector.http.security.oidc.token.request | optional | OIDC `Token Request` body in `application/x-www-form-urlencoded` encoding |
| gid.connector.http.security.oidc.token.endpoint.url | optional | OIDC `Token Endpoint` url, to which the token request will be issued |
| gid.connector.http.security.oidc.token.expiry.reduction | optional | OIDC tokens will be requested if the current time is later than the cached token expiry time minus this value. |
| gid.connector.http.security.oidc.token.request | optional | OIDC `Token Request` body in `application/x-www-form-urlencoded` encoding |
| gid.connector.http.security.oidc.token.endpoint.url | optional | OIDC `Token Endpoint` url, to which the token request will be issued |
| gid.connector.http.security.oidc.token.expiry.reduction | optional | OIDC tokens will be requested if the current time is later than the cached token expiry time minus this value. |
| gid.connector.http.source.lookup.request.timeout | optional | Sets HTTP request timeout in seconds. If not specified, the default value of 30 seconds will be used. |
| gid.connector.http.source.lookup.request.thread-pool.size | optional | Sets the size of pool thread for HTTP lookup request processing. Increasing this value would mean that more concurrent requests can be processed in the same time. If not specified, the default value of 8 threads will be used. |
| gid.connector.http.source.lookup.response.thread-pool.size | optional | Sets the size of pool thread for HTTP lookup response processing. Increasing this value would mean that more concurrent requests can be processed in the same time. If not specified, the default value of 4 threads will be used. |
Expand Down
20 changes: 19 additions & 1 deletion src/main/java/com/getindata/connectors/http/internal/utils/JavaNetHttpClientFactory.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import java.net.http.HttpClient;
import java.net.http.HttpClient.Redirect;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties;
Expand Down Expand Up @@ -73,7 +74,8 @@ public static HttpClient createClient(Properties properties, Executor executor)
* @return new {@link SSLContext} instance.
*/
private static SSLContext getSslContext(Properties properties) {
SecurityContext securityContext = createSecurityContext(properties);
String keyStorePath =
properties.getProperty(HttpConnectorConfigConstants.KEY_STORE_PATH, "");

boolean selfSignedCert = Boolean.parseBoolean(
properties.getProperty(HttpConnectorConfigConstants.ALLOW_SELF_SIGNED, "false"));
Expand All @@ -88,6 +90,22 @@ private static SSLContext getSslContext(Properties properties) {
String clientPrivateKey = properties
.getProperty(HttpConnectorConfigConstants.CLIENT_PRIVATE_KEY, "");

if (StringUtils.isNullOrWhitespaceOnly(keyStorePath)
&& !selfSignedCert
// checking the property in this way so that serverTrustedCerts is not left and null
// or empty, which causes the http client to error.
&& (properties.getProperty(HttpConnectorConfigConstants.SERVER_TRUSTED_CERT)
== null)
&& StringUtils.isNullOrWhitespaceOnly(clientCert)
&& StringUtils.isNullOrWhitespaceOnly(clientPrivateKey)) {
try {
return SSLContext.getDefault();
} catch (NoSuchAlgorithmException e) {
throw new RuntimeException(e);
}
}

SecurityContext securityContext = createSecurityContext(properties);
for (String cert : serverTrustedCerts) {
if (!StringUtils.isNullOrWhitespaceOnly(cert)) {
securityContext.addCertToTrustStore(cert);
Expand Down

0 comments on commit eac08ed

Please sign in to comment.