Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Support to configure socket timeout for write #122

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

banmoy
Copy link
Collaborator

@banmoy banmoy commented Sep 19, 2024

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Which issues of this PR fixes :

Fixes #

Problem Summary(Required) :

Add a configuration starrocks.write.socket.timeout.ms. It's the timeout in milliseconds that the http client waits for data or, put differently, a maximum period inactivity between two consecutive data packets. The default value -1 is same as that of apache http client which is interpreted as undefined (system default if applicable). A timeout value of zero is interpreted as an infinite timeout. You can use this option to fail the stream load from the connector side if the http client does not receive response from StarRocks before timeout. The other option starrocks.write.properties.timeout take effects on the StarRocks side, but the response to the client may delay in some unexpected cases. If you want to have a strict timeout from the connector side, you can set this option to an acceptable value.

After reaching the timeout, there will be an exception like this

java.net.SocketTimeoutException: Read timed out
	at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_345]
	at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_345]
	at java.net.SocketInputStream.read(SocketInputStream.java:171) ~[?:1.8.0_345]
	at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_345]
	at com.starrocks.streamload.shade.org.apache.http.impl.conn.LoggingInputStream.read(LoggingInputStream.java:84) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:137) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:153) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:138) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]
	at com.starrocks.streamload.shade.org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:108) ~[starrocks-stream-load-sdk-1.0-SNAPSHOT-jar-with-dependencies.jar:?]

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr will affect users' behaviors
  • This pr needs user documentation (for new or modified features or behaviors)
  • I have added documentation for my new feature or new function

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant