[MRESOLVER-396] Back off on too many requests #326
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Provide a way to "back off", and lower the request pace if remote claims "too many requests" --- https://issues.apache.org/jira/projects/MRESOLVER/issues/MRESOLVER-396
kwin
requested changes
Sep 4, 2023
maven-resolver-api/src/main/java/org/eclipse/aether/ConfigurationProperties.java
Show resolved
Hide resolved
...resolver-transport-http/src/main/java/org/eclipse/aether/transport/http/HttpTransporter.java
Outdated
Show resolved
Hide resolved
* exponential backoff * Retry-After header * Maximum of retryInterval, when request should be rather failed
kwin
approved these changes
Sep 4, 2023
kwin
reviewed
Sep 4, 2023
src/site/markdown/configuration.md
Outdated
| @@ -43,8 +43,11 @@ Option | Type | Description | Default Value | Supports Repo ID Suffix | |||
| `aether.connector.http.preemptiveAuth` | boolean | Should HTTP client use preemptive-authentication for all HTTP verbs (works only w/ BASIC). By default is disabled, as it is considered less secure. | `false` | yes | |||
| `aether.connector.http.preemptivePutAuth` | boolean | Should HTTP client use preemptive-authentication for HTTP PUTs only (works only w/ BASIC). By default is enabled (same as Wagon). | `true` | yes | |||
| `aether.connector.http.retryHandler.count` | int | The maximum number of times a request to a remote HTTP server should be retried in case of an error. | `3` | yes | |||
| `aether.connector.http.retryHandler.interval` | long | The initial retry interval if server responds with "too many requests". Is multiplied by 1, 2,.. on each try. | `5000` | yes | |||
There was a problem hiding this comment.
Suggested change
| `aether.connector.http.retryHandler.interval` | long | The initial retry interval if server responds with "too many requests". Is multiplied by 1, 2,.. on each try. | `5000` | yes | |
| `aether.connector.http.retryHandler.interval` | long | The initial retry interval if server responds with "too many requests". Previous value is multiplied by 2 on each retry. | `5000` | yes |
There was a problem hiding this comment.
No, is not, initial value is multiplied with 1, 2, 3...
There was a problem hiding this comment.
Isn’t this what my suggestion says as well?
There was a problem hiding this comment.
I think not, you proposed "exponential", where you have some sort of loop like this:
retryInterval = initial
loop:
retryInterval = retryInterval * counter
For input 1, 2, 3 and initial=5000 this produces 5000, 10000, 30000
What I did is slightly different:
retryInterval;
loop:
retryInterval = initial * counter
For input 1,2,3 and initial=5000 this produces 5000, 10000, 15000 etc
IMHO, "real exponential" does not really make sense here: your build is time sensitive, if you are forced to wait (exponentially) for server as it is overloaded, then its better to make your IT env/vendor fix it 😄
There was a problem hiding this comment.
sorry, you are right. Let's leave it like you proposed.
gnodet
reviewed
Sep 4, 2023
...resolver-transport-http/src/main/java/org/eclipse/aether/transport/http/HttpTransporter.java
Show resolved
Hide resolved
gnodet
approved these changes
Sep 4, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Provide a way to "back off", and lower the request pace if remote claims "too many requests"
https://issues.apache.org/jira/projects/MRESOLVER/issues/MRESOLVER-396