Adjust vheap sizes for message handling processes on Erlang/OTP 27 #12208
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kjnilsson
force-pushed
the
qq-otp27-conf
branch
from
f08cdd2
to
f24b02c
Compare
|
64B messages (represented differently within OTP than binaries above 64B): 65B messages: 100B messages: 1kb messages: 5kb messages: Full results: |
|
My interpretation of the benchmarks can be summarized as: with this QQ process configuration the throughput becomes comparable, if not better (with smallest messages), on Erlang 27 but latency is worse, in particular the 99% percentile/max. I think it's a reasonable outcome and will allow us to support Erlang 27 in 4.0 without major regressions. |
michaelklishin
approved these changes
Sep 5, 2024
kjnilsson
force-pushed
the
qq-otp27-conf
branch
from
d9e839f
to
bebd21d
Compare
kjnilsson
force-pushed
the
qq-otp27-conf
branch
from
bebd21d
to
295ff3a
Compare
kjnilsson
changed the title
mkuratczyk
approved these changes
Sep 12, 2024
mkuratczyk
force-pushed
the
qq-otp27-conf
branch
from
773a3f9
to
3b7cdde
Compare
michaelklishin
force-pushed
the
qq-otp27-conf
branch
from
3b7cdde
to
bf6ad89
Compare
|
The forced push was a rebase. @mkuratczyk I assume this change is still relevant, correct? |
|
Yes, I think we can merge it. @kjnilsson anything else you wanted to do here? |
OTP 27 reset all assumptions on how the vm reacts to processes that buffer and process a lot of large binaries. Substantially increasing the vheap sizes for such process restores most of the same performance by allowing processes to hold more binary data before major garbage collections are triggered. This introduces a new module to capture process flag configurations. The new vheap sizes are only applied when running on OTP 27 or above.
michaelklishin
force-pushed
the
qq-otp27-conf
branch
from
bf6ad89
to
6a7f8d0
Compare
|
Another rebase. |
michaelklishin
added a commit
that referenced
this pull request
Oct 11, 2024
Adjust vheap sizes for message handling processes in OTP 27 (backport #12208)
michaelklishin
changed the title
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
OTP 27 reset all assumptions on how the vm reacts to processes that
buffer and process a lot of large binaries.
Substantially increasing the vheap sizes for such process restores
most of the same performance by allowing processes to hold more binary
data before major garbage collections are triggered.
This introduces a new module to capture process flag configurations.
The new vheap sizes are only applied when running on OTP 27 or
above.