Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Refactor] add a config to ignore rpc overcrowded problem #47332

Merged
merged 1 commit into from
Jul 1, 2024
Merged

[Refactor] add a config to ignore rpc overcrowded problem #47332

merged 1 commit into from
Jul 1, 2024

Conversation

stdpain
Copy link
Contributor

@stdpain stdpain commented Jun 21, 2024 •
edited
Loading

Why I'm doing:

If the amount of data to be sent by a single channel of brpc exceeds brpc_socket_max_unwritten_bytes
it will cause rpc to report an error. We add configuration to ignore rpc overload.
This may cause process memory usage to rise.
In the future we need to count the memory on each channel of the rpc. To do application layer rpc flow limiting.

What I'm doing:

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@stdpain stdpain requested review from a team as code owners June 21, 2024 08:22
@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

✅ pass : 6 / 6 (100.00%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/storage/segment_replicate_executor.cpp 1 1 100.00% []
🔵 be/src/exec/dictionary_cache_writer.cpp 1 1 100.00% []
🔵 be/src/exec/tablet_sink_index_channel.cpp 3 3 100.00% []
🔵 be/src/exec/pipeline/exchange/sink_buffer.cpp 1 1 100.00% []

@stdpain stdpain changed the title [Enhancement] add a config to ignore rpc overcrowded problem [Refactor] add a config to ignore rpc overcrowded problem Jun 26, 2024
@stdpain stdpain changed the title [Refactor] add a config to ignore rpc overcrowded problem [Refactor] add a config to ignore rpc overcrowded problem Jun 28, 2024
@meegoo meegoo enabled auto-merge (squash) July 1, 2024 02:26
@meegoo meegoo merged commit ea923b7 into StarRocks:main Jul 1, 2024
85 of 92 checks passed
@stdpain
Copy link
Contributor Author

stdpain commented Jul 1, 2024

https://github.com/Mergifyio backport branch-3.3 branch-3.2 branch-3.1 branch-3.0 branch-2.5

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 1, 2024 •
edited
Loading

backport branch-3.3 branch-3.2 branch-3.1 branch-3.0 branch-2.5

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Jul 1, 2024
@stdpain @mergify
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
f26dfd3 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit ea923b7)
mergify bot pushed a commit that referenced this pull request Jul 1, 2024
@stdpain @mergify
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
bbdf920 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit ea923b7)

# Conflicts:
#	be/src/exec/dictionary_cache_writer.cpp
This was referenced Jul 1, 2024
Merged
Closed
mergify bot pushed a commit that referenced this pull request Jul 1, 2024
@stdpain @mergify
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
775b3c5 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit ea923b7)

# Conflicts:
#	be/src/exec/dictionary_cache_writer.cpp
#	be/src/exec/tablet_sink_index_channel.cpp
@mergify mergify bot mentioned this pull request Jul 1, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Jul 1, 2024
@stdpain @mergify
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
daeee1d 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit ea923b7)

# Conflicts:
#	be/src/common/config.h
#	be/src/exec/dictionary_cache_writer.cpp
#	be/src/exec/tablet_sink_index_channel.cpp
mergify bot pushed a commit that referenced this pull request Jul 1, 2024
@stdpain @mergify
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
8fbf4e1 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit ea923b7)

# Conflicts:
#	be/src/common/config.h
#	be/src/exec/dictionary_cache_writer.cpp
#	be/src/exec/tablet_sink_index_channel.cpp
This was referenced Jul 1, 2024
Closed
Closed
wanpengfei-git pushed a commit that referenced this pull request Jul 1, 2024
@mergify @stdpain
[Refactor] add a config to ignore rpc overcrowded problem (backport #…
020510f 
…47332) (#47668)

Co-authored-by: stdpain <[email protected]>
stdpain added a commit that referenced this pull request Jul 11, 2024
@stdpain
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
dca5344 
Signed-off-by: stdpain <[email protected]>

Signed-off-by: stdpain <[email protected]>
stdpain added a commit that referenced this pull request Jul 11, 2024
@stdpain
[Refactor] add a config to ignore rpc overcrowded problem (#47332)
ea26cc8 
Signed-off-by: stdpain <[email protected]>

Signed-off-by: stdpain <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Jul 11, 2024
@stdpain
[Refactor] add a config to ignore rpc overcrowded problem (backport #…
4966fcb 
…47332) (#48232)

Signed-off-by: stdpain <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Jul 12, 2024
@mergify @stdpain
[Refactor] add a config to ignore rpc overcrowded problem (backport #…
3349f59 
…47332) (#47670)

Signed-off-by: stdpain <[email protected]>
Co-authored-by: stdpain <[email protected]>
stdpain added a commit to stdpain/starrocks-2 that referenced this pull request Jul 29, 2024
@stdpain
[Refactor] add a config to ignore rpc overcrowded problem (StarRocks#…
eb8da4f 
…47332)

Signed-off-by: stdpain <[email protected]>

Signed-off-by: stdpain <[email protected]>
@stdpain stdpain mentioned this pull request Jul 29, 2024
Merged
24 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kevincai kevincai kevincai approved these changes

@meegoo meegoo meegoo approved these changes

@silverbullet233 silverbullet233 silverbullet233 approved these changes

@Seaven Seaven Seaven approved these changes

Assignees

@stdpain stdpain

Labels
3.1-merged 3.2-merged 3.3-merged title needs [type]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@stdpain @kevincai @meegoo @silverbullet233 @Seaven