Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg #48618

Merged
merged 1 commit into from
Jul 22, 2024
Merged

[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg #48618

merged 1 commit into from
Jul 22, 2024

Conversation

stdpain
Copy link
Contributor

@stdpain stdpain commented Jul 19, 2024 •
edited
Loading

Why I'm doing:

In past implementations, Function States would not be released.

For sorted streaming agg or spill aggregates, they will not be released even if destroy is called.

What I'm doing:

This PR adds a remove state interface to Function States.

After calling destroy, it will call function states-> remove.

Fixes #issue

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.3
    • 3.2
    • 3.1
    • 3.0
    • 2.5

@stdpain stdpain requested a review from a team as a code owner July 19, 2024 07:47
@stdpain stdpain force-pushed the fix_udf_spill_case branch 2 times, most recently from 16be96b to 0ae3d8e Compare July 19, 2024 11:34
@stdpain
[BugFix] Fix Java UDAF OOM when spill
be7d803 
In past implementations, Function States would not be released.

For sorted streaming agg or spill aggregates, they will not be released even if destroy is called.
This PR adds a remove state interface to Function States.

After calling destroy, it will call function states-> remove.

Signed-off-by: stdpain <[email protected]>
@sonarcloud SonarCloud
Copy link

sonarcloud bot commented Jul 20, 2024

Quality Gate Passed Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

✅ pass : 58 / 65 (89.23%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/exprs/agg/java_window_function.cpp 0 4 00.00% [68, 69, 70, 71]
🔵 be/src/exprs/agg/java_udaf_function.h 19 22 86.36% [276, 277, 355]
🔵 be/src/exprs/agg/java_udaf_function.cpp 4 4 100.00% []
🔵 be/src/udf/java/java_udf.cpp 24 24 100.00% []
🔵 be/src/exprs/function_context.cpp 5 5 100.00% []
🔵 be/src/exec/aggregator.cpp 6 6 100.00% []

@kangkaisen kangkaisen merged commit 36b3aae into StarRocks:main Jul 22, 2024
47 of 48 checks passed
@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.3

@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.2

@github-actions github-actions bot removed the 3.3 label Jul 22, 2024
@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.1

@github-actions GitHub Actions
Copy link

@Mergifyio backport branch-3.0

@github-actions github-actions bot removed the 3.0 label Jul 22, 2024
@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 22, 2024 •
edited
Loading

backport branch-3.3

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 22, 2024 •
edited
Loading

backport branch-3.2

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 22, 2024 •
edited
Loading

backport branch-3.1

✅ Backports have been created

@mergify Mergify
Copy link
Contributor

mergify bot commented Jul 22, 2024 •
edited
Loading

backport branch-3.0

✅ Backports have been created

mergify bot pushed a commit that referenced this pull request Jul 22, 2024
@stdpain @mergify
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (#48618)
395aa7a 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit 36b3aae)
@mergify mergify bot mentioned this pull request Jul 22, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Jul 22, 2024
@stdpain @mergify
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (#48618)
0447776 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit 36b3aae)
@mergify mergify bot mentioned this pull request Jul 22, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Jul 22, 2024
@stdpain @mergify
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (#48618)
882ceee 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit 36b3aae)
@mergify mergify bot mentioned this pull request Jul 22, 2024
Merged
42 tasks
mergify bot pushed a commit that referenced this pull request Jul 22, 2024
@stdpain @mergify
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (#48618)
8f87f45 
Signed-off-by: stdpain <[email protected]>
(cherry picked from commit 36b3aae)

# Conflicts:
#	be/src/exprs/function_context.cpp
@mergify mergify bot mentioned this pull request Jul 22, 2024
Closed
42 tasks
wanpengfei-git pushed a commit that referenced this pull request Jul 22, 2024
@mergify @stdpain
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (backport #…
39d01f0 
…48618) (#48671)

Co-authored-by: stdpain <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Jul 22, 2024
@mergify @stdpain
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (backport #…
0925431 
…48618) (#48670)

Co-authored-by: stdpain <[email protected]>
wanpengfei-git pushed a commit that referenced this pull request Jul 24, 2024
@mergify @stdpain
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (backport #…
c26381a 
…48618) (#48669)

Co-authored-by: stdpain <[email protected]>
dujijun007 pushed a commit to dujijun007/starrocks that referenced this pull request Jul 29, 2024
@stdpain @dujijun007
[BugFix] Fix Java UDAF OOM in spill or sorted streaming agg (StarRock…
345c77c 
…s#48618)

Signed-off-by: stdpain <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Seaven Seaven Seaven approved these changes

Assignees

@stdpain stdpain

Labels
3.1-merged 3.2-merged 3.3-merged
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@stdpain @Seaven @kangkaisen