Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Replace SubstreamCancelStrategy with SupervisionDecider for Split #252

Merged

Conversation

mdedetrich
Copy link
Contributor

@mdedetrich mdedetrich commented Mar 18, 2023

This PR is the equivalent of akka/akka#31415 but for Pekko, I would recommend reading the conversation thread from that PR to get the context/justification for the feature.

The very brief summary is that the SubstreamCancelStrategy abstraction that is only used when creating a SubFlow (via usage of splitWhen and other associated functions) is not needed since Pekko already provides a generic streams abstraction for this called a SupervisionDecider. It can even be said that the SubstreamCancelStrategy is worse than pointless duplication because unlike SupervisionDecider you can't do things like log the thrown exception in the SubFlow (this is actually the main impetus for the change, because as user I had the situation where SubFlow was throwing exceptions and I had no idea what was going on because it wasn't logging them via a custom SupervisionDecider which I defined, even worse because its not possible to change the already defined SubstreamCancelStrategy you have to do annoying workarounds to log any potentially thrown exceptions in a SubFlow).

Note that this change although not source/binary breaking it does change behaviour because the default SupervisionDecider is to stop in case of an exception where as the default SubstreamCancelStrategy default is to ignore and keep on going (this is also the reason behind the test changes in the PR). As stated in the original PR, even though it is a breaking behavioural change, since it fails more explicitly its much easier for a user to notice (see akka/akka#31415 (review)).

Due to this behaviour change the PR was intended to be looked at by @raboof and @jrudolph to get buy in/opinion whether this change is acceptable (see akka/akka#31415 (comment)), would be great to get feedback on this PR's premise (because otherwise we have to create an entirely new set of functions which overall I think is worse proposition).

PR is currently draft because I didn't update the docs for the behavioural change. If @jrudolph / @raboof are happy with this change (along with anyone else that has strong experience with akka/pekko-streams, i.e. @He-Pin ) then I will update the PR with proper documentation explaining all the changes. It will also target Pekko 1.1.x.

@mdedetrich mdedetrich added enhancement New feature or request help wanted Extra attention is needed labels Mar 18, 2023
@mdedetrich mdedetrich marked this pull request as draft March 18, 2023 11:37
@mdedetrich mdedetrich added this to the 1.1.0 milestone Mar 18, 2023
@mdedetrich mdedetrich force-pushed the use-supervisiondecider-with-split branch from fcf46b5 to b7ff620 Compare March 18, 2023 11:43
@He-Pin
Copy link
Member

He-Pin commented Mar 18, 2023

I will read this PR in more detail, and add feedback later.

@He-Pin
Copy link
Member

He-Pin commented Mar 18, 2023

The original issue is : akka/akka#31394

@He-Pin
Copy link
Member

He-Pin commented Mar 18, 2023

Pekko-HTTP is using the splitAfter and splitWhen IIRC, how about running a snapshot against it too?

@mdedetrich
Copy link
Contributor Author

mdedetrich commented Mar 18, 2023

Pekko-HTTP is using the splitAfter and splitWhen IIRC, how about running a snapshot against it too?

Sure I can do this, I would expect that pekko-http 1.1.x would depend on pekko 1.1.x so if there are any breaking changes in pekko-http it would just be for 1.1.x.

I can check locally what impact this change would have against pekko-http

@mdedetrich
Copy link
Contributor Author

Pekko-HTTP is using the splitAfter and splitWhen IIRC, how about running a snapshot against it too?

See apache/pekko-http#371 which is related

@mdedetrich mdedetrich force-pushed the use-supervisiondecider-with-split branch from b7ff620 to 93710fd Compare December 8, 2023 03:00
@mdedetrich mdedetrich marked this pull request as ready for review December 8, 2023 03:00
@mdedetrich
Copy link
Contributor Author

mdedetrich commented Dec 8, 2023

So I have decided to finish off this PR (which just involved updating the documentation) since it makes sense to release this as part of Pekko 1.1.0. Of note is that I had to add a new migration-guide-1.0.x-1.1.x.md which follows the same style Akka used (I copied it from https://github.com/mdedetrich/akka-apache/blob/main/akka-docs/src/main/paradox/project/migration-guides.md and https://github.com/mdedetrich/akka-apache/blob/main/akka-docs/src/main/paradox/project/migration-guide-2.5.x-2.6.x.md which is the latest Apache 2.0 release of Akka).

Regarding the behavioural changes, as you can see from https://github.com/mdedetrich/akka-apache/blob/main/akka-docs/src/main/paradox/project/migration-guide-2.5.x-2.6.x.md?plain=1#L17-L18, doing behavioural changes across minor updates do happen as long as there is a justification for it and I believe this PR does hit that threshold. These changes are clearly documented in migration-guide-1.0.x-1.1.x.md.

There is one other final note, the current implementation of this PR alias's the behaviour of Supervision.restart to Supervision.stop in the context of SubFlow because at least on the surface to me, Supervision.restart doesn't make any real sense for errors thrown in the resulting SubFlow's. If I am wrong here (either in that it should instead be aliased to Supervision.resume or that retrying in the context of SubFlow does make sense in which case I should implement it) then let me know.

@mdedetrich mdedetrich force-pushed the use-supervisiondecider-with-split branch 3 times, most recently from 1487124 to f03c736 Compare December 8, 2023 03:59
@mdedetrich
Copy link
Contributor Author

Pinging @He-Pin , @pjfanning , @jrudolph and @raboof since I forgot to do so earlier.

@mdedetrich mdedetrich force-pushed the use-supervisiondecider-with-split branch from f03c736 to 32c5d2c Compare December 17, 2023 22:00
@mdedetrich
Copy link
Contributor Author

@pjfanning All apostrophe's have been removed

Copy link
Contributor

@pjfanning pjfanning left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm - seems sensible to not need the SubstreamCancelStrategy. I would like to see some other approvals though.

Copy link
Contributor

@nvollmar nvollmar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@He-Pin
Copy link
Member

He-Pin commented Dec 18, 2023

@mdedetrich I have left some comments

@mdedetrich
Copy link
Contributor Author

@mdedetrich I have left some comments

I can't see any, I think you forgot to submit?

Copy link
Member

@He-Pin He-Pin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I updated my comments

@mdedetrich
Copy link
Contributor Author

mdedetrich commented Dec 20, 2023

@He-Pin Responded to comments. I think that when you looked at the PR you didn't expand out the diff's and missed the fact that a lot of the code you commented on was within @InternalApi making the comments irrelevant.

Copy link
Member

@He-Pin He-Pin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this one is ok, I missed it's was internalApi.

@mdedetrich mdedetrich removed the help wanted Extra attention is needed label Dec 20, 2023
@mdedetrich
Copy link
Contributor Author

I think this one is ok, I missed it's was internalApi.

Thanks

I will leave the PR open till the end of the month (given that its christmas) to see if anyone else wants to comment on it, at that point I will merge it.

@mdedetrich mdedetrich force-pushed the use-supervisiondecider-with-split branch from 32c5d2c to 2b2958b Compare December 20, 2023 22:51
@mdedetrich mdedetrich force-pushed the use-supervisiondecider-with-split branch from 2b2958b to 5df953f Compare December 20, 2023 23:04
@mdedetrich
Copy link
Contributor Author

The commit I just pushed was adding @deprecation in some places that I missed beforehand, otherwise PR is the same.

@mdedetrich
Copy link
Contributor Author

mdedetrich commented Dec 28, 2023

Im going to go ahead and merge this, largely so that migration-guide-1.0.x-1.1.x.md file which this PR introduces can be used for any future possible behaviourally incompatible changes but also so I can test the changes sooner rather than later to iron out any potential problems (of particular note is pekko-http which actually uses sub flow).

@mdedetrich mdedetrich merged commit b818925 into apache:main Dec 28, 2023
18 checks passed
@mdedetrich mdedetrich deleted the use-supervisiondecider-with-split branch December 28, 2023 04:46
@pjfanning pjfanning added the release notes Need to release note label Dec 28, 2023
@mdedetrich
Copy link
Contributor Author

Just wanted to mention that since the changes in this PR were published as a snapshot, pekko-http tests ran against these changes and passed without any problems (see https://github.com/apache/incubator-pekko-http/actions/runs/7353051075/job/20018534864)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants