Define and document a procedure for following up on reviews

[ezio-melotti]

I noticed several instances in which someone (typically a reviewer) suggests a change/improvement or points out other areas affected by the same issue, but those changes are deemed out of the scope of the current PR. Even if there is consensus that these changes should take place, often no follow-up takes place.

When this happens, there are usually different actors involved:

• the author of the PR
• the reviewer that suggested/requested the change (sometimes this is the author themselves)
• whoever suggested to handle the change in a separate PR (might be the author or a reviewer)

There are at least a couple of reasons why the follow-up doesn't happen:

• it is not clear whose responsibility it is to follow up
• people might only be interested in the initial issue, and don't have time/desire to fix the requested changes

In order to solve these issues, I suggest we adopt and document the following procedure:

1. When a situation arises where a follow-up is necessary, the author of the PR should either volunteers to follow up, or ask one of the other actors if they would like to do it. This solves the first issue by ensuring that one of actors does follow up.
2. The appointed person should then create a new issue (or even a PR) that links to the original comment where the follow-up was requested, in order to provide context. It is not expected that whoever creates the follow-up issue/PR invests any more time in fixing it (even though it would be appreciated). This solves the second issue by setting expectations.

The goal here is both to create a reminder and to provide context through the reference to the previous PR.

Generally, the follow-up should be create when the suggestion is a defined action item. If for example the changes in the PR are "good enough" but could be better (such as when the code was optimized, but could be optimized further), and it's not clear how to make them better, then it's not always necessary to create a new follow-up issue.

Click for a few real-world examples

Disclaimer: with these examples, it is not my intent to blame anyone as there is currently no defined procedure to handle this situation. These are only to illustrate the very problem that I'm suggesting we address.

During a review to a PR made by Hugo, I noticed and brought up a couple of issues. Petr suggested to handle them in a separate PR (which sounds reasonable), however none of use followed up on this after the PR was merged.

Three weeks ago, Nikita created a PR to document dict.fromkeys params as pos-only and Erlend suggest to do the same for the other dict methods. Nikita replied that he would prefer to keep the PR focused (which is reasonable), and the PR was merged. I'm not aware of any follow-up, and if there is, it doesn't have a link to this PR.

Last month, Irit moved the compiler docs from the devguide to the cpython repo, and updated them. Even though there might be more updates and improvement that could be made to this page, there are no specific problems to be fixed, so in this case a follow up is not necessary.

In this final example, Erlend created two follow-up issues based on the review comments he received on his PR, by simply copy-pasting the comments and linking to the original PR. Even though the issues have been opened for over 7 months now, they document the problems and provide context about the previous discussions.

[willingc]

@ezio-melotti I agree that opening an issue is a good way to keep track of future action items. I would be in favor of a lightweight suggestion in the docs section of the dev guide about etiquette for future actions.

Let's discuss at the next Docs community meeting.