[Bug]: Many add-ons have been erroneously added to the review queue for overgrowth

[wagnerand]

What happened?

A lot of add-ons were recently added to the queue for growing over the average growth. This happened in error.

After an initial investigation, we believe this is happening because of an unanticipated negative hotness. That needs to be confirmed.

What did you expect to happen?

Only add-ons that really grew over the average should have been flagged.

Is there an existing issue for this?

• I have searched the existing issues

┆Issue is synchronized with this Jira Task

[diox]

We also need to clear problematic NeedsHumanReview instances and call reset_due_date() on affected versions. That would happen automatically if the NeedsHumanReview are modified, but in this case we likely want those instances gone entirely. A migration can't easily do that because reset_due_date() method wouldn't be available, need to figure out if we could have a management command or admin tool that automatically resets due date for a given set of versions...

[diox]

After an initial investigation, we believe this is happening because of an unanticipated negative hotness. That needs to be confirmed.

This is definitely the case.

If the average hotness in a tier is -0.1, and the percentage set to 50 for that tier, with the current algorithm the threshold will be -0.0995. We likely want -0.05 in that case instead, since we want to consider increases in growth over the average.

[diox]

There is another issue with that algorithm: if the average hotness in a tier is 0.00, then multiplying by it won't do anything... We could work around that as well, but I think the true solution is going to be look at the Xth percentile instead.

[diox]

Command to fix the NHR/due dates:
manage.py process_addons --task=delete_erroneously_added_overgrowth_needshumanreview --with-deleted

We successfully ran that in prod (and stage before that) today.

[diox]

The data has been fixed. We'll file another issue for the new algorithm.

[wagnerand]

The command above missed an edge case where a new version was submitted, causing the NHR flag to move from one version to another but changing the reason from "overgrowth" to "previous version had NHR set".

We identified and fixed those manually. The NHR flags for them have not been removed but marked inactive using the admin tools (NeedsHumanReview model page)