Bug/Feature: NumPrevPRsTransformer in TTM model

[oindrillac]

The custom transformer NumPrevPRsTransformer is a column feature transformer that is currently designed to go through the number of occurrences of the particular PR's creator in the input dataframe (minus that particular PR) and assign it that value.

I'm wondering if this will appropriately apply for a model service where we are expecting results per PR? In cases where we are not giving rows of PRs or a dump of PRs. Will it always be 0 in those cases where the model is predicting per row?

Proposed solutions

• For a repo or org specific model, create a dictionary of creator/PRs mapping to compare the PR with and update that dictionary iteratively.
• Exclude that as a feature, or only enable for a model which is sending batch requests

@chauhankaranraj what are your thoughts on this?

[chauhankaranraj]

@chauhankaranraj what are your thoughts on this?

My initial thought is to exclude it as a feature and measure the performance difference. Based on this we can evaluate whether it'd make sense to set up a different process for calculating this feature. wdyt?

[oindrillac]

Model with the feature:

Model without the feature:

Excluding it as a feature doesn't seem to create that big a difference. So we can exclude it for now. For the future, to improve model performance, we know that we can explore this avenue.