Add Pretranslation Monitoring spec

[mathjazz]

Fix #2894.

A couple of differences from what we discussed at the meeting:

• There is no breakdown of data per project on the Team dashboard and no breakdown of data per team on the Project dashboard.
  • Instead, we plot the Pretranslation review activity chart also on the Project-Locale dashboard.
  • Also, (separately from this feature) we could add data filters to all Insights panels.
• No "One dashboard" with the overall data on pretranslation (Pontoon-level oversight).
  • Instead, I'd like to approach this more holistically as part of KR 1.1.
  • Also, what do we need this information for? Would it be enough to expose numbers over the API (or gather them via a script)?

TODO: I need to make another pass at the colors, but that's not blocking the review.

[flodolo]

Why the hidden column?

[mathjazz]

Because it doesn't have impact on the quality metrics (approval ratio and chrF++ score).

It's also consistent with New suggestions and New source strings in the Review/Translation activity panels.

[flodolo]

No "One dashboard" with the overall data on pretranslation (Pontoon-level oversight).
Instead, I'd like to approach this more holistically as part of KR 1.1.

Can you expand on this? KR 1.1 is about community metrics collection, which shouldn't involve dashboards? Also, only the review part would probably be part of that.

Also, what do we need this information for?

I was thinking that you can't go through 50+ locales to check if there are issues: you would check the main dashboard, then switch to single locale or project dashboards to look at data in more detail. For example:

• One graph with average approval rate over time per locale (line chart, one line per locale).
• Same graph, but per project.

The question is if these two charts are going to get too crowded.

Also, given I expect this type of question to get asked:

• Histogram chart over time with number of pretranslated strings and approval rate across all locales and projects.

Would it be enough to expose numbers over the API (or gather them via a script)?

That information wouldn't be easily accessible to all PMs, especially the latter.

[mathjazz]

No "One dashboard" with the overall data on pretranslation (Pontoon-level oversight).
Instead, I'd like to approach this more holistically as part of KR 1.1.

Can you expand on this? KR 1.1 is about community metrics collection, which shouldn't involve dashboards? Also, only the review part would probably be part of that.

As discussed at the team call last week, let's drop this idea.

Also, what do we need this information for?

I was thinking that you can't go through 50+ locales to check if there are issues: you would check the main dashboard, then switch to single locale or project dashboards to look at data in more detail. For example:

* One graph with average approval rate over time per locale (line chart, one line per locale).

* Same graph, but per project.

The question is if these two charts are going to get too crowded.

Also, given I expect this type of question to get asked:

* Histogram chart over time with number of pretranslated strings and approval rate across all locales and projects.

I'll update the spec with the new /insights page. It will consist of two graphs mentioned above:

• one with approval rate over time line for each locale,
• one with approval rate over time line for each project.

I'll also add "average" to each chart, which will represent the global approval rate across all locales and projects.

If these charts turn out to be unusable during the implementation phase, we can create tabular views, simiar to /teams and /projects, which will show approval rate charts for each locale and project in a separate line.

[mathjazz]

I haven't added the mockup for the /insights page. It's quite obvious how it would look like, and I'd rather not spend time working on the design, before figuring out if it even works with the real data. So I'd prefer to go straight to implementation, revisit the design decision and update the spec accordingly.

[flodolo]

Thanks, LGTM