Create db to track stats

[miketaylr]

Stuff we want, at least right now:

• http://bornawesome.com/adam/moz/weekly/ (Weekly number of new bugs, broken down by milestone)
• burn down stats for needsdiagnosis

cc @adamopenweb

[adamopenweb]

To expand a bit the reports should answer these questions:

1. How many reports do we get each week? And what's the breakdown of valid to not valid (invalid / duplicate / worksforme / non-compat / wontfix)?
   Notes: Similar to the old report. A view of all the reports created within x weeks, sorted by the week created, grouped by the current milestone. Maybe switch to a stacked line chart.

2. What do the burn down stats look like this week for each milestone? And how has that changed over time?
   Notes: Same format as 1 but is all the open bugs at each week interval, grouped by the milestone they were set to at that given week. This is basically a snapshot of the milestone counts each week, stored into a DB or file.

[karlcow]

The time is in JST.

more ../../milestones/needsdiagnosis.txt 
2018-02-08T22:12:17 822 
2018-02-09T00:00:01 819 -
2018-02-10T00:00:03 806 -
2018-02-11T00:00:02 805 -
2018-02-12T00:00:03 804 -
2018-02-13T00:00:03 800 -
2018-02-14T00:00:02 791 -
2018-02-15T00:00:25 783 -
2018-02-15T22:25:58 776 - (min)
2018-02-16T00:00:01 777 +
2018-02-17T00:00:03 780 +
2018-02-18T00:00:03 781 +
2018-02-19T00:00:03 782 +
2018-02-20T00:00:03 768 -
2018-02-21T00:00:02 768
2018-02-22T00:00:04 764 -
2018-02-23T00:00:03 759 -
2018-02-23T05:02:26 755 - (min)
2018-02-24T00:10:19 758 +
2018-02-25T00:00:03 758 
2018-02-25T20:45:12 758
2018-02-26T00:00:01 758
2018-02-27T00:00:03 751 - (min)
2018-02-28T00:00:02 758 +
2018-03-01T00:00:02 772 +
2018-03-02T16:00:03 761 -
2018-03-04T16:00:01 759 -
2018-03-05T16:05:19 766 +
2018-03-06T16:00:04 768 +
2018-03-07T16:00:02 766 -
2018-03-10T00:00:03 753 -
2018-03-11T00:00:02 749 - (min)
2018-03-12T00:00:02 751 +
2018-03-13T00:00:04 759 +
2018-03-14T00:00:02 759
2018-03-15T00:00:03 760 +
2018-03-16T00:00:04 752 -
2018-03-17T00:00:02 751 -
2018-03-18T00:00:03 749 -
2018-03-19T00:00:02 749
2018-03-20T00:00:04 746 - (min)
2018-03-21T00:00:02 740 - (min)
2018-03-22T00:00:03 744 +
2018-03-23T00:00:02 743 -
2018-03-24T00:00:02 747 +
2018-03-25T00:00:03 747
2018-03-26T00:15:02 751 +
2018-03-27T00:00:04 741 -
2018-03-28T00:00:02 737 - (min)
2018-03-29T00:00:03 721 - (min)
2018-03-30T00:00:02 712 - (min)
2018-03-31T00:00:03 700 - (min)
2018-04-01T00:00:03 700
2018-04-02T00:00:03 700
2018-04-03T00:00:03 688 - (min)
2018-04-04T00:00:02 680 - (min)
2018-04-05T00:00:02 672 - (min)
2018-04-06T00:00:04 670 - (min)
2018-04-07T00:00:05 645 - (min)
2018-04-08T00:00:04 644 - (min)
2018-04-09T00:00:02 644
2018-04-10T00:00:03 625 - (min)
2018-04-11T00:00:03 619 - (min)
2018-04-12T00:00:05 622 +
2018-04-13T00:48:02 626 +
2018-04-14T00:00:04 624 -
2018-04-15T00:00:03 623 -
2018-04-16T00:00:03 624 +
2018-04-17T00:00:04 599 - (min)
2018-04-18T00:00:03 592 - (min)
2018-04-19T00:00:03 577 - (min)
2018-04-20T00:00:02 580 +
2018-04-21T00:00:03 574 - (min)
2018-04-22T00:00:02 573 - (min)
2018-04-23T00:00:01 573
2018-04-24T00:00:04 572 - (min)
2018-04-25T00:00:02 565 - (min)
2018-04-26T00:00:03 551 - (min)
2018-04-27T00:00:02 548 - (min)
2018-04-28T00:00:01Z 549 +
2018-04-29T00:00:00Z 549
2018-04-30T00:00:01Z 549
2018-05-01T00:00:01Z 548 - 
2018-05-02T00:00:01Z 541 - (min)
2018-05-03T00:00:03Z 543 +
2018-05-04T00:00:00Z 538 - (min)
2018-05-05T00:00:00Z 527 - (min)
2018-05-06T00:00:00Z 527
2018-05-07T00:07:13Z 526 - (min)
2018-05-08T00:00:00Z 520 - (min)
2018-05-09T00:00:00Z 511 - (min)
2018-05-10T00:00:00Z 505 - (min)
2018-05-11T00:00:00Z 507 +
2018-05-12T00:00:00Z 510 +
2018-05-14T00:00:00Z 508 -
2018-05-15T00:00:01Z 499 - (min)
2018-05-16T00:00:00Z 485 - (min)
2018-05-17T00:00:00Z 471 - (min)
2018-05-18T00:00:06Z 473 +
2018-05-19T00:00:00Z 476 +
2018-05-20T00:00:00Z 476
2018-05-21T00:00:01Z 474 -
2018-05-22T00:01:08Z 467 - (min)
2018-05-23T00:00:00Z 453 - (min)
2018-05-24T00:00:00Z 442 - (min)
2018-05-26T00:00:01Z 435 - (min)
2018-05-27T00:00:00Z 435
2018-05-29T00:00:01Z 432 - (min)
2018-05-30T00:00:00Z 425 - (min)
2018-05-31T00:00:01Z 389 - (min)
2018-06-01T00:00:00Z 392 +
2018-06-02T00:00:00Z 392
2018-06-03T00:00:00Z 387 - (min)
2018-06-04T00:00:01Z 387
2018-06-05T00:00:01Z 383 - (min)
2018-06-06T00:00:01Z 382 - (min)
2018-06-07T00:00:01Z 374 - (min)
2018-06-08T00:00:01Z 370 - (min)
2018-06-10T00:59:59 370
2018-06-11T00:59:59 370
2018-06-12T00:59:59 370
2018-06-13T00:59:59 377 +
2018-06-14T00:59:59 380 +
2018-06-15T00:59:59 381 +
2018-06-16T00:59:59 381 
2018-06-17T00:59:59 381
2018-06-18T00:59:59 381
2018-06-19T00:59:59 378 -
2018-06-20T00:59:59 377 -
2018-06-21T00:59:59 367 - (min)

Produced by https://gist.github.com/karlcow/92347ffc71e5b59ba21cb3e061a76e59

Currently run on my laptop from a launchd command (aka new cron for macos)

[karlcow]

http://bornawesome.com/adam/moz/weekly/ (Weekly number of new bugs, broken down by milestone)

and broken down by source. We can/could track this without Google Analytics.

[karlcow]

A better velocity model for this script would be to add what's in and what's out.
We could track that quite easily with a webhook monitoring the milestones event.

[karlcow]

still adding data to the #2 (comment) so to have a better idea. Added indicators for trend.

• - going down. Good
• + going up. Not good
•   stable
• (min) New minimum since the start. Very good.