Frequency variance

[joeflack4]

Overview

For each of the count categories (1-4) above, when is the occurrence of these topics, when are they more frequent / less frequent.

Additional thoughts

I think that the first dataset to generate would have the following fields / types:

• 1. year: string: YYYY format
• 2. month: string: MM format
• 3. day: string: DD format
• 4. keyword: string
• 5. n_messages: integer

Dates dataframe
I think the first thing to do would be to create a dataframe with just columns 1-3. The existence of leap year makes this kind of annoying. For our initial implementation, we could just pretend every year has a leap day, or leave it out entirely. We could generate this initial dataframe a number of different ways.

Messages
Once we have that, then we can iterate over our cached messages.

We would look at the timestamp column, and then we could extract the year, month, and day. We could then update the given row in the dates dataframe, and increment the n_messages

Or, we could simplify this dataframe to be a similar format to dates dataframe, and then do something like a JOIN.