This is a proof of my skills regarding data engineering.
To have some kind of goal to this project, I have to determine the objective which I should achieve or at least get closer and closer to.
Based on what I consider interesting, I have decided to build a pipeline around the Spotify Web API. I would like to generate analysis about how my listening habits have changed over time, what are my "hot" hours and stuff like that. I should not think too much about the analysis phase to focus on the data engineering.
I decided to do a report of this project for a course "Informaatioanalytiikan työkurssi" (Work Course for Information Analytics).
- I would like to be able to update the data daily. Also, I would like to be able to backfill the data from the past.