In this project, I have build an ETL(extract,Transform,Load) pipeline using the spotify API on local machine. The Pipeline will retrive data from the Spotify API, Tranform it to a desired format, and load it into an Mongodb database. For user authentication I have used flask python framework only for one time authorisation.
Spotify Web API : Retrieve metadata from Spotify content, control playback or get recommendations
Create an App : The app provides, among others, the Client ID and Client Secret needed to implement any of the authorization flows.
Authorization : Authorization refers to the process of granting a user or application access permissions to Spotify data and features (e.g your application needs permission from a user to access their playlists).
Spotify implements the OAuth 2.0 authorization framework.
Authorization Code Flow : We use the authorization code flow because it is suitable for long-running applications where the user grants permission only once.
Scopes : Scopes provide Spotify users using third-party apps the confidence that only the information they choose to share will be shared, and nothing more.
If the user is not logged in, they are prompted to do so using their Spotify credentials. When the user is logged in, they are asked to authorize access to the data sets or features defined in the scopes.
Refreshing tokens :Access tokens are intentionally configured to have a limited lifespan (1 hour), at the end of which, new tokens can be obtained by providing the original refresh token acquired during the authorization token request response.
PyMongo : PyMongo is a Python distribution containing tools for working with MongoDB, and is the recommended way to work with MongoDB from Python. This documentation attempts to explain everything you need to know to use PyMongo.
Flask : Flask is a micro web framework written in Python.
- Python / Notebook / Flask
- MongoDB
- Spotify API
To get a local copy up and running follow these simple steps:
- Clone the repo
git clone https://github.com/codeXXripper/Spotify--Data-Pipeline- Run the flask python script first
- Execute the ETL python notebook
- Enjoy!
Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.
- Fork the Project
- Create your Feature Branch (
git checkout -b feature/AmazingFeature) - Commit your Changes (
git commit -m 'Add some AmazingFeature) - Push to the Branch (
git push origin feature/AmazingFeature) - Open a Pull Request
Distributed under the MIT License. See LICENSE for more information.