This is the repository for the backend which powers Milou, running on
an Elastic Beanstalk environment on AWS. It includes a MongoDB database
connection and can be interfaced via HTTPS requests. It is also fetching from different financial APIs such
as Alpha Vantage, Finnhub, Benzinga
and News API
to retrieve information/news about stocks and companies. For a proper manipulation of different currencies real time rate data is collected from Rates API to convert prices to Euro.
Furthermore, finAPI
is used to enable users to connect their bank-accounts, which allows the application to import their real portfolios.
Besides, the Analytics part of Milou is integrated in a sub-folder called data-analytics and one file
called analyticsRoutes.js, which offers a connection to the Analyzer part of Milou.
What's not working yet?
- automatic tests for the routes need to be implemented
- the formula for calculating the portfolio score needs to be revised
Among others, we use express, swagger, passportJS, fetch and axios, aws-sdk, dotenv and mongoose.
Express is our main backend-application framework. We use Swagger 3.0 OpenAPI for the documentation of the API. PassportJS is used for user-authorization. Fetch is used by the backend application to retrieve data from other APIs, the analytics team uses Axios to do so. Aws-sdk and dotenv are used to access keys and secrets from AWS Secret Manager. Mongoose is used to access our MongoDB.
You can install our dependencies by running npm install.
Except for importing the secrets (next step), no major effort is required to run the project locally.
npm i
npm startFor local development you should keep in mind, that the application is connected to the database (if your aws-secrets are correct). Thus, you are able to run CRUD operations on the real database using Postman, for example.
In order to connect to AWS and retrieve the secrets (in local development), the application needs to get the AWS
credentials. It looks for them in the file ~/.aws/credentials on Linux/macOS and in the file
C:\Users\USER_NAME\.aws\credentials. That file must contain aws_access_key_id and aws_secret_access_key.
As already mentioned, our code is deployed on an Elastic Beanstalk instance. To do so
a Connection is set up, deploying
changes on master within seconds using
AWS CodePipeline.
The backend's database is running on MongoDB Atlas. It is used to cache information about stocks, companies and currencies, enabling us to update analysis of portfolios, even tough we are limited in terms of calls per minute to our financial APIs.
It also stores the user profiles and all the portfolios of the users, as well as the analysis values of the portfolios, calculated by the analysis team.
All data is stored encrypted (MongoDB automatically encrypts all data), and passwords are stored hashed.
These two endpoints1 and these five fields2 of stocks are updated every night at 4:00 am via CronJobWorker from Alpha Vantage API, because they are important for frontend and require daily updates. Other fields/endpoints of the stocks do not require daily update and hence information is static.
The information about stock values in the portfolio and the analysis of portfolios are updated every night at 2:00 am, in order to keep the values of the stock prices and of the performance chart up to date.
1 Endpoints: /stocks/charts/historic , /stocks/charts/analysts
2 Fields: price, per1d, per7d, per30d, date
The Authentication of users can be viewed at auth/auth.js. It uses a generator as well as a secret to generate
a JSON web token (JWT), which has to be passed as a bearer token in the header of requests, that should
only work for signed-in users. This JWT token is valid for 24 hours, afterwards it has to be renewed.
| Endpoint | Parameters | Description | Example response |
|---|---|---|---|
| /stocks/list | country, currency, industry, mc | Returns a stock list according to filter. | Click |
| /stocks/search | id (can be name, ISIN1, WKN1, symbol) | Returns stock list with search parameter. | Click |
| /stocks/overview | id (symbol of a stock) | Returns a stock overview with given id. | Click |
| /stocks/details | id (symbol of a stock) | Returns details of a stock with given id. | Click |
| /stocks/charts/historic | id (symbol of a stock), max (all data points if true, else only last 5 years) | Get the performance of the stock from beginning or last 5 years. | Click |
| /stocks/charts/key_figures | id (symbol of a stock), max (all data points if true, else only last 5 years) | Get the key figures of the stock from beginning or last 5 years. | Click |
| /stocks/charts/dividend | id (symbol of a stock), max (all data points if true, else only last 5 years) | Get the key figures of the stock from beginning or last 5 years. | Click |
| /stocks/charts/detailed-analysts | id (symbol of a stock) | Get the analysts recommendation from Benzinga 2. | Click |
| /stocks/charts/analysts | id (symbol of a stock) | Get the analysts recommendation from finAPI. | Click |
| /stocks/news | id (symbol of a stock) | Returns news list with given stock symbol from News API. | Click |
| /stocks/balanceSheet | id (symbol of a stock) | Get balance sheet. | Click |
| /stocks/incomeStatement | id (symbol of a stock) | Get income statement. | Click |
| /stocks/cashFlow | id (symbol of a stock) | Get cash flow. | Click |
| /stocks/finanzen | id (symbol e.g. dax, sp500) | Get finanzen data. | Click |
1 Since ISIN and WKN are licensed and are not provided by any of free APIs, we mocked ISIN and WKN fields of stocks with random data in a proper format.
2 Benzinga is on free student trial currently and valid until 19.04.2021. After this date, detailed-analysts endpoint will not be updated.
Endpoints for reading, modifying, creating and deleting portfolios. For accessing these routes, a valid access_token of a specific user must be passed in the header of the request. A user can view/modify only their own portfolios. Only virtual portfolios can be modified. These routes are used by the frontend portfolio application.
| Endpoint | Parameters | Description |
|---|---|---|
| /portfolio/list | - | Returns all portfolios of the authorized user |
| /portfolio/details/:portfolioId | ID of a portfolio | Returns a global overview of a portfolio, positions, risk, analytics etc. |
| /portfolio/performance/:portfolioId | ID of a portfolio | Returns data points to display a performance chart. |
| /portfolio/create | Name of a portfolio to create | Creates a new, empty, virtual portfolio and saves the timestamp of the portfolio creation in the portfolio's history. |
| /portfolio/:portfolioId | ID of a portfolio to delete | A request to delete a portfolio with a given ID, if exists. |
| /portfolio/rename/:portfolioId | ID of a portfolio to rename, a new name | Renames a specific portfolio with a given name. |
| /portfolio/modify/:portfolioId | ID of a portfolio to modify, symbols of positions and a new quantity for them. | Modifies a virtual portfolio, if exists, by adjusting a quantity of specified positions and updates timestamp of the last change. |
| /portfolio/duplicate/:portfolioId | ID of a portfolio to duplicate, name for a new one. | Creates a new virtual portfolio as a duplicate of a real or virtual portfolio. Changes to a real portfolio will not be tracked in the duplicated version. |
| /portfolio/stock/:symbol | Symbol/name/isin/wkn (symbol preferred) of the specified stock | Get request. After adding a stock to his portfolios returns portfolio name and quantity of a specified stock for all portfolios of the authorized user. |
| /portfolio/stock/:symbol | Symbol/name/isin/wkn (symbol preferred) of the specified stock | Put request. Modifies a stock's quantity within multiple portfolios simultaneously. |
The User Routes enable the user to interact with his identity model, which consists of an email and a password. First- and lastname are optional and can be set later on via edit. In this part of the documentation it can be reviewed. Users can register, login and confirm their emails via a token that gets send, once registered. Also the token can be resent. If a password was forgotten, a reset process can be started, wich sends an email with a link in it, including a token used for verification of the user.
Besides that main user functionality also a user is able to search for banks and add, view, remove and refresh bank-connections.
Of course a user can delete the account, including all user-related data on the database, such as portfolios, user-information, bank-information as well as the account on finAPI.
The application can be found at:
Our documentation can be viewed at: