Skip to content

Milestone 4

Jump to bottom
Chirag Galani edited this page Apr 29, 2019 · 8 revisions

Revised architecture diagram and user stories

Please refer Home

Task accomplished in this milestone

  • OpenID Connect for Authentication
  • Setup HAProxy on IU & TACC
  • Setup Nginx Reverse Proxy for intra-network cluster communication
  • Setup Blue-Green deployment on Kubernetes
  • Setup CI/CD to build & deploy Blue, Green Branches for each microservice
  • Revamp the website for additional information along with Landing Page

Detailed Information

OpenID Connect for Authentication

We used OpenID connect protocol for Authentication. We used the Auth0 service for authentication. Our website technically doesn't have anything logged in user specific, but to fulfill the assignment four requirements we have incorporated the feature.

Test URL: http://149.165.168.185/dashboards/login

Setup HAProxy on IU & TACC

We've set up the High Availablity Proxy application on an instance of IU and TACC. This is mapped to a DNS service using GoDaddy on http://www.schwenck.live

IU URL: http://149.165.170.43

IU stats: http://149.165.170.43/haproxy?stats

TACC URL: http://129.114.16.248

TACC stats: http://129.114.16.248/haproxy?stats

The configuration file for this service has been uploaded in the configuration_files branch

Setup Nginx Reverse Proxy for intra-network cluster communication

We've set up the Nginx Reverse Proxy to ensure that the intra-network cluster communication among pods happens within the localhost and not via a hardcoded IP addresses. This is done primarily for 2 reasons:

  • To prevent exposing the ports and IP addresses of the services
  • To ensure communication between microservices happens within the same network resulting in lower latency

Setup Blue-Green deployment on Kubernetes

We've also set up the blue and green branches for each microservice to ensure that one of them is ahead of the other for every release version. So, at any point of time, 2 versions of each microservice deployed on the Kubernetes master ensure that if the newer update has some issues, traffic can be routed to the other deployment Eg. The backend microservice has 2 versions: v1.0 (backend-green deployment) and v1.1 (backend-blue deployment) active on Kubernetes master. If there is some issue with the latest version [v1.1 (backend-blue deployment)], we can set the service to route all the traffic to v1.0 (backend-green deployment)

Setup CI/CD to build & deploy Blue, Green Branches for each microservice

We've also setup Jenkins Pipelines for each variant (blue and green) for every microservice which builds and deploys the new code on the server. Every time, there is a commit on either deployment is pushed on the Kubernetes master, traffic is routed to the latest version of the microservice.

Revamp the website for additional information along with Landing Page

To improve the user experience, we added a landing page on the front end. We also added a few enhancements to make UI better.

Improvements from Project 3

  • We worked on the front end to make it more user-friendly. We also improved diet and fitness microservices to increase security.
  • The project need not be manually installed on one machine to be able to work completely and can be accessed from anywhere.
  • We have embraced the distributed systems architecture to enhance the project's scalability and availability.

Steps to run the project

You can test this system either using the Dashboard or by applying a service file:

Dashboard

Creating a Service File

  • In order to test Blue-Green deployment for Backend Service, perform the following steps:
    • Visit Backend Test URL
    • If it says blue deployment, then run the following command: kubectl apply -f serviceBackendServerGreen.yaml
    • If it says green deployment, then run the following command: kubectl apply -f serviceBackendServerBlue.yaml
    • Now revisit the UI URL

Development Branches are as follows:

Configuration_Files

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/configuration_files

Purpose: This branch contains all the configuration files for HAProxy and Nginx Reverse Proxy.

DevOps

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/dev_ops

Purpose: This branch contains all the different YAML files which were used for the deployment of different microservices including the front-end. It also contains configuration maps and deployment files for Kube Monkey which performs load balancing testing on the system

Backend Server

Test URL: http://149.165.168.185:30032/

Purpose: This server is built using the NodeJS Express Framework. It involves fetching data stored on the mongo databases received from the cron jobs. It also facilitates image management from the Media management microservice. This is the only point of contact for the front end of the application to fetch and send data.

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/server-dev-green

Jenkins Pipeline Name: Backend_green_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Backend_green_Pipeline/

Blue Deployement

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/server-dev-blue

Jenkins Pipeline Name: BackEnd_Blue_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Backend_Blue_Pipeline/

Backend Data Microservice

Test URL: http://149.165.168.185:30072/

Purpose: This microservice transmits the data from the database and sends it to the backend server. It is created to prevent the bottleneck creation on the backend server and avoid a single point of failure on the backend server.

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/server-dev-green

Jenkins Pipeline Name: Backend_green_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Backend_green_Pipeline/

Blue Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/server-dev-blue

Jenkins Pipeline Name: BackEnd_Blue_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Backend_Blue_Pipeline/

Backend Dropbox

Test URL: http://149.165.168.185:30092/

Purpose: This automatically (via a cron job) pulls in the respective data sets from various devices to be added to the DB:

  • blood pressure data
  • cardio mood data

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/server-dev-green

Jenkins Pipeline Name: Backend_green_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Backend_green_Pipeline/

Blue Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/server-dev-blue

Jenkins Pipeline Name: BackEnd_Blue_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Backend_Blue_Pipeline/

Fitness Service

Test URL: http://149.165.168.185:30062/get

Purpose: This microservice is developed in Python using the Flask framework. It runs as a cron job which fetches the latest diet data from Strava API. The data includes the outdoor activities performed recently by the user which can be Biking, Running, etc.

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/fitness-service-green

Jenkins Pipeline Name: Fitness_Green_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Fitness_Green_Pipeline/

Blue Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/fitness-service-blue

Jenkins Pipeline Name: Fitness_Blue_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Fitness_Blue_Pipeline/

Diet Service

Test URL: http://149.165.168.185:30082/add

Purpose: This microservice is developed in Python using the Flask framework. It runs as a cron job which fetches the latest diet data from the MyFitnessPal database. The data includes the number of calories, nutritional information, and food items.

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/diet-service-green

Jenkins Pipeline Name: Diet-green-pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Diet-green-pipeline/

Blue Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/diet-service-blue

Jenkins Pipeline Name: Diet-blue-pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Diet-blue-pipeline/

Fitbit Data Tracker

Test URL: http://149.165.168.185:30052/getsleep

http://149.165.168.185:30052/getstat

http://149.165.168.185:30052/getheartrate

Purpose: This microservice is used to track sleep data, heart rate data and intraday heart rate from Fitbit app. It is developed in python Flask Framework. It runs as a cron job which fetches the latest data from the Fitbit database.

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/fitbit-service-green

Jenkins Pipeline Name: Fitbit_green_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Fitbit_Green_Pipeline/

Blue Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/fitbit-service-blue

Jenkins Pipeline Name: Fitbit_blue_Pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Fitbit_blue_Pipeline/

Front End

Test URL: http://149.165.168.185:30042/

Purpose: This is the front end of the application which displays all the integrated output of the application

Green Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/milestone_4_green

Jenkins Pipeline Name: Front_end_M4_Green_pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Front_end_M4_Green_pipeline/

Blue Deployment

Branch URL: https://github.com/airavata-courses/Gateway-Falcons/tree/milestone_4_blue

Jenkins Pipeline Name: Front_end_M4_Blue_pipeline

Jenkins Pipeline URL: http://149.165.170.222:8080/job/Front_end_M4_Blue_pipeline/

Challenges faced

OpenID Connect for Authentication

We faced a lot of issues while setting up the OpenID connect as we were using a hash router. We need to change the router to browser router to manipulate the history.

Front-end

Learning and eventually customizing layouts to improve user experience.

Milestone evolution

  • In the 1st milestone, this project started with a tightly coupled application which needed to be installed manually on each system. That had multiple issues, as we faced platform-specific issues.
  • In the 2nd milestone, we deployed all the microservices on every IU Jetstream instance. We also used Jenkins to automate CI/CD process and deploy on their existing JetStream instances.
  • In the 3rd milestone, we dockerized all the microservices into their respective containers and deployed on a Kubernetes Master to make the application plug & play. We also used Kubemonkey (a variant of Netflix's ChaosMonkey) to test fault tolerance and JMeter for capacity testing of the system to check the microservice's capacity limitations.
  • In the 4th milestone, we deployed on TACC jetstream servers as well. We also developed Blue Green deployment in the Kubernetes cluster for all deployments to ensure that there is an application which has an older version of the application actively running in the background. We also added OpenID authentication to authenticate the user with different sign-in methods. We also used HAProxy to load balance between the Kubernetes master hosted on different data centers. Lastly, we also implemented Nginx for intra-network communication between front-end and other microservices
Clone this wiki locally