Skip to content

VEuPathDB/vdi-service

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

VEuPathDB Dataset Installer Service (VDI)

Tagging, Deployment, and Branch Rules

Branch Rules

qa

The qa branch is to reflect what is currently on QA. Code may only be pushed to this branch via pull request which depends on both approval and a successful workflow build of the code.

production

The production branch should always reflect what is on production. This branch is to remain locked until we are ready to move code from QA to production, at which point the following steps will be performed:

  1. The production branch will be unlocked via the repository settings.

  2. The qa branch will be merged into production

  3. The production branch will be re-locked via the repository settings.

These steps are not the service deployment steps, they are simply an outline of steps strictly related to getting the production branch up to date with the qa branch. For deployment procedure steps see the Deployment section below.

main

The main branch is for active development. It is strongly advised that fixes and features be developed on branches forked from main and then merged back into main when complete to avoid conflict problems with other active development.

Tagging

Release tags v* are only to be made on the production branch.

Other tags should be avoided.

Deployment

The following is a full list of steps necessary to perform a deployment of the QA VDI service code onto production.

  1. Unlock the production branch via the repository settings. This setting can be found in the "Branches" menu under the "production" branch rule. The specific setting to uncheck is "Lock Branch".

  2. Merge the qa branch into production.

  3. Re-lock the production branch via the setting described in step 1.

  4. Wait for the production branch to build in Jenkins.

  5. Create a new release tag on the merge commit from qa to production.

  6. Wait for the new tag to build in Jenkins.

  7. Create a PR on the Tagger repository, forked from and targeting the prod branch, updating the vdi-service tag to the new tag created in step 5.

  8. Assign the PR to systems and notify them of the PR.

⚠️

Steps 4 and 6 are necessary to avoid known issues with our current Jenkins build setup for containerized services.

Repo Structure

The VDI service repository is divided into 4 categories: components, daemons, lanes, and the core bootstrapper.

Components are small shared libraries that contain code specific to a purpose that is not the core focus of the VDI service.

Daemons are background processes that perform maintenance tasks such as dataset reconciliation or pruning old, soft-deleted datasets from S3.

Lanes are event handlers that are tailored to individual event types, performing some process or processes on a dataset specified in the subscribed events.

The bootstrapper is the core of the service, and the root level Gradle project. This piece is responsible for starting up and shutting down the various project modules.

Daemons

Background tasks that run unsupervised.

Lanes

Dataset event handlers. Each lane is a separate process that subscribes to a Kafka channel and operates on datasets whose information is provided in the incoming events.

Rest Service

The rest service is the public API through which users and administrators communicate with and operate on the VDI system.

Bootstrapper

The bootstrapper is the core of the service, and the root level Gradle project. This piece is responsible for starting up and shutting down the various project modules.

Development

Running the Stack

  1. make build
    Builds the VDI service docker image.

  2. make up
    Spins up the service.

  3. make down
    Shuts down the service and removes all the vdi-specific containers, volumes, and networks.