Setting up a local instance of Whole Tale

This directory contains scripts required to run a full instance of the Whole Tale platform on a single system (e.g., laptop or VM). The installation uses a predefined domain *.local.wholetale.org which maps to localhost. This means that VM-based installations will require port forwarding to access (see below).

System requirements

• Linux (tested on Ubuntu 18.04 - 22.04)
• docker 17.04.0+, swarm mode (23.0.5 recommended)
• python, requests
• jq
• make
• Globus OAuth credentials (pinned at #core-dev)
• SSL certs for .local.wholetale.org (pinned at #core-dev)
• Default user with uid:1000 and gid:100
• On Ubuntu, apt-get install davfs2 fuse libfuse-dev

Installing (Ubuntu)

This assumes a standard JS2 Ubuntu 22.04 cloud instance.

Install required packages:

sudo apt-get update -y 
sudo apt-get -y install python-is-python3 python3-pip jq make davfs2 fuse libfuse-dev

Install Docker https://docs.docker.com/engine/install/ubuntu/).

Add ubuntu user to docker group:

sudo usermod -G docker ubuntu

Initialize your system for Swarm:

docker swarm init

Deployment process

The deployment process does the following:

• Pulls all required images
• Starts the WT stack via Docker swarm
• Starts the Celery worker (gwvolman)
  • WARNING: This container has fairly elavated privileges:
    • it has access to host's filesystem
    • it can perform multiple system calls as de facto host's root
  • WARNING: During celery_worker's initialization host's /usr/local directory is overshadowed by the content of /usr/local from the container.
• Runs setup_girder.py to initialize the instance

Add default user (1000) and group (100), if not present:

[[ -z $(getent group 100) ]] && sudo groupadd -g 100 wtgroup
[[ -z $(getent passwd 1000) ]] && sudo useradd -g 100 -u 1000 wtuser

Export Globus Oauth ID and secret (available on #core-dev):

export GLOBUS_CLIENT_ID=<client ID>
export GLOBUS_CLIENT_SECRET=<client secret>

Export ORCID ID and secret (available on #core-dev):

export ORCID_CLIENT_ID=<client ID>
export ORCID_CLIENT_SECRET=<client secret>

Clone this repository:

git clone https://github.com/whole-tale/deploy-dev
cd deploy-dev/

Create the traefik acme directory. Download SSL certs from #core-dev and copy to acme.json in this directory. Change permissions and ownership:

mkdir traefik/acme
sudo chown root:root traefik/acme/acme.json
sudo chmod 0600 traefik/acme/acme.json

Run:

make dev

or

DATACAT=1 make dev

for WholeTale flavor operating on local data.

To confirm things are working, all REPLICAS should show 1/1

$ docker service ls
ID                  NAME                MODE                REPLICAS            IMAGE                        PORTS
jpzhf12jh6wj        wt_dashboard        replicated          1/1                 wholetale/dashboard:latest
26gv8qfb85sq        wt_girder           replicated          1/1                 wholetale/girder:latest
irdizcla8jal        wt_mongo            replicated          1/1                 mongo:3.2
p46dbxbgcae3        wt_redis            replicated          1/1                 redis:latest
vyf55zx0x95y        wt_registry         replicated          1/1                 registry:2.6                 *:443->443/tcp
41u4zyqrrv79        wt_traefik          replicated          1/1                 traefik:alpine               *:80->80/tcp, *:8080->8080/tcp

The celery_worker runs outside of swarm, confirm that it's running:

$ docker ps | grep celery_worker
0e8124024f03        wholetale/gwvolman:latest                                "python3 -m girder_w…"   15 hours ago        Up 15 hours                             celery_worker

Note: If you're running on a VM, you'll need to setup forwarding for port 443 (requires sudo):

$ sudo ssh -L 443:localhost:443 user@IP

You should now be able to open a browser to https://dashboard.local.wholetale.org to access your running instance of Whole Tale.

Modifications

Anytime you modify docker-stack.yml, you will need to run make dev or use docker stack commands directly to see those changes reflected in Docker.

Development Modifications

If you plan to modify and rebuild the dashboard code, you can follow these steps:

1. First, run a make rebuild_dashboard
2. Under dashboard in docker-stack.yml, uncomment the relevant volumes section u
3. Run make dev to deploy docker-stack.yml to the swarm

NOTE: if you fail to run make rebuild_dashboard first, Docker may create your dist/ directory with incorrect permissions (which can lead to headaches later when writing build artifacts or serving built artifacts via NGINX)

Uninstall

The following will remove all services and delete the volume data:

make clean