Skip to content

Installing a local curator and tree browser test server

Jump to bottom Edit New page
Jim Allman edited this page Jul 16, 2015 · 12 revisions

Requirements

Install and patch web2py

NOTE that we're currently pinned to web2py 2.8.2 for deployment, so it's best to stick to this version for your local test environment!

We apply two "patch" files to fix web2py bugs in authentication (OAuth) and URL rewriting:

cp -p $PATH_TO_OPENTREE_REPO/oauth20_account.py $PATH_TO_WEB2P/gluon/contrib/login_methods/
cp -p $PATH_TO_OPENTREE_REPO/rewrite.py $PATH_TO_WEB2P/gluon/

($PATH_TO_WEB2PY is the directory path where web2py has been installed.)

Copy secure files from a live OpenTree server

(This assumes that you have permission to connect to the current server via SSH. If not, you'll need to create your own keys and register your webapps with GitHub.)

mkdir -p ~/.ssh/opentree
cd ~/.ssh/opentree

scp -i ~/.ssh/opentree/opentree.pem \
 [email protected]:/home/opentree/.ssh/OPENTREEAPI_OAUTH_TOKEN \
 .

scp -i ~/.ssh/opentree/opentree.pem \
 [email protected]:repo/opentree/webapp/private/GITHUB_CLIENT_SECRET \
 ./treeview-GITHUB_CLIENT_SECRET-devtree.opentreeoflife.org

scp -i ~/.ssh/opentree/opentree.pem \
 [email protected]:repo/opentree/curator/private/GITHUB_CLIENT_SECRET \
 ./curation-GITHUB_CLIENT_SECRET-devtree.opentreeoflife.org

Modify local /etc/hosts file

To enable login via OAuth+GitHub, add an entry like the following to your local /etc/hosts file:

127.0.0.1   devtree.opentreeoflife.org

To test a local API server, add this entry:

127.0.0.1   devapi.opentreeoflife.org

To avoid confusion, remember to comment these out when you're done testing local servers!

Why do we need to do this? When using GitHub OAuth for authentication, the domain name is part of a redirect URL back to (in our case) the web app's login page. This line in the app config specifies the redirect:

github_redirect_uri = http://devtree.opentreeoflife.org/opentree/user/login

This must (exactly) match the URL stored on GitHub for a registered app. So it assumes that the user's browser, at least, will recognize this URL and do the right thing when redirected by the OAuth server.

Create local webapp config files

This can be done by copying config.example to config in these directories:

  • $PATH_TO_OPENTREE_REPO/webapp/private for the synthetic-tree viewer
  • $PATH_TO_OPENTREE_REPO/curator/private for the study curation app

For the synthetic-tree viewer (as of July 2015), update these settings in webapp/private/config:

[apis]
...
github_client_id = 8cdc1fa7f5a3a416f958
github_redirect_uri = http://devtree.opentreeoflife.org/opentree/user/login
...
[domains]
treemachine = //devapi.opentreeoflife.org/treemachine/v1
taxomachine = //devapi.opentreeoflife.org/taxomachine/v1
oti = //devapi.opentreeoflife.org/oti/v1
opentree_api = //devapi.opentreeoflife.org/phylesystem/v1
# Cached versions of some APIs will speed up repeated calls (see below).
# These use a simple web2py cache implemented in phylesystem-api
CACHED_treemachine = //devapi.opentreeoflife.org/cached/treemachine/v1
CACHED_taxomachine = //devapi.opentreeoflife.org/cached/taxomachine/v1

For the curation app (as of July 2015), update these settings in curator/private/config:

[apis]
...
github_client_id = d731965529a15ef9d529
github_redirect_uri = http://devtree.opentreeoflife.org/curator/user/login
...
[domains]
treemachine = //devapi.opentreeoflife.org/treemachine/v1
taxomachine = //devapi.opentreeoflife.org/taxomachine/v1
oti = //devapi.opentreeoflife.org/oti/v1
opentree_api = //devapi.opentreeoflife.org/phylesystem/v1

Add GitHub client-secret files alongside config files

These are required for OAuth on GitHub. For each webapp, we need to copy the client-secret files (from above) into the private/ folder, alongside the web app's configuration file:

cd $PATH_TO_OPENTREE_REPO
cp ~/.ssh/opentree/treeview-GITHUB_CLIENT_SECRET-devtree.opentreeoflife.org \
   webapp/private/GITHUB_CLIENT_SECRET
cp ~/.ssh/opentree/curation-GITHUB_CLIENT_SECRET-devtree.opentreeoflife.org \
   curator/private/GITHUB_CLIENT_SECRET

Create Python virtual environment

As usual, it's best to run this project with Python's virtualenv. To do this, create an isolated environment called opentree (preferably using python2.7 for best web2py compatibility):

cd path/to/my/local/project
virtualenv --verbose --python=/usr/local/bin/python2.7 --no-site-packages --distribute opentree
# If that fails, try using the system's default python/version:
virtualenv --verbose --no-site-packages --distribute opentree

Now you can activate this virtualenv like so ($PATH_TO_MY_PROJECT is a directory path where your venv settings will be kept):

cd $PATH_TO_MY_PROJECT
. opentree/bin/activate

This should change your system prompt to indicate that you're inside the virtual environment. To leave it, just call deactivate from anywhere in the shell, and your normal prompt should re-appear.

From within this virtual environment, install all requirements for the two main webapps:

cd $PATH_TO_MY_PROJECT
pip install -r opentree/requirements.txt
pip install -r opentree/curator/requirements.txt

Install peyotl (required for curation app)

See the main peyotl README for installation instructions. In short, clone the repo to your system, then from inside your virtualenv:

$ cd $PATH_TO_PEYOTL_REPO
$ pip install -r requirements.txt
$ python setup.py develop

For full(er) documentation, check out our http://opentreeoflife.github.io/peyotl/ documentation site.

Add symlinks for our web2py applications

Create the sym links for the main web app and the study curation tool. This "installs" the apps in web2py while keeping our opentree repo distinct in the local filesystem.

cd $PATH_TO_WEB2PY/applications
ln -s $PATH_TO_OPENTREE_REPO/webapp opentree
ln -s $PATH_TO_OPENTREE_REPO/curator curator

($PATH_TO_WEB2PY is a directory path where web2py has been previously installed.)

Customize web2py's site-wide routing behavior

We'll use our modified SITE.routes.py instead of the default. This routing file works in tandem with the opentree app router and lets us have proper URLs with hyphens instead of underscores.

# return to main web2py directory
cd $PATH_TO_WEB2PY
cp $PATH_TO_OPENTREE_REPO/SITE.routes.py routes.py

Run the local app server

Now it's time to start your local server (and webapps). Since we've modified /etc/hosts to point to devtree.opentreeoflife.org on localhost, we need run web2py on port 80. This in turn requires us to use sudo to start the server:

sudo python $PATH_TO_WEB2PY/web2py.py -p 80 -a '<recycle>'

For convenience, you can invoke this from a tiny shell script run-local-test-server.sh:

sudo python $PATH_TO_WEB2PY/web2py.py -p 80 -a '<recycle>'

Again, you'll want to replace $PATH_TO_WEB2PY to reflect your own system, or define it in your environment.

Add support for HTTPS (secure connections)

If you need to support secure connections to the local test server (needed for many features), we'll need to add a web2py options file to bind port 443 with our supporting SSL key + certs. This, combined with the DNS tweaks above, will give the expected behavior.

This multiport_options.py file lists the ports needed, plus the location of key and cert files in the local filesystem:

import os
interfaces = [(
                 '0.0.0.0',
                 80
              ),
              (
                 '0.0.0.0',
                 443,
                 '~/.ssh/opentree/ssl-keys/opentreeoflife.org.key',
                 '~/.ssh/opentree/ssl-keys/STAR_opentreeoflife_org.crt'
              )]
password = '<recycle>'
pid_filename = 'httpserver.pid'
log_filename = 'httpserver.log'
folder = os.getcwd()

Here's another tiny shell script run-secure-test-server.sh that I use to launch web2py using these options:

sudo python $PATH_TO_WEB2PY/web2py.py --verbose -L multiport_options.py
# NOTE that multiport_options.py *must* be located in web2py/ (alongside web2py.py) to import properly!
Add a custom footer
Add a custom sidebar
Clone this wiki locally