Skip to content

Commit

Permalink
Merge pull request #166 from Autodesk/polishing
Browse files Browse the repository at this point in the history
Polishing
  • Loading branch information
avirshup authored Jun 28, 2017
2 parents 939ed29 + dbedb8e commit 27046a4
Show file tree
Hide file tree
Showing 101 changed files with 6,919 additions and 1,108 deletions.
4 changes: 2 additions & 2 deletions DockerMakefiles/DockerMake.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,11 +114,11 @@ deploy_requirements:
python_deploy_base_py2:
requires:
- miniconda
- miniconda_py2
- deploy_requirements

python_deploy_base:
requires:
- miniconda_py2
- miniconda
- deploy_requirements

84 changes: 67 additions & 17 deletions DockerMakefiles/NWChem.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ nwchem_build:
build: |
RUN apt-get install -y patch curl perl
WORKDIR /opt
ENV NWCHEM_MODULES="nwdft driver stepper" \
ENV NWCHEM_MODULES="nwdft driver stepper mm qmmm nwmd esp " \
PYTHONVERSION=2.7 \
PYTHONHOME="/usr" \
USE_PYTHONCONFIG=Y \
Expand All @@ -45,25 +45,32 @@ nwchem_build:
&& cd nwchem \
&& svn co --depth empty https://svn.pnl.gov/svn/nwchem/trunk/src \
&& cd src \
&& svn update GNUmakefile nwchem.F config \
tools include basis geom inp input \
&& svn update GNUmakefile nwchem.F config mm qmmm nwmd \
tools include basis geom inp input qhop fft cafe \
pstat rtdb task symmetry util peigs perfm bq cons blas lapack \
NWints atomscf cphf ddscf driver gradients hessian nwdft optim property stepper symmetry vib \
NWints atomscf cphf ddscf driver gradients hessian nwdft \
optim property stepper symmetry vib space esp \
&& rm -rf /opt/nwchem/.svn \
&& cd tools \
&& svn export --non-interactive --username nwchem --password nwchem \
https://svn.pnl.gov/svn/hpctools/branches/ga-5-5 \
&& cd .. \
#strip native compiler options as Fedora does
&& sed -i 's|-march=native||' config/makefile.h \
&& sed -i 's|-mtune=native|-mtune=generic|' config/makefile.h \
&& sed -i 's|-mfpmath=sse||' config/makefile.h \
&& sed -i 's|-msse3||' config/makefile.h \
&& make nwchem_config && make -j3 \
#clean unnecessary source to reduce docker size
&& rm -rf tce tools nwdft NWints geom symmetry util nwxc ddscf lapack blas rism \
argos peigs rmdft gradients symmetry property smd lucia dplot propery \
hessian ccsd mp2_grad moints cafe analyz dimqm /opt/nwchem/lib
&& sed -i 's|-msse3||' config/makefile.h
RUN cd /opt/nwchem/src \
&& make nwchem_config \
&& make -j 8
RUN mkdir nwchem/contrib \
&& cd nwchem/contrib \
&& svn co https://svn.pnl.gov/svn/nwchem/trunk/contrib/mov2asc \
&& cd mov2asc \
&& FC=gfortran make \
&& mv mov2asc asc2mov /opt/nwchem/bin/LINUX64
RUN git clone https://github.com/NWChem-Python/nwapi \
&& cd nwapi \
&& make -f makefile.linux \
Expand All @@ -76,19 +83,62 @@ nwchem:
description: Deployable NWChem image
build_directory: buildfiles/nwchem/
requires:
- python_deploy_base
- python_deploy_base_py2
- nwchem_requirements
build: |
COPY run.sh run.py getresults.py /usr/bin/
RUN chmod +x /usr/bin/getresults.py /usr/bin/run.py /usr/bin/run.sh
RUN pip install pint
RUN pip install pint cclib
ENV NWCHEMFILE="From https://svn.pnl.gov/svn/nwchem/trunk/src"
ENV PYTHONPATH=$PYTHONPATH:/opt
ENV PYTHONPATH=$PYTHONPATH:/opt/nwapi
COPY run.py runqmmm.py getresults.py /usr/local/bin/
RUN chmod +x /usr/local/bin/getresults.py \
/usr/local/bin/run.py \
/usr/local/bin/runqmmm.py
copy_from:
nwchem_build:
/usr/local/lib/python2.7/dist-packages: /usr/local/lib/python2.7/
/opt/nwchem: /opt
/opt/lib: /opt
copy_from:
/opt/nwapi: /opt
openblas:
/opt/lib: /opt

#################################
# Experimental QM/MM images below
nwchem_amber_qmmmm:
description: Image with Amber and NWChem for QM/MM
requires:
- nwchem
build: |
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
gcc \
gfortran \
&& cleanapt
copy_from:
amber_dev_branch_build:
/opt/amber16: /opt
/root/.bash_profile: /root


amber_dev_branch_build:
build_directory: buildfiles/nwchem
requires:
- buildbase
- ambertools_requirements
build: |
ADD amber-dev.tgz /opt
RUN apt-get install -y \
flex \
bison \
csh \
gfortran \
g++ \
make \
patch \
python-dev
WORKDIR /opt/amber16
RUN echo N | ./configure -noX11 --with-python /usr/bin/python --with-netcdf /usr/ gnu \
&& make -j6 install \
&& rm -rf test AmberTools doc # reduce image size for copying to deployment image
RUN echo "test -f /opt/amber16/amber.sh && source /opt/amber16/amber.sh" >> /root/.bash_profile
7 changes: 5 additions & 2 deletions DockerMakefiles/PythonTools.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,8 +27,11 @@ chem_python_conda:
description: All of the external python chemistry libraries in one place
build: |
RUN conda install -qy -c openbabel openbabel=2.4.1
RUN conda install -qy -c omnia biopython openmm parmed pdbfixer
RUN conda install -qy -c omnia \
biopython=1.68 \
openmm=7.1.1 \
parmed=2.7.3 \
pdbfixer=1.4
chem_python_requirements:
requires:
Expand Down
48 changes: 39 additions & 9 deletions DockerMakefiles/README.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,51 @@
# Docker images for MDT

The Molecular Design Toolkit uses a lot of functionality from other open source chemistry packages, such as molecular dynamics from OpenMM, Quantum chemistry from OpenMM, etc., etc.
The Molecular Design Toolkit does NOT implement molecular modeling algorithms - it instead provides an intuitive, general interface for running calculations with other open source chemistry packages, such as molecular dynamics from OpenMM, Quantum chemistry from PySCF, etc., etc.

To make these calculations 1) easy, 2) portable, and 3) reproducible, users don't need to compile any of these by themselves. Instead, they're provided as docker images, which (as of this writing) are freely available from a Dockerhub repository (see https://hub.docker.com/r/autodesk/moldesign/). Users probably won't need to pull the images manually - they'll be automatically pulled whenever they're needed.
To make these calculations 1) easy, 2) portable, and 3) reproducible, users don't need to compile any other software. Instead, external open source packages are provided as docker images, which MDT automatically downloads from a public [DockerHub repository](https://hub.docker.com/r/autodesk/moldesign/).

These images are built from definitions contained in the DockerMakefiles in this directory (everythign with a `.yml` extension).

# About the images
MDT relies on two types of dependencies:

`docker-make` is used to build and push the images to the public repositories. The Dockerfiles for all images are defined in the `DockerMake.yml` and its associates YAML sources in this directory.
1. CLI executables (e.g., NWChem, AmberTools utilities, etc.). CLI executables come in their own lightweight images. These images let you run the executables without compiling or installing them yourself.
2. Python libraries (OpenMM, PySCF, etc.). To interact with these, we build a rather large docker image named `moldesign_complete` that includes MDT _and_ all interfaced Python libraries. When MDT needs to use a Python library that isn't installed on your machine, it will run the necessary code in a `moldesign_complete` docker container instead.

If you're just *using* MDT, you'll be relying on the prebuilt MDT images hosted on [DockerHub](https://hub.docker.com/r/autodesk/moldesign/). However, if you are developing MDT, you may need to install docker-make by running `pip install dockermake>=0.5`.

# Development and building images

MDT relies on two types of dependencies:
**NOTE**: If you're just *using* MDT, you don't need to do this - MDT will automatically download the images hosted on [DockerHub](https://hub.docker.com/r/autodesk/moldesign/).

##### Install DockerMake:
We use the `docker-make` command line utility to build and manage MDT's docker images. To install it:
```bash
pip install "DockerMake>=0.5.6"
```

##### Activate MDT "devmode":
To tell MDT to use your own, locally built docker images, run:
```bash
echo "devmode:true" >> ~/.moldesign/moldesign.yml
```

Use the following commands to build and manage the docker images. They should be run in this directory (`[...]/molecular-design-toolkit/DockerMakefiles`)

##### List docker images:
```bash
docker-make --list
```

1. CLI executables (e.g., ambertools utilities like antechamber, tleap). CLI executables come in their own lightweight images.
2. Python libraries (openmm, pyscf, etc.). To interact with these, we build a rather large docker image named "moldesign_complete" that includes both moldesign and all of its dependencies. This allows MDT to call functions from libraries that aren't installed on your machine! Instead, those function calls will be evaluated in the moldesign_complete docker image, and the results will be returned.
##### Build all docker images:
```bash
docker-make --tag dev --all
```

In the future, we plan to isolate the python dependencies into smaller, lightweight installations that don't include MDT. This will require a common data format with translation layers for each dependency.
##### Build the MDT docker image for python 3:
```bash
docker-make --tag dev moldesign_complete
```

##### Build the MDT docker image for python 2:
```bash
docker-make --tag dev moldesign_complete_py2
```
Loading

0 comments on commit 27046a4

Please sign in to comment.