Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

conda/anaconda docs #288

Open
wants to merge 2 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
82 changes: 82 additions & 0 deletions docs/Anaconda_easyblocks.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
.. _anaconda:

Anaconda
=========

`Anaconda <https://www.continuum.io/downloads/>` is a scientific distribution of python distributed by Continuum Analytics.

.. contents::
:depth: 3
:backlinks: none

Similarities to Python
-----------------------

Like python, anaconda comes in two main flavors, Anaconda2 and Anaconda3. The greatest differences lie in requiring system libraries or admin privileges. Most linux OSes come with Python. In order to install python packages with the OS python you must have admin privileges, and python itself requires admin privileges. Anaconda, however, does not. In the case of Easybuild, using the Anaconda easyblock is a possible alternative to Python.

Anaconda Install Process
-------------------------

Anaconda is a bootstrapped install process, that includes its own prepackaged application libraries, and does not require any compiler toolchains. The easyblock itself inherits from the Binary easyblock.

Conda Package Manager
------------------------
Anaconda comes with conda, the anaconda package manager. It is similar in functionality to `pip <https://pypi.python.org/pypi/pip>`. Like anaconda, conda does not require root access. Conda can create environments and install packages into any of those environments.

You can find out more about using conda with Easybuild here (How do i add a url in rst?).

.. _examples:

Anaconda2 EasyConfig Example
+++++++++++++++++++++++++++++

easyblock = 'EB_Anaconda'

name = 'Anaconda2'
version = '4.0.0'

homepage = 'https://www.continuum.io/anaconda-overview'
description = """Built to complement the rich, open source Python community,
the Anaconda platform provides an enterprise-ready data analytics platform
that empowers companies to adopt a modern open data science analytics architecture.
"""

toolchain = {'name': 'dummy', 'version': 'dummy'}

source_urls = ['http://repo.continuum.io/archive/']
sources = ['%(name)s-%(version)s-Linux-x86_64.sh']
checksums = ['31ed3ef07435d7068e1e03be49381b13']

postinstallcmds = ["conda install -f -c conda conda-env=2.5.2"]


moduleclass = 'lang'

Anaconda3 EasyConfig Example
+++++++++++++++++++++++++++++

easyblock = 'EB_Anaconda'

name = 'Anaconda3'
version = '4.0.0'

homepage = 'https://www.continuum.io/anaconda-overview'
description = """Built to complement the rich, open source Python community,
the Anaconda platform provides an enterprise-ready data analytics platform
that empowers companies to adopt a modern open data science analytics architecture.
"""

toolchain = {'name': 'dummy', 'version': 'dummy'}

source_urls = ['http://repo.continuum.io/archive/']
sources = ['%(name)s-%(version)s-Linux-x86_64.sh']
checksums = ['546d1f02597587c685fa890c1d713b51']

postinstallcmds = ["conda install -f -c conda conda-env=2.5.2"]

moduleclass = 'lang'

Resources
=========

`All the versions! <https://repo.continuum.io/archive/>` <- Check out all the versions and their md5s on the anaconda archive page.
86 changes: 86 additions & 0 deletions docs/Conda.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
Using the Conda package manager in Easybuild
===========================================

`Conda <https://github.com/conda/conda>` is Anaconda package manager. Conda is different from most programming language package managers because it aims to be modular. To use conda, you must have an Easybuild Anaconda2 or Anaconda3 module built. Conda is also a lightweight module manager, but since we are using Environment Modules or Lmod for our environments, we will not be using this functionality.

Conda uses two commands to create environments. The first is using `conda create`, and the second is using `conda env create`. The two share most functionality. Conda env uses a configuration file for its channels and dependendies.Conda env also gives you the ability to export an environment and share it on `anaconda cloud <https://anaconda.org/nyuad-cgsb/environments>`. If you are using the default conda-create this becomes a two step process where first we create the environment and then install the packages.

Create Easyblocks with Conda Create
++++++++++++++++++++++++++++++++++++

If you are using `conda create` you can optionally supply a list of requirements, as you would with pip. Please note that the requirements a separate file, but is an actual list of required packages.

Conda Create EasyConfig Example
++++++++++++++++++++++++++++++++

easyblock = 'Conda'

name = "perl-app-cpanminus"
version = "1.7039"
variant = "Linux-x86_64"

homepage = 'https://github.com/miyagawa/cpanminus'
description = """ cpanm - get, unpack build and install modules from CPAN """

toolchain = {'name': 'dummy', 'version': ''}

requirements = "%(name)s=%(version)s"
channels = ['bioconda']

builddependencies = [('Anaconda2', '4.0.0')]

sanity_check_paths = {
'files': ['bin/cpanm', 'bin/perl'],
'dirs': ['lib', 'lib/perl5']
}

moduleclass = 'tools'

The perl-app-cpanminus conda example would execute

conda create -p $EASYBUILD_PREFIX/software/perl-app-cpanminus/1.7039
conda install -c bioconda perl-app-cpanminus=1.7039

Conda Env Create EasyConfig Example
++++++++++++++++++++++++++++++++

easyblock = 'Conda'

name = "gencore_variant_detection"
version = "1.0"
variant = "Linux-x86_64"

homepage = "https://nyuad-cgsb.github.io/variant_detection/public/index.html"
description = """ This is a bundled install of software packages for doing variant detection analysis. """

toolchain = {'name': 'dummy', 'version': ''}

builddependencies = [('Anaconda3', '4.0.0')]

# Use one of the following - either an environment.yml file or a remote environment definition
#environment_file = '/path/to/conda-environment.yml'
remote_environment = "nyuad-cgsb/%(name)s_%(version)s"

sanity_check_paths = {
'files': ["bin/conda"],
'dirs': []
}

moduleclass = 'tools'

The gencore_variant_detection conda example would execute

conda env create nyuad_cgsb/gencore_variant_detection_1.0 -p $EASYBUILD_PREFIX/software/gencore_variant_detection/1.0

In this particular case we are fetching a remote environment from anaconda cloud. Alternately, create an environment from a environment.yml file.

Loaded the module in the usual fashion.

module load gencore_variant_detection/1.0

For users who have experience with anaconda/conda, running `module load gencore_variant_detection/1.0` is functionally equivalent to `source activate $EASYBUILD_PREFIX/software/gencore_variant_detection/1.0`.

Resources
=========

`Conda Resource <http://conda.pydata.org/docs/test-drive.html>`