Re-add Tool Dependencies slides to admin

- Move Tool Dependencies slides from "Tool Shed" to "Tool Management" - Restore and update slides deleted in galaxyproject#1831 - Change slide deck and tutorial title to "Galaxy Tool Management with Ephemeris"
nsoranzo · Jun 12, 2020 · baf70d8 · baf70d8
1 parent 3e8a658
commit baf70d8
Show file tree

Hide file tree

Showing 4 changed files with 167 additions and 48 deletions.
diff --git a/topics/admin/tutorials/connect-to-compute-cluster/slides.html b/topics/admin/tutorials/connect-to-compute-cluster/slides.html
@@ -296,7 +296,7 @@
 
 ---
 
-## Dependency Resolution
+## Tool Dependency Resolution
 
-- Galaxy can resolve dependencies from Conda, Singularity, Docker, and other systems like Modules
-- You can read more about this in the [Toolshed slides](../toolshed/slides.html)
+- Galaxy can resolve dependencies using Conda, Singularity, Docker, and other systems like Modules
+- You can read more about this in the [Galaxy Tool Management with Ephemeris slides](../tool-management/slides.html)
diff --git a/topics/admin/tutorials/tool-management/slides.html b/topics/admin/tutorials/tool-management/slides.html
@@ -5,67 +5,74 @@
 - /topics/admin/tutorials/tool-install/slides
 
 
-title: "Ephemeris for Galaxy Tool Management"
+title: "Galaxy Tool Management with Ephemeris"
 questions:
   - How to install, update, and maintain Galaxy tools?
   - How to extract a list of tools from a workflow or Galaxy instance?
+  - How are tool dependencies resolved by Galaxy?
 objectives:
-  - Learn about Ephemeris
   - Extract a list of tools from a workflow
   - Install these tools into a given Galaxy
+  - Understand the tool dependency resolution process
 key_points:
   - Ephemeris and automation help with the tool management on Galaxy
   - There are tool management best practices you can learn from
-  - Do not manage your Galaxy's tools manually
+  - Dependency resolvers have different priorites
 contributors:
   - mvdbeek
   - martenson
   - slugger70
   - hexylena
+  - nsoranzo
 ---
 
-# Galaxy Vocabulary
+## Galaxy Vocabulary
 
 * `tool` or `tool wrapper` - XML file that describes to Galaxy how the underlying software works.
 * `repository` - Versioned code archive in Tool Shed containing Galaxy tool(s).
+* `requirement` or `dependency` - the underlying software needed to execute the tool command
 
 ---
+class: left
 
 # Ways to add tools
 
 You can add tools to Galaxy either
 
-* Manually - Trivial for trivial tools and useful for tool development.
-* Using Tool Shed
-  * Through admin UI in Galaxy
+* Manually - useful for tool development.
+* From the Tool Shed
+  * Through the admin UI in Galaxy
   * Using **ephemeris** (recommended)
 
 ---
 
-# How to add tools manually
+## How to add tools manually
 
 - By default Galaxy loads all tools in `tool_conf.xml.sample` into tool panel.
-- To add local tools by hand:
+- To add a local tool by hand:
   - Make a copy `$ cp tool_conf.xml.sample tool_conf.xml`.
-  - Add your tool entries to the `tool_conf.xml`.
-  - Refresh the site
+  - Add your tool entry to the `tool_conf.xml`.
 - To add local tools with the [galaxyproject.galaxy Ansible role][galaxy-role]:
   - Set `galaxy_local_tools`
   - Run the playbook
+- Tool dependencies need to be installed separately, unless
+  `conda_auto_install: true` is set in `galaxy.yml` (not recommended for
+  production)
+
 
 [galaxy-role]: https://galaxy.ansible.com/galaxyproject/galaxy
 
 ---
 
-# How to install from the Tool Shed
+## How to install from the Tool Shed
 
 * Find the repository you want to install.
 * From Galaxy's admin UI find the repo in the connected TS.
 * Install the repo into Galaxy.
 
 ---
 
-# How to install with Ephemeris
+## How to install with Ephemeris
 
 * Find the repository you want to install.
 * Run `shed-tools install` with the repository name
@@ -76,7 +83,7 @@
 
 Small Python library for Galaxy Management
 
-* Install:
+* Can install:
 	- Tools
 	- Reference data
 	- Workflows
@@ -98,7 +105,7 @@
 (use `workflow-to-tools` to extract list of tools from a workflow)
 
 ---
-## Get Galaxy's tool list
+## Get installed tool list for a Galaxy instance
 
 ```console
 get-tool-list [-h] [-v] [-g GALAXY] [-u USER] [-p PASSWORD] [-a API_KEY]
@@ -233,10 +240,9 @@
 
 # More on Tool Management:
 
-- tool installation
-- configuration
-- suites
-- dependencies
+- Tool installation
+- Configuration
+- Dependencies
 
 ---
 
@@ -250,9 +256,11 @@
 [conda]: https://conda.io/
 
 ---
+class: left
 
 ## Example config entry
- one entry in `shed_tool_conf.xml`
+
+One entry in `shed_tool_conf.xml`:
 
 ```xml
 <section id="filter" name="Filter and Sort" version="">
@@ -281,3 +289,136 @@
     <tool id="testtoolshed.g2.bx.psu.edu/repos/devteam/bamtools_filter/bamFilter/0.0.1" />
 </section>
 ```
+
+---
+
+## Tool dependency resolution
+
+* We aim to make Galaxy resolver-independent but Conda-oriented.
+* What resolver is going to be used for the tool dependencies is determined at runtime
+according to the order specified in `dependency_resolvers_conf.xml`.
+
+```xml
+<dependency_resolvers>
+  <tool_shed_packages />
+  <galaxy_packages />
+  <conda />
+  <galaxy_packages versionless="true" />
+  <conda versionless="true" />
+<!-- other resolvers
+  <lmod />
+  <lmod versionless="true" />
+  <modules modulecmd="/opt/Modules/3.2.9/bin/modulecmd" />
+  <modules modulecmd="/opt/Modules/3.2.9/bin/modulecmd" versionless="true" default_indicator="default" />
+  <tool_shed_tap />
+  <homebrew />
+-->
+</dependency_resolvers>
+```
+
+[Full documentation](https://docs.galaxyproject.org/en/master/admin/dependency_resolvers.html)
+
+---
+class: left
+
+## Dependency resolution example
+
+Tool XML contains:
+
+```xml
+<tool id="foo">
+    <requirements>
+        <requirement type="package" version="4.2">bar</requirement>
+    </requirements>
+</tool>
+```
+
+---
+class: left
+
+## `<tool_shed_packages />`
+
+Did tool `foo` depend on a Tool Shed package providing `bar` 4.2 when installed?
+- Yes: Query install DB for `repo_owner`, `repo_name`, `changeset_id`, then source:
+  - `<tool_dependencies_dir>/bar/4.2/<repo_owner>/<repo_name>/<changeset_id>/env.sh`
+- No: continue
+
+---
+class: left
+
+## `<galaxy_packages />`
+
+Does `<tool_dependency_dir>/bar/4.2/env.sh` exist?:
+- Yes: source it
+- No: Does `<tool_dependency_dir>/bar/4.2/bin/` exist?:
+  - Yes: prepend it to `$PATH`
+  - No: continue
+
+Good for dependencies manually installed by an administrator
+
+---
+class: left
+
+## `<conda />`
+
+Does `bar==4.2` exist in Conda?:
+- Yes: use `conda create` to create a Conda env in `<conda_prefix>/envs/` (if it doesn't exist already), then activate it
+- No: continue
+
+[Conda for tool dependencies FAQ](https://docs.galaxyproject.org/en/master/admin/conda_faq.html)
+
+---
+class: left
+
+## `<galaxy_packages versionless="true" />`
+
+Does `<tool_dependency_dir>/bar/default/` exist and is it a symlink?:
+
+- Yes: Does `<tool_dependency_dir>/bar/default/env.sh` exist?:
+  - Yes: source it
+  - No: Does `<tool_dependency_dir>/bar/default/bin/` exist?:
+      - Yes: prepend it to `$PATH`
+      - No: continue
+- No: continue
+
+---
+class: left
+
+## `<conda versionless="true" />`
+
+Does `bar` (any version) exist in Conda?:
+- Yes: use `conda create` to create env in job working directory
+- No: continue
+
+---
+class: left
+
+## `<modules modulecmd="..." />`
+
+Is `bar/4.2` a registered module?:
+- Yes: Load module `bar/4.2`
+- No: continue
+
+See: [Environment Modules](http://modules.sourceforge.net/)
+
+---
+class: left
+
+## `<modules modulecmd="..." versionless="true" />`
+
+Is `bar/default` a registered module?:
+- Yes: Load module `bar/default`
+- No: continue
+
+---
+
+## If dependency is not resolved
+
+Submit the job anyway, maybe it's available on `$PATH`
+
+---
+
+## Using containers for Tool Dependencies
+
+- Galaxy can also use **Docker** or **Singularity** containers to resolve dependencies
+- You can read more about this in the [Tool Dependencies and Containers slides](../../../dev/tutorials/containers/slides.html)
diff --git a/topics/admin/tutorials/tool-management/tutorial.md b/topics/admin/tutorials/tool-management/tutorial.md
@@ -1,7 +1,7 @@
 ---
 layout: tutorial_hands_on
 
-title: "Ephemeris for Galaxy Tool Management"
+title: "Galaxy Tool Management with Ephemeris"
 questions:
   - How to install, update, and maintain Galaxy tools?
   - How to extract a list of tools from a workflow or Galaxy instance?

diff --git a/topics/admin/tutorials/toolshed/slides.html b/topics/admin/tutorials/toolshed/slides.html
@@ -171,30 +171,8 @@
 </requirements>
 ```
 
-Galaxy aims to be dependency resolution agnostic.
-
----
-## Dependency resolution
-
-* We aim to make Galaxy resolver-independent but Conda-oriented.
-* What resolver is going to be used for the tool dependency is determined at runtime
-and prioritised in the config file `dependency_resolvers_conf.xml`.
-
-```xml
-<dependency_resolvers>
-  <tool_shed_packages />
-  <galaxy_packages />
-  <conda />
-  <galaxy_packages versionless="true" />
-  <conda versionless="true" />
-  <modules modulecmd="/opt/Modules/3.2.9/bin/modulecmd" />
-  <modules modulecmd="/opt/Modules/3.2.9/bin/modulecmd" versionless="true" default_indicator="default" />
-<!-- other resolvers
-  <tool_shed_tap />
-  <homebrew />
--->
-</dependency_resolvers>
-```
+- Galaxy can resolve dependencies using Conda, Singularity, Docker, and other systems like Modules
+- You can read more about this in the [Galaxy Tool Management with Ephemeris slides](../tool-management/slides.html)
 
 ---