Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Various updates #45

Merged
merged 5 commits into from
Nov 20, 2023
Merged

Various updates #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
86fce19
add linux-aarch64 to local channels
daler Nov 19, 2023
737c423
remove MINICONDA_VER
daler Nov 19, 2023
4f1d78e
update readme
daler Nov 19, 2023
5faa156
update readme
daler Nov 19, 2023
c26746f
add configure-conda to test
daler Nov 19, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,3 +12,4 @@ jobs:
- name: Test install bioconda-utils
run: |
COMMON_GIT_REF=${GITHUB_SHA} bash install-and-set-up-conda.sh
bash configure-conda.sh
34 changes: 32 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,34 @@
This repository contains common definitions for the bioconda build system, e.g., the bioconda-utils tag to use and the miniconda version.
# Overview

This repository acts as a central "source of truth" which can be used by
various components of the Bioconda build system.

- `common.sh` contains environment variables that control versions
# Components

**`common.sh`:** contains environment variables that control versions.

**`install-and-set-up-conda.sh`:**
- downloads and installs conda using the Mambaforge distribution
- installs bioconda-utils dependencies
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should not use mambaforge anymore. Conda-forge discourages it and advertises miniforge.

With the release of Miniforge3-23.3.1-0, that incorporated the changes in conda-forge/miniforge#277, the packages and configuration of Mambaforge and Miniforge3 are now identical. The only difference between the two is the name of the installer and, subsequently, the default installation directory.
Given its wide usage, there are no plans to deprecate Mambaforge. If at some point we decide to deprecate Mambaforge, it will be appropriately announced and communicated with sufficient time in advance.
As of September 2023, the new usage of Mambaforge is thus discouraged. Bug reports specific to Mambaforge will be closed as won't fix.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was not aware of this, thanks for pointing it out.

This PR was mostly about documentation; do you think it's worth changing mambaforge -> miniforge in this PR? Theoretically it should not change anything in the build process since we're already on 23.3.1, so this is more a question of "do you think it will actually not change anything".

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A different PR is ok. But we should get rid of it at some point.

- installs the version of bioconda-utils specified in common.sh via pip install

**`configure-conda.sh`:** ensures channels are configured correctly.

# Typical usage

In practice, a typical CI environment will do the following:

1. Download the files, typically via:

```bash
wget https://raw.githubusercontent.com/bioconda/bioconda-common/master/{common,install-and-set-up-conda,configure-conda}.sh
```
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe 3 separate commands? I guess the intention here is to copy-and-paste?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was documenting what we currently have. Could go either way, but I guess it's slightly easier to copy-paste this single line.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. Agree.


2. Run `install-and-set-up-conda.sh` (which sources `common.sh`).

3. Use the CI platform's mechanism to cache the conda install dir (which is
configured in `common.sh`).

4. After a cache restore, run `configure-conda.sh`. The reason for this is that
the `--system` arg for `conda config` is not supported by mamba, and so the
channel config ends up in the home directory which is not cached.
1 change: 0 additions & 1 deletion common.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
MINICONDA_VER=py37_4.8.3
BIOCONDA_UTILS_TAG=v2.9.0
MAMBAFORGE_VER=23.1.0-1
MAMBAFORGE_INSTALLATION_DIR="/opt/mambaforge"
2 changes: 1 addition & 1 deletion install-and-set-up-conda.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ if [ ${BIOCONDA_DISABLE_BUILD_PREP:=0} == 0 ]; then

# Set local channel as highest priority (requires conda-build, which is
# installed as a dependency of bioconda-utils)
mkdir -p "${MAMBAFORGE_INSTALLATION_DIR}/conda-bld/{noarch,linux-64,osx-64}"
mkdir -p "${MAMBAFORGE_INSTALLATION_DIR}/conda-bld/{noarch,linux-64,osx-64,linux-aarch64}"
conda index "${MAMBAFORGE_INSTALLATION_DIR}/conda-bld"
conda config --add channels "file://${MAMBAFORGE_INSTALLATION_DIR}/conda-bld"
fi
Expand Down
Loading