Skip to content

Commit

Permalink
upd(alps): more Alps doc, remove part under ICON
Browse files Browse the repository at this point in the history
  • Loading branch information
leclairm committed Aug 30, 2024
1 parent b16fe11 commit 3987bf5
Show file tree
Hide file tree
Showing 3 changed files with 64 additions and 46 deletions.
63 changes: 52 additions & 11 deletions docs/alps/uenvs.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,23 +38,59 @@ It can be used to
- run a single command with that uenv mounted
- access uenv modules

For now, `uenv` isn't available on Balfrin and requires (easy) [manual installation :material-open-in-new:](https://eth-cscs.github.io/uenv/#getting-uenv){:target="_blank"}.
For now, `uenv` isn't available on Balfrin and requires (easy) [manual installation :material-open-in-new:](https://eth-cscs.github.io/uenv/#getting-uenv){:target="_blank"}:
```shell
git clone https://github.com/eth-cscs/uenv.git && ./uenv/install --local
```

### Quickstart

!!! note "TODO: Quick start"
In the following, using `name/version:tag` as metadata to target a uenv can be replaced by the absolute path to the image if necessary.


#### Create a local images repository
In order to be able to use uenv images, you first need to `pull`them into a local repository. First create the repo with
```shell
uenv repo create
```

- [ ] `uenv image repo create`
- [ ] `uenv image find`
- [ ] `uenv image pull`
- [ ] `uenv start`
- [ ] `uenv run`
#### List available images
```shell
uenv image find
```

## `--uenv` SLURM plugin
#### Pull an image in the local repo
```shell
uenv image pull name/version:tag
```

#### Start a uenv for interactive use
This will spawn a new shell with the uenv mounted at `/mount-point` (defaulting to `/user-environment`)
```shell
uenv start name/version:tag [/mount-point]
```
Exit with a usual `exit` or `uenv stop`.

#### Use modules
Software in a uenv can be accessed by using it as a `spack` upstream (see [`spack-c2sm` integration](#spack-c2sm-integration)) but also by loading modules. For that, use the `--view=modules` option:
```shell
uenv start --view=modules name/version:tag [/mount-point]
```
A usual `module avail` command would then show you the different modules added by the uenv on top of the original environment

#### launch a subprocess with a uenv mounted
```shell
uenv run name/version:tag -- my_command
```
will run `my_command` (can be executing a script) with the uenv mounted. This is particularly useful when inside scripts.

## The `--uenv` SLURM plugin

CSCS developped the `--uenv` slurm plugin in order to mount uenvs at runtime. Either use the name of a uenv in your local repo or the full absolute path to the uenv.

```shell
sbatch --uenv uenv_name:/mount/point
sbatch --uenv /image/path:/mount/point
sbatch --uenv name/version:tag[:/mount/point]
sbatch --uenv /image/path[:/mount/point]
```

`:mount/point` can always be omitted and defaults to `:/user-environment`.
Expand All @@ -72,8 +108,13 @@ Uenvs cannot be mounted *anywhere*. They are generated with a predefined install

## `spack-c2sm` integration

User environments are used as a spack upstream with
User environments are supported in non-yet released `spack-c2sm` tags. For testing purposes, you can use `v0.21.1.2`, so
```shell
git clone --depth 1 --recurse-submodules --shallow-submodules -b v0.21.1.2 https://github.com/C2SM/spack-c2sm.git
```
Then start the user environment and use it as a spack upstream with
```shell
uenv start name/version:tag
source spack-c2sm/setup-env.sh /user-environment
```

Expand Down
12 changes: 12 additions & 0 deletions docs/alps/vclusters.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,3 +72,15 @@ Santis is dedicated to Weather and Climate and is deployed on ~400 Grace-Hopper
!!! note "TODO"

- [ ] Storage


## Tödi

Tödi is the testing vCluster and is currently deployed on the entire machine.

### Uenvs

| uenv | activity |
|--------------------------|-------------------------------|
| icon-wcp/v1:rc4 | build and run icon |
| netcdf-tools/2024:v1-rc1 | pre and post processing tools |
35 changes: 0 additions & 35 deletions docs/models/icon.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,41 +54,6 @@ spack install cosmo-eccodes-definitions
srun -N 1 -c 12 --mem-per-cpu=20G spack install -v -j 12
```

### Temporary Tödi

!!! warning "These are only temporary guidelines during the acceptance phase of Alps"

- Santis will be the official W&C vcluster but Tödi has to be deployed on the entire Alps system for the acceptance phase. These guidelines are for the ones who have access to Tödi.
- The resulting executable is not tested.
- Anything can go wrong

- Get `spack-c2sm`
```shell
git clone --depth 1 --recurse-submodules --shallow-submodules https://github.com/C2SM/spack-c2sm.git
```

- Start user environment
```shell
uenv start /capstor/scratch/cscs/leclairm/uenvs/images/icon_c2sm_v1_rc3.sqfs
```

- Setup `spack-c2sm`
```shell
source spack-c2sm/setup-env.sh /user-environment
```

- Adapt your spack environment following `/capstor/scratch/cscs/leclairm/icon-nwp/config/cscs/spack/uenv/mch_gpu_double/spack.yaml` and activate it as usual with `spack env activate`.

- Build as usual from your icon root dir
```shell
spack install
```

- Submit job with uenv
```shell
sbatch --uenv /capstor/scratch/cscs/leclairm/uenvs/images/icon_c2sm_v1_rc3.sqfs my_run_script
```

## Run test case with ICON
In the *run* folder, you find many prepared test cases, which you can convert into run scripts. To generate the runscript of one of the experiment files, e.g. *mch_ch_lowres*, you can use the `make_runscripts` function.

Expand Down

0 comments on commit 3987bf5

Please sign in to comment.