-
Notifications
You must be signed in to change notification settings - Fork 2
JASMIN
The JASMIN platform is the Natural Environment Research Council (NERC)'s compute service. If you are interested in its metrics, look here https://mon.jasmin.ac.uk/.
The start of JASMIN's documentation is here: https://help.jasmin.ac.uk/category/158-getting-started. There are video instructions.
Note: in the process of being updated: This GitHub has recordings for various workshops with accompanying exercises, in particular:
- Accessing and using the JASMIN “scientific analyis” severs
- Copy data to JASMIN to share with collaborators
- Processing on the scientific analysis servers
- A script to extract 1 variable from a file in the CEDA Archive
- Batch run a script on LOTUS
- Choosing the right storage for your workflow
- Manage a multi-step workflow
- Building your own Python 3 environment
To access JASMIN, you must sign up for an account following the instructions here: https://help.jasmin.ac.uk/article/4435-get-a-jasmin-account
In the 'What will you be using JASMIN for' box, put something along the lines of: Environmental data analysis tasks forming part of my MRes training in the Artificial Intelligence for Environmental Risk (AI4ER, https://ai4er-cdt.esc.cam.ac.uk/) Centre for Doctoral Training hosted at the University of Cambridge and the British Antarctic Survey.
Once you've confirmed your email address, you'll be asked for some account details, including an SSH Public Key. If you're not sure about this, see here: https://help.jasmin.ac.uk/article/185-generate-ssh-key-pair . If using MobaXterm (https://help.jasmin.ac.uk/article/4832-mobaxterm, https://mobaxterm.mobatek.net/documentation.html) on Windows, first choose a persistent home directory in Settings/ Configuration / General for MobaXterm.
Your account so far is just a profile on the accounts portal but has no privileges. If you intend to use JASMIN computing resources you need to apply for the jasmin-login service, which will allow you to connect to JASMIN machines using ssh: https://help.jasmin.ac.uk/article/161-get-login-account. When asked for supporting information provide the same information as when applying for the account. This service allocates you a home area of 100GB, and you can access the login servers using SSH. You can submit jobs to LOTUS using the batch scheduler SLURM. To submit python scripts, this might be useful: http://gws-access.jasmin.ac.uk/public/bas_climate/files/scott/pybsub.py
You can apply to access our 10TB Group Work Space (GWS) following these instructions https://help.jasmin.ac.uk/article/3801-apply-for-access-to-a-gws-new searching for ai4er or follow this link https://accounts.jasmin.ac.uk/services/group_workspaces/ai4er/. For supporting information provide the same information as when applying for the account.
Another service to apply for is the Jupyter Notebook service. Follow instructions here: https://help.jasmin.ac.uk/article/4851-jasmin-notebook-service.
The approval process can take a couple of days.
To access the CEDA data archive (https://help.jasmin.ac.uk/article/3838-ceda-archive) you need a CEDA account. You register here https://services.ceda.ac.uk/cedasite/register/info/. After this you need to link your CEDA account to your JASMIN account. To do so, when logged into the JASMIN account portal and on your profile page, click the "link now" button. Once linked you are a member of the Unix group open. To check just type groups. You can apply for access to further data sets. https://help.ceda.ac.uk/article/137-ceda-data-catalogue describes how to use the CEDA data catalogue.
JASMIN restricts login access by maintaining an "allow list" of network domains which are allowed to make SSH connections to the JASMIN login gateways. All .ac.uk network domains are already registered. Thus access JASMIN from an .ac.uk network: https://help.jasmin.ac.uk/article/190-check-network-details. If connecting from home, connect to your institution before making an outgoing SSH connection to a JASMIN login server.
Log-in instructions are here: https://help.jasmin.ac.uk/article/187-login. If encountering "Could not add identity "~.ssh/id_rsa_jasmin": agent refused operation", it might already be loaded. Check with ssh-add -l. Otherwise, check whether ssh-agent is running with Get-Service ssh-agent or if on Windows type services in the command box and look for OpenSSH Authentication Agent. If this still does not resolve the problem check whether the permissions on the key file are not too open. Further trouble shooting https://help.jasmin.ac.uk/article/848-login-problems.
Note that you cannot do any actual work on the login servers. You need to log in to one of the scientific analysis servers https://help.jasmin.ac.uk/article/121-sci-servers.
The introduction is found here https://help.jasmin.ac.uk/article/199-introduction-to-group-workspaces. https://help.jasmin.ac.uk/article/3839-gws-etiquette describes how information about data in a GWS should be shared. Please follow the instruction on directory structure and meta data.
The path for the AI4ER GWS is /gws/nopw/j04/ai4er. Members of the UNIX group gws_ai4er have access. Instructions on creating directories and making contents writable to other users are here https://help.jasmin.ac.uk/article/203-managing-a-gws. Note that the default set of permissions is set to umask 002 meaning that members of our group can create and modify data files.
TO DO: Showcases can be made accessible to a wider audience via https://help.jasmin.ac.uk/article/202-share-gws-data-via-http. Back-up arrangements.
A list of data transfer help pages is here: https://help.jasmin.ac.uk/category/217-data-transfer. A list of transfer servers is here: https://help.jasmin.ac.uk/article/192-transfer-servers. rsync (https://help.jasmin.ac.uk/article/3810-data-transfer-tools-rsync-scp-sftp) is recommended, because it is simple and generally applicable. Remember -v for verbose, -r recursive for directories.
This page provides details of the software environments on JASMIN: https://help.jasmin.ac.uk/article/4729-jaspy-envs. Since Python is open source, there are subtle dependencies between package versions. We strongly recommend that you are aware of versions and use module load jaspy/<python_version>/<release> for consistency and continuity. If you need a new package, or an updated version, to the jaspy environments on JASMIN, please follow the procedure as described on the page. You can load different environments and run your code there to check dependencies on versions.
If you are using Matplotlib to visualise data, there is some advice here: https://help.jasmin.ac.uk/article/4733-matplotlib. Especially on the batch server LOTUS, you need to set the rendering backend matplotlib.use('agg') before importing matplotlib.pyplot (https://matplotlib.org/faq/usage_faq.html#what-is-a-backend).
Instructions are here https://help.jasmin.ac.uk/article/4851-jasmin-notebook-service. An introduction is here https://github.com/cedadev/ceda-notebooks/blob/master/notebooks/intro/notebook-tour.ipynb.
Once you have the JupyterLab interface, the left hand panel will show your home directory. For security reasons, it is not possible for a user to open a notebook from outside of their home directory. Notebooks can access files from the CEDA Archive, and also have read-only access to group workspaces. Launch a new notebook and write the code
with open('/gws/nopw/j04/ai4er/wiki_link.txt') as file: # file closed after with block
print(file.read())to check that you can access the GWS from the notebook service.
You can execute shell commands in a Python console, or in a Python cell in a notebook by prefixing the line with !, e.g. ! ls /gws/nopw/j04/ai4er. You can copy notebooks into your home directory to work on and run them, e.g. ! cp /gws/nopw/j04/ai4er/notebook-examples/plot_polar_era5_hourly_mslp_t2m_and_uv10.ipynb $HOME.
plot_polar_era5_hourly_mslp_t2m_and_uv10.ipynb (kindly provided by Tony Phillips) makes a polar stereographic plot of ERA5 hourly MSLP, 2m temperature and/or 10m wind
Further example notebooks are at https://github.com/cedadev/ceda-notebooks/blob/master/notebooks. Remember that you need a linked CEDA account.
The following servers are available via jasmin, given you have the right permissions:
- login = login servers: jasmin-login1.ceda.ac.uk, cems-login1.cems.ac.uk
- sci = scientific analysis servers: jasmin-sci[1-5].ceda.ac.uk, cems-sci[1-2].cems.rl.ac.uk
- transfer = data transfer servers: jasmin-xfer[1-2].ceda.ac.uk, cems-xfer1.cems.rl.ac.uk
- LOTUS = LOTUS batch processing cluster (all cluster nodes)
For more info on the specification of the scientific analysis servers, see the following table (table taken from here)
| Server name | Virtual/Physical (OperatingSystem) | Processor model | CPU Cores | RAM (GB) | /tmp (GB) |
| sci1.jasmin.ac.uk |
V (CentOS7) | Xeon(R) Silver 4114 CPU @ 2.20GHz | 8 | 16 | NA |
| sci2.jasmin.ac.uk |
V (CentOS7) | Xeon(R) E5-2630 @ 2.20GHz | 8 | 16 | NA |
| sci3.jasmin.ac.uk | P (CentOS7) | AMD EPYC 7402 | 48 | 1000 | - |
| sci4.jasmin.ac.uk | V (CentOS7) | Xeon E5-2660 @ 2.20GHz | 8 | 32 | NA |
| sci5.jasmin.ac.uk | V (CentOS7) | Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz | 8 | 32 | NA |
| sci6.jasmin.ac.uk | P (CentOS7) | AMD EPYC 7402 | 48 | 1000 | - |
| sci8.jasmin.ac.uk | P (CentOS7) | Intel(R) Xeon(R) Gold 5118 CPU @ 2.30GHz |
24 | 383. | - |