-
Notifications
You must be signed in to change notification settings - Fork 2
JASMIN
The JASMIN platform is the Natural Environment Research Council (NERC)'s compute service. If you are interested in its metrics, look here https://mon.jasmin.ac.uk/.
The start of JASMIN's documentation is here: https://help.jasmin.ac.uk/category/158-getting-started. There are video instructions.
To access JASMIN, you must sign up for an account following the instructions here: https://help.jasmin.ac.uk/article/4435-get-a-jasmin-account
In the 'What will you be using JASMIN for' box, put something along the lines of: Environmental data analysis tasks forming part of my MRes training in the Artificial Intelligence for Environmental Risk (AI4ER, https://ai4er-cdt.esc.cam.ac.uk/) Centre for Doctoral Training hosted at the University of Cambridge and the British Antarctic Survey.
Once you've confirmed your email address, you'll be asked for some account details, including an SSH Public Key. If you're not sure about this, see here: https://help.jasmin.ac.uk/article/185-generate-ssh-key-pair . If using MobaXterm (https://help.jasmin.ac.uk/article/4832-mobaxterm, https://mobaxterm.mobatek.net/documentation.html) on Windows, first choose a persistent home directory in Settings/ Configuration / General for MobaXterm.
Your account so far is just a profile on the accounts portal but has no privileges. If you intend to use JASMIN computing resources you need to apply for the jasmin-login service, which will allow you to connect to JASMIN machines using ssh: https://help.jasmin.ac.uk/article/161-get-login-account. When asked for supporting information provide the same information as when applying for the account. This service allocates you a home area of 100GB, and you can access the login servers using SSH. You can submit jobs to LOTUS using the batch scheduler SLURM. To submit python scripts, this might be useful: http://gws-access.jasmin.ac.uk/public/bas_climate/files/scott/pybsub.py
You can apply to access our 10TB Group Work Space (GWS) following these instructions https://help.jasmin.ac.uk/article/3801-apply-for-access-to-a-gws-new searching for ai4er or follow this link https://accounts.jasmin.ac.uk/services/group_workspaces/ai4er/. For supporting information provide the same information as when applying for the account.
Another service to apply for is the Jupyter Notebook service. Follow instructions here: https://help.jasmin.ac.uk/article/4851-jasmin-notebook-service.
The approval process can take a couple of days.
JASMIN restricts login access by maintaining an "allow list" of network domains which are allowed to make SSH connections to the JASMIN login gateways. All .ac.uk network domains are already registered. Thus access JASMIN from an .ac.uk network: https://help.jasmin.ac.uk/article/190-check-network-details. If connecting from home, connect to your institution before making an outgoing SSH connection to a JASMIN login server.
Log-in instructions are here: https://help.jasmin.ac.uk/article/187-login. If encountering "Could not add identity "~.ssh/id_rsa_jasmin": agent refused operation", it might already be loaded. Check with ssh-add -l. Otherwise, check whether ssh-agent is running with Get-Service ssh-agent or if on Windows type services in the command box and look for OpenSSH Authentication Agent. If this still does not resolve the problem check whether the permissions on the key file are not too open. Further trouble shooting https://help.jasmin.ac.uk/article/848-login-problems.
Note that you cannot do any actual work on the login servers. You need to log in to one of the scientific analysis servers https://help.jasmin.ac.uk/article/121-sci-servers.
The introduction is found here https://help.jasmin.ac.uk/article/199-introduction-to-group-workspaces. https://help.jasmin.ac.uk/article/3839-gws-etiquette describes how information about data in a GWS should be shared. Please follow the instruction on directory structure and meta data.
The path for the AI4ER GWS is /gws/nopw/j04/ai4er. Members of the UNIX group gws_ai4er have access. Instructions on creating directories and making contents writable to other users are here https://help.jasmin.ac.uk/article/203-managing-a-gws. Note that the default set of permissions is set to umask 002 meaning that members of our group can create and modify data files.
TO DO: Showcases can be made accessible to a wider audience via https://help.jasmin.ac.uk/article/202-share-gws-data-via-http. Back-up arrangements.
A list of data transfer help pages is here: https://help.jasmin.ac.uk/category/217-data-transfer. A list of transfer servers is here: https://help.jasmin.ac.uk/article/192-transfer-servers. rsync (https://help.jasmin.ac.uk/article/3810-data-transfer-tools-rsync-scp-sftp) is recommended, because it is simple and generally applicable. Remember -v for verbose, -r recursive for directories.
This page provides details of the software environments on JASMIN: https://help.jasmin.ac.uk/article/4729-jaspy-envs. Since Python is open source, there are subtle dependencies between package versions. We strongly recommend that you are aware of versions and use module load jaspy/<python_version>/<release> for consistency and continuity. If you need a new package, or an updated version, to the Jaspy environments on JASMIN, please follow the procedure as described on the page. You can load different environments and run your code there to check dependencies on versions.
If you are using Matplotlib to visualise data, there is some advice here: https://help.jasmin.ac.uk/article/4733-matplotlib. Especially on the batch server LOTUS, you need to set the rendering backend matplotlib.use('agg') before importing matplotlib.pyplot (https://matplotlib.org/faq/usage_faq.html#what-is-a-backend).