This repo is created as a part of the CSDE Workshop on Introduction to the UNIX/Linux Shell.
It includes a sample script for working with UW HYAK -- on-campus high-performance computing platform for scalable scientific computing.
Object: Running a simple regression model and calculating Bootstrap Confidence Intervals.
- First we need to log-in to our HYAK instance. Open a terminal instance and paste the following:
ssh [UW-NetID]@mox.hyak.uw.edu-
Enter your password. Then, you will be directed to 2-Factor Authentication via Duo Mobile.
-
Once you are successfully logged in, navigate to your project directory. (Hint:
ls,cd,pwdcommands are your best friends.) -
We can use
sftporgit(recommended) for file transfer between local and HYAK. (Hint: You can uselcd,lls,lpwdto navigate through local files.)
sftp [UW-NetID]@mox.hyak.uw.edu
put -r [PATH-TO-LOCAL-FILE]- Then, you can submit your job via this command:
sbatch submit.slurm-
If successfuly, you should receive a message with your job id:
Submitted batch job [JOBID] -
You can track the progress of your currents job via:
squeue -u [UW-NetID]or including past jobs:
sacct - Once finished, you can do quickly check the output in command line:
cat slurm-*- To export final output into our local machines, we can use
sftporgit(recommended) again:
get -r [PATH-TO-REMOTE-FILE]example.R: Demonstration of the task, interactively.bootstrap.R: Bootstrapping script designed to be used with a slurm script.submit.slurm: Slurm script for a single batch submission.slurm-[JOBID].out: Once our job is finished, the console output is stored in here.output/: The resulting data.frames with bootstrap estimates.plots/: Plots that show the distribution of bootstrap estimates.bootstrapArray.R: Bootstrapping script modified for array batch submissionsubmitArray.slurm: Slurm script modified for array batch submission.slurm-[JOBID]_N.out: Outputs of array batches stored in this format.