Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added autoformatting and updated README #1

Merged
merged 13 commits into from
Sep 7, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[flake8]
extend-ignore = E203
max-line-length = 100
12 changes: 12 additions & 0 deletions .github/workflows/black.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
name: Lint

on: [push, pull_request]

jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: psf/black@stable
with:
src: "./src"
164 changes: 164 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,164 @@
### Python template
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be foud at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/

# User-specific stuff
*.whl
15 changes: 15 additions & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
repos:
- repo: https://github.com/psf/black-pre-commit-mirror
rev: 23.7.0
hooks:
- id: black
language_version: python3.9
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
- id: check-added-large-files
- id: check-executables-have-shebangs
- id: check-shebang-scripts-are-executable
- id: check-yaml
- id: end-of-file-fixer
- id: trailing-whitespace
116 changes: 68 additions & 48 deletions README.md
100755 → 100644
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
hyakvnc
=======
# hyakvnc

Create and manage VNC Slurm jobs on UW HYAK Klone cluster.

Expand All @@ -15,94 +14,115 @@ time.
### Prerequisites

Before running `hyakvnc`, you'll need the following:

- SSH client
- VNC client/viewer
- TigerVNC viewer is recommended for all platforms
- TigerVNC viewer is recommended for all platforms
- HYAK Klone access with compute resources
- VNC Apptainer with TigerVNC server and a desktop environment
- Install additional tools and libraries to the container as required by
programs running within the VNC session.
- xstartup script used to launch a desktop environment
- Install additional tools and libraries to the container as required by programs running within the VNC session.
- `xstartup` script used to launch a desktop environment
- A Python interpreter (version **3.9** or higher)

### Building

Update pip:
`hyakvnc` is a Python package that can be installed with `pip`. The minimum Python version required is **3.9**. You can check the version of the default Python 3 interpreter with:

```bash
python3 -m pip install --upgrade --user pip
python3 -V
```

As of 2021-09-30, the default Python 3 interpreter on Klone is version 3.6.8. Because `hyakvnc` requires version 3.9 or higher, it is necessary to specify the path to a Python 3.9 or newer interpreter when installing `hyakvnc`. You can list the Python 3 interpreters you have available with:

```bash
compgen -c | grep '^python3\.[[:digit:]]$'
```

At this time, Klone supports Python 3.9, which can be run with the command `python3.9`. The following instructions are written with `python3.9` in mind. If you use another version, such as `python3.11`, you will need to substitute `python3.9` with, e.g., `python3.11` in the instructions.

```bash
python3.9 -m pip install --upgrade --user pip
```

Build and install the package:

```bash
python3 -m pip install --user git+https://github.com/uw-psych/hyakvnc
python3.9 -m pip install --user git+https://github.com/uw-psych/hyakvnc
```

Or, clone the repo and install the package locally:

```bash
git clone https://github.com/uw-psych/hyakvnc
python3 -m pip install --user .
python3.9 -m pip install --user .
```

If successful, then `hyakvnc` should be installed to `~/.local/bin/`.

#### Optional dependencies for development

The optional dependency group `[dev]` in `pyroject.toml` includes dependencies useful for development, including [pre-commit](https://pre-commit.com/) hooks that run in order to commit to the `git` repository.
These apply various checks, including running the `black` code formatter before the commit takes place.

To ensure `pre-commit` and other development packages are installed, run:

```bash
python3.9 -m pip install --user '.[dev]'
```

### General usage

`hyakvnc` is command-line tool that only works while on the login node.

### Creating a VNC session

1. Start a VNC session with the `hyakvnc create` command followed by arguments
to specify the Slurm account and partition, compute resource needs,
reservation time, and paths to a VNC apptainer and xstartup script.
1. Start a VNC session with the `hyakvnc create` command followed by arguments to specify the Slurm account and partition, compute resource needs, reservation time, and paths to a VNC apptainer and xstartup script.

```bash
# example: Starting VNC session on `ece` compute resources for 5 hours on a
# node with 16 cores and 32GB of memory
hyakvnc create -A ece -p compute-hugemem \
-t 5 -c 16 --mem 32G \
--container /path/to/container.sif \
--xstartup /path/to/xstartup
```
```bash
# example: Starting VNC session on `ece` compute resources for 5 hours on a
# node with 16 cores and 32GB of memory
hyakvnc create -A ece -p compute-hugemem \
-t 5 -c 16 --mem 32G \
--container /path/to/container.sif \
--xstartup /path/to/xstartup
```

2. If successful, `hyakvnc` should print a unique port forward command:

```
...
=====================
Run the following in a new terminal window:
ssh -N -f -L AAAA:127.0.0.1:BBBB [email protected]
then connect to VNC session at localhost:AAAA
=====================
```
```text
...
=====================
Run the following in a new terminal window:
ssh -N -f -L AAAA:127.0.0.1:BBBB [email protected]
then connect to VNC session at localhost:AAAA
=====================
```

Copy this port forward command for the following step.
Copy this port forward command for the following step.

3. Set up port forward between your computer and HYAK login node. On your
machine, in a new terminal window, run the the copied command.
3. Set up port forward between your computer and HYAK login node. On your machine, in a new terminal window, run the the copied command.

Following the example, run:
Following the example, run:

```bash
ssh -N -f -L AAAA:127.0.0.1:BBBB [email protected]
```
```bash
ssh -N -f -L AAAA:127.0.0.1:BBBB [email protected]
```

Alternatively, for PuTTY users, navigate to
PuTTY Configuration->Connection->SSH->Tunnels, then set:
- source port to `AAAA`
- destination to `127.0.0.1:BBBB`
Alternatively, for PuTTY users, navigate to `PuTTY Configuration->Connection->SSH->Tunnels`, then set:
- source port to `AAAA`
- destination to `127.0.0.1:BBBB`

Press `Add`, then connect to Klone as normal. Keep this window open as it
maintains a connection to the VNC session running on Klone.
Press `Add`, then connect to Klone as normal. Keep this window open as it
maintains a connection to the VNC session running on Klone.

4. Connect to the VNC session at instructed address (in this example:
`localhost:AAAA`)

5. To close the VNC session, run the following

```bash
hyakvnc kill-all
```
```bash
hyakvnc kill-all
```

### Checking active VNC sessions

Expand Down Expand Up @@ -135,7 +155,7 @@ re-established with new port mappings via the `hyakvnc repair` command.

Before repairing:

```
```text
Active hyakvnc jobs:
Job ID: NNNNNNNN
SubNode: n3050
Expand All @@ -145,7 +165,7 @@ Active hyakvnc jobs:

After repairing port forwarding:

```
```text
Active hyakvnc jobs:
Job ID: NNNNNNNN
SubNode: n3050
Expand All @@ -162,7 +182,7 @@ which may require the user to set up new port forward(s).

### Override Apptainer bind paths

If present, `hyakvnc` will use `APPTAINER_BINDPATH` or `SINGULARITY_BINDPATH` to
If present, `hyakvnc` will use the [environment variables](https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_02.html) `APPTAINER_BINDPATH` or `SINGULARITY_BINDPATH` to
determine how paths are mounted in the VNC container environment. If neither is
defined, `hyakvnc` will use its default bindpath.

Expand Down
Loading