Skip to content

Latest commit

 

History

History
231 lines (161 loc) · 6.73 KB

Developing.md

File metadata and controls

231 lines (161 loc) · 6.73 KB
Raw

Instructions for CrayCLI Developers

Development Process

Using the Cray Service Generator is the preferred method of integrating with the CLI. This will automatically take your Swagger file and bootstrap a new CLI module from it, creating a pull request into the CLI along the way. The only thing you need to do after this process is create functional tests and add them to your pull request.

If you choose to manually integrate and not use the generator, our development process is based on git flow. Do the following:

  • Create a feature branch off of master
  • Integrate your CLI, note that integrations will not be approved without functional tests.
  • Run nox -s tests locally to ensure the CLI tests pass
  • Create a pull request to have your changes merged into master.

Testing against a local API server

If you would like to point the CLI at a local server that is running an API so that you don't have to develop with an entire Cray system up and running, you can configure the CLI like this:

cray config set core hostname=http://localhost:8080

if your server is running on port 8080. Since this localhost server is not running with https, you need to set OAUTHLIB_INSECURE_TRANSPORT=1 in your shell.

Command:

export OAUTHLIB_INSECURE_TRANSPORT=1
cray config get core.hostname

Output:

http://localhost:8080

Command:

cray mymodule things list

Output:

[]

Your CLI commands are now routed to the local server.

Testing against a hardware system

To test against a real hardware system, build the source as a wheel and install in a virtual environment on the target system.

python -m pip install build
python -m build --wheel
scp ./dist/cray-<version>.whl <target system>
ssh <target system>
python3 -m venv venv
source venv/bin/activate
pip install cray-<version>.whl

Now running the cray command will test your source against the target system.

Unit/Functional Testing

The CLI Framework is using nox to run tests, linting, and building documentation. This means you only need to worry about writing unit/functional tests and we take care of testing on each python version, auto-doc new CLI commands, etc. Test cases should be added for each new module. Pull requests cannot be merged until a 95% code coverage is achieved.

To run tests:

nox -s tests cover

Installation for Development

For development, we recommend a virtualenv with python3:

NOTE It is recommended to install the highest version of Python supported by the application (e.g. Python 3.10) For MacOS users we recommend using pyenv, which can be installed via Homebrew (brew). Using pyenv enables installing every version of Python there is, and avoids interactions with the system Python. For help using pyenv, see setting up python.

Setting up Python

These steps are specific to MacOS. For other distros, please ask in the CrayCLI Slack channel.

pyenv is a Python version manager for MacOS, it allows users to install multiple versions of Python. It is not recommended to use the system's Python, since it often requires root privileges and is a fragile dependency of the OS itself. pyenv installs Python versions into the user directory on the system, keeping the Python versions local to the user session.

  1. Install bew using the steps outlined on Homebrew (brew)'s homepage.

  2. Install pyenv, using the directions on their MacOS page here.

    NOTE There are automatic installers mention on the linked page above.

  3. Open a new shell, and invoke pyenv to make sure it works.

  4. Install Python 3.10

    # Find latest Python 3.10
pyenv install -l | grep 3.10

pyenv install 3.10.10 # or w/e the latest one was

  5. Install virtualenv

    NOTE Optionally you can run pyenv global 3.10.10 and then python -m pip install virtualenv. The step below uses the installed path instead because not all users wnat to change their global Python version for their user account.

    ~/.pyenv/versions/3.10.10/bin/python -m pip install virtualenv

  6. Create a virtualenv for craycli

    mkdir -p ~/.virtualenvs
~/.pyenv/versions/3.10.10/bin/python -m virtualenv ~/.virtualenvs/craycli

Installing craycli

  1. Load the virtualenv (the example below uses the virtualenv created in setting up python).

    source ~/.virtualenvs/craycli/bin/activate

  2. Clone craycli (if it isn't already cloned somewhere)

    git clone https://github.com/Cray-HPE/craycli.git
cd craycli

  3. Install craycli

    python -m pip install .
cray --version

Building

We are using pyinstaller to generate a binary, and then installing it on systems using an RPM.

Bugs

If you find a bug in the craycli framework, feel free to open a bug in the CASMCLOUD project. We'll triage it within a day or two. File Bug

How to update your Swagger

Assuming you're using the .remote option for your module: (from the root of the forked project)

Convert to Swagger

NOTE Remember to activate your virtualenv, or create one following installation for development.

  • Install CI tools.

    python -m pip install .[ci]

  • Generate Swagger.

    nox -s swagger -- my_service_name path/to/api/file

    Potential output:

    nox > Running session swagger
nox > Creating virtual environment (virtualenv) using python3 in .nox/swagger
nox > /bin/bash utils/convert.sh cray/modules/ims swagger3.json
... 
Wrote /Users/rusty/gitstuffs/cray-shasta/craycli/cray/modules/my_service_name/swagger3.json
nox > Session swagger was successful.

Running nox (unit tests)

  • Install CI tools.

    python -m pip install .[ci]

  • Run unit tests.

    nox -s tests

    Potential output:

    nox > Running session tests
nox > Creating virtual environment (virtualenv) using python3 in .nox/tests
nox > python -m pip install '.[test]'
nox > python -m pip install .
nox > pytest --quiet --cov=cray --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=85 cray
... [100%]
Required test coverage of 85% reached. Total coverage: 91.83%
523 passed in 211.76s (0:03:31)
nox > Session tests was successful.