Update Development docs

In the previous refactor (#81) the development documentation and scripts needed more attention. This updates the `Developing.md` with up-to-date information, as well as updating the `utils/` items. The `devenv` container is no longer used, instead the virtualenv path is now recommended. The swagger container was updated and pushed to our Artifactory on algol60.net. This also updates the `--version` and `--help` (`-h`) arguments to leverage `click`'s builtins, removing the funky `build_version` handling (which doesn't work anyway given we use `setuptools_scm[toml]` now). `Makefile` was updated for maintainers to make and publish new swagger Docker images as needed. Add setup-python directions
Cray-HPE · May 10, 2023 · cab5b33 · cab5b33
1 parent 49d3d90
commit cab5b33
Show file tree

Hide file tree

Showing 9 changed files with 179 additions and 143 deletions.
diff --git a/Developing.md b/Developing.md
@@ -31,22 +31,26 @@ running with https, you need to set `OAUTHLIB_INSECURE_TRANSPORT=1` in your
 shell.
 
 Command:
+
 ```bash
 export OAUTHLIB_INSECURE_TRANSPORT=1
 cray config get core.hostname
 ```
 
 Output:
+
 ```text
 http://localhost:8080
 ```
 
 Command:
+
 ```bash
 cray mymodule things list
 ```
 
 Output:
+
 ```text
 []
 ```
@@ -59,8 +63,8 @@ To test against a real hardware system, build the source as a wheel and install
 in a virtual environment on the target system.
 
 ```bash
-pip install wheel
-python setup.py bdist_wheel
+python -m pip install build
+python -m build --wheel
 scp ./dist/cray-<version>.whl <target system>
 ssh <target system>
 python3 -m venv venv
@@ -86,70 +90,142 @@ nox -s tests cover
 
 ## Installation for Development
 
-For development, we recommend the container below or a virtualenv with python3:
+For development, we recommend a virtualenv with python3:
 
-```bash
-virtualenv -p python3 cli
-cd cli
-source bin/activate
-git clone https://github.com/Cray-HPE/craycli.git
-cd craycli
-python -m pip install .
-```
+> ***NOTE*** It is recommended to install the highest version of Python supported by the application \(e.g. Python 3.10\)
+> For MacOS users we recommend using [`pyenv`](https://github.com/pyenv/pyenv#homebrew-in-macos), which can be installed via
+> [Homebrew (`brew`)](https://brew.sh/). Using `pyenv` enables installing every version of Python there is, and avoids 
+> interactions with the system Python. For help using `pyenv`, see [setting up python](#setting-up-python).
 
-### Dev Container
+### Setting up Python
 
-We created a container that includes all dependencies required to run nox and
-develop for the cli:
+These steps are specific to MacOS. For other distros, please ask in the CrayCLI Slack channel.
 
-```bash
-git clone https://github.com/Cray-HPE/craycli.git
-cd craycli
-utils/devenv.sh
-```
+[`pyenv`](https://github.com/pyenv/pyenv) is a Python version manager for MacOS, it allows users to
+install multiple versions of Python. It is not recommended to use the system's Python, since it often requires root
+privileges and is a fragile dependency of the OS itself. `pyenv` installs Python versions into the user directory on
+the system, keeping the Python versions local to the user session.
+
+1. Install bew using the steps outlined on [Homebrew (`brew`)](https://brew.sh/)'s homepage.    
+
+1. Install `pyenv`, using the directions on their MacOS page [here](https://github.com/pyenv/pyenv#homebrew-in-macos).
+
+    > ***NOTE*** There are automatic installers mention on the linked page above.
+
+1. Open a new shell, and invoke `pyenv` to make sure it works.
+1. Install Python 3.10
+
+    ```bash
+    # Find latest Python 3.10
+    pyenv install -l | grep 3.10
+
+    pyenv install 3.10.10 # or w/e the latest one was
+    ```
+
+1. Install `virtualenv`
+
+    > ***NOTE*** Optionally you can run `pyenv global 3.10.10` and then `python -m pip install virtualenv`. The step below
+    > uses the installed path instead because not all users wnat to change their global Python version for their user account. 
+
+    ```bash
+    ~/.pyenv/versions/3.10.10/bin/python -m pip install virtualenv
+    ```
+
+1. Create a `virtualenv` for `craycli`
+
+    ```bash
+    mkdir -p ~/.virtualenvs
+    ~/.pyenv/versions/3.10.10/bin/python -m virtualenv ~/.virtualenvs/craycli
+    ```
+
+### Installing craycli
+
+1. Load the `virtualenv` (the example below uses the virtualenv created in [setting up python](#setting-up-python)).
+
+    ```bash
+    source ~/.virtualenvs/craycli/bin/activate
+    ```
+
+1. Clone `craycli` (if it isn't already cloned somewhere)
+
+    ```bash
+    git clone https://github.com/Cray-HPE/craycli.git
+    cd craycli
+    ```
+
+1. Install `craycli`
+
+    ```bash
+    python -m pip install .
+    cray --version
+    ```
 
 ## Building
 
-We are using pyinstaller to generate a binary and wrapping into an RPM.
-This is integrated into the DST jenkins pipeline.
+We are using `pyinstaller` to generate a binary, and then installing it on systems using an RPM.
 
 ## Bugs
 
-If you find a bug in the craycli framework, feel free to open a bug in the
-casmcloud project.  We'll triage it within a day or two.
+If you find a bug in the `craycli` framework, feel free to open a bug in the
+CASMCLOUD project.  We'll triage it within a day or two.
 [File Bug](https://github.com/Cray-HPE/craycli/issues/new)
 
 ## How to update your swagger
 
 Assuming you're using the .remote option for your module: (from the root of the forked project)
 
+#### Convert to Swagger
 
-__Generate the swagger__
+> ***NOTE*** Remember to activate your `virtualenv`, or create one following
+> [installation for development](#installation-for-development).
 
-```
-$> ./utils/devenv.sh
-bash-4.4$ nox -s swagger -- my_service_name
-nox > Running session swagger
-nox > Creating virtualenv using python3.6 in /work/.nox/swagger
-... 
-nox > Session swagger was successful.
-```
+- Install CI tools.
 
-__Run normal tests__
+    ```bash
+    python -m pip install .[ci]
+    ```
 
-```
-bash-4.4$ nox -s tests
-nox > Running session tests-2.7
-nox > Creating virtualenv using python2.7 in /work/.nox/tests-2-7
-nox > pip install --upgrade -r requirements-test.txt
-nox > pip install --upgrade -e .
-nox > py.test --quiet --cov=cray --cov=tests --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=95 tests
-... [100%]
-Required test coverage of 95% reached. Total coverage: 95.60%
-460 passed in 110.78 seconds
-nox > Session tests-2.7 was successful.
-bash-4.4$ exit
-$>
-```
+- Generate swagger.
+
+    ```bash
+    nox -s swagger -- my_service_name path/to/api/file
+    ```
+  
+    Potential output:
+
+    ```text
+    nox > Running session swagger
+    nox > Creating virtual environment (virtualenv) using python3 in .nox/swagger
+    nox > /bin/bash utils/convert.sh cray/modules/ims swagger3.json
+    ... 
+    Wrote /Users/rusty/gitstuffs/cray-shasta/craycli/cray/modules/my_service_name/swagger3.json
+    nox > Session swagger was successful.
+    ```
+
+#### Running `nox` (unit tests)
+
+- Install CI tools.
+
+    ```bash
+    python -m pip install .[ci]
+    ```
+
+- Run unit tests.
+
+    ```bash
+    nox -s tests
+    ```
+
+    Potential output:
 
-If they pass open the PR and we’ll merge it asap.
+    ```text
+    nox > Running session tests
+    nox > Creating virtual environment (virtualenv) using python3 in .nox/tests
+    nox > python -m pip install '.[test]'
+    nox > python -m pip install .
+    nox > pytest --quiet --cov=cray --cov-append --cov-config=.coveragerc --cov-report= --cov-fail-under=85 cray
+    ... [100%]
+    Required test coverage of 85% reached. Total coverage: 91.83%
+    523 passed in 211.76s (0:03:31)
+    nox > Session tests was successful.
+    ```
diff --git a/Makefile b/Makefile
@@ -45,7 +45,7 @@ export VERSION := $(shell python3 -m setuptools_scm 2>/dev/null | tr -s '-' '~'
 endif
 
 ifeq ($(VERSION),)
-$(error VERSION not set! Verify setuptools_scm[toml] is installed and try again.)
+$(warning VERSION not set! Verify setuptools_scm[toml] is installed and try again.)
 endif
 
 #############################################################################
@@ -123,6 +123,7 @@ help:
 	@echo '    help               	Show this help screen.'
 	@echo '    clean               	Remove build files.'
 	@echo
+	@echo '    image                Build and publish the swagger testing image.'
 	@echo '    rpm                	Build a YUM/SUSE RPM.'
 	@echo '    all 					Build all production artifacts.'
 	@echo
@@ -132,6 +133,10 @@ help:
 	@echo '    rpm_build            Builds the RPM.'
 	@echo '    rpm_build_source		Builds the SRPM.'
 	@echo '    rpm_package_source   Creates the RPM source tarball.'
+	@echo
+	@echo '    image_login   		Logs into the Docker registry for pulling and publishing images.'
+	@echo '    image_build   		Builds the swagger testing image.'
+	@echo '    image_publish   		Builds and publishes the testing image.'
 	@echo ''
 
 clean:
@@ -172,3 +177,18 @@ rpm_build:
 
 snyk:
 	$(MAKE) -s image | xargs --verbose -n 1 snyk container test
+
+#############################################################################
+# RPM targets
+#############################################################################
+
+image_login:
+	docker login artifactory.algol60.net
+
+image_build:
+	docker build -t artifactory.algol60.net/csm-docker/stable/craycli/swagger2openapi:latest utils/
+
+image_publish: image_build
+	docker push artifactory.algol60.net/csm-docker/stable/craycli/swagger2openapi:latest
+
+image: image_publish
diff --git a/README.md b/README.md
@@ -67,21 +67,21 @@ The version is derived from Git by the `setuptools_scm` Python module.
 
     ```bash
     # Format       {tag}.post1.dev{distance}+{scm letter}{revision hash}
-    canu, version 1.6.28.post1.dev14+g818da8a
+    cray, version 1.6.28.post1.dev14+g818da8a
     ```
 
 - **(unstable/dev)** No distance and not clean; the build came from a commit that has a git-tag, and the repository had uncommitted changes.
 
     ```bash
     # Format       {tag}.dYYYYMMDD
-    canu, version 1.6.28.d20230123
+    cray, version 1.6.28.d20230123
     ```
 
 - **(unstable/dev)** Distance and not clean; the build came from one or more commits after a git-tag, and the repository had uncommitted changes.
 
     ```bash
     # Format       {tag}.post1.dev{distance}+{scm letter}{revision hash}.dYYYYMMDD
-    canu, version 1.6.28.post1.dev3+g3071655.d20230123
+    cray, version 1.6.28.post1.dev3+g3071655.d20230123
     ```
 
 The `setuptools_scm` module is configured by `pyproject.toml`.

diff --git a/cray/cli.py b/cray/cli.py
@@ -27,7 +27,6 @@
 
 import os
 import re
-import sys
 import click
 
 from cray.auth import AuthUsername
@@ -51,9 +50,12 @@
         'config': Config('', '', raise_err=False),
         'token': None,
         'auth': None
-    }, 'auto_envvar_prefix': NAME.upper()
+    },
+    'auto_envvar_prefix': NAME.upper(),
+    'help_option_names': ['-h', '--help'],
 }
 
+CONTEXT_SETTINGS = {}
 
 def rsa_required(config):
     """Get the value for 'auth.login.rsa_required' from the CLI
@@ -84,6 +86,7 @@ def rsa_required(config):
     context_settings=CONTEXT_SETTING
 )  # pragma: NO COVER
 @click.pass_context
+@click.version_option()
 def cli(ctx, *args, **kwargs):
     """ Cray management and workflow tool"""
     pass
@@ -212,17 +215,3 @@ def cli_cb(ctx, result, **kwargs):
     # Use click echo instead of our logging because we always want to echo
     # our results
     click.echo(format_result(result, ctx.obj['globals'].get('format')))
-
-
-if getattr(sys, 'frozen', False):
-    version = None
-    if '--version' in sys.argv:
-        # Only bother opening the file if actually asking for version
-        path = os.path.join(os.path.dirname(__file__), 'build_version')
-        if os.path.isfile(path):
-            with open(path, encoding='utf-8') as v:
-                version = v.read()
-    click.version_option(version)(cli)
-    cli(sys.argv[1:])
-else:
-    click.version_option()(cli)
diff --git a/noxfile.py b/noxfile.py
@@ -59,7 +59,7 @@ def generate(session):
 """
 
     if len(session.posargs) != 2:
-        msg = 'Usage: nox -s generate -- [module name] [stash link to swagger]'
+        msg = 'Usage: nox -s generate -- [module name] [path to API file]'
         global ERROR_ON_GENERATE
         if ERROR_ON_GENERATE:
             raise Exception(msg)
@@ -82,7 +82,6 @@ def generate(session):
 
     is_local_file = os.path.exists(swagger_file)
 
-
     if not os.path.exists(module_path):
         os.makedirs(module_path)
     if not os.path.isfile(init_file):
@@ -107,7 +106,7 @@ def generate(session):
     convert_file(session, module_path, convert_file_name)
 
 
-@nox.session(python=None)
+@nox.session(python='3')
 def swagger(session):
     """Run each swagger file through the converter in case anything changed.
     This should be run before running unit tests"""
@@ -130,7 +129,7 @@ def swagger(session):
         swagger_file = None
         remote_file = None
         for f in files:
-            if f.startswith('swagger.'):
+            if f.startswith('swagger'):
                 swagger_file = f
                 break
             if f == '.remote':