refactor: Setup package structure to follow the Scientific Python Lib…

…rary Development Guide (Ciela-Institute#103)

* feat: Setup packaging and Github Codespaces dev environment (#26)

* Update README.md

Add SSEC badge

* Move project to src/ directory

* Update Codespaces link

* Update README.md

* Update README.md

Update Codespaces link

* Update devcontainer

* ci: Create cd.yml for building and pushing package

* Update caustic name to caustics

* Add PyLint and Black formatting features to devcon

* Closes issue #16

* Updated test/ -> tests/

* Move project to src/ directory

* Update README.md

* Update README.md

Update Codespaces link

* Create noxfile.py

* Add .readthedocs.yaml

* Create .git_archival.txt

* Create .gitattributes

* Create .pre-commit-config.yaml

* Add sphinx and myst-parser support

* style: pre-commit fixes

* Update docs

* style: pre-commit fixes

* Update .github Actions

* Update environment.yml
- Add jupyter-book

* Create CONTRIBUTING.md

* Update README.md

* chore: resolve conflicts

* chore: Remove sources

* test: remove test lenses

* test: fix test_simulator

* chore: Set min python to 3.9

* fix: change pytest to point to tests/

* build: update dep list to use reqs.txt

* ci: Don't fail fast

* chore(deps): Pin astropy < 6

* fix: change instances of caustic to caustics

* fix: Perform pre-commit

* fix: update path to conf.py

* update README

* style: pre-commit fixes

* docs: Add contributor graph

* chore: Setup dynamic versioning

* ci: Update fetch depth to 0 to get correct version

* Update README.md

Update Codespaces button to new repo name

---------

Co-authored-by: Cordero Core <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* build: remove dir sources (#27)

* fix: Add project urls to empty slots

* ci: update yaml configuration for CI (#28)

* ci: update yaml configuration for CI

* style: pre-commit fixes

---------

Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>

* Update README.md

Added issue to contribution section

* Update conf.py

Co-authored-by: Connor Stone, PhD <[email protected]>

* Update pixelated_convergence.py

Co-authored-by: Connor Stone, PhD <[email protected]>

* Update test_parametrized.py

Co-authored-by: Connor Stone, PhD <[email protected]>

* Update test_parametrized.py

Co-authored-by: Connor Stone, PhD <[email protected]>

* Update cd.yml

* Update cd.yml

* Update cd.yml

* Update README.md

* docs: Grab version from package __version__

* Update README.md

* Add nox dependency

* Update noxfile.py

* Update CONTRIBUTING.md

Temporarily removing instructions for `docs`

* Remove `docs` instructions (temp)

* Add tutorials and update nox instructions

* Update `nox` instructions in CONTRIBUTING.md (#3)

* Updated local nox instructions

* Remove tutorials

* Update nox install instruction and noxfile

---------

Co-authored-by: Cordero Core <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Connor Stone, PhD <[email protected]>

.devcontainer/apt.txt

@@ -0,0 +1,4 @@
+git
+ncdu
+wget
+curl

.devcontainer/devcontainer.json

@@ -0,0 +1,21 @@
+// For format details, see https://aka.ms/devcontainer.json. For config options, see the
+{
+  "image": "quay.io/pangeo/pytorch-notebook:latest",
+
+  "customizations": {
+    "vscode": {
+      "extensions": [
+        "ms-toolsai.jupyter",
+        "ms-python.python",
+        "ms-vsliveshare.vsliveshare",
+        "DavidAnson.vscode-markdownlint",
+        "GitHub.copilot"
+      ]
+    }
+  },
+  "postCreateCommand": "sh .devcontainer/postBuild.sh",
+  "features": {
+    "ghcr.io/devcontainers-contrib/features/black:2": {},
+    "ghcr.io/devcontainers-contrib/features/pylint:2": {}
+  }
+}

.devcontainer/environment.yml

@@ -0,0 +1,16 @@
+channels:
+  - conda-forge
+dependencies:
+  - python>=3.10
+  - astropy
+  - jupyterlab
+  - matplotlib
+  - numpy
+  - pandas
+  - scipy
+  - h5py
+  - nox
+  - sphinx
+  - myst-parser
+  - jupyter-book
+  - pip

.devcontainer/postBuild.sh

@@ -0,0 +1,5 @@
+# For writing commands that will be executed after the container is created
+
+# Installs `caustic` as local library without resolving dependencies (--no-deps)
+python3 -m pip install -e /workspaces/caustics --no-deps
+python3 -m pip install -e ".[dev]"

.devcontainer/start

@@ -0,0 +1,12 @@
+#!/bin/bash -l
+
+# ==== ONLY EDIT WITHIN THIS BLOCK =====
+
+export CAUSTICS_ENV="caustics"
+if ! [[ -z "${CAUSTICS_SCRATCH_PREFIX}" ]] && ! [[ -z "${JUPYTERHUB_USER}" ]]; then
+    export CAUSTICS_SCRATCH="${CAUSTICS_SCRATCH_PREFIX}/${JUPYTERHUB_USER}/"
+fi
+
+# ==== ONLY EDIT WITHIN THIS BLOCK =====
+
+exec "$@"

.git_archival.txt

@@ -0,0 +1,4 @@
+node: $Format:%H$
+node-date: $Format:%cI$
+describe-name: $Format:%(describe:tags=true,match=*[0-9]*)$
+ref-names: $Format:%D$

.gitattributes

@@ -0,0 +1 @@
+.git_archival.txt  export-subst

.github/CONTRIBUTING.md

@@ -0,0 +1,113 @@
+See the [Scientific Python Developer Guide][spc-dev-intro] for a detailed
+description of best practices for developing scientific packages.
+
+[spc-dev-intro]: https://scientific-python-cookie.readthedocs.io/guide/intro
+
+# Quick development
+
+The fastest way to start with development is to use `nox`. This will set up a
+virtual environment for you to run all the checks and tests. There are 2 ways to
+install `nox`:
+
+## Codespaces
+
+Nox is pre-installed in the Codespaces environment. So, after activating a Codespace,
+you can just open the terminal and run `nox` to run all the checks and tests.
+
+## Local
+
+If you don't have nox, you can do the following to install `nox`:
+
+```bash
+pip install nox
+```
+
+If you use macOS, then `nox` is in brew:
+
+```bash
+brew install nox
+```
+
+## Nox basics
+
+### What is it?
+
+`nox` is a command-line tool that automates testing in multiple Python environments,
+similar to tox. Unlike tox,
+Nox uses a standard Python file for configuration,
+you can find this configuration in [`noxfile.py`](../noxfile.py).
+
+### How do I use it?
+
+To use, run `nox`. This will lint and test using every installed version of
+Python on your system, skipping ones that are not installed. You can also run
+specific jobs:
+
+```bash
+nox -s lint  # Lint only
+nox -s tests  # Python tests
+nox -s build  # Make an SDist and wheel
+```
+
+Nox handles everything for you, including setting up a temporary virtual
+environment for each run.
+
+# Setting up a development environment manually
+
+You can set up a development environment by running:
+
+```bash
+python3 -m venv .venv
+source ./.venv/bin/activate
+pip install -v -e .[dev]
+```
+
+If you have the
+[Python Launcher for Unix](https://github.com/brettcannon/python-launcher), you
+can instead do:
+
+```bash
+py -m venv .venv
+py -m install -v -e .[dev]
+```
+
+# Post setup
+
+You should prepare pre-commit, which will help you by checking that commits pass
+required checks:
+
+```bash
+pip install pre-commit # or brew install pre-commit on macOS
+pre-commit install # Will install a pre-commit hook into the git repo
+```
+
+You can also/alternatively run `pre-commit run` (changes only) or
+`pre-commit run --all-files` to check even without installing the hook.
+
+# Testing
+
+Use pytest to run the unit checks:
+
+```bash
+pytest
+```
+
+# Coverage
+
+Use pytest-cov to generate coverage reports:
+
+```bash
+pytest --cov=caustics
+```
+
+# Pre-commit
+
+This project uses pre-commit for all style checking. While you can run it with
+nox, this is such an important tool that it deserves to be installed on its own.
+Install pre-commit and run:
+
+```bash
+pre-commit run -a
+```
+
+to check all files.

.github/dependabot.yml

@@ -0,0 +1,7 @@
+version: 2
+updates:
+  # Maintain dependencies for GitHub Actions
+  - package-ecosystem: "github-actions"
+    directory: "/"
+    schedule:
+      interval: "weekly"

.github/matchers/pylint.json

@@ -0,0 +1,32 @@
+{
+  "problemMatcher": [
+    {
+      "severity": "warning",
+      "pattern": [
+        {
+          "regexp": "^([^:]+):(\\d+):(\\d+): ([A-DF-Z]\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
+          "file": 1,
+          "line": 2,
+          "column": 3,
+          "code": 4,
+          "message": 5
+        }
+      ],
+      "owner": "pylint-warning"
+    },
+    {
+      "severity": "error",
+      "pattern": [
+        {
+          "regexp": "^([^:]+):(\\d+):(\\d+): (E\\d+): \\033\\[[\\d;]+m([^\\033]+).*$",
+          "file": 1,
+          "line": 2,
+          "column": 3,
+          "code": 4,
+          "message": 5
+        }
+      ],
+      "owner": "pylint-error"
+    }
+  ]
+}

.github/workflows/cd.yml

@@ -0,0 +1,99 @@
+name: CD
+
+on:
+  workflow_dispatch:
+  push:
+    branches:
+      - main
+      - dev
+  release:
+    types:
+      - published
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  FORCE_COLOR: 3
+
+jobs:
+  dist:
+    name: Distribution build
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Build sdist and wheel
+        run: pipx run build
+
+      - uses: actions/upload-artifact@v3
+        with:
+          path: dist
+
+      - name: Check products
+        run: pipx run twine check dist/*
+
+  test-built-dist:
+    needs: [dist]
+    name: Test built distribution
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write
+    steps:
+      - uses: actions/[email protected]
+        name: Install Python
+        with:
+          python-version: "3.10"
+      - uses: actions/download-artifact@v3
+        with:
+          name: artifact
+          path: dist
+      - name: List contents of built dist
+        run: |
+          ls -ltrh
+          ls -ltrh dist
+      - name: Publish to Test PyPI
+        uses: pypa/[email protected]
+        with:
+          repository-url: https://test.pypi.org/legacy/
+          verbose: true
+          skip-existing: true
+      - name: Check pypi packages
+        run: |
+          sleep 3
+          python -m pip install --upgrade pip
+
+          echo "=== Testing wheel file ==="
+          # Install wheel to get dependencies and check import
+          python -m pip install --extra-index-url https://test.pypi.org/simple --upgrade --pre caustics
+          python -c "import caustics; print(caustics.__version__)"
+          echo "=== Done testing wheel file ==="
+
+          echo "=== Testing source tar file ==="
+          # Install tar gz and check import
+          python -m pip uninstall --yes caustics
+          python -m pip install --extra-index-url https://test.pypi.org/simple --upgrade --pre --no-binary=:all: caustics
+          python -c "import caustics; print(caustics.__version__)"
+          echo "=== Done testing source tar file ==="
+
+  publish:
+    needs: [dist, test-built-dist]
+    name: Publish to PyPI
+    environment: pypi
+    permissions:
+      id-token: write
+    runs-on: ubuntu-latest
+    if: github.event_name == 'release' && github.event.action == 'published'
+
+    steps:
+      - uses: actions/download-artifact@v3
+        with:
+          name: artifact
+          path: dist
+
+      - uses: pypa/[email protected]
+        if: startsWith(github.ref, 'refs/tags')

.github/workflows/ci.yml

@@ -0,0 +1,74 @@
+# This workflow will install Python dependencies, run tests and lint with a single version of Python
+# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python
+
+name: CI
+
+on:
+  workflow_dispatch:
+  pull_request:
+  push:
+    branches:
+      - main
+      - dev
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
+env:
+  FORCE_COLOR: 3
+  PROJECT_NAME: "caustics"
+
+jobs:
+  build:
+    runs-on: ${{matrix.os}}
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.9", "3.10", "3.11"]
+        os: [ubuntu-latest, windows-latest, macOS-latest]
+
+    steps:
+      - name: Checkout caustics
+        uses: actions/checkout@v3
+        with:
+          fetch-depth: 0
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v4
+        with:
+          python-version: ${{ matrix.python-version }}
+          allow-prereleases: true
+
+      - name: Record State
+        run: |
+          pwd
+          echo github.ref is: ${{ github.ref }}
+          echo GITHUB_SHA is: $GITHUB_SHA
+          echo github.event_name is: ${{ github.event_name }}
+          echo github workspace: ${{ github.workspace }}
+          pip --version
+
+      - name: Install dependencies
+        run: |
+          python -m pip install --upgrade pip
+          pip install pytest pytest-cov torch wheel
+
+      # We only want to install this on one run, because otherwise we'll have
+      # duplicate annotations.
+      - name: Install error reporter
+        if: ${{ matrix.python-version == '3.10' }}
+        run: |
+          python -m pip install pytest-github-actions-annotate-failures
+
+      - name: Install Caustics
+        run: |
+          pip install -e ".[dev]"
+          pip show ${{ env.PROJECT_NAME }}
+
+      - name: Test with pytest
+        run: |
+          pytest -vvv --cov=${{ env.PROJECT_NAME }} --cov-report=xml --cov-report=term tests/
+
+      - name: Upload coverage reports to Codecov with GitHub Action
+        uses: codecov/codecov-action@v3