cmem-plugin-template

This repository contains a copier template.

You can use it to bootstrap the following types of project:

• eccenca Corporate Memory Python Plugin: Bootstrapping and keeping plugin repositories up-to-date was the initial idea of this template.
• Generic Python Project: After tons of commits, we wanted to use this template not only for plugins, but for all kinds of python projects.

Table of contents

• Features
• Usage
  • Project Initialization
  • Template Updates
  • Other Tasks
• Setup
  • Local Requirements
  • Integration Tests
  • Plugins only: Corporate Memory Environment
  • CI Build Plan
  • Editor / IDE Support
    • PyCharm

Features

• Python / poetry project with
  • pytest (incl. memray + pytest-dotenv + code coverage) as the testing framework,
  • ruff as all-hands linter and formatter,
  • mypy as type checker, and
  • safety as dependency vulnerability scanner.
• Build plans for
  • gitlab,
  • github, and
  • locally with task (tested for Linux, MacOS and Windows/MinGW).
  • Including
    • badge generation,
    • JUnit XML file and
    • coverage stat generation.

Usage

Project Initialization

The following command will create a new project directory with the latest released template. This produces a plugin which is compatible the latest release of eccenca Corporate Memory.

Note: Select 'Generic Python Project' as an answer to the initial question to skip creation of plugin specific code.

$ copier copy gh:eccenca/cmem-plugin-template cmem-plugin-my

The following command will use the latest develop version of the template: This produces a plugin which is compatible the latest development snapshot of eccenca Corporate Memory.

$ copier copy -r develop gh:eccenca/cmem-plugin-template cmem-plugin-my

After that, you need to initialize the repository and optionally install the git pre-commit hooks:

$ cd cmem-plugin-my
$ git init; git add .; git commit -m "init"
$ pre-commit install

Then you can run the local test suite and build a first deployment artefact:

$ task check build

Template Updates

We continuously update this repository. This includes maintenance of dependencies, build plan updates and the adoption of new features from the plugin base library.

In order to upgrade your project to the latest template release, use the following command:

$ copier update

In order to prepare your project for the upcoming next release, use this command:

$ copier update -r develop

Please also have a look at the copier documentation.

Other Tasks

The available tasks for your project can be listed like this (note that there are more tasks, prefixed with plugin: in case you started a plugin project):

∴ task
task: Available tasks for this project:
* build:                   Build a tarball and a wheel package
* check:                   Run whole test suite incl. unit and integration tests
* clean:                   Removes dist, *.pyc and some caches
* check:linters:           Run all linter and static code analysis tests
* check:mypy:              Complain about typing errors
* check:pytest:            Run unit and integration tests
* check:ruff:              Complain about everything else
* check:safety:            Complain about vulnerabilities in dependencies
* format:fix:              Format Python files and fix obvious issues
* format:fix-unsafe:       Format Python files and fix 'unsafe' issues

You can extend this task list by creating a file TaskfileCustom.yaml in your repository root:

$ cat TaskfileCustom.yaml
---
version: '3'

tasks:

  ttt:
    desc: just a test
    cmds:
      - task --list

Setup

Local Requirements

The following tools are needed for local task execution:

• Python 3.11+
• copier (>= v9) for project template rendering and updating
• task (>= v3.29) for running build tasks (make sure to follow the installation instructions to avoid confusion with taskwarrior)
• poetry (>= v1.7) for packaging and dependency managing (+ dynamic versioning plugin)
• pre-commit (>= v2.20) for managing pre-commit hooks (optional)

Example installation of the requirements with pipx on Ubuntu:

$ sudo sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b /usr/local/bin
$ python3 -m pip install --user pipx
$ python3 -m pipx ensurepath
$ pipx install copier
$ pipx install pre-commit
$ pipx install poetry
$ poetry self add "poetry-dynamic-versioning[plugin]"

Integration Tests

This template uses the pytest testing framework. Testing your plugin is crucial and should be done locally as well as integrated with eccenca Corporate Memory.

In order to setup access to a Corporate Memory deployment, you need to provide correct environment variables. Without these variables, only standalone tests can be executed (see 1 skipped):

$ task check:pytest
...
... ===== 3 passed, 1 skipped in 0.09s =====

Plugins only: Corporate Memory Environment

By providing the correct cmemc environment variables in an .env file or directly in your environment, your plugin can be tested in an integrated way:

# Environment as direct variables:
$ export CMEM_BASE_URI="https://cmem.example.org"
$ export OAUTH_CLIENT_ID="cmem-service-account"
$ export OAUTH_CLIENT_SECRET="..."
$ export OAUTH_GRANT_TYPE="client_credentials"

# Environment as .env files
$ cat .env
CMEM_BASE_URI="https://cmem.example.org"
OAUTH_CLIENT_ID="cmem-service-account"
OAUTH_CLIENT_SECRET="..."
OAUTH_GRANT_TYPE="client_credentials"

CI Build Plan

The gitlab workflow as well as the github action pipelines need the same environment variables as secrets:

• For github, go to Settings > Secret > Actions > New Repository Secret
• For gitlab, go to Settings > CI/CD > Variables (Expand) > Add Variable (protected, masked, all environments)

An example github pipeline can be seen here.

In addition to the eccenca Corporate Memory credential secrets, a PYPI_TOKEN secret can be set in order to use the publish task/workflow.

Editor / IDE Support

PyCharm

In order to have the best PyCharm experience, when starting a project with this template, we suggest the following PyCharm plugins:

• Ruff will provide the linting hints which will be raised by the pipeline anyway.
• Taskfile will allow for starting tasks.