Updated Release Management, Build Base, and Build Service GitHub Acti…

…ons. The Release Management updates include fixes to GitHub Action syntax on the and playbook documentation. The Build Base Dockerfile was moved to a base directory and GitHub action updated. The Release Management GitHub action is using a composite GitHub action that does not support specifying a Dockerfile file name. The Build Service Dockerfile was updated to remove the .activator suffix to accomodate and the Build Service GitHub action updated.

.github/workflows/build-base.yml

@@ -4,13 +4,13 @@ on:
   push:
     paths:
       - '.github/workflows/build-base.yml'
-      - 'Dockerfile.main'
+      - 'base/Dockerfile'
     branches:
       - main
   pull_request:
     paths:
       - '.github/workflows/build-base.yml'
-      - 'Dockerfile.main'
+      - 'base/Dockerfile'
   workflow_dispatch:
     inputs:
       stackTag:
@@ -44,6 +44,7 @@ jobs:
           username: ${{ github.actor }}
           password: ${{ secrets.GITHUB_TOKEN }}
       - name: Determine base image eups tag
+        working-directory: base
         run: |
           if [[ -n "$STACK_TAG" ]]; then
             if [[ "$STACK_TAG" == "*_latest" ]]; then
@@ -60,15 +61,17 @@ jobs:
           echo "Eups tag = $(< eups.tag)"
       - name: Build image
         # Context-free build
+        working-directory: base
         run: |
           docker build - \
             --build-arg "STACK_TAG=$(< lsst.docker.tag)" \
             --tag $IMAGE_NAME \
             --label "runnumber=${GITHUB_RUN_ID}" \
             --label "stacktag=$(< stack.tag)" \
             --label "eupstag=$(< eups.tag)" \
-            < Dockerfile.main
+            < Dockerfile
       - name: Push image to registries
+        working-directory: base
         run: |
           MAKE_LATEST="${{ inputs.makeLatest }}"
           [[ -n "$MAKE_LATEST" ]] || MAKE_LATEST="false"

.github/workflows/build-service.yml

@@ -8,13 +8,13 @@ on:
       - '.github/workflows/build-service.yml'
       - 'pipelines/**'
       - 'python/activator/**'
-      - 'Dockerfile.activator'
+      - 'Dockerfile'
   pull_request:
     paths:
       - '.github/workflows/build-service.yml'
       - 'pipelines/**'
       - 'python/activator/**'
-      - 'Dockerfile.activator'
+      - 'Dockerfile'
   workflow_dispatch:
 
 permissions:
@@ -98,7 +98,7 @@ jobs:
           echo "Eups tag = $(< eups.tag)"
       - name: Build image
         run: |
-          docker build . -f Dockerfile.activator \
+          docker build . -f Dockerfile \
             --build-arg "BASE_TAG=$BASE_TAG" \
             --tag $IMAGE_NAME \
             --label "runnumber=${GITHUB_RUN_ID}" \

.github/workflows/ci-release.yaml

@@ -15,6 +15,10 @@ name: Release CI
       - "*"
   pull_request: {}
 
+env:
+  # Base tag to run tests against.
+  BASE_TAG: "latest"
+
 jobs:
   build:
     runs-on: ubuntu-latest
@@ -26,6 +30,29 @@ jobs:
     steps:
       - uses: actions/checkout@v3
 
+      - name: Fix permissions
+        run: chmod -R a+rwX $GITHUB_WORKSPACE
+
+      - name: Login to GitHub Container Registry
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+
+      - name: Run tests
+        run: |
+          docker run \
+              -v $GITHUB_WORKSPACE:/home/lsst/prompt_processing \
+              ghcr.io/${{ github.repository_owner }}/prompt-base:$BASE_TAG \
+              bash -c '
+                  cd /home/lsst/prompt_processing
+                  source /opt/lsst/software/stack/loadLSST.bash
+                  setup -r .
+                  # Fix permissions; arg must be absolute path.
+                  git config --global --add safe.directory /home/lsst/prompt_processing
+                  scons'
+
       - uses: lsst-sqre/build-and-push-to-ghcr@v1
         id: build
         with:

Dockerfile.activator → Dockerfile

@@ -1,9 +1,5 @@
 ARG BASE_TAG=latest
 FROM ghcr.io/lsst-dm/prompt-base:${BASE_TAG}
-ENV PYTHONUNBUFFERED True
-ENV APP_HOME /app
-ENV PROMPT_PROCESSING_DIR $APP_HOME
-ARG PUBSUB_VERIFICATION_TOKEN
 ARG PORT
 WORKDIR $APP_HOME
 COPY python/activator activator/

doc/playbook.rst

@@ -55,6 +55,34 @@ To build the service container:
 
    The ``PYTHONUNBUFFERED`` environment variable defined in the Dockerfiles for the containers ensures that container logs are emitted in real-time.
 
+To build a release:
+
+Releases are largely automated through GitHub Actions (see the `ci-release.yaml <https://github.com/lsst-dm/prompt_processing/actions/workflows/ci-release.yaml>`  workflow file for details). When a semantic version tag is pushed to GitHub, Prompt Processing Docker images are published on GitHub and Docker Hub with that version.
+
+Regular releases happen from the ``main`` branch after changes have been merged.  From the ``main`` branch you can release a new major version (``X.0.0``), a new minor version of the current major version (``X.Y.0``), or a new patch of the current major-minor version (``X.Y.Z``).  Release tags are semantic version identifiers following the `pep 440 <https://peps.python.org/pep-0440/>` specification.
+
+1. Create a Release
+
+On GitHub.com, navigate to the main page of the repository.  To the right of the list of files, click **Releases**.  At the top of the page, click **Draft a new release**.  Type a tag using semantic versioning described in the previous section.  The Target should be the main branch.  
+
+Select **Generate Release Notes**.  This will generate a list of commit summaries and of submitters.   Add text as follows.
+* Add the specific motivation for the release(for example, including a specific feature, preparing for a specific observing run))
+* Science Pipelines version and rubin-env version
+* Changes to the APDB and Alerts schemas, to be replaced by version numbers once we have them.
+
+Select **Publish Release**.
+
+The `ci-release.yaml <https://github.com/lsst-dm/prompt_processing/actions/workflows/ci-release.yaml>` GitHub Actions workflow uploads the new release to GitHub packages.
+
+2. Tag the release
+
+At the HEAD of the ``main`` branch, create and push a tag with the semantic version:
+
+.. code-block:: sh
+
+   git tag -s X.Y.Z -m "X.Y.Z"
+   git push --tags
+
 Stable Base Containers
 ----------------------
 
@@ -65,7 +93,6 @@ Any subsequent builds of the service container will build against both bases.
 
 This is the only situation in which a change to ``BASE_TAG_LIST`` should be committed to ``main``.
 
-
 Buckets
 =======