Skip to content

Commit

Permalink
first cut at API checking
Browse files Browse the repository at this point in the history
It turns out there is a consumer of cabal-install-solver, so I have
added it to API generation and checking.
  • Loading branch information
geekosaur committed Sep 15, 2024
1 parent efa04f7 commit 1f21acd
Show file tree
Hide file tree
Showing 8 changed files with 22,109 additions and 0 deletions.
49 changes: 49 additions & 0 deletions .github/workflows/check-api.skip.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@

name: Check API Skip

# This Workflow is special and contains a workaround for a known limitation of GitHub CI.
#
# The problem: We don't want to run the "check-api" jobs on PRs which contain only changes
# to the docs, since these jobs take a long time to complete without providing any benefit.
# We therefore use path-filtering in the workflow triggers for the bootstrap jobs, namely
# "paths-ignore: doc/**". But the "Check API post job" is a required job, therefore a PR cannot
# be merged unless the "Check API post job" completes succesfully, which it doesn't do if we
# filter it out.
#
# The solution: We use a second job with the same name which always returns the exit code 0.
# The logic implemented for "required" workflows accepts if 1) at least one job with that name
# runs through, AND 2) If multiple jobs of that name exist, then all jobs of that name have to
# finish successfully.
on:
push:
paths:
- 'doc/**'
- '**/README.md'
- 'CONTRIBUTING.md'
- "changelog.d/**"
# only top level for these, because various test packages have them too
- "*/ChangeLog.md"
- "*/changelog.md"
- "release-notes/**"
branches:
- master
pull_request:
paths:
- 'doc/**'
- '**/README.md'
- 'CONTRIBUTING.md'
- "changelog.d/**"
- "*/ChangeLog.md"
- "*/changelog.md"
- "release-notes/**"
release:
types:
- created

jobs:
check-api-post-job:
if: always()
name: Check API post job
runs-on: ubuntu-latest
steps:
- run: exit 0
136 changes: 136 additions & 0 deletions .github/workflows/check-api.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
name: Check API

on:
push:
paths-ignore:
- "doc/**"
- "**/README.md"
- "CONTRIBUTING.md"
- "changelog.d/**"
# only top level for these, because various test packages have them too
- "*/ChangeLog.md"
- "*/changelog.md"
- "release-notes/**"
branches:
- master
pull_request:
paths-ignore:
- "doc/**"
- "**/README.md"
- "CONTRIBUTING.md"
- "changelog.d/**"
- "*/ChangeLog.md"
- "*/changelog.md"
- "release-notes/**"
release:
types:
- created
workflow_call:

jobs:
check-api:
name: Check API using ${{ matrix.sys.os }} ghc-${{ matrix.ghc }}
runs-on: ${{ matrix.sys.os }}
strategy:
matrix:
# we check API only on one platform and ghc release, since it shouldn't
# vary elsewhere and the API tracer is sensitive to both
sys:
- { os: ubuntu-latest }
ghc:
[
# print-api only supports a small subset of ghc versions
"9.10.1",
]

steps:

- uses: actions/checkout@v4

- uses: haskell-actions/setup@v2
id: setup-haskell
with:
ghc-version: ${{ matrix.ghc }}
cabal-version: 3.12.1.0 # see https://github.com/haskell/cabal/pull/10251
ghcup-release-channel: https://raw.githubusercontent.com/haskell/ghcup-metadata/master/ghcup-prereleases-0.0.8.yaml

# I was going to use the canned action, but it only supports a single package and reinstalls the same binary each time
- name: Install print-api
run: |
wget -q https://github.com/Kleidukos/print-api/releases/download/v0.1.0.1/print-api-0.1.0.1-Linux-static-${{ matrix.ghc }}-x86_64.tar.gz
tar -xzf print-api-0.1.0.1-Linux-static-${{ matrix.ghc }}-x86_64.tar.gz
mkdir -p "$HOME/.local/bin"
mv print-api "$HOME/.local/bin/print-api"
chmod +x "$HOME/.local/bin/print-api"
echo "$HOME/.local/bin" >> $GITHUB_PATH
# print-api needs environment files. It also doesn't make a lot of sense to use the cached builds, sadly,
# since they're special in different ways (bootstrap and validate) and we want a vanilla build. And there
# isn't enough cache space to make a third cache, even though this is a very limited build.
- name: Build Cabal with environment files
run: cabal build Cabal-syntax Cabal cabal-install-solver --write-ghc-environment-files=always

- name: Generate Cabal-syntax, Cabal, and cabal-install-solver APIs
run: make generate-api

# upload the new API records as artifacts, since there's no guarantee that a contributor could produce
# them (wrong platform or ghc version). This must happen _before_ we check the API, because the
# point is to have them available on API mismatch so they can be updated.
- uses: actions/upload-artifact@v4
with:
name: Cabal-api
path: '*.api'

- name: Check Cabal-syntax, Cabal, and cabal-install-solver APIs
run: |
rc=0
if diff -c Cabal-syntax/Cabal-syntax-${{ matrix.ghc }}.api Cabal-syntax-${{ matrix.ghc }}.api >api.tmp; then
:
else
echo "Cabal-syntax API changed"
if [ $(wc -l < api.tmp) -lt 50 ]; then
cat api.tmp
else
echo Diff too large for GitHub viewer
fi
rc=1
fi
if diff -c Cabal/Cabal-${{ matrix.ghc }}.api Cabal-${{ matrix.ghc }}.api >api.tmp; then
:
else
echo "Cabal API changed"
if [ $(wc -l < api.tmp) -lt 50 ]; then
cat api.tmp
else
echo Diff too large for GitHub viewer
fi
rc=1
fi
if diff -c cabal-install-solver/cabal-install-solver-${{ matrix.ghc }}.api cabal-install-solver-${{ matrix.ghc }}.api >api.tmp; then
:
else
echo "cabal-install-solver API changed"
if [ $(wc -l < api.tmp) -lt 50 ]; then
cat api.tmp
else
echo Diff too large for GitHub viewer
fi
rc=1
fi
if [ $rc -ne 0 ]; then
echo "The new APIs are in the artifact uploaded in the previous step."
exit $rc
fi
# See check-api.skip.yml for why we need this
check-api-post-job:
if: always()
name: Check API post job
runs-on: ubuntu-latest
needs: check-api

steps:
- run: |
echo "jobs info: ${{ toJSON(needs) }}"
- if: contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')
run: exit 1
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -107,3 +107,6 @@ bench.html

# ignore the downloaded binary files
scripts/release/binary-downloads/

# ignore generated API files
/*.api
38 changes: 38 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,44 @@ We use automated whitespace convention checking. Violations can be fixed by
running [fix-whitespace](https://hackage.haskell.org/package/fix-whitespace). If
you push a fix of a whitespace violation, please do so in a _separate commit_.

API Changes and Check API job
-----------------------------

The `Check API` job tests the `Cabal`, `Cabal-syntax`, and `cabal-install-solver`
packages for API changes. It's useful to indicate when a changelog is needed and
which PRs aren't appropriate for backports.

If the `Check API` job fails, you will find in its build artifacts (at the bottom
of the "upload artifacts" step, immediately before the actual API check) a ZIP file
containing the new API records. You can download this and replace the existing API
descriptions, which can be found in the package top level directories, with `.api`
suffixes. Generating them locally is possible with the [check-api tool](https://github.com/Kleidukos/print-api), but
is not guaranteed to produce the same result as the CI job does.

If you do wish to generate a local API record, install [`print-api`](https://github.com/Kleidukos/print-api/releases/tag/v0.1.0.1) and
run it on the `Cabal`, `Cabal-syntax`, and `cabal-install-solver` packages, from
the top level directory of the Cabal repo:

make generate-api

You will need `ghc-9.10.1` to be on `$PATH`; `ghcup` is the easiest way to do this.

The resulting `Cabal-syntax.api`, `Cabal.api`, and `cabal-install-solver` files
can then be compared to the ones in the `Cabal-syntax`, `Cabal`, and
`cabal-install-solver` package directories.

make check-api

If necessary, you can then install the API records:

make update-api

It is also possible to do this individually; see the `Makefile`.

Note that different compiler versions and different architectures will alter the
output. It is not expected that different Linux distributions will, but you may
need to use the static build if you aren't using Ubuntu 22.04.

Other Conventions
-----------------

Expand Down
Loading

0 comments on commit 1f21acd

Please sign in to comment.