Skip to content

Commit

Permalink
ci: Split cabal and stack work caches from regular cache
Browse files Browse the repository at this point in the history
This is a first step to split up the cabal and stack caches in separate
pieces. Here we split the work folder, which just contains the
postgrest-specific build artifacts, into a separate cache.

More fine-grained caching should give us better cache hits and much
fewer upload size in the regular case, improving CI performance.

Since the work file caches are very small (about 30-40 MB) they are
cached for PRs, too. This will allow the majority of PRs, which only
change source code files, but no dependencies, to still have cached
their build files for additional commits.
  • Loading branch information
wolfgangwalther committed May 12, 2024
1 parent 80df14c commit d02d6d5
Show file tree
Hide file tree
Showing 3 changed files with 38 additions and 12 deletions.
16 changes: 14 additions & 2 deletions .cirrus.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,20 @@ build_task:
stack_cache:
folders: /.stack
fingerprint_script: cat postgrest.cabal stack.yaml.lock
reupload_on_changes: false
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock

stack_work_cache:
folders: .stack-work
fingerprint_script:
- echo $CIRRUS_OS
- stack --version
- md5sum postgrest.cabal
- md5sum stack.yaml.lock
- find main src -type f -iname '*.hs' -exec md5sum "{}" +

build_script: |
stack build -j 1 --local-bin-path . --copy-bins
Expand Down
10 changes: 7 additions & 3 deletions .github/actions/cache-on-main/action.yaml
Original file line number Diff line number Diff line change
@@ -1,29 +1,33 @@
name: Cache on main

description: Stores caches on main only, but restores them on all branches.
description: Stores caches on main and release branches only, but restores them on all branches.

inputs:
path:
description: Path(s) to cache
required: true
save-prs:
description: Whether to additionally store the cache in a pull request, too. Should only be used for very small caches.
default: false
prefix:
description: Cache key prefix to be used in both primary key and restore-keys.
required: true
suffix:
description: Cache key suffix to be used only in primary key.
required: true

runs:
using: composite
steps:
- uses: actions/cache@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ startsWith(github.ref, 'refs/heads/') }}
if: ${{ startsWith(github.ref, 'refs/heads/') || (inputs.save-prs && github.head_ref) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
restore-keys: |
${{ runner.os }}-${{ inputs.prefix }}-
- uses: actions/cache/restore@0c45773b623bea8c8e75f6c82b208c3cf94ea4f9 # v4.0.2
if: ${{ !startsWith(github.ref, 'refs/heads/') }}
if: ${{ !startsWith(github.ref, 'refs/heads/') && !(inputs.save-prs && github.head_ref) }}
with:
path: ${{ inputs.path }}
key: ${{ runner.os }}-${{ inputs.prefix }}-${{ inputs.suffix }}
Expand Down
24 changes: 17 additions & 7 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -91,35 +91,39 @@ jobs:
runs-on: ubuntu-22.04
cache: |
~/.stack
.stack-work
# no artifact for Linux, because we use the static build

- name: MacOS
runs-on: macos-12
cache: |
~/.stack
.stack-work
artifact: postgrest-macos-x64

- name: Windows
runs-on: windows-2022
cache: |
~\AppData\Roaming\stack
~\AppData\Local\Programs\stack
.stack-work
deps: Add-Content $env:GITHUB_PATH $env:PGBIN
artifact: postgrest-windows-x64

name: Stack - ${{ matrix.name }}
runs-on: ${{ matrix.runs-on }}
steps:
- uses: actions/checkout@44c2b7a8a4ea60a981eaca3cf939b5f4305c123b # v4.1.5
- name: Stack working files cache
- name: Cache ~/.stack
uses: ./.github/actions/cache-on-main
with:
path: ${{ matrix.cache }}
prefix: stack
suffix: ${{ hashFiles('stack.yaml.lock') }}
suffix: ${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
- name: Cache .stack-work
uses: ./.github/actions/cache-on-main
with:
path: .stack-work
save-prs: true
prefix: stack-work-${{ hashFiles('postgrest.cabal', 'stack.yaml.lock') }}
suffix: ${{ hashFiles('main/**/*.hs', 'src/**/*.hs') }}
- name: Install dependencies
if: matrix.deps
run: ${{ matrix.deps }}
Expand Down Expand Up @@ -169,15 +173,21 @@ jobs:
- name: Fix caching
run: |
mkdir ~/.cabal
- name: Cache
- name: Cache .cabal
uses: ./.github/actions/cache-on-main
with:
path: |
~/.cabal/packages
~/.cabal/store
dist-newstyle
prefix: cabal-${{ matrix.ghc }}
suffix: ${{ hashFiles('postgrest.cabal', 'cabal.project', 'cabal.project.freeze') }}
- name: Cache dist-newstyle
uses: ./.github/actions/cache-on-main
with:
path: dist-newstyle
save-prs: true
prefix: cabal-${{ matrix.ghc }}-dist-newstyle-${{ hashFiles('postgrest.cabal', 'cabal.project', 'cabal.project.freeze') }}
suffix: ${{ hashFiles('**/*.hs') }}
- name: Install dependencies
run: |
cabal update
Expand Down

0 comments on commit d02d6d5

Please sign in to comment.