Actions

Actions
Recommendations for multi-job pipeline
Run locally
Contributing
Need Help?
License

Actions

Affected Action

This task is designed for projects in mono repos that are not fully covered by build tools similar to Make, Bazel, or Nx. It helps track the dependency graph and streamline your pipeline by identifying and executing only the steps impacted by recent changes.

Key Features

Dependency Graph Optimization: Generates a JSON object to identify dependencies impacted by changes, allowing you to skip unnecessary steps and focus only on what needs to be executed.
Commit Alignment: Aligns Git commits with images using recommended_imagetags and shas. These hashes represent the state of the dependency graph, based on defined rules, ensuring consistency across your workflow.

Recommendations:

Use changes for pull requests to detect and act upon specific updates.
Use shas for core branches like main, develop, and prod as a key for caching purposes, improving build efficiency.

This approach helps optimize pipelines, reduce execution time, and maintain reliable caching across your development workflow.

jobs:
  init:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0 # fetch all history for accurate change detection
          # If you have multi-job workflow add affected task to an init step to avoid redundant checkouts.
          # If you are using path triggers the diff is limited to 300 files.
          # @see: https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#git-diff-comparisons
          # With this task you can get all the changes.

      - name: calculate affected
        id: affected
        uses: leblancmeneses/actions/apps/affected@main
        with:
          verbose: false # optional
          recommended-imagetags-tag-prefix: '' # optional; The prefix to add to the image tag. target:<prefix>sha1
          recommended-imagetags-tag-suffix: '' # optional; The suffix to add to the image tag. target:sha1<suffix>'
          recommended-imagetags-registry: '' # optional; used in recommended_imagetags.
          recommended-imagetags-tag-truncate-size: 0 # optional; The number of characters to keep from the sha1 value.
          changed-files-output-file: '' # optional; The path to write the file containing the list of changed files.
          rules-file: '' # optional; The path to the file containing the rules if you perfer externalizing the rules for husky integration.
          rules: |
            peggy-parser: 'apps/affected/src/parser.peggy';
            peggy-parser-checkIf-incomplete: peggy-parser AND (!'apps/affected/src/parser.ts' OR !'apps/affected/src/parser.spec.ts');
              # peggy was updated but not the generated parser file or its tests.

            markdown: '**/*.md';

            third-party-deprecated: 'libs/third-party-deprecated/**';
            ui-core: 'libs/ui-core/**';
            ui-libs: ui-core third-party-deprecated;

            <project-ui>: ui-libs 'project-ui/**' EXCEPT (markdown '**/*.spec.ts');
            <project-api>: 'project-api/**' EXCEPT ('**/README.md');
            <project-dbmigrations>: './databases/project/**';

Rule DSL

These rules map a project name and the expression to check for changes and to generate an sha1 hash of the dependency graph.

The left side of the colon : is the rule key, while the right side specifies the expression to match files.
Rule keys with brackets <> will appear in the JSON object under recommended_imagetags or shas, and changes.
Rule keys without brackets will only appear in changes but not in recommended_imagetags or shas.
Glob expressions use picomatch for matching.

Composing Rules

The project-ui rule is composed of ui-libs and project-ui's definition, enabling you to reference and combine multiple expressions. For example, project-ui runs when files change in any of these projects but excludes runs triggered by markdown or test only changes.

Expressions can combine multiple conditions using AND or OR operators. If no operator is specified, OR is used by default.

Literal Expression

Literal expressions are string-based and can be enclosed in single or double quotes. For example:

'file.ts' OR "file.ts"

By default, literal expressions are case-sensitive. To make them case-insensitive, append the i flag:

Example: "readme.md"i will match README.md, readme.md, or rEaDme.mD.

Regex Expression

Regex expressions allow for more flexible matching and are defined using the standard JavaScript regex syntax. For example:

/readme\.md/i

This regex will match README.md, readme.md, or rEaDme.mD. Internally, the expression is converted to a JavaScript RegExp object, ensuring full compatibility with JavaScript’s native regex functionality.

Suffix for Literal and Regex Expressions

By default, all expressions match files regardless of their Git status code. However, you can add a suffix to the expression to filter matches based on specific Git status codes. The suffixes are A for added, M for modified, D for deleted, R for renamed, C for copied, U for unmerged, T for typechange, X for unknown, B for broken.

Usage with Literal Expressions:

Default behavior: 'file.ts' matches files with any Git status code.
With status suffix: 'file.ts':M matches only files with the "modified" status.
Case-insensitive matching: 'file.ts'i:A matches "added" files, ignoring case.

Usage with Regular Expressions:

Default behavior: /readme\.md/ matches files with any Git status code.
With status suffix: /readme\.md/:M matches only "modified" files.
Case-insensitive matching: /readme\.md/i:A matches "added" files, ignoring case.

Key Notes:

Suffix Syntax: Add a colon : followed by the desired status code to filter matches.
Case Insensitivity: Use the i flag before the colon to make the match case-insensitive.

Negate Expression

The ! operator is used to exclude specific files or directories from matching criteria. This ensures that certain files or directories are not modified in a pull request.

Example: !'dir/file.js' ensures that changes to dir/file.js are not allowed in a pull request.

Except Expression

The EXCEPT operator removes files or directories from the expression.

  markdown: '**/*.md';
  <project-ui>: 'project-ui/**' EXCEPT (markdown '**/*.spec.ts');

Wrapping up example

Assuming a changelist contains the following files:

[
  "project-ui/file1.js",
  "project-api/README.md",
]

The affected action will generate the following JSON objects:

{
  "changes": {
    "peggy-parser": false,
    "peggy-parser-checkIf-incomplete": false,
    "markdown": true,
    "project-api": false,
    "project-ui": true,
    "project-dbmigrations": false,
    "third-party-deprecated": false,
    "ui-core": false,
    "ui-libs": false
  },
  "shas": {
    "project-ui": "38aabc2d6ae9866f3c1d601cba956bb935c02cf5",
    "project-api": "dd65064e5d3e4b0a21b867fa02561e37b2cf7f01",
    "project-dbmigrations": "7b367954a3ca29a02e2b570112d85718e56429c9"
  },
  "recommended_imagetags": {
    "project-ui": [
      "project-ui:38aabc2d6ae9866f3c1d601cba956bb935c02cf5",
      "project-ui:pr-6"
    ],
    "project-api": [
      "project-api:dd65064e5d3e4b0a21b867fa02561e37b2cf7f01",
      "project-api:pr-6"
    ],
    "project-dbmigrations": [
      "project-dbmigrations:7b367954a3ca29a02e2b570112d85718e56429c9",
      "project-dbmigrations:pr-6"
    ],
  }
}

Consuming the JSON object

      - name: example affected output
        run: |
          echo "affected: "
          echo '${{ steps.affected.outputs.affected }}' | jq .

          # You can use env values for naming complex expressions.
          HAS_CHANGED_PROJECT_UI=$(echo '${{ steps.affected.outputs.affected }}' | jq -r '.changes["project-ui"]')
          echo "HAS_CHANGED_PROJECT_UI=$HAS_CHANGED_PROJECT_UI" >> $GITHUB_ENV

      - name: ui tests
        if: ${{ !failure() && !cancelled() && fromJson(steps.affected.outputs.affected).changes.project-ui }}
        run: npx nx run project-ui:test

Real world usage

jobs:
  vars:
    uses: ./.github/workflows/template.job.init.yml
    secrets:
      GCP_GITHUB_SERVICE_ACCOUNT: ${{secrets.GCP_GITHUB_SERVICE_ACCOUNT}}

  build-ui:
    needs: [vars, lint-ui, lint-api]
    uses: ./.github/workflows/template.job.build.yml
    if: |
      !failure() && !cancelled() && needs.lint-ui.result != 'failure'
    with:
      ENABLED: ${{fromJson(needs.vars.outputs.affected).changes.app-ui}}
      FORCE_BUILD: ${{ github.event.inputs.MANUAL_FORCE_BUILD == 'true' ||
        fromJson(needs.vars.outputs.pragma).FORCE-BUILD == true }}
      PRE_BUILD_HOOK: .github/_prebuild.app-ui.sh
      DOCKER_FILE: "./app-ui/Dockerfile"
      DOCKER_CONTEXT: "./app-api"
      DOCKER_BUILD_ARGS: "ENV_TYPE=production"
      DOCKER_LABELS: ${{needs.vars.outputs.IMAGE_LABELS}}
      DOCKER_IMAGE_TAGS: ${{ fromJson(needs.vars.outputs.affected).recommended_imagetags.app-ui &&
           toJson(fromJson(needs.vars.outputs.affected).recommended_imagetags.app-ui) || '[]' }}
      CHECKOUT_REF: ${{needs.vars.outputs.CHECKOUT_REF}}
    secrets:
      GCP_GITHUB_SERVICE_ACCOUNT: ${{secrets.GCP_GITHUB_SERVICE_ACCOUNT}}

  # ...

Run locally for Husky integration

After installing Husky in your project, you can integrate the affected action.

Benefits of This Approach

Speed: Only runs checks on changed files, making pre-commit hooks faster.
Efficiency: Avoids running checks on the entire codebase unnecessarily.
Automation: Automatically adds fixed files back to the staging area, streamlining the commit process.

Our rule-based approach standardizes the process to identify which targets have changed, making it adaptable to diverse tech stacks and monorepo structures.

See Husky Example.

# runs cli version of the tool
# https://hub.docker.com/repository/docker/leblancmeneses/actions-affected/general
docker run --rm -v ./:/app -w /app leblancmeneses/actions-affected:v3.0.4-60aac9c calculate --rules-file ./.github/affected.rules > affected.json

Pragma Action

This GitHub Action allows pull requests to change behavior allowing builds to accept [skip,deploy,force] flags.

Features

Pull Request Overrides: Extracts variables from pull request descriptions using a specific pattern (x__key=value).
Key Standardization: Ensures all keys are converted to uppercase to avoid case-sensitivity issues.
Merged Configuration: Combines default variables with overrides, giving precedence to pull request variables.
Flexible Value Types: Automatically converts values to appropriate types (boolean, number, or string).

Inputs

Name	Required	Description
`variables`	Yes	A string containing INI-formatted variables as default values.

Outputs

Name	Description
`pragma`	A JSON object containing the merged configuration variables.

Pull Request Override Usage

Developers can override default variables by adding variables prefixed with x__ to the pull request description. These variables will take precedence over the defaults specified in the variables input. For example:

      - name: calculate pragma
        id: pragma
        uses: leblancmeneses/actions/apps/pragma@main
        with:
          variables: | # INI format to initialize default variables
            lint-appname-ui = ''
            force = false
            deploy = "${{ github.ref == 'refs/heads/dev' || github.ref == 'refs/heads/prod' }}"

Pull request description:

PR description

...

x__lint-appname-ui=skip

Merged Result

The final merged output for this example would be:

{
  "LINT-APPNAME-UI": "skip",
  "FORCE": false,
  "DEPLOY": false
}

Consuming the JSON object

This will override the LINT-APPNAME-UI variable to skip the linting step.

      - name: lint appname-ui
        if: ${{ !failure() && !cancelled() && fromJson(steps.pragma.outputs.pragma).LINT-APPNAME-UI != 'skip' }}
        run: npm run lint:appname-ui

Version Autopilot Action

This is perfect for packages that are not meant to be consumed by other packages, like a website or a mobile app, where semantic versioning is not required and is continuously deployed.

This will automatically increment the version on every run of your github action pipeline.

  - name: calculate version autopilot
    id: version-autopilot
    uses: leblancmeneses/actions/apps/version-autopilot@main
    with:
      major: 0
      minor: 0
      shift: 50 # remove if this is a brand new application. Otherwise, use this to match your current version.

  - name: example in README.md output
    run: |
      echo "github.run_number: ${{ github.run_number }}"

      # useful for container image and package names
      echo "version_autopilot_string_recommended: ${{ steps.version-autopilot.outputs.version_autopilot_string_recommended }}"

      # base to derive your own versioning naming scheme
      echo "version_autopilot_string: ${{ steps.version-autopilot.outputs.version_autopilot_string }}"

      # android and ios version codes
      echo "version_autopilot_code: ${{ steps.version-autopilot.outputs.version_autopilot_code }}"

      # json object with all fields
      echo '${{ steps.version-autopilot.outputs.version_autopilot }}' | jq .

If you have an existing application you can modify the major.minor and shift inputs to match the current version of your application. See our .github/workflows/tests.version-autopilot.yml for how rollover works. We leverage ${{github.run_number}} internally to increment the version.

If you are looking for semantic versioning research git tags and release pipelines.

Real world usage

For Docker image tagging

  - name: myapp containerize and push
    uses: docker/build-push-action@v5
    with:
      platforms: linux/amd64
      push: true
      tags: ${{ env.ARTIFACT_REGISTRY }}/myapp:${{ steps.version-autopilot.outputs.version_autopilot_string_recommended }}
      context: ./apps/myapp
      file: ./apps/myapp/Dockerfile-myapp

For Android APK generation:

  - name: apk generation for PR
    if: github.event_name == 'pull_request'
    run: bash ./gradlew assembleDebug --stacktrace
    env:
      APP_VERSION_CODE: ${{ steps.version-autopilot.outputs.version_autopilot_code }}
      APP_VERSION_STRING: ${{ steps.version-autopilot.outputs.version_autopilot_string_recommended }}
      BASE_URL: https://xyz-${{github.event.number}}-api.<project>.nobackend.io/

For IOS IPA build

  - name: archive and export IPA
    run: |
      xcodebuild \
        -workspace MyApp.xcworkspace \
        -scheme MyApp \
        -configuration Release \
        -destination 'generic/platform=iOS' \
        CURRENT_PROJECT_VERSION=${{ steps.version-autopilot.outputs.version_autopilot_code }} \
        MARKETING_VERSION=${{ steps.version-autopilot.outputs.version_autopilot_string }} \
        PROVISIONING_PROFILE_SPECIFIER=${{ github.ref_name == 'prod' && 'distribution-profile' || 'adhoc-profile' }} \
        -archivePath ./build/MyApp.xcarchive \
        archive | xcpretty --simple --color
      ....

For a chrome extension:

  - name: update manifest version
    run: |
      manifest=tabsift/extension/manifest.json
      jq --arg version "${{ steps.version-autopilot.outputs.version_autopilot_string }}" '.version = $version' $manifest > tmp.json && mv tmp.json $manifest

Recommendations for multi-job pipeline

A single job pipeline is a great starting point for CI/CD workflows. However, as your project evolves, you may need to divide your pipeline into multiple jobs to enhance performance, maintainability, and accommodate different operating systems for various tools.

Create an init job to calculate variables needed across multiple jobs. This will avoid redundant checkouts and calculations across each job.

Generate an init.yml file with the following content:

name: template.job.init

on:
  workflow_call:
    outputs:
      affected:
        value: ${{ jobs.init.outputs.affected }}
      pragma:
        value: ${{ jobs.init.outputs.pragma }}
      version-autopilot:
        value: ${{ jobs.init.outputs.version-autopilot }}

jobs:
  init:
    runs-on: ubuntu-latest
    outputs:
      affected: ${{steps.affected.outputs.affected}}
      pragma: ${{steps.pragma.outputs.pragma}}
      version-autopilot: ${{steps.version-autopilot.outputs.version_autopilot}}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: calculate pragma outputs
        id: pragma
        uses: leblancmeneses/actions/apps/pragma@main
        with:
          variables: |
            ...

      - name: calculate affected outputs
        id: affected
        uses: leblancmeneses/actions/apps/affected@main
        with:
          changed-files-output-file: .artifacts/affected.json
          rules: |
            ...

      - name: upload affected output
        uses: actions/upload-artifact@v4
        with:
          name: affected
          if-no-files-found: ignore
          retention-days: 1
          path: .artifacts/**
          include-hidden-files: true

      - name: calculate version-autopilot outputs
        id: version-autopilot
        uses: leblancmeneses/actions/apps/version-autopilot@main
        with:
          major: 0
          minor: 0
          shift: 0

      # Add more steps or calculations here to validate run.
      # ...

name: build-app-name

on:
  push:
    # ...
  pull_request:
    # ...
  workflow_dispatch:
    # ...

jobs:
  vars:
    uses: ./.github/workflows/template.job.init.yml

  example:
    needs: [vars]
    runs-on: ubuntu-latest
    steps:
      - name: checkout code
        uses: actions/checkout@v4
        with:
          persist-credentials: false

      - name: download affected
        uses: actions/download-artifact@v4
        with:
          name: affected
          path: .artifacts/

      - name: example output
        run: |
          echo "affected: "
          echo '${{ needs.vars.outputs.affected }}' | jq .
          echo "pragma: "
          echo '${{ needs.vars.outputs.pragma }}' | jq .
          echo "version-autopilot: "
          echo '${{ needs.vars.outputs.version-autopilot }}' | jq .

          cat ./.artifacts/affected.json
          for file in $(jq -r '.[] | .file' ./.artifacts/affected.json); do
            echo "processing: $file"
          done

We recommend locking the uses: clause to a specific tag or sha to avoid pipeline breakage due to future changes in the action.

uses: leblancmeneses/actions/apps/<taskname>@main # latest
uses: leblancmeneses/actions/apps/<taskname>@v1.1.1 # specific tag
uses: leblancmeneses/actions/apps/<taskname>@commit-sha # specific sha

Run locally

nvm use
pnpm i
pnpm nx run-many --target=test --parallel

Contributing

Contributions are welcome! Please open an issue or submit a pull request if you have suggestions or improvements.

Need Help?

Large language models (LLMs) cannot solve your organization's people problems. If your software teams are struggling and falling behind, consider engaging an actual human expert who can identify product and development issues and provide solutions.

Common areas where we can assist include DSL development, continuous delivery, cloud migrations, Kubernetes cluster cost optimizations, GitHub Actions and GitHub Codespaces.

Contact us at improvingstartups.com.

License

This project is licensed under the MIT License.

Name		Name	Last commit message	Last commit date
Latest commit History 27 Commits
.github		.github
.husky		.husky
.vscode		.vscode
apps		apps
dist/apps		dist/apps
docs/graphics		docs/graphics
.editorconfig		.editorconfig
.eslintignore		.eslintignore
.eslintrc.json		.eslintrc.json
.gitignore		.gitignore
.nvmrc		.nvmrc
.prettierignore		.prettierignore
.prettierrc		.prettierrc
LICENSE		LICENSE
README.md		README.md
jest.config.ts		jest.config.ts
jest.preset.js		jest.preset.js
nx.json		nx.json
package.json		package.json
pnpm-lock.yaml		pnpm-lock.yaml
tsconfig.base.json		tsconfig.base.json

License

leblancmeneses/actions

Folders and files

Latest commit

History

Repository files navigation

Actions

Affected Action

Key Features

Recommendations:

Rule DSL

Composing Rules

Literal Expression

Regex Expression

Suffix for Literal and Regex Expressions

Usage with Literal Expressions:

Usage with Regular Expressions:

Key Notes:

Negate Expression

Except Expression

Wrapping up example

Consuming the JSON object

Real world usage

Run locally for Husky integration

Benefits of This Approach

Pragma Action

Features

Inputs

Outputs

Pull Request Override Usage

Merged Result

Consuming the JSON object

Version Autopilot Action

Real world usage

Recommendations for multi-job pipeline

Run locally

Contributing

Need Help?

License

About

Topics

Resources

License

Stars

Watchers

Forks

Releases 9

Packages 0

Languages

Packages