Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Running CI/CD on org-runners. Refactor for GH workflow #2193

Merged
merged 26 commits into from
Oct 23, 2023

Conversation

jcortejoso
Copy link
Member

@jcortejoso jcortejoso commented Oct 10, 2023

Description

Run the workflow on org-runners instead of repo-runners. This help on faster scale and easier management. This requires specifying the container image at job level.

Advantages:

  • The workflow is less dependent (or non-dependent) on dependencies installed in the runner -> We can even switch to GitHub's runner and it should work (slower)
  • We can share resources (runners) across projects so it can scale faster and in general be more resilient.

Disadvantages:

  • The jobs need to pull an extra image (the runner-base image) which may add a few seconds to the execution time (it seems to me from other repositories that it'd be ~5-8 secs).

About the git config --global --add safe.directory '*' step: we're disabling a git warning when the user (uid) is not the owner of the files. The reason to this to happen is because when using the job.*.container field together with actions-runner-controller, the runner (pod) starts first with a generic image, authorizes with GitHub and fetches the files, and then starts the container inside this runner pod with a command like (you can check the command on the step Initialize containers on any job):

/usr/local/bin/docker create --name 16dc96ff1e654dd88a85f09e6552117f_uswest1dockerpkgdevdevopsreactionsrunnercontrollerblockchainlatest_77a975 --label 60e226 --workdir /__w/celo-blockchain/celo-blockchain --network github_network_f91380036362482db6f2676f3cae45ae --user root -e "HOME=/github/home" -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/runner/_work":"/__w" -v "/runner/externals":"/__e":ro -v "/runner/_work/_temp":"/__w/_temp" -v "/runner/_work/_actions":"/__w/_actions" -v "/opt/hostedtoolcache":"/__t" -v "/runner/_work/_temp/_github_home":"/github/home" -v "/runner/_work/_temp/_github_workflow":"/github/workflow" --entrypoint "tail" us-west1-docker.pkg.dev/devopsre/actions-runner-controller/blockchain:latest "-f" "/dev/null"

The workdir for the steps are inside a shared folder between the runner and the container, and because there is a mismatch between the uid owner of the folder in the runner and the user in the container (run as root, uid=1), this cause that we need to disable the warning from git about ownership of files. Personally I don't see any impact of doing this, as the workspace is ephemeral and there is not risk of breaking the permissions of the files for future usages or other users.
More info.

@github-actions
Copy link

github-actions bot commented Oct 10, 2023

Coverage from tests in ./e2e_test/... for ./consensus/istanbul/... at commit 4e9521d

coverage: 45.8% of statements across all listed packages
coverage:  57.2% of statements in consensus/istanbul
coverage:  23.7% of statements in consensus/istanbul/announce
coverage:  54.3% of statements in consensus/istanbul/backend
coverage:   0.0% of statements in consensus/istanbul/backend/backendtest
coverage:  24.3% of statements in consensus/istanbul/backend/internal/replica
coverage:  61.1% of statements in consensus/istanbul/core
coverage:  45.0% of statements in consensus/istanbul/db
coverage:   0.0% of statements in consensus/istanbul/proxy
coverage:  64.2% of statements in consensus/istanbul/uptime
coverage:  51.8% of statements in consensus/istanbul/validator
coverage:  79.2% of statements in consensus/istanbul/validator/random

@github-actions
Copy link

github-actions bot commented Oct 10, 2023

5871 passed, 45 skipped

@codecov
Copy link

codecov bot commented Oct 10, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (5c44f5d) 55.16% compared to head (9731a5a) 55.10%.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #2193      +/-   ##
==========================================
- Coverage   55.16%   55.10%   -0.07%     
==========================================
  Files         682      682              
  Lines      114447   114447              
==========================================
- Hits        63136    63062      -74     
- Misses      47432    47492      +60     
- Partials     3879     3893      +14     

see 25 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jcortejoso jcortejoso changed the title Running CI/CD on org-runners. Small refactor for GH workflow Running CI/CD on org-runners. Refactor for GH workflow Oct 11, 2023
Copy link

@rkachowski rkachowski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the explanation in the PR description, it really outlines why + how things work :)

@jcortejoso jcortejoso merged commit 4e9521d into master Oct 23, 2023
27 of 28 checks passed
@jcortejoso jcortejoso deleted the jcortejoso/org-runners branch October 23, 2023 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants