Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workflow to auto-patch vendored Samba code #768

Merged
merged 2 commits into from
Aug 21, 2023

Conversation

GabrielNagy
Copy link
Contributor

Since our vendored Samba code has a few patches, a good way to ensure we keep it in sync with upstream is to apply an auto-patch workflow where we take the original files and apply a series of patches to get the
files to the final state. This way we ensure that we don't miss any of the changes that happen upstream.

The implementation is self-explanatory for the most part, taking inspiration from our other automated PR workflows. Auto-merging is disabled to give maintainers the opportunity to review (and test) the changes before merging anything in.

This means, in addition to vendoring the "final" versions in ./internal/policies/certificate, we also need to vendor the upstream versions, and the series of patches to apply.

As there's no reliable way to trigger this workflow only on upstream code changes (i.e. webhooks), the next best thing is to have the workflow run on schedule. We don't expect changes to the vendored part of our codebase given that it's been around 5-6 months since the last commits, so running it on a weekly cadence should suffice. I'm also leaving the workflow_dispatch trigger on in case we want to run it on demand.

If the patching fails, the PR body will contain the hunks that failed to apply. In this case, the developer is expected to manually perform the actions of the workflow, updating the patches so they are applicable to the new Samba version.

Fixes UDENG-1113

For each of the vendored and modified Samba files in
`internal/policies/certificate`, add the unmodified versions from the
official Samba repository, alongside the set of patches needed to bring
them to the "final" state.
Since our vendored Samba code has a few patches, a good way to ensure we
keep it in sync with upstream is to apply an auto-patch workflow where
we take the original files and apply a series of patches to get the
files to the final state. This way we ensure that we don't miss any of
the changes that happen upstream.

The implementation is self-explanatory for the most part, taking
inspiration from our other automated PR workflows. Auto-merging is
disabled to give maintainers the opportunity to review (and test) the
changes before merging anything in.

This means, in addition to vendoring the "final" versions in
`./internal/policies/certificate`, we also need to vendor the upstream
versions, and the series of patches to apply.

As there's no reliable way to trigger this workflow only on upstream
code changes (i.e. webhooks), the next best thing is to have the
workflow run on schedule. We don't expect changes to the vendored part
of our codebase given that it's been around 5-6 months since the last
commits, so running it on a weekly cadence should suffice. I'm also
leaving the `workflow_dispatch` trigger on in case we want to run it on
demand.

If the patching fails, the PR body will contain the hunks that failed to
apply. In this case, the developer is expected to manually perform the
actions of the workflow, updating the patches so they are applicable to
the new Samba version.

Fixes UDENG-1113
@codecov-commenter
Copy link

Codecov Report

Merging #768 (ff32d16) into main (3bed352) will not change coverage.
Report is 2 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #768   +/-   ##
=======================================
  Coverage   86.07%   86.07%           
=======================================
  Files          77       77           
  Lines        8552     8552           
=======================================
  Hits         7361     7361           
  Misses        868      868           
  Partials      323      323           

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@GabrielNagy GabrielNagy marked this pull request as ready for review August 16, 2023 08:05
@GabrielNagy GabrielNagy requested a review from a team as a code owner August 16, 2023 08:05
@GabrielNagy
Copy link
Contributor Author

Note that the patch series is still subject to changes regarding commit messages and splitting, but the code changes are final (and what we currently ship in ./internal/policies/certificate).

Copy link
Member

@denisonbarbosa denisonbarbosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Did you test it in a separate repo? I'd like to see how the PR would look like when there's changes to be made.

@GabrielNagy
Copy link
Contributor Author

Yes, sorry should have linked it: GabrielNagy#25 in this case I just deleted the comment lines from .github/samba/python/samba/gp/gp_cert_auto_enroll_ext.py and the PR added them back.

You can cycle through the PR comment edits to see the different states:

  • if the patch is successful the body looks like this:

Automated changes to vendored Samba code - c01c206d7652e5bc5d4134f786522a17729de403

  • if some hunks fail to apply, the body looks like this:

Automated changes to vendored Samba code - c01c206d7652e5bc5d4134f786522a17729de403

Rejected hunks:

--- python/samba/gp/gp_cert_auto_enroll_ext.py
+++ python/samba/gp/gp_cert_auto_enroll_ext.py
@@ -173,8 +173,7 @@ def fetch_template_attrs(ldb, name, attrs=None):
         return {'msPKI-Minimal-Key-Size': ['2048']}

 def format_root_cert(cert)
-    cert = base64.b64encode(cert.encode())
-    return cert_wrap % re.sub(b"(.{64})", b"\\1\n", cert, 0, re.DOTALL)
+    return cert_wrap % re.sub(b"(.{64})", b"\\1\n", cert.encode(), 0, re.DOTALL)
 
 def find_cepces_submit():
     certmonger_dirs = [os.environ.get("PATH"), '/usr/lib/certmonger',
  • if the job fails for another reason, the body looks like this:

Automated changes to vendored Samba code - c01c206d7652e5bc5d4134f786522a17729de403
No rejected hunks, please check job output for failure details: GabrielNagy/adsys/actions/runs/5876067857

Copy link
Member

@denisonbarbosa denisonbarbosa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Thanks for linking the example PR!

@GabrielNagy GabrielNagy merged commit e779bcd into main Aug 21, 2023
4 checks passed
@GabrielNagy GabrielNagy deleted the samba-auto-update-workflow branch August 21, 2023 08:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants