Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Tooling] Release-on-CI automation improvements #23585

Merged
merged 16 commits into from
Sep 19, 2024
Merged

[Tooling] Release-on-CI automation improvements #23585

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
0f820b1
Use release-toolkit backmerge PR action
iangmaia Sep 9, 2024
734e218
Annotate builds with triggered builds
iangmaia Sep 9, 2024
844c4d4
Move PRs to next milestone during code freeze
iangmaia Sep 9, 2024
d7c868e
Improve lanes error handling
iangmaia Sep 9, 2024
4d4e64c
Ensure release branch does not already exist at code freeze
iangmaia Sep 11, 2024
87f0e5d
Prevent manual re-trigger of release pipelines
iangmaia Sep 11, 2024
e41f2b7
Fix Rubocop violations
iangmaia Sep 12, 2024
d9201bd
Update release-toolkit / Dangermattic
iangmaia Sep 11, 2024
7b031df
Add release publishing lane
iangmaia Sep 11, 2024
1f1e5c2
Fix Rubocop violation
iangmaia Sep 12, 2024
6c72417
Add publish-release.yml pipeline
iangmaia Sep 12, 2024
80c9069
Fix typo
iangmaia Sep 16, 2024
355c3d5
Update release-toolkit
iangmaia Sep 16, 2024
e77b97b
[Tooling] Automate Release Publishing (#23592)
iangmaia Sep 16, 2024
b318b70
Apply suggestions from code review
iangmaia Sep 19, 2024
536b1c0
Address a trailing whitespace via `bundle exec rubocop -a`
mokagio Sep 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 10 additions & 2 deletions .buildkite/release-builds.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,18 @@ steps:
command: ".buildkite/commands/release-build-wordpress.sh $BETA_RELEASE"
plugins: [$CI_TOOLKIT_PLUGIN]
notify:
- slack: "#build-and-ship"
- slack: "#build-and-ship"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On seeing this I wondered "How has it worked all this time?!" but it turns out YAML (or at least the Ruby parser and Buildkite's parser) are flexible on array indentation:

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, I was also surprised 🙂

Copy link
Contributor

@AliSoftware AliSoftware Sep 19, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was surprised as well 😅

💡 Tip of the day: If you want to test how buildkite-agent itself would interpret your YAML (and thus be sure to rely on its implementation of YAML parser), you can use buildkite-agent pipeline upload --dry-run to print its interpretation of the pipeline. It prints the interpreted pipeline to stdout—in JSON by default, but you can use --format=yaml if you prefer YAML output that would be closer to the input file you provided.

For example, using a pipe to pass the YML to parse:

# Note: you need to provide the `-i` flag (but not add the `-t` flag!) for `docker run` to be able to read from STDIN through the pipe properly
echo "notify:\n- x" | docker run --rm -i buildkite/agent pipeline upload --dry-run
cat .buildkite/release-builds.yml | docker run --rm -i buildkite/agent pipeline upload --dry-run --format=yaml

Or if you want to pass it a file path to parse:

docker run --rm -v ./:/app -w /app buildkite/agent pipeline upload --dry-run --format=yaml .buildkite/pipeline.yml

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On seeing this I wondered "How has it worked all this time?!" but it turns out YAML (or at least the Ruby parser and Buildkite's parser) are flexible on array indentation:

image

retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false

- label: ":jetpack: :testflight: Jetpack Release Build (App Store Connect)"
command: ".buildkite/commands/release-build-jetpack.sh"
plugins: [$CI_TOOLKIT_PLUGIN]
notify:
- slack: "#build-and-ship"
- slack: "#build-and-ship"
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
4 changes: 4 additions & 0 deletions .buildkite/release-pipelines/code-freeze.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,7 @@ steps:
echo '--- :shipit: Run code freeze'
bundle exec fastlane code_freeze skip_confirm:true
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
12 changes: 10 additions & 2 deletions .buildkite/release-pipelines/complete-code-freeze.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,17 @@ steps:
- label: Complete Code Freeze
key: complete_code_freeze
plugins: [$CI_TOOLKIT_PLUGIN]
command: ".buildkite/commands/complete-code-freeze.sh $RELEASE_VERSION"
command: .buildkite/commands/complete-code-freeze.sh $RELEASE_VERSION
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false

- label: Log Outdated Pods
depends_on: complete_code_freeze
plugins: [$CI_TOOLKIT_PLUGIN]
command: ".buildkite/commands/log-outdated-pods.sh $RELEASE_VERSION"
command: .buildkite/commands/log-outdated-pods.sh $RELEASE_VERSION
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
6 changes: 5 additions & 1 deletion .buildkite/release-pipelines/finalize-hotfix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@ steps:
queue: mac
env:
IMAGE_ID: $IMAGE_ID
command: ".buildkite/commands/finalize-hotfix.sh $VERSION"
command: .buildkite/commands/finalize-hotfix.sh $VERSION
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
6 changes: 5 additions & 1 deletion .buildkite/release-pipelines/finalize-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,8 @@ steps:
queue: mac
env:
IMAGE_ID: $IMAGE_ID
command: ".buildkite/commands/finalize-release.sh $RELEASE_VERSION"
command: .buildkite/commands/finalize-release.sh $RELEASE_VERSION
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
4 changes: 4 additions & 0 deletions .buildkite/release-pipelines/new-beta-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,7 @@ steps:
echo '--- :shipit: Deploy new beta'
bundle exec fastlane new_beta_release skip_confirm:true
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
4 changes: 4 additions & 0 deletions .buildkite/release-pipelines/new-hotfix.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,3 +19,7 @@ steps:
echo '--- :shipit: Start new hotfix'
bundle exec fastlane new_hotfix_release skip_confirm:true version:"$VERSION"
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
31 changes: 31 additions & 0 deletions .buildkite/release-pipelines/publish-release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# yaml-language-server: $schema=https://raw.githubusercontent.com/buildkite/pipeline-schema/main/schema.json
---

agents:
queue: mac

env:
IMAGE_ID: $IMAGE_ID

steps:
- label: Publish Release
plugins: [$CI_TOOLKIT_PLUGIN]
command: |
echo '--- :git: Configure Git for release management'
.buildkite/commands/configure-git-for-release-management.sh
echo '--- :git: Checkout release branch'
.buildkite/commands/checkout-release-branch.sh $RELEASE_VERSION
echo '--- :ruby: Setup Ruby tools'
install_gems
echo '--- :closed_lock_with_key: Access secrets'
bundle exec fastlane run configure_apply
echo '--- :package: Publish Release'
bundle exec fastlane publish_release skip_confirm:true
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
4 changes: 4 additions & 0 deletions .buildkite/release-pipelines/update-app-store-strings.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,7 @@ steps:
echo '--- :shipit: Update relaese notes and other App Store metadata'
bundle exec fastlane update_appstore_strings skip_confirm:true
retry:
manual:
# If those jobs fail, one should always prefer re-triggering a new build from ReleaseV2 rather than retrying the individual job from Buildkite
allowed: false
4 changes: 2 additions & 2 deletions Gemfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
source 'https://rubygems.org'

gem 'cocoapods', '~> 1.14'
gem 'danger-dangermattic', '~> 1.0'
gem 'danger-dangermattic', '~> 1.1'
gem 'dotenv'
# 2.221.0 includes a fix for an ASC-interfacing bug
#
Expand All @@ -15,7 +15,7 @@ gem 'fastlane-plugin-sentry'
# This comment avoids typing to switch to a development version for testing.
#
# gem 'fastlane-plugin-wpmreleasetoolkit', git: 'https://github.com/wordpress-mobile/release-toolkit', ref: ''
gem 'fastlane-plugin-wpmreleasetoolkit', '~> 11.0'
gem 'fastlane-plugin-wpmreleasetoolkit', '~> 12.0'
gem 'rake'
gem 'rubocop', '~> 1.60'
gem 'rubocop-rake', '~> 0.6'
Expand Down
52 changes: 26 additions & 26 deletions Gemfile.lock
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,17 @@ GEM
base64
nkf
rexml
activesupport (7.1.3.4)
activesupport (7.2.1)
base64
bigdecimal
concurrent-ruby (~> 1.0, >= 1.0.2)
concurrent-ruby (~> 1.0, >= 1.3.1)
connection_pool (>= 2.2.5)
drb
i18n (>= 1.6, < 2)
logger (>= 1.4.2)
minitest (>= 5.1)
mutex_m
tzinfo (~> 2.0)
securerandom (>= 0.3)
tzinfo (~> 2.0, >= 2.0.5)
addressable (2.8.7)
public_suffix (>= 2.0.2, < 7.0)
algoliasearch (1.27.5)
Expand All @@ -24,17 +25,17 @@ GEM
ast (2.4.2)
atomos (0.1.3)
aws-eventstream (1.3.0)
aws-partitions (1.962.0)
aws-sdk-core (3.201.3)
aws-partitions (1.975.0)
aws-sdk-core (3.205.0)
aws-eventstream (~> 1, >= 1.3.0)
aws-partitions (~> 1, >= 1.651.0)
aws-sigv4 (~> 1.8)
aws-sigv4 (~> 1.9)
jmespath (~> 1, >= 1.6.1)
aws-sdk-kms (1.88.0)
aws-sdk-core (~> 3, >= 3.201.0)
aws-sdk-kms (1.91.0)
aws-sdk-core (~> 3, >= 3.205.0)
aws-sigv4 (~> 1.5)
aws-sdk-s3 (1.157.0)
aws-sdk-core (~> 3, >= 3.201.0)
aws-sdk-s3 (1.162.0)
aws-sdk-core (~> 3, >= 3.205.0)
aws-sdk-kms (~> 1)
aws-sigv4 (~> 1.5)
aws-sigv4 (1.9.1)
Expand Down Expand Up @@ -91,7 +92,7 @@ GEM
colored2 (3.1.2)
commander (4.6.0)
highline (~> 2.0.0)
concurrent-ruby (1.3.3)
concurrent-ruby (1.3.4)
connection_pool (2.4.1)
cork (0.3.0)
colored2 (~> 3.1)
Expand Down Expand Up @@ -204,7 +205,7 @@ GEM
fastlane-plugin-appcenter (2.1.2)
fastlane-plugin-sentry (1.24.0)
os (~> 1.1, >= 1.1.4)
fastlane-plugin-wpmreleasetoolkit (11.1.0)
fastlane-plugin-wpmreleasetoolkit (12.1.0)
activesupport (>= 6.1.7.1)
buildkit (~> 1.5)
chroma (= 0.2.0)
Expand All @@ -213,7 +214,7 @@ GEM
git (~> 1.3)
google-cloud-storage (~> 1.31)
java-properties (~> 0.3.0)
nokogiri (~> 1.11, < 1.17)
nokogiri (~> 1.11)
octokit (~> 6.1)
parallel (~> 1.14)
plist (~> 3.1)
Expand Down Expand Up @@ -265,29 +266,29 @@ GEM
os (>= 0.9, < 2.0)
signet (>= 0.16, < 2.a)
highline (2.0.3)
http-cookie (1.0.6)
http-cookie (1.0.7)
domain_name (~> 0.5)
httpclient (2.8.3)
i18n (1.14.5)
i18n (1.14.6)
concurrent-ruby (~> 1.0)
java-properties (0.3.0)
jmespath (1.6.2)
json (2.7.2)
jwt (2.8.2)
jwt (2.9.0)
base64
kramdown (2.4.0)
rexml
kramdown-parser-gfm (1.1.0)
kramdown (~> 2.0)
language_server-protocol (3.17.0.3)
logger (1.6.1)
mini_magick (4.13.2)
mini_mime (1.1.5)
mini_portile2 (2.8.7)
minitest (5.24.1)
minitest (5.25.1)
molinillo (0.8.0)
multi_json (1.15.0)
multipart-post (2.4.1)
mutex_m (0.2.0)
nanaimo (0.3.0)
nap (1.1.0)
naturally (2.2.1)
Expand All @@ -303,7 +304,7 @@ GEM
options (2.3.2)
optparse (0.5.0)
os (1.1.4)
parallel (1.25.1)
parallel (1.26.3)
parser (3.3.4.1)
ast (~> 2.4.1)
racc
Expand All @@ -325,8 +326,7 @@ GEM
trailblazer-option (>= 0.1.1, < 0.2.0)
uber (< 0.2.0)
retriable (3.1.2)
rexml (3.3.6)
strscan
rexml (3.3.7)
rmagick (5.3.0)
pkg-config (~> 1.4)
rouge (2.0.7)
Expand All @@ -352,6 +352,7 @@ GEM
sawyer (0.9.2)
addressable (>= 2.3.5)
faraday (>= 0.17.3, < 3)
securerandom (0.3.1)
security (0.1.5)
signet (0.19.0)
addressable (~> 2.8)
Expand All @@ -361,7 +362,6 @@ GEM
simctl (1.6.10)
CFPropertyList
naturally
strscan (3.1.0)
terminal-notifier (2.0.0)
terminal-table (3.0.2)
unicode-display_width (>= 1.1.1, < 3)
Expand All @@ -375,7 +375,7 @@ GEM
tzinfo (2.0.6)
concurrent-ruby (~> 1.0)
uber (0.1.0)
unicode-display_width (2.5.0)
unicode-display_width (2.6.0)
word_wrap (1.0.0)
xcodeproj (1.25.0)
CFPropertyList (>= 2.3.3, < 4.0)
Expand All @@ -394,12 +394,12 @@ PLATFORMS

DEPENDENCIES
cocoapods (~> 1.14)
danger-dangermattic (~> 1.0)
danger-dangermattic (~> 1.1)
dotenv
fastlane (~> 2.221)
fastlane-plugin-appcenter (~> 2.1)
fastlane-plugin-sentry
fastlane-plugin-wpmreleasetoolkit (~> 11.0)
fastlane-plugin-wpmreleasetoolkit (~> 12.0)
rake
rmagick (~> 5.3.0)
rubocop (~> 1.60)
Expand Down
5 changes: 0 additions & 5 deletions fastlane/Fastfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -314,8 +314,3 @@ end
def editorial_branch_name(version: release_version_current)
"release_notes/#{version}"
end

def ensure_git_branch_is_release_branch
# Verify that the current branch is a release branch. Notice that `ensure_git_branch` expects a RegEx parameter
ensure_git_branch(branch: '^release/')
end
2 changes: 1 addition & 1 deletion fastlane/lanes/build.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@
release_version = release_version_current

version = options[:beta_release] ? build_code : release_version
release_url = create_release(
release_url = create_github_release(
repository: GITHUB_REPO,
version: version,
release_notes_file_path: WORDPRESS_RELEASE_NOTES_PATH,
Expand Down
6 changes: 1 addition & 5 deletions fastlane/lanes/localization.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,11 +243,7 @@ def generate_strings_file(gutenberg_path:, derived_data_path:)

push_to_git_remote(tags: false)

pr_url = create_release_management_pull_request(
release_version: release_version,
base_branch: compute_release_branch_name(options: options, version: release_version),
title: "Merge editorialized release notes in #{release_version}"
)
pr_url = create_backmerge_pr

message = <<~MESSAGE
Release notes and metadata localization sources successfully generated. Next, review and merge the [integration PR](#{pr_url}).
Expand Down
Loading