[Tooling] Release-on-CI automation improvements

[iangmaia]

This PR implements a few additions for our release automation on CI:

• Automatically move PRs from the current release milestone to the next one
• Uses the shared create_release_backmerge_pull_request Action from release-toolkit, assigning a milestone to the backmerge PR.
• Make sure release tasks won't fail due to milestones already being closed
• Annotate builds triggering other builds with their URL
• Close hotfix milestone
• Updated release pipelines to not allow manual retry
• Added ensure_branch_does_not_exist in the beginning of code_freeze, to avoid unexpected behavior and errors.
• Removed the :trigger_ testing lanes, as testing can happen either via Rv2 or the Buildkite web UI. (moved to the PR Remove unused release_management_in_ci.rb lanes #23589)

Update:

• Merged [Tooling] Automate Release Publishing #23592 into this PR (without realizing it wasn't for trunk), so it now also contains release publishing.

[wpmobilebot]

📲 You can test the changes from this Pull Request in Jetpack Alpha by scanning the QR code below to install the corresponding build.

	App Name	Jetpack Alpha
	Configuration	Release-Alpha
	Build Number	pr23585-536b1c0
	Version	25.3
	Bundle ID	com.jetpack.alpha
	Commit	536b1c0
	App Center Build	jetpack-installable-builds #9758

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[wpmobilebot]

📲 You can test the changes from this Pull Request in WordPress Alpha by scanning the QR code below to install the corresponding build.

	App Name	WordPress Alpha
	Configuration	Release-Alpha
	Build Number	pr23585-536b1c0
	Version	25.3
	Bundle ID	org.wordpress.alpha
	Commit	536b1c0
	App Center Build	WPiOS - One-Offs #10715

Automatticians: You can use our internal self-serve MC tool to give yourself access to App Center if needed.

[mokagio]

Looking good. However it's always hard to tell withe these, short of running them 😞 Easy to miss a rename or trivial things like that.

None of my suggestions and nitpicks are blocking, so leaving it up to you whether to merge or to apply first.

Probably a conversation for an RFC in P2, but I wonder what you think of my approach from Simplenote Android where:

• create_backmerge_pr is create_backmerge_pr! to account for the UI.user_error! call
• We also have create_backmerge_prs! to differentiate between lanes that expect one backmerge PR only vs lanes that expect more (e.g. finalizing a hotfix)

I think it would be quite unlikely, especially in WordPress which now releases slowly, but if by chance we get a pr_url array instead of a single element, the annotation we then print it in might be confusing.

[mokagio]

On seeing this I wondered "How has it worked all this time?!" but it turns out YAML (or at least the Ruby parser and Buildkite's parser) are flexible on array indentation:

[iangmaia]

Yep, I was also surprised 🙂

[AliSoftware]

Was surprised as well 😅

💡 Tip of the day: If you want to test how buildkite-agent itself would interpret your YAML (and thus be sure to rely on its implementation of YAML parser), you can use buildkite-agent pipeline upload --dry-run to print its interpretation of the pipeline. It prints the interpreted pipeline to stdout—in JSON by default, but you can use --format=yaml if you prefer YAML output that would be closer to the input file you provided.

For example, using a pipe to pass the YML to parse:

# Note: you need to provide the `-i` flag (but not add the `-t` flag!) for `docker run` to be able to read from STDIN through the pipe properly
echo "notify:\n- x" | docker run --rm -i buildkite/agent pipeline upload --dry-run
cat .buildkite/release-builds.yml | docker run --rm -i buildkite/agent pipeline upload --dry-run --format=yaml

Or if you want to pass it a file path to parse:

docker run --rm -v ./:/app -w /app buildkite/agent pipeline upload --dry-run --format=yaml .buildkite/pipeline.yml

[mokagio]

On seeing this I wondered "How has it worked all this time?!" but it turns out YAML (or at least the Ruby parser and Buildkite's parser) are flexible on array indentation: