Enforce post-release activities

[ohltyler]

Currently many post-release activities are not being enforced and subsequently not done by component owners.
The 3 main ones I've seen that are not being done are:

1. Not version bumping the .x branches to the next minor version / developer iteration
2. Not adding component to the manifest of the next minor version / developer iteration
3. Not publishing the github release

When Steps 1 & 2 are not done, it blocks components with dependencies from building the next version, until all of the dependencies have performed Steps 1 & 2. Version bumping is needed for constructing local build environments (like cloning core Dashboards and bootstrapping a Dashboards plugin), and manifest changes are needed to produce builds with the next version that can be consumed (like running AD tests using an OpenSearch cluster with job-scheduler and common-utils plugins).

When Step 3 is not done, it is a bad look for viewers of the plugin, when releases are missing or outdated, or inconsistencies from one plugin to the next.

[dblock]

I believe all these things are being automated, cc: @prudhvigodithi, and it wasn't running on 2.4 because of leftoever monorepos and such - what's left for that?

[ohltyler]

I believe all these things are being automated, cc: @prudhvigodithi, and it wasn't running on 2.4 because of leftoever monorepos and such - what's left for that?

For the most part yes, just a few missing things I've noticed. And the current automation only opens the PRs, but right now there's no enforcement on getting them merged and typically they end up sitting for awhile where they still block folks.

And to add, there is no version bump automation for Dashboards plugins yet. @prudhvigodithi @gaiksaya is there a tracking issue regarding this?

[gaiksaya]

I was not able to find it. Saw this opensearch-project/OpenSearch-Dashboards#1801 which was opened as a proposal. @prudhvigodithi might have more details.

Regarding 3. Not publishing the github release proposing an approach:
Creating tags is an automated process. Hence each repo can have a release.yml github workflow similar to https://github.com/opensearch-project/opensearch-js/blob/main/.github/workflows/release-drafter.yml that can create a release once a tag is pushed.
Let me know what you think. We can create a campaign and PRs should be easy enough.

[ohltyler]

@gaiksaya I think that makes sense for #3, as long as there would be a clean way to fetch the actual release notes for the tag in releases/ dir. Reason being, typically the drafted release notes have some issues showing the particular commits for a release, due to the new branching strategy, and for cherry-picking the commits the component owners want. A fresh look at the release draft workflow might be needed.

[gaiksaya]

I believe there is a mechanism to attach your own release notes instead of asking the GH to generate its own. Example: https://github.com/softprops/action-gh-release#-external-release-notes
We can fetch that when the workflow is run as each repo is expected to have a release_notes.md file. Might be an exception for patch releases in which case, auto generated ones might be the best ones.

[ohltyler]

SGTM! :)

[prudhvigodithi]

Hey @ohltyler yes for OpenSearch the version increment is in place, for OpenSearch Dashboards the task to increment the version is merged, however we need to call the that task in an automated way by integrating it to same workflow that does the version increment for the OpenSearch.
But for all this work in a seamless manner the .x branch should be cut at an early stage for every plugin repo close to same time, so that when the version increment workflow is called for both OpenSearch and OpenSearch Dashboards the PR is open for review and merge, once this is there we can automate to merge the PR after having certain checks and time slots so that the end to end version increment is done, but again for this to happen there could be chances the PR CI checks could fail which has dependency with other plugins that has to be pre built and then consumed by the downstream plugins for the CI checks to pass ( catch 22 situation :) ) .
To avoid over complexity what we can do is once the version increment PR's are open and once we can confident with few data points we can proceed to auto merge, then eventually once the plugins built and pushed the downstream plugins can consume them and CI should pass.
@dblock @bbarani @peterzhuamazon @gaiksaya WDTY?

[dblock]

I think we should continue pushing automation here. I don't know what the next step is, but @prudhvigodithi I believe it's yours!

The catch-22 problem was discussed in #2706. I suggest solving it with an auto-rebase automation for any open version increment PR.

[gaiksaya]

Hi @saratvemulapalli, Looks like you are already taking of automating releases on GitHub as a part of this task? #1898 (comment)

[saratvemulapalli]

Hi @saratvemulapalli, Looks like you are already taking of automating releases on GitHub as a part of this task? #1898 (comment)

Yup I am. Assigned it to myself.

Not publishing the github release

@ohltyler this is automated for OpenSearch. You can replicate it the repo's you care about.
See more details at: opensearch-project/opensearch-plugins#198

[bbarani]

@saratvemulapalli Are you planning to create a campaign to automate releases on GitHub across all repos?

[saratvemulapalli]

@bbarani there has been a campaign: opensearch-project/opensearch-plugins#201
Few plugins have already onboarded.