Publish packages to Sonatype

[StepanBrychta]

What does this change?

Remove the Buildkite pipeline and replace it by three GitHub workflows:

• run-tests: Runs when a PR is created/updated
• report-evictions: Runs when a PR is created/updated
• release: Runs only on push to main

run-tests

This action replaces the "Test" group from the original Buildkite pipeline and mirrors its implementation.

report-evictions

This action replaces the "Report evictions" group from the original Buildkite pipeline and is implemented in a similar way. The final eviction report is posted as a comment on the PR (see comment below). If changes are made to the PR, the existing comment is updated as needed.

release

This replace the "cut release" and "Publish" steps from the original Buildkite pipeline. The new action implements several major changes:

• Packages are now published to our public Sonatype repository (instead of a private S3 bucket): https://central.sonatype.com/namespace/org.wellcomecollection
• The existing Python scripts have been refactored to introduce a separation of concerns and improve maintainability. The refactored script (create_release.py) is now only concerned with updating the CHANGELOG.md file and the build.sbt file. Surrounding logic and git commands have been extracted into the parent GitHub action.

How to test

Testing the release action is tricky, but I did some testing on this branch to verify that the publishing process works as expected.

How can we measure success?

All scala-libs packages are successfully published to Sonatype whenever we merge a new version to main.

Have we considered potential risks?

Releasing the packages to a public repository might increase the risks of leaking secrets and other confidential information. However, since this repository is already public and the source code doesn't contain any confidential information, this should not be an issue.

[github-actions]

Suspected binary incompatible evictions across all projects (summary)

• org.scala-lang:scala-library:2.12.17 is selected over {2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.16}
• org.scala-lang:scala-library:2.12.18 is selected over {2.12.15, 2.12.15, 2.12.17, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.15, 2.12.17, 2.12.17, 2.12.16, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.17, 2.12.17, 2.12.17, 2.12.16, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.17, 2.12.17, 2.12.17, 2.12.16, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.18, 2.12.18, 2.12.18, 2.12.8, 2.12.17, 2.12.15, 2.12.15, 2.12.18, 2.12.18, 2.12.18, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.18, 2.12.18, 2.12.18, 2.12.8, 2.12.17, 2.12.15, 2.12.15, 2.12.18, 2.12.18, 2.12.18, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.17, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.12, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.15, 2.12.17, 2.12.17, 2.12.16, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.18, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.18, 2.12.18, 2.12.18, 2.12.18, 2.12.8, 2.12.17, 2.12.15, 2.12.15, 2.12.17, 2.12.17, 2.12.16, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.18, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.15, 2.12.18, 2.12.18, 2.12.18, 2.12.18, 2.12.8, 2.12.17, 2.12.15, 2.12.15, 2.12.17, 2.12.17, 2.12.16, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.15, 2.12.15}
• org.scala-lang:scala-library:2.12.19 is selected over {2.12.15, 2.12.15, 2.12.8, 2.12.15, 2.12.15}
• org.slf4j:slf4j-api:2.0.4 is selected over {1.7.36, 1.7.36, 1.7.9, 1.7.30, 1.7.30, 1.7.30, 1.7.30}
• org.slf4j:slf4j-api:2.0.4 is selected over {1.7.36, 1.7.36, 1.7.9, 1.7.30, 1.7.30, 1.7.30}
• org.slf4j:slf4j-api:2.0.7 is selected over {1.7.9, 1.7.30, 1.7.30, 1.7.30}
• org.slf4j:slf4j-api:2.0.7 is selected over {2.0.4, 1.7.36, 1.7.36, 1.7.9, 1.7.30, 1.7.30, 1.7.30}

See individual evictions stages for more detail

[kenoir]

You might elsewhere (in a different stack) be able to terraform these values on to the GitHub repository so there existence is codified and updated values in GitHub can be done with a terraform apply.

[agnesgaroux]

Here a GHA secret is added at the org level and its access configured. Its value is also created by the tf so we can have it as module.gha_scala_formatting_role.role_arn, which I believe you can't do here?

[kenoir]

This is really fantastic work.

Only comment is it might be good to add some of the context about what's going on to a README somewhere, and mention the Sonatype account setup too.

[kenoir]

Part of wellcomecollection/platform#5783