Implement GPG signing of the binary build artifacts

[sxa]

First stage in solving adoptium/temurin-build#1275

Will require:

• a change in refactor_openjdk_release_tool for them to be published (Done)
• Changes in the API/website to make them downloadable (Done)
• Likely subsequent changes in the underlying signing job which should be moved into an external script
• Potential changes to where the job is run to avoid deadlocks occurring with the master nodes in jenkins (Hasn't been a problem ... so far!)
• Documentation on how users can use these signatures once they are made available.
• Potentially look at changing this to run from the individual platform build pipelines instead of the top level one - currently the sha256sums are only generated in the top level pipeline so are not available before that point (Except on mac/windows - slack thread) (Done in this PR)
• Look at aligning more with how the OpenJ9 team have done this as they cannot currently reuse the contents of this PR and will be skipping it

But this is the first step and is a prerequisite for the others. Note that this functionality will always be executed and is not currently gated by the enableSigner checkbox on the pipeline jobs. That can be added in a subsequent PR if desired.

This has been successfully trialled on the pipeline jobs.

[github-actions]

Thank you for creating a pull request!

Please check out the information below if you have not made a pull request here before (or if you need a reminder how things work).

Code Quality and Contributing Guidelines

If you have not done so already, please familiarise yourself with our Contributing Guidelines and Code Of Conduct, even if you have contributed before.

Tests

Github actions will run a set of jobs against your PR that will lint and unit test your changes. Keep an eye out for the results from these on the latest commit you submitted. For more information, please see our testing documentation.

In order to run the advanced pipeline tests (executing a set of mock pipelines), I require an admin to post run tests on this PR.
If you are not an admin, please ask for one's attention in #infrastructure on Slack or ping one here.

[karianna]

LGTM - do we need to keep this in draft so we execute the right chain of events?

[sxa]

LGTM - do we need to keep this in draft so we execute the right chain of events?

No, this can safely go in without the other bits being present. It just won't put the artifacts anywhere other than in jenkins at the moment.

[tellison]

Sign the hashcode files - that's unusual isn't it?
I'd only expect a signature for the release artefacts themselves.

[sxa]

Sign the hashcode files - that's unusual isn't it? I'd only expect a signature for the release artefacts themselves.

Not really - here are a couple of examples where the hash which is verified by the signature rather than the main download.

• https://github.com/nodejs/node#verifying-binaries
• https://wiki.eclipse.org/Platform-releng/How_to_check_integrity_of_downloads

The GPG verification process will be more iCPU ntensive than the SHA sum (and our files are quite big, but I appreciate that on modern hardware it likely isn't too much of an overhead) and this way you're getting some guarantee that both are integral without having to clog up the number of artifacts we have available to download by having double the number of GPG signature files.

Obviously either can be technically done (I initially prototyped doing this with the full archive file, but switched over after a recommendation from Eclipse)

[tellison]

Like you, I'm not too concerned about the CPU load for signing the full artefacts. I hadn't come across people generally signing the hash as a proxy for the full artefact, so wondering why that would be preferred.

In my mind, the hash is a statement of fact - the bytes I (Adoptium) have can be described by this succinct and very-likely-to-be-unique value (sha256) and you (end user) can compute the same thing to check that the file you have match mine. Whereas, the signature is an authentication check that you (end user) can verify based upon a secret I (Adoptium) have, to ensure the bytes are the ones I declare to have released.

Signing the hashcode is akin to "authenticating a summary" of the actual artefact, not the artefact itself. Given we can easily verify the binary itself why not do that. Again, in a practical sense I guess there is little difference by this extra layer being introduced, and I've not seen it done before.

I'm reading a few of the best practices and guidelines for code signing, and they don't referer to signing the hash only signing the code. Maybe we can seek input from folks more experienced in this area as I don't want to derail our SSDF initiative.

[sxa]

@tellison FYI The artifacts that get signed aren't affected by this PR. I've adjusted the jenkins job that does the work to pick up all the tar.gz/zip/msi/pkg files instead of the SHAsums and modified this PR to copy back all of the .sig files

[EDIT: Adjusted again to do the signing of the binary artefacts in the platform build pipelines after they've been built]

[gnu-andrew]

The way I've always done it for IcedTea and OpenJDK releases is to sign the binary itself, and then compute the hashsums of both the binary and the signature. The signature is a check that the binary is one published by someone with access to the corresponding private key and the hashsums are a check that both files have downloaded correctly.

However, I can also see that, assuming no hash collisions, the binary and its hashsum are equivalents; only that binary can produce that hashsum. The advantage of signing the binary is there is no obligation to download the hashsum to run signature verification. The advantage of signing the hashsum is in computation time, as mentioned. I have no issue with just leaving it open to picking up signatures for whatever file or files they correspond to.

[sxa]

@AdamBrousseau Does OpenJ9 use the signing options in these pipelines and would therefore be affected by this unless an additional guard was put in place around it?

[andrew-m-leonard]

lgtm

[karianna]

@sxa will let you merge at a good time to monitor it.

[sxa]

@sxa will let you merge at a good time to monitor it.

Yeah I'm holding off pending verification that it can be done without breaking OpenJ9

[sxa]

Confirmed with OpenJ9 that it will impact them but we have a mitigation and I'll aim to come back to this alongside all the other items in the description.