Caddy 2.6.4 Signature Verification (Mac arm64) throws certificate expired

[lantrix]

Following the asset verification documentation and using the latest (non-beta) release for mac_arm64 on release v2.6.4 does not work, although it's now the way you've (excellently!) implemented binary verification:

As of Caddy v2.6.0, CI/CD release artifacts are signed using project Sigstore technology, which issues certificates containing details about the subject to whom the certificate is issued.

Initially the latest cosign v2 (I tried v2.1.1) will not verify with cosign verify-blob due to sigstore/cosign#2632, the same as mentioned in sigstore/cosign#2632 (comment)

Switching to installation of the v1 cosign cli the Mac arm64 release throws the below:

go install github.com/sigstore/cosign/cmd/cosign@latest
cosign version
  ______   ______        _______. __    _______ .__   __.
 /      | /  __  \      /       ||  |  /  _____||  \ |  |
|  ,----'|  |  |  |    |   (----`|  | |  |  __  |   \|  |
|  |     |  |  |  |     \   \    |  | |  | |_ | |  . `  |
|  `----.|  `--'  | .----)   |   |  | |  |__| | |  |\   |
 \______| \______/  |_______/    |__|  \______| |__| \__|
cosign: A tool for Container Signing, Verification and Storage in an OCI registry.

GitVersion:    v1.13.1
GitCommit:     unknown
GitTreeState:  unknown
BuildDate:     unknown
GoVersion:     go1.20.5
Compiler:      gc
Platform:      darwin/arm64

cosign verify-blob --certificate caddy_2.6.4_mac_arm64.pem --signature caddy_2.6.4_mac_arm64.tar.gz.sig caddy_2.6.4_mac_arm64.tar.gz
Error: verifying blob [caddy_2.6.4_mac_arm64.tar.gz]: certificate expired before signatures were entered in log: 2023-02-14T19:53:54Z is before 2023-07-31T01:30:30+10:00
main.go:62: error during command execution: verifying blob [caddy_2.6.4_mac_arm64.tar.gz]: certificate expired before signatures were entered in log: 2023-02-14T19:53:54Z is before 2023-07-31T01:30:30+10:00

I tried the latest beta for the same arch/platform and same error:

cosign verify-blob --certificate ./caddy_2.7.0-beta.2_mac_arm64.pem --signature caddy_2.7.0-beta.2_mac_arm64.tar.gz.sig caddy_2.7.0-beta.2_mac_arm64.tar.gz
Error: verifying blob [caddy_2.7.0-beta.2_mac_arm64.tar.gz]: certificate expired before signatures were entered in log: 2023-06-22T00:51:24Z is before 2023-07-31T01:26:32+10:00
main.go:62: error during command execution: verifying blob [caddy_2.7.0-beta.2_mac_arm64.tar.gz]: certificate expired before signatures were entered in log: 2023-06-22T00:51:24Z is before 2023-07-31T01:26:32+10:00

In essence I'm not confident in your Mac arm64 binary signatures according to your own verification documentation.

certificate expired before signatures were entered in log: 2023-02-14T19:53:54Z is before 2023-07-31T01:30:30+10:00

[mohammed90]

I'll look into it, but this

certificate expired before signatures were entered in log: 2023-02-14T19:53:54Z is before 2023-07-31T01:30:30+10:00

doesn't make sense. The signature log should be on the 14th of February. It can't be in the future. The future hasn't happened yet.

I'd check the certificate expiry and the Rekor entry for the asset. If you can't do that, I'll check it later tonight.

[mohammed90]

I'm completely not able to reproduce it. Here's my reproducible script:

wget https://github.com/caddyserver/caddy/releases/download/v2.6.4/caddy_2.6.4_mac_arm64.tar.gz
wget https://github.com/caddyserver/caddy/releases/download/v2.6.4/caddy_2.6.4_mac_arm64.tar.gz.sig
wget https://github.com/caddyserver/caddy/releases/download/v2.6.4/caddy_2.6.4_mac_arm64.pem
base64 -d < caddy_2.6.4_mac_arm64.pem > cert.pem

cosign verify-blob \
--certificate-oidc-issuer https://token.actions.githubusercontent.com \
--certificate-github-workflow-name "Release" \
--certificate-github-workflow-ref refs/tags/v2.6.4 \
--certificate-identity-regexp caddyserver/caddy \
--certificate ./cert.pem \
--signature ./caddy_2.6.4_mac_arm64.tar.gz.sig \
--verbose \
./caddy_2.6.4_mac_arm64.tar.gz

For the asset verification point, that's on me. I promised to update the page in caddyserver/website#312. I'll do this promptly.

[lantrix]

I'll look into it, but this

certificate expired before signatures were entered in log: 2023-02-14T19:53:54Z is before 2023-07-31T01:30:30+10:00

doesn't make sense. The signature log should be on the 14th of February. It can't be in the future. The future hasn't happened yet.

I'd check the certificate expiry and the Rekor entry for the asset. If you can't do that, I'll check it later tonight.

Yeah that's strange. The 2023-07-31T01:30:30+10:00 is the time when I ran the command (hence the +10:00 as my TZ). But 2023-02-14T19:53:54Z is in the past and lines up with the release date:

github-actions released this Feb 15

I'm not sure where the future part is coming from.

Also the asset verification docs notes something totally different to what you ran, specifically noting

We run the following command (notice it uses the undecoded cert)

You're using the decoded cert. Thanks for noting you have an outstanding doc update 🎉
When I run the verify-blob command you've provided with the v2 cosign CLI, I get OK verification.

[mohammed90]

Also the asset verification docs notes something totally different to what you ran,

The cosign CLI changed between cosign v1 (when I wrote the docs) and their v2.

specifically noting\n\nWe run the following command (notice it uses the undecoded cert)\n\n

You're using the decoded cert.

Nope, that cert is in base64. It's encoded in base64, thus _un_decoded.

Thanks for noting you have an outstanding doc update 🎉
When I run the verify-blob command you've provided with the v2 cosign CLI, I get OK verification.

I'm glad it's working fine for you now! Thanks