Updating cacert with new one from https://curl.se/docs/caextract.html

[XaF]

^ To update the store that hasn't been updated in a while, and that are not containing the new Let's Encrypt root certs
This only updates cacert.pem though as I'm not sure what the cacert1024.pem entails

[cmcculler]

the DST Root CA X3 cert should be removed as well per https://letsencrypt.org/docs/dst-root-ca-x3-expiration-september-2021/

[XaF]

Just took the last update from https://curl.se/docs/caextract.html
I'd prefer avoiding to touch the file content myself, knowing that DST Root CA X3 is expired anyway :)

[jmreid]

🤞

[jamesyale]

+1 for the problem, but manually patching in the proposed cacert.pem into the gem doesn't seem to resolve the certificate verify failed issues for Lets Encrypt certificates, so not sure it works?

[FraDim]

@nahi: Can you please take a look?

[XaF]

+1 for the problem, but manually patching in the proposed cacert.pem into the gem doesn't seem to resolve the certificate verify failed issues for Lets Encrypt certificates, so not sure it works?

This PR has been tested and succeeded in solving the problem for cases of using that library.
A number of other systems hardcoding cacert.pem have been affected though. Long term solution should be not to hardcode those anymore, and use the ones from the system!

[jrochkind]

Long term solution should be not to hardcode those anymore, and use the ones from the system!

As far as this gem, then, as a step in that direction, would it sense to have a PR so it could be configured globally to use system certs vs bundled certs? I think right now it requires a "monkey-patch" to configure this globally in httpclient?

[jamesyale]

Long term solution should be not to hardcode those anymore, and use the ones from the system!

As far as this gem, then, as a step in that direction, would it sense to have a PR so it could be configured globally to use system certs vs bundled certs? I think right now it requires a "monkey-patch" to configure this globally in httpclient?

Monkey patch wise we just worked round this in production using this (livelink@c1b97a7#diff-2a6765023b9beb8295399d5c04286045360ce90aabfffead2dba5c2c3ef71173) - it uses the system CA certs rather than the bundled files, so as long as your system is up to date, it just works ™️

Might be useful to anyone seeing this right now until a proper patch is released :)

[Tensho]

As I see from the latest Mozilla bundle there is no DST Root CA X3 cert in it, because it's essentially expired on 30 Sep 2021. So this bundle should not work for Lets Encrypt intermediate certs, which are cross-signed with DST Root CA X3. 🤔

[XaF]

Cross-signature allows to trust the ISRG cert without having it in your root store. The fact that ISRG is signed by DST becomes irrelevant as soon as you trust ISRG directly. That's why it works at the moment with both those roots. Ultimately, the expire root should be removed from the cacert.pem though, but not a blocker/failure.

[Tensho]

It didn't work for me until I removed expired DST cert from CA bundle provided in this PR. In the first comment @cmcculler mentioned it should be deleted too. Sorry, I understand that theoretically in case of cross-signing the "OR" logic is considered – trust at least one of the signing certs – direct-sign or cross-sign. If one is expired and another not – trust chain is not entirely broken for you (of course if active cert is in place). However, practically I can't grasp how that works for you? Please could you elaborate a little bit more on this? 🙇

[XaF]

If things are done as they should by the client, the chain of trust should be followed until you are able to reach a trusted issuer. That means that the client will look at the first certificate in the chain (initially, the one about your website), and compare the issuer with a list of trusted Certificate Authorities (what we call Root CA). If we don't find a match, we check if any of the Root CA has signed the issuing CA certificate. And we continue doing so until we find a trusted root or reach the end of the trust chain (in which case, we didn't get to a point of trust).

With Let's Encrypt having setup the cross-signing of their new Root CA (ISRG) by the old Root CA (DST), it allowed old browsers and libraries, not up to date, to keep trusting more recently obtained certificate. Let's consider a trust chain: DST signs ISRG signs LE R3 signs WebsiteCert. Now an example validation:

• For old client; prior to Sept 30th, 2021:

Is WebsiteCert in Root CA ? No.
Who signs WebsiteCert ? LE R3
Is LE R3 in Root CA ? No.
Who signs LE R3 ? ISRG
Is ISRG in Root CA ? No.
Who signs ISRG  ? DST
Is DST in Root CA ? Yes, trust found.

• For updated client; prior to Sept 30th, 2021:

Is WebsiteCert in Root CA ? No.
Who signs WebsiteCert ? LE R3
Is LE R3 in Root CA ? No.
Who signs LE R3 ? ISRG
Is ISRG in Root CA ? Yes, trust found.

Now between the announcement of Let's Encrypt and the expiration, they would have expected everyone to have their clients updated, which would have the same effect as the last example above, where we don't need to check ISRG's signatures because we already trust ISRG as a Root CA. It's kind of like not asking your best friend for a reference everytime you meet with them, because you directly trust them.
The problem comes for client that haven't been updated and still hold old Root CA stores, which will end-up in that situation:

• For old client; after Sept 30th, 2021:

Is WebsiteCert in Root CA ? No.
Who signs WebsiteCert ? LE R3
Is LE R3 in Root CA ? No.
Who signs LE R3 ? ISRG
Is ISRG in Root CA ? No.
Who signs ISRG  ? DST
Is DST in Root CA ? Yes, but no more valid.
Unable to reach trust.

Now if you're looking at a valid cert using directly openssl, you'll still see the warnings about the Root CA expiry for certs that still have the DST in their chain, but these are not invalidity issue, just information that the cert in the chain is expired, but in the end not required.

Of course, these are how things should be (and I probably took some shortcuts in explaining), but there might still be clients doing things slightly differently (probably also for old systems) and that would check all the certificates in the list, but that's not what should happen (and even not-so-recent browsers behave better). I've seen instances of old versions of curl doing that though (but recent ones don't seem to).

So this is why, in practice, the fact of having the DST in the Root CA store shouldn't change anything. All that being said, https://curl.se/docs/caextract.html seems to have published a new version of the root cert store a bit after my PR last week, so I'll see tomorrow to update that file to the most recent one!

[jrochkind]

The pertinent question here, I think, is not how any browsers behave, recent or not, but how the HTTPClient code behaves. That seems the only thing relevant to the cert store embedded with HTTPClient, no? Can it connect to letsencrypt certs with the cert store it's using, is all that matters, I think?

[XaF]

And since that PR is working, it's behaving the correct way 😊

[Tensho]

@XaF Thanks, Raphaël, for the detailed explanation. Now I clearly understand what's going on. My case was exactly about client that haven't been updated and still hold old Root CA stores.

After all, the latest Mozilla CA bundle without expired DST feels the best option for all kind of old clients (prior and after 30 Sep). Would be glad to see it in this PR 🙂

BTW, do you know the gem release cadence?

[XaF]

BTW, do you know the gem release cadence?

No idea, unfortunately. 😔