gh-84559: multiprocessing: detect if forkserver cannot work due to missing hmac-sha256

[gpshead]

Default to the spawn start method in that scenario.

Mostly I want to see if this meaningfully lets the odd configuration FIPS buildbot make further progress or not.

This PR is now stacked on top of #127492.

• Craft a unittest for this scenario. Probably by mocking out hmac.new and raising ValueError in a child process before the multiprocessing.context import.
• merge the parent PR gh-127298: Refactor test_hashlib for better usedforsecurity & openssl fips mode env support. #127492, merge main & unstack.

• Issue: multiprocessing's default posix start method of 'fork' is broken: change to `'forkserver' || 'spawn' #84559

[gpshead]

!buildbot FIPS

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 7474d73 🤖

The command will test the builders whose names match following regular expression: FIPS

The builders matched are:

• AMD64 CentOS9 FIPS Only Blake2 Builtin Hash PR
• AMD64 RHEL8 FIPS No Builtin Hashes PR
• AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
• AMD64 CentOS9 FIPS No Builtin Hashes PR

[gpshead]

!buildbot FIPS

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @gpshead for commit 740cb40 🤖

The command will test the builders whose names match following regular expression: FIPS

The builders matched are:

• AMD64 CentOS9 FIPS Only Blake2 Builtin Hash PR
• AMD64 RHEL8 FIPS No Builtin Hashes PR
• AMD64 RHEL8 FIPS Only Blake2 Builtin Hash PR
• AMD64 CentOS9 FIPS No Builtin Hashes PR

[xnox]

On hosts without the hmac-sha256 algorithm available (rare)

Is this artificially created host, or something realistic? Without sha256 there would not be any realistic TLS working. And HMAC construction should also be always available.

Would it be a host with tampered/missing openssl & no built-in fallbacks? But then like nothing else is working either (hashlib, ssl)

I'm trying to understand if this is really more "support multiprocessing without openssl & without fallback crypto algorithms" as in no cryptography whats-so-ever, rather than "missing sha256" which I would have thought is a broken, and hopefully impossible, situation.

[gpshead]

This is a with a dumb "openssl fips mode" config that blocks all ciphers, running a python also configured not to have a fallback builtin sha256 module. configure --without-builtin-hashlib-hashes, and set the OPENSSL_CONF= environment variable to point to a file like https://github.com/python/cpython/blob/905ba7f06c12f7ef9985ccaa2bf24229f759a2e1/Lib/test/hashlibdata/openssl.cnf and you can repro this kind of setup.

this is primarily a "get the test suite to work in this situation" as we have a buildbot running with that type of config to better understand odd config failure modes.

i don't expect anyone to actually run their system with such a borked config. I'm not spending a lot of time on this, but I do consider it a "nice to have" for the redistributors who make strange "fips mode" envrionments for customers. they should wind up with less pain this way. Even though it isn't how any CPython binary release we ship would ever be configured. ie, it should unblock #127298.

[gpshead]

IIRC, anything involving a (non-fork start method?) multiprocessing Connection will fail without the relevant hmac auth algorithm. But not all multiprocessing features rely on those. I do not intend to document what does and doesn't work in this situation - we don't make any guarantees here.

[xnox]

This is a with a dumb "openssl fips mode" config that blocks all ciphers, running a python also configured not to have a fallback builtin sha256 module. configure --without-builtin-hashlib-hashes, and set the OPENSSL_CONF= environment variable to point to a file like https://github.com/python/cpython/blob/905ba7f06c12f7ef9985ccaa2bf24229f759a2e1/Lib/test/hashlibdata/openssl.cnf and you can repro this kind of setup.

this is primarily a "get the test suite to work in this situation" as we have a buildbot running with that type of config to better understand odd config failure modes.

i don't expect anyone to actually run their system with such a borked config. I'm not spending a lot of time on this, but I do consider it a "nice to have" for the redistributors who make strange "fips mode" envrionments for customers. they should wind up with less pain this way. Even though it isn't how any CPython binary release we ship would ever be configured. ie, it should unblock #127298.

Ack, whilst SHA-2 is still valid for post-2035 as per https://csrc.nist.gov/pubs/ir/8547/ipd it could be the case that people move on to just SHA-3 & SHAKE, at which point SHA-2 hmac may become irrelevant in favour of KMAC-256. And these changes will make things limp along.

For pre-2035 horizon, the relevant changes that we do need from this pr is the test/ changes and the multiprocessing is nice-to-have (or borderline theoretical / making it easier to reason about by using absolutely crypto-less config AKA zero-knowledge as to what future holds).

So can this be marked not-draft? what is missing to land the testsuite changes of this PR, or even this whole PR to unblock #127298?

[xnox]

Oh I see this PR is in draft, because it is based on #127492 and that one needs to land first, which indeed has the required perquisites. Ok I am catching up to all the things now.