Compatibility between devices (Apple and Microsoft have incompatible encodings)

[hakavlad]

All password inputs must be converted into Unicode Normalization Form KC, NFKC, and then encoded as UTF-8 bytes. This is to ensure compatibility between devices (Apple and Microsoft have incompatible encodings).

https://github.com/covert-encryption/covert/blob/main/docs/Specification.md#argon2-password-hashing

What do you think about it? Why doesn't Kryptor do Normalization?

[samuel-lucas6]

It sounds like that's a sensible idea (it's recommended by NIST), although it's unclear to me how big of an issue this is. It might be problematic with non-English keyboards, which is something I've never tested.

That just never occurred to me. It doesn't seem that widely discussed, and it looks like age and probably lots of programs don't do normalization either.

[hakavlad]

See also:

https://datatracker.ietf.org/doc/html/rfc8265
Preparation, Enforcement, and Comparison of Internationalized Strings Representing Usernames and Passwords
RFC 8265
RFC - Proposed Standard
"It represents the consensus of the IETF community."

   4.  Normalization Rule: Unicode Normalization Form C (NFC) MUST be
       applied to all strings.

It doesn't seem that widely discussed

However, it has been discussed for over 20 years (at least since RFC 3454 and RFC 4013) and is widely recommended by NIST and IETF.

[samuel-lucas6]

However, it has been discussed for over 20 years (at least since RFC 4013) and is widely recommended by NIST and IETF.

Regrettably, I'm not omniscient. There are thousands of standards and documents, each tens or hundreds of pages long. It's completely unrealistic to be aware of or care about all of them.

I've heard of the NIST standard, so I should've at least skim read that (there's lots of irrelevant information). However, I've never heard of those RFCs.

People at NIST didn't originally include that in NIST SP 800-63B, it's not in NIST SP 800-132, it's not mentioned on the OWASP password hashing page, it's not mentioned in the Argon2/scrypt/PBKDF2 RFCs, string.Normalize() is barely mentioned in the dotnet/runtime repo, there's no mention of normalization on the Encoding.GetBytes() documentation, etc. It could clearly be advertised better. It seems to mostly come up if you search 'normalization', which you wouldn't do if you weren't aware of this problem in the first place.

[hakavlad]

I'm not omniscient.

OK. Let's just stick to best practices going forward.

[samuel-lucas6]

I'm not convinced normalization is actually the solution here because it doesn't completely solve the problem. The better approach is probably to restrict the input to ASCII characters, which is I believe what a lot of websites do. Something to think about for the next major version.