Display and use SHA256 hashes

[daveschaefer]

Currently Perspectives uses MD5 certificate hashes. We should upgrade to SHA1, and at least give the option of displaying the SHA1 hashes. This will need to be rolled out in stages:

• Client is able to read MD5 and SHA1 and handle either
• Servers begin storing SHA1 hashes

[danwent]

If we are going to handle multiple hashes, I'd also suggest including a more secure hash, like SHA-256, if we can get the corresponding hashes within the browser (seems likely)

[daveschaefer]

Good idea - it may be neat to be able to view many hashes.

My thoughts for SHA1 were that it tends to be the default hash shown by Firefox, SeaMonkey, and Chrome when you click 'View Certificate', so it would be easy for people to compare against to make sure Perspectives is doing the right thing. In Firefox and SeaMonkey MD5 and SHA1 are also both easily retrievable by calling one function from the nsISSLStatusProvider interface, the place we currently use the get the certificate fingerprint in the client. i.e. you can call cert.md5Fingerprint or cert.sha1Fingerprint.

For firefox it looks like we could compute a fingerprint with multiple different hashes using the nsICryptoHash interface - https://developer.mozilla.org/en-US/docs/XPCOM_Interface_Reference/nsICryptoHash

I'm not sure if that would be computationally expensive.

[bencomp]

In a comment in danwent/Perspectives#139, @d7415 notes

At a glance, it seems the server can be changed by simply replacing all occurrences of "md5" with e.g. "sha256". My preliminary tests shows that it runs and generates the hashes like this.

Without having looked at the project architecture or code very closely, could storing all of md5, sha1, sha256 and sha512 fingerprints work?

[daveschaefer]

Hey @bencomp - yes, we will want all of the notaries to store hashes for each hashing algorithm we support. For example, they might store the MD5, SHA1, and SHA256 hashes for each site they scan.

[d7415]

Ok. Is anyone working on this? If not I might give it a go. A start, anyway.

Would we like it to return all hashes, or one as requested? My gut feeling would be to store all but only return one requested by the client. This way the default could still be MD5, but when the client upgrades its default can be changed as required. It wouldn't help with new clients and old servers, but we can't please everyone.

Thoughts?

Update: Looked at the actual code, and it seems the client should just ignore if given the wrong hash type. I'm not convinced it does after my little test, and I'm not well versed in extensions, but I think it should. Same for mismatched version.

So if we're working with multiple hash types, we can either change the request and keep the existing v1 response or we can change to a v2 response and supply them all.

In theory we could keep the same basic format, supplying the existing fields (including type) for MD5 and add new "sha256_timestamp" etc, but that would mean changing the response format without incrementing the version, which would be nasty.

[daveschaefer]

Hey @d7415 , thanks for the interest.

I would agree that notaries should store multiple hashes but only return one requested by the client. For backwards compatibility that probably means returning md5 hashes unless an additional parameter is sent. This way we can upgrade the servers first, have the collect data, upgrade the clients to start using the new hashes, and then eventually discontinue the MD5 code altogether.

As you noted in client ticket 139 the actual code changes on the server to collect new hashes is not hard at all - we can simply change the call from hashlib.md5() to hashlib.sha256(). The part that is more work, however, is restructuring the database to store this extra information.

I created ticket #30 for the work of updating the database schema, moving the data, and adding a place to store new hashes. I have local draft code that works, but it needs to be cleaned up before it can be sent as a pull request.

Upgrading the hashes is very high on my priority list. Perhaps it would help if I updated the Perspectives roadmap to show the current priorities. Give me a few hours and I will create a page for it here.

[daveschaefer]

Since SHA1 hashes are being sunset and considered more and more insecure, I vote we jump over them entirely and upgrade straight to SHA256. For now, without any performance tests, I vote that we simply add a single new type of hash to the server - SHA256 - and leave SHA512 for later (perhaps once MD5 is no longer being used at all).

Firefox 32 now shows the SHA256 hashes of certificates when you view certificate information, which is great, so it should be easy for people to view and verify the Perspectives SHA256 hash and the browser SHA256 hash side by side.

[daveschaefer]

I have updated this ticket title to reflect using SHA256 hashes rather than SHA1.

[daveschaefer]

@d7415 I have created a page on the Perspectives wiki that outlines the current priority of tasks - https://github.com/danwent/Perspectives/wiki/Perspectives-Roadmap . Comments are welcome. Hopefully that gives a bit of an idea about what we are planning