Smart Google people worked really hard to get this right. But they used the wrong comparison function --- one that leaked timing information --- and so attackers could forge messages.
Atwood really thinks we should be trying to educate developers about how to build crypto?
This is an incredibly dumb bug -- while Nate claims that "The lesson from this is that crypto flaws can be very subtle", this isn't really a subtle or novel flaw from a cryptographic perspective: Replace "HMAC" with "password", and this attack was published decades ago and is probably the most commonly mentioned side channel attack in the cryptographic literature.
Atwood really thinks we should be trying to educate developers about how to build crypto?
I'm not sure where Atwood comes into this; but I don't see how having non-skilled developers stay away from cryptographic code would have helped here. Yes, this is the sort of mistake which people with a little bit of cryptographic knowledge ("just enough to be dangerous") can make easily -- of course, that's as much an argument for giving them more knowledge as it is for preventing them from having any at all -- but Stephen Weis is not such a person. He did a PhD in cryptography, supervised by Rivest; no amount of saying "crypto is hard, you should use existing libraries instead of writing your own" would have kept him away from making this mistake.
I have found this exact bug (MAC compare timing attack) in three different products over the past few years. I could only talk publicly about this one. The fact that it's so common indicates it hasn't been publicized well enough, especially for crypto implementers.
My recommendation to use libraries is so all engineers can benefit from the review. The more crypto libraries there are, the less reviewed each is. I get all twitchy when I see how many different single sign-on and cookie verification libraries there are out there. There's even duplication between OAUTH and OpenID.
We need libraries to operate at a higher level. We need less of them (proliferation). We need more review of those libraries. The latter two things go hand-in-hand.
As the person who made the coding mistake, I think the lesson drawn from this is the benefit of open sourcing Keyczar. I've seen this exact type of timing attack before, but overlooked it in my own code. There's a real advantage to having fresh eyes find bugs like this.
Thanks to Nate and other people who have been contributing. Keep the bug reports coming.
There is obviously plenty of literature about side channel attacks, particularly timing attacks (which you have contributed to ;-)).
But, among 'lay programmers' I've met, I would say that more than 9/10 have no idea what a 'side channel attack' is. In my experience, its unusual (~ 3/10) for a programmer to know what asymmetric encryption is. I don't expect most programmers to know the particular math behind RSA (let alone ECC/etc), but many don't even the general idea of having a function and its inverse with properties A, B, and C.
I suspect that the majority of people who "use" RSA don't feel like they need to understand the math behind it, even though the math behind it has practical security consequences.
I think that plenty of people feel confident rolling RSA or DH into a protocol design precisely because they do understand the math. The structure of both algorithms is very simple (In the case of DH maybe even trivial), but if you don't understand dozens of (not-very-obvious) attacks against both algorithms you're never going to be able to use them or implement them correctly.
I don't see how your argument coheres. Someone who did a PhD under Rivest made a trivially exploitable crypto error, which is evidence that lay programmers should be implementing crypto too?
No -- it's evidence that having more eyes looking at code is good, and that the "experts" aren't necessarily better than the "non-experts" at avoiding dumb mistakes.
I think there's also an argument to be made that cryptographic code (or security code in general) should be written by people working 9-5 who are forbidden from every doing any overtime. The risk that a security flaw will get in because someone was sleepy far outweighs any benefits of getting code out the door sooner.
I disagree with your first statement. Non-experts will make more trivial errors. But I think we do agree that being an expert does not change the need for third-party review.
While I don't believe Atwood should have anything to do with educating developers about how to build crypto, the fact is that the available libraries are remarkably poor, and as a developer I've been placed in the unfortunate position of "building crypto", with my OpenSSL reference and Applied Cryptography in-hand.
Until we developers have books that better elucidate the pitfalls of implementation, or better libraries that avoid the pitfalls on our behalf, we'll be stuck "building crypto", somewhat blindly.
I've seen you recommend GPGME previously. It's practice of executing the GPG command line tool is just not workable in many implementation environments. In my most recent use-case (several years ago), if there had been a decent portable, liberally licensed stream-supporting implementation of CMS (RFC3852), I would have gladly, readily used it.
> Until we developers have books that better elucidate the pitfalls of implementation, or better libraries that avoid the pitfalls on our behalf, we'll be stuck "building crypto", somewhat blindly.
There are a lot of really good books about cryptography now.
This book is the best one that I've seen for coverage of the pitfalls of implementation.
I missed the context in which tptacek was recommending GPGME, but if your problem is something other than authenticating downloaded packages or signing and encrypting email, designing with GPG as a 'primitive' is probably not such a great idea. I have seen some attempts to repurpose GPG into new applications and every time these protocols have been badly flawed because the authors don't even realize that they are inventing a brand new protocol.
I haven't recommended GPGME, but one of my standard recommendations is, "TLS in motion, GPG at rest". A lot of common problems devolve to "encrypted signed blob" and "encrypted mail".
I agree with you that building a protocol based on GPG is a bad idea. Building a crypto protocol of any sort is a bad idea.
Atwood really thinks we should be trying to educate developers about how to build crypto?