keynote - closing the tls authentication gap

Closing the TLS Authentication Gap

Marsh Ray

Steve Dispensa

PhoneFactor

www.phonefactor.com

Who are we, anyway?

We’re going to tell a story

• Finding the flaw• Deciding how to address it• Private disclosure• Public disclosure• Post-disclosure work• Lessons learned

FINDING THE PROBLEMAugust 11, 2009

So how did this happen?

• It’s Microsoft’s fault!

• Answered a question in a forum…

• Which turned into a series of interesting discussions over the summer about MitM

• Eventually, Marsh got fed up and went spelunking in mod_ssl

/* To allow authentication to complete in this auth hook, the  * solution used here is to fill a (bounded) buffer with the  * request body, and then to reinject that request body later. */

if (renegotiate && !renegotiate_quick && (apr_table_get(r->headers_in, "transfer-encoding") || (apr_table_get(r->headers_in, "content-length") && strcmp(apr_table_get(r->headers_in, "content-length"), "0"))) && !r->expecting_100) {

int rv; /* Fill the I/O buffer with the request body if

possible. */ rv = ssl_io_buffer_fill(r);

...

Apache 2.2 mod_ssl documentation

That can’t be right…

• Buffering and replaying a request seemed… scary

• So, I decided to make sure authentication continuity was maintained across the renegotiation.

Imagine my surprise when I couldn’t find a clear answer.

RFC 5246: Strangely quiet on renegotiation

7.4.1. Hello Messages

... compression algorithms are initialized to null. The current connection state is used for renegotiation messages.

7.4.1.2. Client Hello

When this message will be sent:

When a client first connects to a server, it is required to send the ClientHello as its first message. The client can also send a ClientHello in response to a HelloRequest or on its own initiative in order to renegotiate the security parameters in an existing connection.

So, Marsh disappeared for a couple of weeks, and out came:

Demo!

We immediately understood the significance.

Disclosure without disclosure, on September 8

Three attacks

1. Client certificate-based attack– Client certificates can trigger renegotiation

2. Upgrade attack– Different-strength crypto requirements can lead to

renegotiation

3. Client-initiated attack– In theory, a client could start a renegotiation at any

time

But wait, there’s more!

• Browsers don’t always validate the server cert before handing out the client cert!– Therefore, a client cert can effectively be forwarded

to any server on the net that accepts it

• Browsers don’t always prompt for client certificates when they make can make an “intelligent” choice– Victim never knows what hit him

OK, does it matter?

We struggled to assess scope and impact…

• Client certificate – first finding; mitigation painful

• Upgrade attack – cute, but… meh…

• Client-initiated – almost an afterthought at this point– Not absolutely sure it would even work

Is Renegotiation worth saving?

• Disabling renegotiation completely would be:– Easy– Effective– Solve about 95% of the problem

• Will it ever come back?– IP Source Routing, anyone?

Some uses of TLS renegotiation

Some uses of TLS renegotiation

Wikipedia

Wild speculation!

DoD Common Access Card System

• 3.5M active cards• 1M card readers

Primarily based on client certificates!

Wikipedia

Doesn’t this make you want a National ID card?

We realized:

• Needed a coordinated effort to fix– Huge leak potential

• We wanted near-simultaneous disclosure

• Wanted a solution before the bug leaked

PROJECT MOGULSeptember 14, 2009

Oh, by the way, it’s not this guy:

Disclosure plan

Decided on a phased disclosure plan:

1. Disclose a few respected security gurus

2. Disclose to SSL code owners and start a fix

3. Widen the circle carefully over time

4. Hope for a controlled public disclosure

The NDA

• Everyone told us we were insane to want an NDA, until…– …they heard about the flaw!

• We wanted pressure on vendors

• Intentionally written to expire on 1/31/2010.

"Both the insider and his friend were active members of the hacking group, and regularly attended the organization’s meetings. They used IRC channels to communicate back and forth with one another and relay information under

assumed hacker names in an attempt to mask their identities."

First, we asked Frank Heidt of Leviathan Security,

Who confirmed our intuition about the impact of this vuln:

Extraordinary claims require extraordinary proof

• Ponytail immediately began flapping wildly. Took up smoking again.

• Frank referred us to lots of helpful people, including:– Jon Callas, as an independent security review– Ben Laurie, for obvious reasons– Dan Geer, Kerberos– Jennifer Granick @ EFF

We thought we needed a plan. Turns out, we needed several.

Plan A:• Get code owners together and tell them all at once,

under NDAs

Drawback:– Needed people besides coders

Plan B:• Get programmers and limited support people

to Mountain View and disclose all at once under NDA

Drawbacks:– Vendors needed to know the bug before committing– Vendors needed some time to assess impact in

order to figure out who needed to be involved

Plan C:• Disclose bug to code owners and limited

support personnel under NDA and then go to Mountain View to work out the details

Drawback:– Had trouble getting some companies under NDA

Plan D:• Disclose in advance

– to the fewest people possible (coders, PSIRT managers, …)

– under group NDAs such as ICASI and Google – then get people to Mountain View to work out the

details in a week or two

Disclosure

• All disclosures were completed within about a week

• Disclosed to Ben Laurie– Reproduced across the Internet, tempting the demo gods

• Tried to disclose to IBM: NDA fail

• Disclosed to Microsoft next

ICASI

• Pointed to ICASI by Frank, IBM, and Microsoft

• Disclosed to Steve Manzuik of Juniper/ICASI, leading to:– Microsoft– Intel– Cisco– Juniper– Nokia– IBM

Exceptions

• There were a few notable exceptions:

– Red Hat lawyers worked the weekend!

– Sun : “Type your vuln here and hit submit ok thx bye”

– Apple: We didn’t realize they had their own TLS code

– Others, due to an attempt to limit scope

Mogul meeting: September 28, 2009

• About 45 people representing about a dozen organizations

• Description and captures, again

• Severity and impact– Lots of time spent on client-initiated renegotiation

• Solution discussion– Rescorla, Oskov, and Dispensa/Ray had identical proposals

Proposed solution

The obvious solution was to bind the cryptographic state from the previous handshake to the current one

This is easy:• Resend the verify_data from the previous Finished

message– Already cryptographically secure– Already under consideration as a “channel binding”

• Not a perfect solution, however:– Requires a TLS extension– Requires additional storage (bad for silicon?)

Post-conference work

Turns out it’s hard to organize a private, cross-vendor, ad-hoc team!

• Manzuik requisitioned help from Paul Vixie / OpSecTrust

• Manzuik set up [mogul-private] and a PGP keyring• We set up a private SILC channel

• Good initial discussion on the lists, but vendor engagement dropped off quickly– No data!

Initial implementations of safe renegotiation

• Nasko Oskov from Microsoft had a working implementation quickly

• Eric Rescorla provided code for OpenSSL

• Dispensa worked up a patch for GNUTLS

• We suspect others were making progress

TLA

[this page intentionally left blank.]

Timeline tension

• Work was going really slowly

• January 31 “couldn’t possibly work”– Not a Patch Tuesday– Not a weekday– BlackHat / ShmooCon

• By late October, Steve was on repeated ICASI calls– Insisting that we postpone publishing

• Our position was unchanged officially– Meanwhile, Marsh and Steve started to argue about scenarios

PUBLIC DISCLOSURENovember 4, 2009

So, we were all minding our own business, when…

To: tls at ietf.orgSubject: [TLS] MITM attack on delayed TLS-client auth through renegotiationFrom: Martin Rex <Martin.Rex at sap.com>Date: Wed, 4 Nov 2009 18:28:00 +0100 (MET)

After elaborating so much about the client cert authentication through renegotiation with Microsoft IIS, I'm beginning to believe that there is a potential security problem with that scheme, because it is susceptible to a MITM attack.

...

[TLS] turns into Full-Disclosure

Hilarity ensues

• Dispensa calls Martin Rex

• Project Mogul is notified in a very tense call– Vendors Hope It Goes Away™

• Vendors end by insisting that it would be “extremely irresponsible” to publish– After all, “nobody will notice.”

So, did anyone notice?

Three hours after Martin Rex’s e-mail…

…it was re-tweeted a few times, too.

Steve gets bored and decides to do something else.

Any guesses as to how long it took for working exploit code to be posted to [full-disclosure]?

18 hours. 34 minutes.

Initial reactions were… mixed.

• “The sky is not falling” –Moxie Marlinspike• It’s just like CSRF! (Whew! … Whew?)

“Most, if not all, major web applications have implementation level protections against CSRF… Those protection measures are effective against this new SSL man in the middle attack. Therefore, this vulnerability has minimal security impact for most websites and Internet users.” –Tom Cross, IBM ISS

A couple of days later…

Coming to terms with the bug

• Yeah, but who cares?– The 41% of users who use the same password for

Twitter as they use for… everything!

• More importantly - you can’t tell what will be broken

• In the end, the confusion was our fault

POST-DISCLOSURE WORK

IETF

• ID ready day 1

• Flawed: undercounted SSLv3– No extensions!– Post-disclosure, Benn Bollay from F5 shared data: 22%!

• Tons of e-mails on the IETF list– Practically a full-time job for Marsh

• Finally, we added SCSV to address old servers

• RFC 5746 very soon!

Patch status

• Several vendors have disabled renegotiation

• A few vendors and projects have implemented the new RFC

• www.phonefactor.com/sslgap

So far, one commercial vendor has shipped:

OPERA FTW!!!

Not everyone has rolled out a fix

SOME LESSONS LEARNED

Security bugs are a no-win situation

• Traumatic for vendors• Not great for researchers• Worst, of course, for the users

• This was a really hard process – hard to balance lots of competing interests

There are other (bigger?) problems with SSL

• PKI is great in theory, but:– ~200 trusted root certificates in Firefox – do you

trust them all?

• There will never be a solution to the dancing bunnies problem– Applies to Business Bunnies too!

• Sometimes, root CA’s do this:

We needed hard data

• We had no success getting vendors to contribute data

• Would have been extremely helpful to know about SSLv3 prevalence before the IETF process

• Does client-initiated renegotiation ever happen?

IETF security process?

It has been suggested that the IETF security review process is broken.

If it is, this bug isn’t why:• SSL was a Netscape creation• SSLv3 was utterly ownerless for years• The IETF did find it, a few months after us

The IETF could have done a better job adopting SSL

One last lesson

This one goes out to the slow/no-disclosure crowd with our compliments:

So Marsh emailed Pavel Kankovsky and:

"I had some free time during the last days of 2006 and wrote the PoC exploit to carry out an experimental verification of the vulnerability. It was easier than I had expected because I found a clever way to make OpenSSL cooperate.

“The exploit was finished on January 3, 2007."

One last question for you

Did we achieve our goal of minimizing the world’s exposure to the bug?

Questions?

marsh@extendedsubset.comdispensa@phonefactor.com

www.phonefactor.com

mogul on silc.hick.orgmogul-open@lists.links.org