"2fac: facebook's internal multi-factor authentication". tim tickel, chad greene,...

Tuesday, October 1, 13

2FAC: Facebook’s internal multi-factor auth platform

C O N F I D E N T I A L

Facebook Security


Agenda

Attacks - A Force for Change

2FAC Authentication

Questions?


Facebook - Big Numbers

1.15B monthly active users

699M daily active users (80+% outside US)

5K+ employees


Identifying weakest points

Red Teams

Incident 1: Spear phishing OWA

Incident 2: Breach identified in January


Red Team Drills - Identify weak points


Incident: Spear Phishing OWA


Incident: Breach discovered in Jan 2013

digitalinsight-ltd


Incident: Breach discovered in Jan 2013


Goal: Protect against remote attackers•Disrupt Lateral Movement phase

•Ensure local user is at keyboard

•Limit origin of illegitimate SSH access

Non-goal: Protect against local attackers

Why 2Fac for SSH?


•Facebook culture: Move Fast

•Intolerant of slowdown

•Highly skilled at finding workarounds

•Primarily work via SSH on dev servers

Engineering @ FB


•Facebook culture: Move Fast

•Intolerant of slowdown

•Highly skilled at finding workarounds

•Primarily work via SSH on dev servers

Goal: Make being secure effortless

Engineering @ FB


State of Multi-Factor


•Easy to use

•Good interoperability

•Synchronization is easy

•Time windows of acceptance

•Only good for infrequent use

Time-based


•Easy to use

•Good interoperability

•Gets out of sync

•Most tokens designed for infrequent use

OTP


•Limited device support

•Security limitations

• False acceptance

• Replay

•Practical Problems: How to biometric auth to remote machine?

•Poor usability

Biometrics


PKI

•Limited device support

•Enrollment is painful

•Management is painful

•Smart Card Proxy attack

PKI


•Easy to setup

•Easy to use

•Push (only on some devices)

•Requires fast, reliable online channel

•Usability is good only for infrequent use

OOB / Mobile


•Usability

• Support Very Frequent use

• Flexible options

•Security

• Require stronger authentication for every session

•Fast Deployment

•Minimal support overhead

Building it Better


•Duo Security + Yubikey Nano

•Flexible Options

•Low operational overhead

•Provisioning process out of the box

•Yubikey is awesome for frequent use

•Bonus: Backup tokens from the start

The Solution


Deployment: Planning


•How is SSH being used?

•Thousands of engineers

•Tens of thousands of sessions per day

•Peak users with >3000 sessions

•Using all authentication mechanisms








•What are they doing?









sshd[87820]: Accepted keyboard-interactive/pam for twt from ::1 port 51317 ssh2sshd[87820]: User child is on pid 87825sshd[87825]: Received disconnect from ::1: 11: disconnected by user



•Add details about what the user is doing

sshd[27587]: Accepted publickey for ::1 port 61447 ssh2sshd[27587]: User child is on pid 27589sshd[27589]: Exec Request for user twt with command uname -a

sshd[8540]: Accepted publickey for twt from ::1 port 50654 ssh2sshd[8540]: User child is on pid 8548sshd[8548]: Allocated pty /dev/pts/18 for user twt session 0sshd[8548]: Shell Request for user twtsshd[8548]: Received disconnect from ::1: 11: disconnected by user

Improving SSH Logs: First Attempt


•Add details about what the user is doing

sshd[27587]: Accepted publickey for ::1 port 61447 ssh2sshd[27587]: User child is on pid 27589sshd[27589]: Exec Request for user twt with command uname -a

sshd[8540]: Accepted publickey for twt from ::1 port 50654 ssh2sshd[8540]: User child is on pid 8548sshd[8548]: Allocated pty /dev/pts/18 for user twt session 0sshd[8548]: Shell Request for user twtsshd[8548]: Received disconnect from ::1: 11: disconnected by user

•Problem: requires multiple log lines with different PIDs for analysis

Improving SSH Logs: First Attempt


•Add sessionization data to SSH logs

sshd[27587]: Accepted publickey for ::1 port 61447 ssh2 session=dev123:52369e5a.c6786sshd[27587]: User child is on pid 27589 session=dev123:52369e5a.c6786sshd[27589]: Exec Request for user twt with command uname -a session=dev123:52369e5a.c6786

sshd[8540]: Accepted publickey for twt from ::1 port 50654 ssh2 session=dev123:5236a24d.3f32sshd[8540]: User child is on pid 8548 session=dev123:5236a24d.3f32sshd[8548]: Allocated pty /dev/pts/18 for user twt session 0 session=dev123:5236a24d.3f32sshd[8548]: Shell Request for user twt session=dev123:5236a24d.3f32sshd[8548]: Received disconnect from ::1: 11: disconnected by user session=dev123:5236a24d.3f32

Sesssionizing SSH Logs



• SFTP

• Random scripts

• TRAMP mode

• Lots of shells

• Using every authentication mechanism

SSH Usage Analysis


Deployment: Implementation


•OpenSSH 6.2 - support for multiple Auth Methods

• Public key, kerberos, password are first factors

• Duo is second factor






•Problem: password and Duo are both handled by keyboard-interactive auth method






•Problem: password and Duo are both handled by keyboard-interactive auth method

•Solutions:

• Submethods for keyboard-interactive/{pam,duo} in OpenSSH 6.2p1

• KerberosAuthentication yes



Handling SFTP


•Clients don’t support multiple auth mechanisms

Handling SFTP



•Primary security concern:

• Single factor command execution

Handling SFTP





•Solution:

• Single factor SFTP chroot

Handling SFTP


Handling scripts + TRAMP mode


•Switch to use SFTP solution?






•Solution:

• SSH whitelists






•Solution:

• SSH whitelists

•New problem:• REGEX:sh -‐c "cd (~/|\w)(((?<!\.\.)/)|((?<!/)\.)|[\w_-‐])+ && grep -‐P '[^']+\\t' tags | head -‐n 10"



•Keyboard layouts

•Exploding computers

•Possessed yubikeys

•Accidental discharge

•Client ssh config problems

•Need moar USB ports

•Enrollment issues

Unexpected Issues


•more 2Fac:

• sudo

• SSH alternatives: mosh, VNC, NX

• priv esc points

• replace/supplement other multi-factor solutions

• 2Fac everywhere

•Get rid of command whitelists

•Make SFTP clients support multi-factor

Ongoing Work


Facebook Security


"2fac: facebook's internal multi-factor authentication". tim tickel, chad greene,...

Technology

thousandsofengineers

allocated

passwordarefirstfactors

3000sessions

user twt sshd

user twt

user child

pid 27589