MINIMALT: Minimal-latency Networking

Through Better Security

W. Michael Petullo1 Xu Zhang2 Jon A. Solworth2

Daniel J. Bernstein2,3 Tanja Lange3

1United States Military Academy

West Point, New York USA

2University of Illinois at Chicago

Chicago, Illinois USA

3TU Eindhoven

Eindhoven, Netherlands

November 6, 2013

1/53

Networking should be:

I Quicker

I Less latency when making a connection

I IP-address mobility (fewer application restarts)

I Universally and more strongly protected

I More confidentiality (e.g., time-based key erasure)

I More integrity (e.g., cryptographic user authentication)

I More DoS protections (e.g., stop RST attack)

I Simpler

I Built on higher-level cryptographic primitives

I Minimal configuration

I Simple design and small code base

2/53

Networking should be:

I Quicker

I Less latency when making a connection

I IP-address mobility (fewer application restarts)

I Universally and more strongly protected

I More confidentiality (e.g., time-based key erasure)

I More integrity (e.g., cryptographic user authentication)

I More DoS protections (e.g., stop RST attack)

I Simpler

I Built on higher-level cryptographic primitives

I Minimal configuration

I Simple design and small code base

3/53

Historical impediments

I Encryption overhead

I Latency

?

Modern CPUs with h

igh-performance

crypto. software:

I Symmetriccrypto. at

Gb/s rates

I Tens of thousands of

public-key

ops. per sec.

I . . . even onconstraine

d devices

Moore’s Law

Image byHannes G

robe

NaCl Crypto Library

Image by Dubravko Soric

4/53

Historical impediments

I Encryption overhead

I Latency

?

Modern CPUs with h

igh-performance

crypto. software:

I Symmetriccrypto. at

Gb/s rates

I Tens of thousands of

public-key

ops. per sec.

I . . . even onconstraine

d devices

Moore’s Law

Image byHannes G

robe

NaCl Crypto Library

Image by Dubravko Soric

5/53

Historical impediments

I Encryption overhead

I Latency

?

Modern CPUs with h

igh-performance

crypto. software:

I Symmetriccrypto. at

Gb/s rates

I Tens of thousands of

public-key

ops. per sec.

I . . . even onconstraine

d devices

Moore’s Law

Image byHannes G

robe

NaCl Crypto Library

Image by Dubravko Soric

6/53

Historical impediments

I Encryption overhead

I Latency

?

Modern CPUs with h

igh-performance

crypto. software:

I Symmetriccrypto. at

Gb/s rates

I Tens of thousands of

public-key

ops. per sec.

I . . . even onconstraine

d devices

Moore’s Law

Image byHannes G

robe

NaCl Crypto Library

Image by Dubravko Soric

7/53

TLS: 4 roundtripsClient Server

UDP DNS req.

UDP DNS resp.

SYN

SYN ACK

(ACK) ClientHello

Server Hello/Session ID, Cert., SK

E, Cert. Request, Done

Cert., CKE, Cert. Verify., Change Cipher Spec., Finish

Change Cipher Spec., Finish

Application data

1

2

3

4

1. DNS lookup.

2. TCP three-way handshake establishesrandom initial sequence number:

I Weak authenticator/liveness check

I Address late packet arrival3–4. Negotiate cipher suite andestablish ephemeral keys

8/53

TLS: 4 roundtripsClient Server

UDP DNS req.

UDP DNS resp.

SYN

SYN ACK

(ACK) ClientHello

Server Hello/Session ID, Cert., SK

E, Cert. Request, Done

Cert., CKE, Cert. Verify., Change Cipher Spec., Finish

Change Cipher Spec., Finish

Application data

1

2

3

4

1. DNS lookup.

2. TCP three-way handshake establishesrandom initial sequence number:

I Weak authenticator/liveness check

I Address late packet arrival3–4. Negotiate cipher suite andestablish ephemeral keys

9/53

TLS: 4 roundtripsClient Server

UDP DNS req.

UDP DNS resp.

SYN

SYN ACK

(ACK) ClientHello

Server Hello/Session ID, Cert., SK

E, Cert. Request, Done

Cert., CKE, Cert. Verify., Change Cipher Spec., Finish

Change Cipher Spec., Finish

Application data

1

2

3

4

1. DNS lookup.

2. TCP three-way handshake establishesrandom initial sequence number:

I Weak authenticator/liveness check

I Address late packet arrival

3–4. Negotiate cipher suite andestablish ephemeral keys

10/53

TLS: 4 roundtripsClient Server

UDP DNS req.

UDP DNS resp.

SYN

SYN ACK

(ACK) ClientHello

Server Hello/Session ID, Cert., SK

E, Cert. Request, Done

Cert., CKE, Cert. Verify., Change Cipher Spec., Finish

Change Cipher Spec., Finish

Application data

1

2

3

4

1. DNS lookup.

2. TCP three-way handshake establishesrandom initial sequence number:

I Weak authenticator/liveness check

I Address late packet arrival

3–4. Negotiate cipher suite andestablish ephemeral keys

11/53

TLS (abbreviated): 2 roundtrips

Client Server

SYN

SYN ACK

(ACK) ClientHello/Session ID

ServerHello, Change Cipher Spec

., Finish

Change Cipher Spec., Finish, Application data

1

2

Only possible on a reconnect

12/53

Existing efforts

Tcpcrypt attempts to

make encryption

universaland does

by modifying TCP

without breaking c

ompatibility with

TCP.

False Start reduces the number of

round trips in TLS.

Snap Start reduces the number ofround trips in TLS.

13/53

Existing efforts

Tcpcrypt attempts to

make encryption

universaland does

by modifying TCP

without breaking c

ompatibility with

TCP.

False Start reduces the number of

round trips in TLS.

Snap Start reduces the number ofround trips in TLS.

14/53

Existing efforts

Tcpcrypt attempts to

make encryption

universaland does

by modifying TCP

without breaking c

ompatibility with

TCP.

False Start reduces the number of

round trips in TLS.

Snap Start reduces the number ofround trips in TLS.

15/53

Additional problems with TLS

I Susceptible to attacks on lower layers, e.g., RST DoS on TCP

I Ephemeral keys last as long as connections/sessions

I Exists as a library, and programmers often improperly use it

(e.g., Georgiev CCS ’12; Fahl and Egele here)

I Many insecure options (e.g., RC4)

I Client authentication is generally password-based

I Implementations have become very large

16/53

MINIMALT

17/53

Encrypted app. data on first packet

I Replace TCP’s handshake with strong cryptography

I Piggyback ephemeral key discovery onto DNS-type lookup

We found that this solution could out-perform evenunprotected TCP’s latency.

18/53

Encrypted app. data on first packet

I Replace TCP’s handshake with strong cryptography

I Piggyback ephemeral key discovery onto DNS-type lookup

We found that this solution could out-perform evenunprotected TCP’s latency.

19/53

MINIMALT design

20/53

MINIMALT objectsI Public keys: identify client users and serversI Ephemeral keys: time-based and cryptographically protectidentifying traffic

I Tunnels: an encrypted channel between two hosts whichmultiplexes connections

I Connections: user-authenticated two-way communicationwithin a tunnel

I Directory and name service: resolve hostnames to IPaddresses and keys

21/53

MINIMALT packet format

Size (bytes)Field First SuccessiveDeliv. Ethernet Header 14 14

IP 20 20

UDP 8 8

&Crypto. Tunnel ID 8 8

Nonce 8 8

Ephemeral public key 32 n/a

Puzzle/solution 148 n/a

Checksum 16 16

Rel. Sequence Num. 4 4

Acknowledgment 4 4

Con.

Connection ID 4 4

RPC variable

Total (except RPC) 282 86

22/53

MINIMALT packet format

Size (bytes)Field First SuccessiveDeliv. Ethernet Header 14 14

IP 20 20

UDP 8 8

&Crypto. Tunnel ID 8 8

Nonce 8 8

Ephemeral public key 32 n/a

Puzzle/solution 148 n/a

Checksum 16 16

Rel. Sequence Num. 4 4

Acknowledgment 4 4

Con.

Connection ID 4 4

RPC variable

Total (except RPC) 282 86

23/53

MINIMALT packet format

Size (bytes)Field First SuccessiveDeliv. Ethernet Header 14 14

IP 20 20

UDP 8 8

&Crypto. Tunnel ID 8 8

Nonce 8 8

Ephemeral public key 32 n/a

Puzzle/solution 148 n/a

Checksum 16 16

Rel. Sequence Num. 4 4

Acknowledgment 4 4

Con.

Connection ID 4 4

RPC variable

Total (except RPC) 282 86

24/53

A tongue twister: MINIMALT

avoids pitfall-plagued

low-level cryptographic

primitives which have

periodically perplexed TLS

25/53

MINIMALT cryptographyHigher-level cryptographic primitives than TLS

I Built on public-key authenticated encryption

I NaCl’s box operation:

m C ′→S ′n ,

takes as input:

I a nonce n ,I C ’s private key,I S ’s public key, andI a messagemto produce an authenticated ciphertext

(including a cryptographic checksum).

26/53

MINIMALT round trips

27/53

MINIMALT roundtripsObtaining D ’s ephemeral key (only at boot time):

C ′ DConn., req. ephemeral key

Ephemeral keyT1

DNS-like lookup (only if tunnel does not yet exist):C ′ D ′

Conn., req. server information

ephemeral key of SIP address, UDP port, key,

T2

Connection establishment:C ′ S ′

application-to-service RPC

Connect, T328/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

29/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

C ′ D ′Conn., req. server information

ephemeral key of SIP address, UDP port, key,

T2

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

30/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

31/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

32/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

33/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

34/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

35/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

36/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

37/53

MINIMALT packet flow

User key U & authenticator

C ′ DConn., req. ephemeral key

Ephemeral keyT1

I Boot time: getdirectory service’s

ephemeral key.

I Connection time:request service

record for some

server.

Directory service

C ′ S ′

application-to-service RPC

Connect, T3

t ,n ,C ′,s , a , nextTid0(t2,C ′),createAuth0(1, serviceName, U , x ),serviceRequest1(. . . )

C ′→S ′n

End server

Now some client application running

as U requests a connection to the

server; creates encrypted tunnel.

t2,n ,C ′, s , a , nextTid0(t3,C ′) C ′→S ′n

MINIMALT fast key erasure:

I Two rolling tunnel IDs: current and

next

I Next key is hash of current

I Transition to new key with

tunnel-creating packet containing

public key & next tunnel ID

Server can cause client to perform

this process with a rekeyNow mes-

sage.

Same technique used to support un-

linkable IP mobility

38/53

MINIMALT uses puzzles to

mitigate DoS

39/53

MINIMALT puzzlesI Server computes puzzle solution:

r = C ′, S ′ kn ′

I From solution, generate puzzle

based on Juels et al.’s client

puzzles

t ,n , [q ,H (r),w ,n ′]

t ,n ,C ′, [r ,n ′], s , a , . . . C ′→S ′n

40/53

MINIMALT puzzlesI Server computes puzzle solution:

r = C ′, S ′ kn ′

I From solution, generate puzzle

based on Juels et al.’s client

puzzles

t ,n , [q ,H (r),w ,n ′]

t ,n ,C ′, [r ,n ′], s , a , . . . C ′→S ′n

41/53

MINIMALT puzzlesI Server computes puzzle solution:

r = C ′, S ′ kn ′

I From solution, generate puzzle

based on Juels et al.’s client

puzzles

t ,n , [q ,H (r),w ,n ′]

t ,n ,C ′, [r ,n ′], s , a , . . . C ′→S ′n

42/53

MINIMALT puzzlesDoS attacks

I Request many tunnels, causing public-key computations (puzzle

in packet header)

I Given tunnel, send garbage data that fails checksum (fast

symmetric cryptography)

I Given tunnel, generate false identities that fail authentication

(puzzle RPC)

I MINIMALT can also use puzzle RPCs to make clients pay to keep

connections alive

I MINIMALT can allow very long-running tunnels, and garbage

collect them only if a client fails to solve a puzzle

43/53

MINIMALT performance

44/53

MINIMALT performanceBenchmark hardware

I 4.3GHz AMD FX-4170

I 64-bit mode

I 16GB memory

I Gb/s Ethernet adapter

I MINIMALT on Ethos

I OpenSSL 1.0.0j on Linux kernel 3.4.4

I TCP/IP benchmark

I Varied latencies using netem

45/53

MINIMALT performance: latencyMinimaLT serial full-connection rate vs. TCP and TLS

0

100

200

300

400

500

1/161/8 1/4 1/2 1 2 4 8 16 32 64 128256

Normalizedconn.rate

MinimaLT

TLS

TCP

One-way additional simulated latency applied to network (ms)

46/53

MINIMALT performance: latencyMinimaLT serial abbreviated-connection rate vs. TCP and TLS

0

100

200

300

400

500

1/161/8 1/4 1/2 1 2 4 8 16 32 64 128256

Normalizedconn.rate

MinimaLT

TLS

TCP

One-way additional simulated latency applied to network (ms)

47/53

MINIMALT performance: connections

Parallel tunnel/connection establishment throughputTunnels User Connections DH perper run Auth. per second conn.One 18,453 0

One X 8,576 1

Many 7,827 1

Many X 4,967 2

Theoretical upper limit is 14,000 DH computations per second(SUPERCOP).

48/53

MINIMALT performance: throughput

Single-connection data throughputSystem Bytes per secondLine speed 125,000,000Unencrypted 117,817,528MLT 113,945,258OpenSSL 111,448,656

49/53

ConclusionImplementation to be published at http://www.ethos-os.org/

I MINIMALT is quicker than TCPI No handshake

I Ephemeral-key negotiation part of DNS lookup

I IP-address mobility

I MINIMALT is stronger than TLSI Built on higher-level cryptographic primitives

I Cryptographic user authentication

I Time-based key erasure (forward secrecy)

I DoS protections

I MINIMALT is simpler than TLS/TCPI Cryptographers choose, we use

I No options (e.g., ECB, RC4)

I Simple design and small code base

50/53

ConclusionImplementation to be published at http://www.ethos-os.org/

I MINIMALT is quicker than TCPI No handshake

I Ephemeral-key negotiation part of DNS lookup

I IP-address mobility

I MINIMALT is stronger than TLSI Built on higher-level cryptographic primitives

I Cryptographic user authentication

I Time-based key erasure (forward secrecy)

I DoS protections

I MINIMALT is simpler than TLS/TCPI Cryptographers choose, we use

I No options (e.g., ECB, RC4)

I Simple design and small code base

51/53

ConclusionImplementation to be published at http://www.ethos-os.org/

I MINIMALT is quicker than TCPI No handshake

I Ephemeral-key negotiation part of DNS lookup

I IP-address mobility

I MINIMALT is stronger than TLSI Built on higher-level cryptographic primitives

I Cryptographic user authentication

I Time-based key erasure (forward secrecy)

I DoS protections

I MINIMALT is simpler than TLS/TCPI Cryptographers choose, we use

I No options (e.g., ECB, RC4)

I Simple design and small code base

52/53

Questions?

53/53