Message-Locked EncryptionandSecure Deduplication

1

Mihir Bellare1

Sriram Keelveedhi1

Thomas Ristenpart2

1University of California, San Diego2University of Wisconsin-Madison

Eurocrypt 2013

Deduplication

2

Storage size after 𝑛 uploads

No deduplication 𝒪(𝑛 ⋅ |𝑓|)

Deduplication 𝒪(|𝑓|)

Bob

Store 𝑓 iff new

𝑓 𝑓

Alice

Server

Store 𝑓 iff new

Google Drive

Storage savings [MB11]

Backup systems 87%

Corporate networks 50%

Avoid storing multiple copies of the same data

Outsourced storage service

Dedup doesn’t work with client-side encryption

3

𝑐𝐴 𝑐𝐵

𝑐𝐴 ← E(𝑘𝐴, 𝑓) 𝑐𝐵 ← E(𝑘𝐵, 𝑓)

ℰ = (K, E, D): Symmetric encryption scheme Bob

Store 𝑓 iff new

Alice

Server

Store 𝑐 iff new

𝑘𝐴 𝑘𝐵𝑐𝐴

Cross-user decryption not possible, Bob still cannot decrypt 𝑐𝐴

⇒Server has to store both 𝑐𝐵 and 𝑐𝐴

Possible fix: Attach file hash H(𝑓) to ciphertext?

Pr 𝑐𝐵 = 𝑐𝐴 is negligible Security of symmetric encryption

Det. PKE [BBO07, MPRS12]Searchable SE [SWP00]Searchable PKE [BBO07]

Rules out

Bob cannot decrypt 𝑐𝐴with 𝑘𝐵

{

Convergent encryption

𝑐𝐴 𝑐𝐵

𝑐𝐴 ← E(H(𝑓), 𝑓) 𝑐𝐵 ← E H 𝑓 , 𝑓= 𝑐𝐴

Bob

Store 𝑓 iff new

Alice

Server

Store 𝑐 iff new

𝑘𝐴 𝑘𝐵𝑐𝐴

Bob can decrypt 𝒄𝑨 with 𝑘 = H(𝑓)

𝐄

𝑚

𝑯 𝑐𝑘

Recipe1. 𝐻: 0,1 ∗ → 0,1 𝑘: Hash function2. ℰ = (K, E, D): Encryption scheme with 𝑘-bit keys

Internet forums,

[DABST02]

Cloud storage

Filesystems Farsite [ABCG*02]

GNUNet

Backup [CTP04][CMN02] [KCP06]

Others [AZ10] [BBST01] [MC11][RCTLL11] [SGLM08]

5

CE has found wide use…

… despite unclear security guarantees

Convergent Encryption

6

• What kind of security can schemes like CE provide?• Are the deployed schemes/variants secure?

CE seems to be widely used, but…

No cryptographic treatment for deduplication over encrypted data

We don’t know!

Our work answers these questions

How to support• Equality checking/deduplication?• Cross-user decryption?

Syntax of such schemes?

Best possible security?

Our work

1. Message-Locked Encryption• Syntax and correctness

• Security goals and notions

7

2. Practical contributions• Attacks and proofs for CE and variants

• New, faster schemes

3. Theoretical contributions• Standard model MLE schemes from

correlated-input hashes and deterministic-PKE

• Relating MLE and other cryptographic primitives

A cryptographic framework for schemes which achievededup over ciphertexts

Message-Locked Encryption

𝐊

𝑚

𝑘

𝐄 𝐃𝑐

Message-derived key

8

𝑚

Key used for encryption is derived from the message itself

𝐓 𝑡 Tag𝐏 𝑝 Public parameter

𝑃, E, K randomized𝐷, 𝑇 deterministic

MLE Scheme ℳ = (P, K, E, D, T)

Convergent encryption as an MLE scheme

𝐇

𝑚

𝑘

𝐄 𝐃𝑐

9

𝑚

𝐇 𝑡𝐏 𝑝 Random 128-bit string

𝐶ℰ = (P, K2, E2, D2, T)

1. 𝐻: 0,1 ∗ → 0,1 𝑘: Hash function2. ℰ = (K, E, D): Encryption scheme with 𝑘-bit keys

Recipe

We will revisit 𝐶ℰ to talk about security

Secure outsourced storage using MLE

Alice Bob

Server

1. MLE Scheme ℳ = P, E, K, D, T

Recipe

2. SE Scheme 𝑆 = (K2, E2, D2)

𝑐𝐴, 𝑐𝐴′

Upload(𝒄𝑩, 𝒄𝑩′ )

Store (𝒇)

𝑘𝑓𝐵 ← K 𝑓

𝑐 ← E 𝑘𝑓𝐵 , 𝑓

c𝐵 ← E2 𝑘𝐵 , 𝑘𝑓𝐵

𝑘𝑓𝐵 ← D2 𝑘𝐵, 𝑐′𝐵

𝑓 ← D 𝑘𝑓𝐵 , 𝑐𝐴

Retreive (𝒌𝑩, 𝒄𝑨, 𝒄𝑩′ )

𝑐𝐵 , 𝑐𝐵′

Store (𝒇)

𝑘𝑓𝐴 ← K 𝑓

𝑐𝐴 ← E 𝑘𝑓𝐴, 𝑓

If T 𝑐𝐴 ≠ T 𝑐𝐵Store 𝑐𝐵

Store 𝑐𝐵′

𝑐𝐴, 𝑐𝐴′𝑐𝐴, 𝑐𝐴

′ , 𝑐𝐵′

𝑐𝐴′ ← E2 𝑘𝐴, 𝑘𝑓

𝐴

𝑐𝐴, 𝑐𝐵′

Requirements1. 𝑓 ← D 𝑘𝑓

𝐵, 𝑐𝐴2. 𝑇 𝑐𝐴 = 𝑇 𝑐𝐵3. 𝑘𝑓

𝐴 = 𝑘𝑓𝐵 ≪ |𝑓|

Bob recovers 𝑓

Deduplication

Storage = |𝑓| + α

MLE Correctness

𝐊𝑚 𝑘

𝐄 𝐃𝑐

𝐓 𝑡

MLE Scheme ℳ = (P, E, K, D, T)

11

𝑚

1. Decryption correctness Any key 𝑘 derived from 𝑚 can decrypt any 𝑚-ciphertext 𝑐

2. Tag correctness All 𝑚 ciphertexts 𝑐 produce the same tag 𝑡

3. Non-triviality All keys 𝑘 are of the same, fixed length

D 𝑘, 𝑐 = 𝑚 ∀ valid messages 𝑚, ∀𝑘 ∈ K 𝑚 , ∀𝑐 ∈ E 𝑘,𝑚

A 𝑥1, … : Set of all outputs of 𝐴 on 𝑥1, …

T 𝑐1 = T(𝑐2) ∀ 𝑚, ∀𝑘1, 𝑘2 ∈ K 𝑚 , ∀𝑐1 ∈ E 𝑘1, 𝑚 , ∀𝑐2 ∈ E 𝑘2, 𝑚

|K 𝑚 | = 𝜅 ∀ 𝑚, ∀𝑘 ∈ K 𝑚

Security, informally

𝐊𝑚 𝑘

𝐄 𝑐 𝐓 𝑡

MLE Scheme ℳ = (P, E, K, D, T)

12

1. PrivacyChosen Distribution vs. Random (CDR)If 𝑚 has high min-entropy, 𝑐indistinguishable from random

2. Consistent tagsTag Consistency (TC)Hard to find 𝑐′ that does not decrypt to 𝑚 but has same tag as 𝑚

Attack runtime = 𝑐 ⋅ 𝑛

Can we get IND-CPA style privacy for MLE?

For 𝑚𝑖 ∈ 𝑆 do𝑚′ ← D K 𝑚𝑖 , 𝑐If 𝑚𝑖 = 𝑚′then return 𝑚𝑖

BruteForc𝑒𝑆(𝑐)

Consider a set 𝑆 = {𝑚1, 𝑚2, … ,𝑚𝑛}

Given 𝑐 ← E K 𝑚𝑖 , 𝑚𝑖 where 𝑖 ← {1,2, … , 𝑛}Find 𝑚𝑖

Has to be super-polynomial

Privacy not possible for predictable messages

No!

A generic brute-force attack:

Message recovery security: MR𝑆,ℳ

MLE Scheme ℳ = (P, E, K, D, T)

Weaker than IND-CPA

Privacy: The CDR notion

𝑝 ← P(); 𝑏 ← 0,1 ; (𝑚1, … ,𝑚𝑛) ← D()For 𝑖 = 1 to 𝑛

𝑘𝑖 ← K 𝑚𝑖 ; 𝑐𝑖1← E 𝑘𝑖 , 𝑚𝑖 ;

𝑐𝑖0← {0,1} 𝑐𝑖

1

No efficient adversary can distinguish encryptions of unpredictable messages from random strings

𝑨𝐈𝐧𝐢𝐭

𝐅𝐢𝐧 Return (𝑏′ = 𝑏)𝑏′

𝑝, 𝑐1𝑏 , … , 𝑐𝑛

𝑏

𝑨𝒅𝒗 A,D = 2 ⋅ Pr CDR(D, A) ⇒ true − 1

Security: No efficient 𝐴 has non-negligible advantage for any unpredictable 𝐷

CDR(A, D)

14

MLE Scheme ℳ = (P, E, K, D, T)

Notion Primitive Style SQ ⇒ MQ

IND[BFOR08] D-PKE Left-Right indist. No

CDA[BBNRSSY09] PKE Left-Right indist. No

CDR [BKR13] MLE Real-random indist. Yes

Comparing with notions that need unpredictability (Discussion in paper)

SQ : Single-query, MQ : Multi-query

D is unpredictable if ∃𝛿 ∈ negl s.t. Pr[𝑚′ ∈ {𝑚1, … , 𝑚𝑛} ∶ 𝑚1, … ,𝑚𝑛 ← D()] ≤ 𝛿 ∀𝑚′

Deduplicability vs. PrivacyDeduplication

Only when messages repeat

15

Privacy

Only when messages unpredictable

Inherent to secure deduplication ⇒ CDR provides best possible security

Encryption for Deduplicated Storage with DupLESS

USENIX Security 2013 Bellare, Keelveedhi, Ristenpart

Security for predictable messages

Data unpredictable to attacker,

not to legitimate clients

Large random file 𝑓

Server

A possible contradiction? NO!

Attacker

CiphertextShared file𝑓

• Shared among group of clients

• Unknown to attacker

Duplicate faking attacks

𝑐′

𝑐′

16

Server

Evil dude

𝑐 ← E 𝐾(𝑓), 𝑓Get 𝑐′ that not decrypt to 𝑓

s.t. T 𝑐′ = T 𝑐

𝑓

1. Attacker stores 𝑐′2. Alice tries to store 𝑐, server already has a matching ciphertext 𝑐′3. When Alice downloads 𝑐′ it decrypts to 𝑓′ ≠ 𝑓

Note: No unpredictability requirement

Alice

𝑓

Store 𝑐 if T(𝑐) is new

𝑐

Noted in [SGL08]

Tag Consistency

𝑝 ← P()

No efficient adversary can find two ciphertexts with matching tagsthat decrypt to different messages

𝑨

𝐈𝐧𝐢𝐭

𝐅𝐢𝐧𝐚𝐥𝐢𝐳𝐞 𝑘 ← K 𝑚 ;𝑚′ ← D(𝑘, 𝑐′)𝑡 ← T E(𝑘, 𝑚) ; 𝑡′ ← T 𝐶′

If 𝑡 ≠ 𝑡′then return falseIf 𝑚 = 𝑚′then return falseIf 𝑚′ =⊥ then return falseReturn true

𝑚, 𝑐′

𝑝

𝑨𝒅𝒗𝐓𝐂 𝐴 = Pr TC(𝐴) ⇒ true

Security: No efficient 𝐴 has non-negligible TC advantage.

TC A

17

MLE Scheme ℳ = (P, E, K, D, T)

In the paper: A stronger tag consistency notion STC

Our work

1. Message-Locked Encryption• Syntax and correctness

• Security goals and notions

18

2. Practical contributions• Attacks and proofs for CE and variants

• New, faster schemes

3. Theoretical contributions• Standard model MLE schemes from

correlated-input hashes and deterministic-PKE

• Relating MLE and other cryptographic primitives

Convergent Encryption

𝐄

𝑚

𝑯 𝑐 𝑡𝑝 𝑯𝑘

19

Encryption in CE

𝐶ℰ = (P, K2, E2, D2, T)

1. 𝐻: 0,1 ∗ → 0,1 𝑘: Hash function2. ℰ = (K, E, D): Encryption scheme with 𝑘-bit keys

Thm: 𝐶ℰ is CDR secure in the 𝐑𝐎 model if ℰ is Real-or-Random secure and Key-Recovery secure.

Thm: 𝐶ℰ is TC secure in the standard model if H is a CR hash.

Recipe

In the paper

Security of other variants of CE, fixes for tag consistency vulnerabilities

Randomized CE One pass, randomized MLE scheme

E𝑚 𝑐1

𝑐2

ℓ

H1

𝑝1

𝑘 H2

𝑡𝒑𝟐 H3

20

1. H1, H2, H3: 0,1∗ → 0,1 𝑘: Hash functions

2. ℰ = (K, E, D): Encryption scheme with 𝑘-bit keys

Thm: 𝑅𝐶ℰ is CDR secure in the 𝐑𝐎 model if ℰ is Real-or-Random secure and Key-Recovery secure.

Thm: 𝑅𝐶ℰ is TC secure in the 𝐑𝐎 model.

Key generation and encryption KE2(𝑝, 𝑚; ℓ)

Recipe

In the paper: Comparison of performance of CE schemes. RCE is fastest.

Our work

1. Message-Locked Encryption• Syntax and correctness

• Security goals and notions

21

2. Practical contributions• Attacks and proofs for CE and variants

• New, faster schemes

3. Theoretical contributions• Standard model MLE schemes from

correlated-input hashes and deterministic-PKE

• Relating MLE and other cryptographic primitives

eXtract Hash and Check

𝑚

𝑿𝑝1𝑯𝑘

22

Encryption in XHC

XHC[𝐻, 𝑋] = (P, K, E, D, T)

1. 𝐻: 0,1 ∗ → 0,1 𝑘: Hash function2. 𝑋: 0,1 𝑘 × 0,1 ∗ → 0,1 𝑘: Extractor

Thm: XHC[𝐻, 𝑋] is CDR∗ secure if 𝐻 is a correlated input hash and 𝑋 is a strong randomness extractor.

Thm: XHC[𝐻, 𝑋] is TC secure.

𝑚1, … ,𝑚𝑖 , … ,𝑚𝑛

𝑘|⟨𝑖⟩|𝑚𝑖

𝑝2

𝑐1, … , 𝑐𝑖 , … , 𝑐𝑛

Recipe Correlated-inputhashes [GOR11]

Decryption in XHC For 𝑖 = 1 to 𝑛If 𝑘|⟨𝑖⟩|0 = 𝑐𝑖 then 𝑚𝑖 = 1Else 𝑚𝑖 = 0Return 𝑚1| 𝑚2| … | 𝑚𝑛

𝑚

If inputs are unpredictable,hashes are pseudorandom

Standard model schemes and relations

23

Correlated-inputhashes

[GOR11]

MLE

Deterministic PKE[BBO07]

XHC

SXE:Sample-Extract-Encrypt

Secure only for independent message-distributions

MLE from extractors and symmetric encryption

In the paper:

Caveat: Don’t know how to build these in standard model with best possible security

[Wi13]Hard to build

Recap

1. Message-Locked Encryption• Syntax and correctness

• Security goals and notions

24

2. Practical contributions• Attacks and proofs for CE and variants

• New, faster schemes

3. Theoretical contributions• Standard model MLE schemes from

correlated-input hashes and deterministic-PKE

• Relating MLE and other cryptographic primitives

A cryptographic framework for schemes which achievededup over ciphertexts

Thank you!

25

Sriram Keelveedhisriramkr@cs.ucsd.edu

Full version: eprint.iacr.org/2012/631

Follow up

• Encryption for Deduplicated Storage with DupLESS• USENIX Security 2013

• Message-Locked Encryption for lock-dependent messages• Abadi, Boneh, Mironov, Raghunathan and Segev in CRYPTO 2013

• Several interesting open problems