an introduction to randomness extractors

An Introduction to Randomness Extractors

Ronen ShaltielUniversity of Haifa

Daddy, how do

computers get random

bits?

Randomized algorithms and protocols

Randomized algorithms/protocols: Receive stream of independent

unbiased coin tosses. Necessary for Crypto.

deterministic algorithm

output

input

Provably help in distributed settings. Randomized algorithms are often simpler and more

efficient than known deterministic ones. (even though we conjecture that BPP=P).

Randomized

“weak source of randomness”

Randomized

algorithm

Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix)These distributions are “somewhat

random” but not “truly random”.Paradigm: randomness extractorsInput: one sample from arbitrary

“weak source of randomness”. Output: independent coin tosses.

How do computers obtain random coin tosses?

RandomnessExtractor

Extensively studied area, dates back to von-Neumann in 1951.

output

input

Coins may be biased and correlated

How do computers obtain random coin tosses?


Randomized

algorithm

RandomnessExtractor

output

input

Computers can sample from: Electro-magnetic noise (Intel) Key strokes of user (Unix) Timing of past events (Unix)These distributions are “somewhat

random” but not “truly random”.Paradigm: randomness extractorsInput: one sample from arbitrary

“weak source of randomness”. Output: independent coin tosses.

Extensively studied area, dates back to von-Neumann in 1951.

Extractors have applications in: Randomized complexity theory. Cryptography. Network design. Ramsey theory. Coding theory. Combinatorics. Algorithm design. Data structures.

Extractors have many applications

Often not directly related to randomness!

Gives additional motivation to extractors (in addition to the initial motivation of extracting randomness for randomized algs).


Randomized algorithm

RandomnessExtractor

input

output

Several notions of extractors

Deterministic extractorsRestrict to specific families of “allowed sources”.

Multiple sources extractors Extractor gets samples from several independent

sources.

Seeded extractors Allow extractor to get a seed of few truly random

bits.

Deterministic extractors: Formal definition


RandomnessExtractor

Distribution X from CDfn: Let C be a set of distributions over {0,1}n (family of “allowed sources”).

E:{0,1}n ! {0,1}m is an extractor for C if 8X2C, random variable E(X) is uniform over {0,1}m.

Two distributions Y,Z over the same domain are ²-close if 8 event A, |Pr[Y 2 A]-Pr[Z 2 A]| ≤ ².

Goal: Design efficiently computable extractors for interesting and general families of sources. Maximize number of extracted bits. Minimize error ².

²-close to

Example: von-Neumann’s sources and extractor (1951!)

Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X1,..,Xn) s.t.

X1,..,Xn i.i.d. p ≤ Pr[Xi=1] ≤ 1-p.

vN extractor E(x) (extracts one bit): on input x2{0,1}n Scan input bits from left to right. If you see pair “01” stop and output “0”. If you see pair “10” stop and output “1”.

Observation: Pr[“01”] = Pr[“10”] (implies correctness).Subsequent work on extracting many bits [Elias72,Peres92].

X has entropy ≥ pn.

Impossibility of extraction from Santha-Vazirani sources

Let 0<p≤½ be a parameter (e.g., p=1/10). A vN-source is a distribution X=(X1,..,Xn) s.t.

X1,..,Xn i.i.d. p ≤ Pr[Xi=1] ≤ 1-p.

An SV-source is a distribution X=(X1,..,Xn) s.t. Source bits can be correlated. Every next bit is somewhat unpredictable. More formally, 8i, 8x1,..,xi-12{0,1},

p ≤ Pr[Xi=1|X1=x1,..,Xi-1=xi-1] ≤ 1-p.

Thm: [SanthaVazirani86] No extractors for such sources.Historically => research on other notions of extractors.

X has entropy ≥ pn.

Bit-fixing sources [ChorGoldreichFriedmanHastadRudichSmolensky85]

Let k be a parameter.

A k-bit-fixing source is a distribution X=(X1,..,Xn) s.t. k bits are uniformly distributed. remaining n-k bits are fixed to arbitrary values.

Easy to extract one bit: E(X1,..,Xn)=parity(X1,..,Xn)

Thm: [CGFHRS] Impossible to extract 2 bits with zero error for k<n/3.

Probably not a good example for “extraction story”.Naturally arise in cryptographic scenarios.

x1

x2

x3

xn

k random bits

(Non-interactive) Privacy amplification

Alice and Bob share a uniformly chosen key Z2{0,1}n.

Can use random key to encrypt communication on public channel.

Eve somehow learns n-k bits of key.

Alice and Bob don’t know which bits.

Eve’s view: Z is a k-bit fixing-source.

Eve’s view: E(Z) is (close to) uniform. E(Z) is a new and secure key.

Motivates extractors: Extract many bits (hopefully k bits). Explicit (poly-time computable).

Alice

Bob

Eavesdropper

public channel

Z2R{0,1}n

Z2R{0,1}n

Use bit-fixing source

extractor .

E(Z) E(Z)

k random bits

From my point of view Z is distributed like:

Extract m=(1-o(1))k bits

[CGFHRS85][CohenWigderson89][KampZuckerman07]

[GabizonRazShaltiel06][Rao09]

Affine sources Let F be a finite field (typically F2={0,1}).

An affine source is a distribution that is uniform over some affine subspace with dimension k of Fn.

Affine sources generalize bit-fixing sources. Extractor is E:Fn!{0,1} is in particular “anti-linear”:

non-constant on any affine subspace of dimension k. (In extractor jargon, this is called a “disperser”).

Exist for k=O(log n) by probabilistic method. Explicit constructions: (poly-time computable)

Extractor : k=o(n) [Bourgain07]. Disperser: k=no(1) (“anti-linear function”) [Shaltiel11].

Feasibly samplable sources [Blum86,TrevisanVadhan00].

Sources defined by considering an allowed “sampling process”.

Source distribution = Sampler(uniform bits). Restrictions on complexity of sampler

induces family of sources. Small space, Small circuits, Constant depth

circuits… [TV00,KampRaoVadhanZuckerman06, KonigMaurer05,Shaltiel06,Viola11,DeWatson11].

Orthogonal notion of “Feasibly recognizable sources” suggested in [Shaltiel09].

Source uniform on {x:P-1(x)=1} for some procedure P. Restrictions on complexity of procedures induce family.




sources.


bits.

Multiple sources extractors No deterministic extractors for SV-sources. Possible if you get samples from two independent

sources!

Can allow a more general family than SV-sources. C={distributions X with “high entropy”}. Best we can hope for.

X

n n

Y

2-sourceextractor

Dfn: (min-entropy)

X has min-entropy ≥ k if ∀x: Pr[X=x] ≤ 2-k

“Can hope to extract k random bits from X”.

Seen examples of sources with min-entropy ≥ k.

vN-sources. SV-sources. Bit-fixing sources. Affine sources.

Another example: flat distributions: X uniformly distributed on a subset of size 2k of {0,1}n.

subset flat distribution

Measuring the entropy of the source distribution


Distribution X over n bits

]Pr[

1logmin)(H

}1,0{ xXX nx

]Pr[

1logExp)(H

}1,0{ xXX nx

size 2k

{0,1}n

A more stringent variant of Shannon entropy

X

Formal definition of Multiple sources extractors

Definition: (emerged from [SanthaVazirani86]) A (k,ε)-2-source-extractor is a function E(x,y) s.t. for every two independent dist. X,Y over n bit strings each having min-entropy ≥ k, E(X,Y) is ε-close to uniform.

Realistic model for generating random bits.Unfortunately, we don’t have good explicit

constructions.

X

n n

Y

2-sourceextractor

Can be generalized to

t>2 sources.

Explicit 2-source extractors imply explicit Ramsey graphs

2-source extractor E(x,y) that outputs one bit is a matrix (w.l.o.g. symmetric)

Property: Every X x Y rectangle of size 2k is balanced.

Þ Every X x X rectangle of size 2k is not monochromatic.

Þ Adjac. matrix of a 2k -Ramsey graph: Graph with no 2k -clique or 2k -independent set.

Explicitly constructing r-Ramsey graphs for small r is a longstanding open problem.

0 0 1 0 0 0 0 1 0

0 1 0 0 1 1 1 0 1

1 0 1 1 1 0 0 1 0

0 1 1 0 0 0 0 1 0

0 1 0 1 0 0 1 1 0

1 0 1 0 0 0 1 0 0

0 1 0 1 0 1 1 0 1

1 0 1 0 1 1 0 1 0

0 1 0 1 0 0 1 0 0

2n

X

Y

x

yX

Explicit constructions of 2-source extractors and Ramsey graphs

2k-Ramsey graphs on 2n nodes

Erdős 47: Probabilstic method achieves k≈log n

Frankl and Wilson 81: Explicit construction k≈(n log n)½

[BKSSW05,BRSW06]: Explicit construction k=no(1)

(Extractor techniques).

Construct bipartite Ramsey graphs (stronger than Ramsey graphs but weaker than 2-source extractors).

(k,ε)-2-source extractors

Probabilstic method achieves k≈log n

Chor and Goldreich 88: E(x,y)=<x,y>mod 2 works for k ≥ n/2.

Bourgain 05: Explicit construction k=0.4999n.

Progress on t-source extractors [BIW04,BKSSW05,Rao06].

Rao06: extract from log n/log k sources with min-ent k.




sources.


bits.

We allow an extractor to also receive an additional seed of (few) independent random bits.

Makes sense as long as: # bits extracted > seed length.

Handle all high min-entropy sources!

Seeded extractors [NisanZuckerman92]

source dist. X on n bits

Extractorseed

Y

random output

Randomness

Definition: A (k,ε)-extractor is a function E(x,y) s.t. for every dist. X with min-entropy ≥ k, E(X,Y) is ε-close to uniform .

Lower bounds [RadhakrishnanTaShma98]: seed length ≥ log(n-k) + 2log(1/ε)

Probabilistic method: Exists optimal extractor which matches lower bound and extracts all the k random bits in the source distribution.

Explicit constructions: E(x,y) can be computed in poly-time.

uniformly distributed

Current milestones in explicit constructions: [LuReingoldVadhanWigderson03, GuruswamiUmansVadhan07,DvirWigderson08,DvirKoppartySarafSudan09].“Optimal up to constants”: seed = O(log(n) + log(1/ε)) output (k) bits.For constant error: seed = O(log(n)) output (1-o(1))∙k bits.

Simulating randomized algorithms using weak random sources

Goal: Run rand algorithm with a weak random source of randomness.

Where can we get a seed?Idea: Go over all seeds. Given sample X from

source. ∀y compute zy= E(X,y) Compute Alg(input,zy) Answer majority vote.

seed=O(log n)=>poly-time.Explicit constructions.

Unsuitable for crypto protocols.


input

output

random coins

RandomnessExtractor

seed

source dist. X on n bits

Something about the tools used in explicit constructions

2-wise independent hash functions [ImpagliazzoLevinLuby89,NisanZuckerman92]. E(x,h)=h(x),h where h is chosen from small family of 2-

wise independent hash functions. Disadvantage: huge seed.

List decodable error correcting codes [Trevisan99]. E(x,y)=Enc(x)y,y where Enc is a binary list decodable error

correcting code. (also works vice-versa). Rate ≥ 1/poly(n) => logarithmic seed. Disadvantage: extract only one additional bit. Can try and exploit properties of specific codes

[TaShmaZuckeramanSafra01,ShaltielUmans01,GuruswamiUmansVadhan07

]. Various composition methods […]

long seedextractor

Composing short seed extractor with long output extractor

x1

x2

x3

xn

k bits of min-entropy

short seedextractor

Short random output

Seeded Extractors are only guaranteed to work when the source and seed are independent.

correlated!

!long random output

Nevertheless, many constructions make this “go through” by modifying initial extractors to have additional properties.

Seeded extractors as graphs with “volume expansion”.

Extractor is a bipartite graph.

Given extractor E(x,y) N=2n (# of inputs) M=2m (# of outputs) K=2k (# of source

elements) D=2d (# of seeds)

Connect x to E(x,1),..,E(x,D).

Small seed length d ~ log n => small deg D ~ log N.

D=2d edges

x

N≈{0,1

}n

M≈{0,1}m

E(x,1)

E(x,D)

..

Extractor graphs: volume expansion property

Extractor property:∀dist X of min-

entropy≥k,E(X,Y) ε-close to

uniform.

=>“expansion” property:

∀set X of size K=2k ,|Γ)X)| ≥ (1-ε)M.

Such graph/function is called “Disperser”.

X

N≈{0,1

}n

M≈{0,1}m

K=2k Γ(X)

(1-ε)M

Extractors and Expander graphs

X

N≈{0,1

}n

M≈{0,1}m

Γ(X)

(1-ε)M

Extractor

N≈{0,1

}n

X Γ(X)

D=2d edges

(1+δ-)Expander

(1+δ)KK

N≈{0,1

}n

K=2k

Volume expansion:

K -> (1-ε)M

K/N -> (1-ε)

Extractors and Expander graphs

X

N≈{0,1

}n

M≈{0,1}m

Γ(X)

(1-ε)M

Extractor

N≈{0,1

}n

X Γ(X)

(1+δ-)Expander

(1+δ)K

N≈{0,1

}n Size expansion:

K -> (1+δ)K

K K=2k

Extractors produce better results in some applications of

expanders

Expanders with expansion that beat the eigenvalue bound [WigdersonZuckerman93]

Goal: Construct low deg expanders with huge expansion.

Line up two low degree extractors.

∀set X of size K , (where K<<N)|Γ)X)| ≥ (1-ε)M > M/2.∀sets X,X’ of size KX and X’ have common

neighbour. Contract middle layer. Bipartite graph in which

every set of size K sees N-K vertices.

Trivially degree ≥ (N-K)/K ≈ N/K.

Obtain low degree ND2/K. Eigenvalue methods cannot

yield graphs with such parameters.

N≈{0,1

}n

N≈{0,1

}n

X

X’

v1

v2 v3

vD

Randomness efficient (oblivious) sampling using expanders [AjtaiKormlosSzemeredi87]

Random walk variables v1..vD behave like i.i.d:

∀A of size ½M Hitting property:

Pr[∀i : vi∊A] ≤ δ = 2-Ω(D). Chernoff style property:

Pr[#i : vi∊A far from exp.] ≤ δ = 2-Ω(D).

# of random bits used for walk: m+O(D)=m+O(log(1/δ))

# of random bits for i.i.d. m∙D=m ∙ O(log(1/δ))

M≈{0,1}m

Random walk on constant degree

expander

Randomness efficient (oblivious) sampling using extractors [Sipser86,Zuckerman96]

Given parameters m,δ: Use E with k=m,

n=m+log(1/δ) ε<½ and small seed d.

Choose random x: m+log(1/δ) random bits.

Set vi=E(x,i)

Expansion property ⇒ Hitting prop.

∀A of size ½MCall x bad if ∀i: E(x,i) inside A.# of bad x’s < K=2k

Pr[x is bad] < 2k/2n = δ

D edges

x

N≈{0,1

}n

M≈{0,1}m

E(x,1)

E(x,D)

..

bad x’s

(1-ε)M

A

Every (oblivious) sampling scheme yields an extractor

An (oblivious) sampling scheme uses a random n bit string x to generated D random variables.

Thm: [Zuckerman06] if the scheme has sampling property then the derived graph is an extractor.

Extractors oblvs Sampling

D=2d edges

x

N≈{0,1

}n

M≈{0,1}m

E(x)1

E(x)D

..

Extractors come in several flavors and have many applications in diverse fields.

Goal: Explicitly construct extractors with parameters that match existential bounds.

Many open problems.

See article in proceedings for more details.

Conclusion



RandomnessExtractor

input

output

Thank You…

Daddy, can you tell me that story

again?

an introduction to randomness extractors

Documents