i h t zr f of lcf sb lh3 2t - csaarindamkhan/courses/toolkit... · 2020. 10. 3. · chernoff and...
TRANSCRIPT
MitzenmacherPROBABILITY REFRESHER upfal
I H T
zrFfield setof Lcf SbeventsLH3 2T
r
E EzEz
A collection F of subsets of 52 is called G field ifi DEFCir A EF ACE F
nCiii Aa Az An E F W A E Fi L
4maynot beUnion bound pairwisedisjoint
Inclusion exclusion principle E E2
chain rule Pr II Ei Pr Ei l j EjPro Ei Ez IED Pro I E n Ez i Pr Ek l ni Ei
Repeatedly choosing random numbers acc toa given distribution samplingwith replacement simple to codewithout replacement gives slightly betterbounds
TLIP
Discrete Random Variables Can2and Expectation M O
Defn A random variable X is a functionR V X I IR A discrete R.V is a R V
that takes finite or countably infinitenumber of values
Defn Expectation E X3 i lPCX i
Properties of expectationthem 2.1 C linearity of ExpectationsFor any finite collection of discrete RVsXa Xz Xn with finite expectations
IE IE Xi IE El XiNote we don't need these Rvs to be independent
Lem 2 2 For any constant and discreteRV X E CX C IEEXT
Defn 2.4 convex function A function f IR IRis said to be convex if for any sea Nz and0 E A E I s f case I a Nz E a fGee t I a fCazn
Fun Intense Is 9.5e g fCa x 24 N x3 is
et HelP for PZ 1 partlyconvex
Tkm 2.4 Jensen's Inequality If f is a convex
function then E f x 3 f E x
corollary E x 233 IEEX
Bernoulli Binomial and Geometric RV
Bernoulli Y be a RV s E Y I w P PO w p y PIE Y p I l P O p
e g one coin toss can be modeled by BernoulliAlso called indicator random variable
Binomial X Bin n p is a random variable
taking the values 0,1 2 N andIP X K Ya pk l P
n k where Ocp 21e g n coin toss How many headsuseful in sampling it x E EhXi EH i
ripGeometric X Geom p is a geometric RVif takes values 1 2 3 with IP X K p l p K
e g numbers of coin flips till you get a first treadIEEXT Hp
BE uIE.ommeenntts and EDeviation 94Variance and moments of a random variable
like KthderivativeK the moment of a RV X is F XK forfunctionsvariance of vars X EfcX E 72 ECx2 CECxDcovariance of Rus X YCor X Y IE CX E X Y EY
Th m 3.2 Van EX Y Var X Var Y t 2 CorCXY
Sf X Y are indep then Cor CX Y O and Thm 3.2is like linearity of expectationThem 3.3 IE X Y E X E Y for indep X Y
Theorem 3 I Markov's inequalityFor a nonnegative random variable X t t 0
pm x t E EI or Fn x E EX E Lt E
G G Define fee q for a Ctro fCx for a t
Define gcse Mt1Fact t gGe Z f Cr
Fact 2 ECfcxD O Procx ET t 2 the x t ppCx t
Prs x t E f CX C Fact 27E E g Cx Fact I
E XIE1 Efx scalingE
Theorem 3 2 Chebyshev's inequalityFor any a O p Ix Ex 1 a s Var
of9th w log assume EX 0
_g qVars X L By scalingSo It XZ L
Define fGet 0 for Kl Ct g se I1 for bet t T2
Pro 1 1 t IE Cf xDE IE Eg xD As 9 a z feelE X E xD 1
ApplicationXi 10 if i'th coin flip is head
X Xi denote headselse in a coin flips
E EX up Mz Var X E var Xi n
Markov IP X 374 E F q 3Msihzeusq.netefx
Chebyshev IP X 3374 E P IX EX 1374Tm.IEaT.ndEYn z Yn4q Inriffed
Chernoff and IHoeffding BoundsChernoff bounds are studied for the tail distributionof a sum of independent O 1 random variableswhich are also known as Poisson trials
Note Bernoulli trials are special cases of Poissontrials where independent O 2 RVs have same distr
O 1 RVsTheorem 4.4 4.5 9Let Xi Xn be independent Poisson trials s tP Xi 1 Pi Let X Xi and µ EX ThenPoisson trials deviations above mean
V 8 O P X Clt 8 te s f g ote orE 0,11 IP X 3 Clt d te s e te 3
for 1 E S g e148 3
for O E S g e StgCI187142
for 1236 te p x R g z R s e04 2 87
Poisson trials deviation below mean
to C Co 1 P C x E l 8 te f f y E et
Combined deviation around mean
it 8 E Q1 IP I x te l 3 orte s 2 e M 13
Additive bounds when s are identical i f E o
a 1pct EX i 7 Pt E e e III P E
b PC's Exists E e GEE
III J
Special cases Not 0 1 RvXa Xz Xn be independent Rvs with X Xi and
Known asIP Xi 1 IP Xi 1 Iz s then RademacherV a 0 IP X a e a42n distr
IP X E a g e a42n
IP 1 1 za E zeaten
IP Xi 1 P Xi O L te EX Zthen
I t a O P Y z te ta s e Zak Additive
T 8 o P y 1 8 µ g e Epe Multiplicative43missingunlike
Ciii t a ECO MI IP y E te a E e Zak AdditiveD A JE CO 17 IPC x e l S te s e 82M multiple
Hoeffding Bounds General RVs w BoundedRangeet X Xn be independent random variabless t V I E i E n E Xi µ and IP a Xie b 1
ThenP Int Xi te e e z e 2nE4Cb
as
more general subsumes most boundsLet X Xn be independent random variabless t V I E i E n E Xi te and IP 9 Exists 1
nn ntheIP IE x E tea I e s ze 2EY.IECbi Ail1 1 i i
There are many more tailors made conc inedsurvey by McDiarmid
Independent Bounded Difference InequalityLet X Xa Xz Xn be a family of independentrandom variables with XjE Aj for j 1 2 n
and 0 Ijn Aj R be a function such that
Cse 0 Can e g whenever the vectors se se
differs only in the j th coordinate Then f t O
IP 4 Cx E Cola t E e HYEin g
Example coin flipFrom Chebyshev IP l X 2 I 3 E 4h
Chernoff IP I X 2 I 3 2 s 2e te843
2eMd 2 exp L y ng 2 3 E 2e n124 tighten
In fact we can show the deviations from the meanP I x 2 13 ziF E 2 exp f 2 64
Yn
We will later use Martingales to obtainconcentration for Rvs that are not independent
Conditional ExpectationDefn E LY I Z z E y P Y y 1 Z 3
yIE Y IZ is a RV taking value IECTI Z Z when Z zFor RVs X and Y E X PmCT y E X Y y
conditional variancevars CX1Y E x21 Y E EXIT
Properties of conditional expectationsLet x Y Z be random variables a b E IR g IR IR then
linearity IE xx BY l Z a E Xl Z BE YI Zmonotonicity X E Y it X I Z E E T I Z
In particulars if X 30 E x I 2 3 O
independence if x z are independent thenE X I Z E X
conditional Jensen's ineg If R IR is convexE 4CX IZ 3 E XIZ
rule of average conditional Law of total expectationEy It XI YI E x used crucially inMartingalesE E x g Y Y g Y E EXITIn particular it g Y ly g y
E I Y gCY E XI Y
E it XI Y Z f Y IE XI Y
Law of Total VarianceVar X Ey Var XI Y Vary IECX1YAbove we assume all the expectations exist