PROBABILITY

Contents

1. Integration - The abstract integral 11.1. Lp-Spaces 11.2. Modes of Convergence 21.3. Limit Theorems 41.4. The Radon-Nikodyn Theorem 62. Product Spaces 72.1. Finite products 7

1. Integration - The abstract integral

1.1. Lp-Spaces. (,F ,P) is a probability space. We can define for all p (0,)and all random variables X : R,(1) Xp = (E{|X|p})1/pprovided that the integral exists-i.e that the |X|pis P-integrable.Definition 1.1. The space Lp(P ) is the collection of all random variables X : R that are p times P -integrable. More precisly, these are the random variableX such that |X|p

2 PROBABILITY

|fgd| f2g2

Definition 1.5. A function : R R is covex if

(2) (x+ (1 )y) (x) + (1 )(y)Jensen Inequality. Suppose is a probability measure. If : R R is

convex and (f) and f are integrable, then(f)d (

fd)

Example 1.6. Since (x) = |x| is convex. Jensens inequality extends the tri-angle inequality:

|f |d | fd|. Another example is the inequality efd exp(

fd), valid because (x) = ex is convex.

Proposition 1.7. If () p > 1, then Lr() Lp(). In fact,(3) fp [()]

1p 1r fr

Fix some p 1 and define d(f, g) = f gp for all f, g Lp(). According toMinkowskis inequality, d has the following properties

(1) d(f, f) = 0;(2) d(f, g) d(f, h) + d(h, g); and(3) d(f, g) = d(g, f).

In other words, if it were the case that d(f, g) = 0 f = g, then d(, ) wouldmetrize Lp(). Unfortunately, the latter property does not hold in general. Foran example consider g = f1A where A 6= and (Ac) = 0. Evidently then g 6= fbut d(f, g) = 0.

Nonetheless, if we can identify the elements of Lp() that are equal to each otheroutside a null set, then the resulting collection of equivalence classes - endowedwith the usual quotient topology and Borel -algebra is indeed a metric space. Itis also complete; i.e., every Cauchy sequence converges.

Theorem 1.8. Let (,F , ) denote a -finite measure space. For any f, g Lp(), write f g iff f = g a.e. That is, ({ : f() 6= g()}) = 0. Then is an equivalent relation on Lp(). Let [f ] denote the -orbit of f ; i.e, f [f ] ifff g. Let Lp() = {[f ] : f Lp()} and define [f ]p = fp. Then, Lp() is acomplete normed linear space. Moreover L2() is a Hilbert space.1.2. Modes of Convergence. Throughout, (,F , ) is a measure space, andf, f1, f2, : R.Definition 1.9. We say that fn converges to f -almost everywhere (written a.e., a.e.[] or even a.e.) if

PROBABILITY 3

(4) ({ : limsupn

|fn() f()| > 0}) = 0

We write {f A} for { : f() A} and {f A} for ({f A}). Inthis way, fn converge to f a.e iff {fn 6 f} = 0. When (, (F ),P) is a probabilityspace and X,X1, X2, . . . are random variables on this space, we say instead thatXn converges to X almost surely (written a.s.).

Definition 1.10. We say that fn f in Lp() if limnfn fp = 0. Also,

fn f in measure if limn

{|fn f | } = 0 for all > 0. If X,X1, X2, . . . arerandom variables on the probability space (,F ,P), the we say that Xn converges toX in probability when Xn X in P -measure; that is, if limnP{|XnX| } = 0for all > 0. We write this as Xn

P X.Theorem 1.11. Either a.e. convergence or Lp-convergence implies convergence inmeasure. Conversely, if sup

jn|fj| 0 in measure, then fn 0 almost everywhere.

Before presenting the proof, we need some concepts.

Definition 1.12. Markovs inequality If f L1(), then for all > 0,

(5) {|f | } 1

{f}

|f |d f1

Proof. Set = {|f | } and note that |f |d

d = (). This yields

the first inequality. The second is obvious.

We can apply the preceding to the function |f |p to deduceDefinition 1.13. Chebyshevs inequality For all p, > 0 and f Lp(),

(6) {|f | } 1p

{|f |}

|f |pd fpp

p

Proof. Theorem 1.11By Chebyshev inequality, Lp()-convergence implies convergence in measure. In

order to prove that a.e.-convergence implies convergence in measure we first needto understand a.e.-convergence a little better.

Note that fn f a.e. if and only if (N=1 n=N {|fn f | }) = 0 for all > 0. Since is continuos,

(7) fn f a.e. limN

(n=N{|fn f | }) = 0

4 PROBABILITY

Because {|fN f | } (nN{|fn f | }, if fN f a.e. then fN f inmeasure. Finally, if sup

jn|fj| 0 in measure then

(8) (N=1 n=N {supjn|fj| }) = lim

N({sup

jN|fj| }) = 0

Thus, the lim supm|fm| lim sup

nsupjn|fj| < a.e. []. If N() denotes the set of

s for which this inequality fails, then Q+N() is a null set of which limm|fm| <

for every rational > 0 i.e., off Q+ we have limm|fm| = 0.

1.3. Limit Theorems.

(1) integration is a positive operation (i.e. if f 0 then fd 0);(2) it is a linear operation

Theorem 1.14. Bounded Convergence.Suppose f1, f2, . . . are measurable functions on (,F) such that supn | fn | is

bounded by a constant K. If fn f in measure [], then limn

fnd =

fd.

Lemma 1.15. Fatous Lemma. If {fi}i=1 is a collection of non negative inte-grable functions on (,F , ), then

limn

inf fnd limn

inf

fnd

Proof. Let gn = infjn

fj and observe that gn f = limn

infkfk asn . Inparticular, for any constant K > 0, (f K gn K) is a bounded measurablefunction that converges to 0 as n. Because gn fn, the bounded convergencetheorem implies that

limn

inf

fnd lim

n

)gn K)d =

(f K)d.

Therefore, it suffices to prove that

(9) limK

(f K)d =

fd.

For all > 0 we can find a simple function S such that:

(1) 0 S f ;(2) there exists C > 0 such that S() C;(3)Sd fd .Now (f K)d (S K)d = Sd fd

if K > C. This proves (9)

PROBABILITY 5

Theorem 1.16. Monotone Convergence TheoremSuppose {fn}n=1 is a sequence of non-negative integrable functions on (,F , )

such that fn(x) fn+1(x) for all n 1 and x , and f(x) = limn

fn(x) exists

for all x . Then limn

fnd =

fd.

Proof. by monotonicity L = limn

fnd exists and is

fd. Apply Fatous

Lemmato deduce the complementary inequality. Theorem 1.17. The Dominated Convergence Theorem. Suppose {fi}i=1is a sequence of measurable functions on (,F) such that sup

m| fm | is integrable

[d]. Then, limn

fnd =

limn

fnd provided that f(x) = limn

fn(x) exists for

all x .Proof. Thanks to Fatous Lemma f L1(). Also, F = sup

i1| fi | L1() by

assumpiton. Apply Fatous Lemma to the non-negative function gn = 2F |fn f | to deduce that lim

n | fn f | d = 0. The dominate convergence

theorem follows from this and the bound

(10) |fnd

fd |

| fn f | d,

which is merely the triangle inequality for integrals.

Proof. Theorem 1.8The fact that Lp(), and hence Lp(), is a linear space has been established in

Theorem (1.3). From the above we know that d(f, g) = f gp is a norm assoon as we prove that d(f, g) = 0 [f ] [g]. In order to establish completenesssuppose that {fn}n=1 is a Cauchy sequence in Lp(). It sufficies to show that fnconverges in Lp(). Recall that {fn}n=1 is Cauchy means that lim

m,nfnfp

0. Thus, we can find a subsequence {nk}k = 1 such that fnk+1 fnkp 2k.Consequently,

k fnk+1 fnkp < . Thanks to Minkoskis inequlity and the

monotone convergence theorem, k=1 | (fnk+1 fnk | p < . In particular,k(fnk+1 fnk) converges -almost everywhere.If f =

k(fnk+1 fnk) then f Lp() by Fatous Lemma. By the triangle

inequality for Lp-norms,

(11) f fnkp

j=k+1

fnk+1 fnkp 0

as k .Minkoskis inequality implies that

6 PROBABILITY

(12) f fNp f fnkp + fnk fNpTherefore we can let N and k tend to to see that fn f in Lp().

By the Holder inequality f, g = fgd is an inner product. Therefore, L2()

is a Hilbert space.

1.4. The Radon-Nikodyn Theorem. Given two measures and v, When canwe find a function pi such that for all measurable sets A, v(A) =

Apid. If

is Lebesgue measure, then the function pi denotes a probability density function,and the prescription v(A) =

Apid defines the probability measure v.

Definition 1.18. Given two measures and v on (,F), we say that v is abso-lutely continuous with respect to (written v

PROBABILITY 7

2. Product Spaces

If A1 and A2 are sets, then their product A1A2 is defined to be the collection ofall ordered pairs (a1, a2) where a1 A1 and a2 A2. We can have infinite-productspaces of the type A1 A2 2.1. Finite products. Suppose (1,F1, 1) and (2,F2, 2) are two finite mea-sure spaces. The is a natural -algebra F1 F2 and a measure 1 2 thatcorresponds to the product set 1 2.

First consider the collection

(14) A0 = {A1 A2 : A1 F1, A2 F2}We add to this all finite disjoint unions of elements of F0 and call the resulting

collection A.Lemma 2.1. The collection A is an algebra, and (A) = (A0).Definition 2.2. We write F1 F2 in place of (A)

Define on A0 as:(15) (A1 A2) = (A1)2(A2) A1 F1, A2 F2

If A1, A2, A0 are disjoint, then we define (i=1Ai) =n

i=1 (Ai). This

constructs on the algebra A in a well-defined manner. Suppose ni=1Ai = mj=1Bjwhere the Ais are disjoint and the Bjs are also disjoint. Then ni=1Ai = ni=1mj=1(Ai Bj) is a disjoint union of nm sets. Therefore,

(16) (ni=1Ai) =ni=1

mj=1

(Ai Bj) = (mj=1Bj),

by symmetry.

Theorem 2.3. There exists a unique measure 12 on (12,F1F2) suchthat 1 2 = on A.Definition 2.4. The measure mu1 2 is called the product measure of 1 and2; the space 1 2 is the corresponding product space, and F1 F2 is theproduct -algebra. The measure space (1 2,F1 F2, 1 2) is the productmeasure space.

Remark 2.5. By induction, we can construct a product measure space (,F , )based on any finite number of measure space i,Fi, i, i = 1, . . . , n : Define =1 2 n,F = F1 F2 Fn and = 1 2 n.Proof. Theorem 2.3