Download - Lec 3 MN Theory Integration-i Statistics
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
1/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
2/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
3/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
4/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
5/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
6/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
7/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
8/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
9/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
10/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
11/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
12/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
13/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
14/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
15/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
16/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
17/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
18/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
19/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
20/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
21/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
22/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
23/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
24/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
25/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
26/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
27/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
28/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
29/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
30/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
31/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
32/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
33/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
34/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
35/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
36/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
37/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
38/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
39/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
40/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
41/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
42/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
43/60
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
44/60
44
Now we can define the integral.
=
=n
iii Ad
1
)(
Since positive measurable functions can be written as theincreasing pointwise limit of simple functions, we define theintegral as
=
d fd f
sup
+ = d f d f fd
Simple Functions: =
=n
i Ai i
1
1
Positive Functions:
For a general measurable function write+ = f f f
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
45/60
45
This picture says everything!!!!
A
f
This is a set! and theintegral is measuringthe size of this set!
Integrals are likemeasures! Theymeasure the size of aset. We just describethat set by afunction.
Therefore, integralsshould satisfy the
properties of measures.
A
fd How should I think about
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
46/60
46
Measures and integrals are different descriptions of of the same concept. Namely, size.
Therefore, they should satisfy the same properties!!
This leads us to another important principle...
Lebesgue defined the integral so that this would betrue!
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
47/60
47
Left Continuous Left Continuous
Bdd Right Cont. Bdd Right Cont.
etc...
Measures are: Integrals are:
A Ai )()(lim A Ai = f f i = fd d f ilim
A Ai
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
48/60
48
The L p Spaces
A function is in L p(,F, ) if =< fgd g f ,
1 p
E l f N d
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
49/60
49
Examples of Normed spacesLp spaces
Lp={f:C| (|f|pd )1/p
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
50/60
50
Examples of Normed spacesLp spaces (Cont.)
The space is very very large.The space is not only important for Kernel methods
but also for the development of Fucntional Analysis aswell as Foundation of Theory of Statistics
When X is finite, , the counting measure and p=2,we have our well-known, Rk
Checking of Existence of Mean of Cauchy Distribution:
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
51/60
51
Checking of Existence of Mean of Cauchy Distribution:
Let X be a standard Cauchy variate withprobability density function,
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
52/60
52
Checking of Existence of Mean of Cauchy Distribution:Lebesgue Integral Way
2
2 2
0 _
2 20
1( ) 1
1 1 11 1
1 1 1
1 1
x xf xdx dx x
x dx x dx x x
x dx x dx x x
+
+
= +
= + +
= + +
Now,
)
2 20 1
2[1, ), 1
1 1
1nn
x x dx dx
x x x
dx x
== +
>+ +
=+
U
put
dz xdx
z x
=
=+
2
1 2
x n+1 1+(n+1) 2
z n 1+n 2( )2
2
1 ( 1)
112
21
1 log2
1 1 ( 1)log2 1
n
x n
n
z
n
n
+ +
+=
=
= + +=
+
Checking of Existence of Mean of Cauchy Distribution:
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
53/60
53
Checking of Existence of Mean of Cauchy Distribution:
Since 22
2
2
21)1(1
log1
)1(1log
n
n
n
n
+++>
+++
And
)(
1
21
1)1(1
loglim
02
2
2
X E
dx x
xn
nn
+
=+
++
does not
existdoes not exist
Therefore mean of the Cauchy distribution doesnot exist
Checking of Existence of Mean of Cauchy Distribution:
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
54/60
54
g yRiemann Integral Way
Method (i):We knowthat,
( )2
2
221
142
2
Butlim1
1lim l og
21 1
limlog2 1 41
2( )
a
a a
a
a a
a
x dx
x
z
a
a
EX
+
+
+ =
+=+
=
does notexist.
Since is an oddfunction
21 x
x
+2l im 0;
1
a
aa
x dx x
=+
Checking of Existence of Mean of Cauchy Distribution:
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
55/60
55
g yRiemann Integral Way
Method (ii):
20 1
( )
x dx x
EX
+
does not exist byquotient test
does notexist
Therefore, mean of the Cauchy distribution doesnot exist.
2 22
2 2
1 lim 1 01 1 1 x
x x x x
x x
x
+ = = =
+ +
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
56/60
56
Measure-Probability Measure-CumulativeDistribution Function
Bounded measure Probabilitymeasure
CumulativeDistribution
Function
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
57/60
57
Broad Categories of Probability Measures
ProbabilityMeasures
Discrete P(A)=1,#(A)=finite or
Continuous
P{x}=0 for all x
AbsolutelyContinous w.r.t.
Lebesgue measureNon A.C.
OnE
u cl i
d e an
s am pl e
s p a c e
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
58/60
58
A random variable is a measurable function.
)( X
The expectation of a random variable is its integral:
= XdP X E )(A density function is the Radon-Nikodym derivative wrtLebesgue measure:
dxdP f X =
== dx x xf XdP X E X )()(
Expectation
=xi p
i
Counting measure p=dP/d
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
59/60
59
Expectation
Change of variables. Let f be measurable from (, F , v)to (,) and g be Borel on (,). Then
i.e. , if either integral exists, then so does the other, and thetwo are the same.
Note that integration domains are indicated on bothsides of 1. This result extends the change of variableformula for Reimann integrals, i.e. ,
(1)
-
7/30/2019 Lec 3 MN Theory Integration-i Statistics
60/60
In statistics we will talk about expectations with respectto different measures.
P = XdP X E P )(Q = XdQ X E Q )(
)()( X E dQ X dQdQdP
X XdP X E Q P
==== where =
dQdP
or dQdP =
And write expectations in terms of the different measures:
Expectation