principle component analysis
DESCRIPTION
Principle Component Analysis. Presented by: Sabbir Ahmed Roll: FH-227. Overview. Variance and Covariance Eigenvector and Eigenvalue Principle Component Analysis Application of PCA in Image Processing. Variance and Covariance(1/2). - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/1.jpg)
Principle Component Principle Component AnalysisAnalysis
Presented by:Sabbir Ahmed
Roll: FH-227
![Page 2: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/2.jpg)
OverviewOverviewVariance and CovarianceEigenvector and EigenvaluePrinciple Component AnalysisApplication of PCA in Image
Processing
2
![Page 3: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/3.jpg)
Variance and Variance and Covariance(1/2)Covariance(1/2)The variance is a measure of
how far a set of numbers is spread out.
The equation of variance is
1)var( 1
n
xxxxx
n
iii
3
![Page 4: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/4.jpg)
Variance and Variance and Covariance(2/2)Covariance(2/2)Covariance is a measure of how
much two random variables change together.
The equation of variance is
1
))((),cov( 1
n
yyxxyx
n
iii
4
![Page 5: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/5.jpg)
Covariance MatrixCovariance MatrixCovariance Matrix is a n*n matrix
where each element can be define as
A covariance matrix over 2 dimensional dataset is
),cov( jiM ij
),cov(),cov(),cov(),cov(yyxyyxxx
M
5
![Page 6: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/6.jpg)
EigenvectorEigenvectorThe eigenvectors of a square matrix A are
the non-zero vectors x such that, after being multiplied by the matrix, remain parallel to the original vector.
1112
33
33
6
![Page 7: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/7.jpg)
EigenvalueEigenvalueFor each Eigenvector, the corresponding Eigenvalue is the factor by which the eigenvector is scaled when multiplied by the matrix.
1112
33
33
1
7
![Page 8: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/8.jpg)
Eigenvector and Eigenvalue Eigenvector and Eigenvalue (1/2)(1/2)The vector x is an eigenvector of
the matrix A with eigenvalue λ (lambda) if the following equation holds:
0)(,0,
xIAorxAxor
xAx
8
![Page 9: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/9.jpg)
Eigenvector and Eigenvalue Eigenvector and Eigenvalue (2/2)(2/2)Calculating Eigenvalues
Calculating Eigenvector
0 IA
0)( xIA
9
![Page 10: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/10.jpg)
Principle Component Analysis Principle Component Analysis (1/3)(1/3)PCA (Principle Component
Analysis) is defined as an orthogonal linear transformation that transforms the data to a new coordinate system such that the greatest variance comes to lie on the first coordinate, the second greatest variance on the second coordinate and so on.
10
![Page 11: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/11.jpg)
Principle Component Analysis Principle Component Analysis (2/3)(2/3)
11
![Page 12: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/12.jpg)
Principle Component Analysis Principle Component Analysis (3/3)(3/3)
12
![Page 13: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/13.jpg)
Principle ComponentPrinciple ComponentEach Coordinate in Principle
Component Analysis is called Principle Component.
Ci = bi1 (x1) + bi2 (x2) + … + bin(xn)
where, Ci is the ith principle component, bij is the regression coefficient for observed variable j for the principle component i and xi are the variables/dimensions.
13
![Page 14: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/14.jpg)
Eigenvector and Principle Eigenvector and Principle ComponentComponentIt turns out that the Eigenvectors
of covariance matrix of the data set are the principle components of the data set.
Eigenvector with the highest eigenvalue is first principle component and with the 2nd highest eigenvalue is the second principle component and so on.
14
![Page 15: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/15.jpg)
Steps to find Principle Steps to find Principle ComponentsComponents1. Adjust the dataset to zero mean
dataset.2. Find the Covariance Matrix M3. Calculate the normalized
Eigenvectors and Eigenvalues of M4. Sort the Eigenvectors according to
Eigenvalues from highest to lowest5. Form the Feature vector F using the
transpose of Eigenvectors.6. Multiply the transposed dataset with
F 15
![Page 16: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/16.jpg)
ExampleExample
X Y2.5 2.40.5 0.72.2 2.91.9 2.23.1 3.02.3 2.72 1.61 1.1
1.5 1.61.1 0.9
X Y0.69 0.49-1.31 -1.210.39 0.990.09 0.291.29 1.090.49 0.790.19 -0.31-0.81 -0.81-0.31 -0.31-0.71 -1.01
Original Data Adjusted Dataset16
AdjustedDataSet = OriginalDataSet - Mean
![Page 17: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/17.jpg)
Covariance Matrix Covariance Matrix
716555556.0615444444.0615444444.060.61655555
M
17
![Page 18: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/18.jpg)
Eigenvalues and Eigenvalues and EigenvectorsEigenvectorsThe eigenvalues of matrix M are
Normalized Eigenvectors with corresponding eigenvales are
28402771.10490833989.0
seigenvalue
735178656.0677873399.0677873399.0735178656.0
rseigenvecto
18
![Page 19: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/19.jpg)
Feature VectorFeature VectorSorted eigenvector
Feature vector
677873399.0735178656.0735178656.0677873399.0
rseigenvecto
677873399.0735178656.0735178656.0677873399.0
,
677873399.0735178656.0735178656.0677873399.0
For
FT
19
![Page 20: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/20.jpg)
Final Data (1/2)Final Data (1/2)
X Y-
0.827970186
-0.17511530
71.77758033 0.14285722
7-
0.992197494
0.384374989
-0.27421041
6
0.130417207
-1.67580142 -0.20949846
1-
0.912949103
0.175282444
-0.09910943
7
-0.34982469
81.14457216 0.04641725
820.43804613
70.01776462
971.22382056 -
0.162675287
FinalData = F x AdjustedDataSetTransposed
20
![Page 21: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/21.jpg)
Final Data (2/2)Final Data (2/2)FinalData = F x
AdjustedDataSetTransposed
21
X-0.8279701861.77758033
-0.992197494-0.274210416-1.67580142-0.9129491030.09910943751.144572160.4380461371.22382056
![Page 22: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/22.jpg)
Retrieving Original Retrieving Original Data(1/2)Data(1/2)
FinalData = F x AdjustedDataSetTransposed
AdjustedDataSetTransposed = F-1 x FinalDatabut, F-1=FT
So, AdjustedDataSetTransposed =FT x FinalData
and, OriginalDataSet = AdjustedDataSet + Mean
22
![Page 23: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/23.jpg)
Retrieving Original Retrieving Original Data(2/2)Data(2/2)
23
![Page 24: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/24.jpg)
Application of PCA in Image Application of PCA in Image ProcessingProcessingPattern RecognitionImage CompressionDetermination of Object
Orientation and Rotation
24
![Page 25: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/25.jpg)
QuestionQuestion
?25
![Page 26: Principle Component Analysis](https://reader035.vdocuments.net/reader035/viewer/2022062323/56815d8c550346895dcb98a9/html5/thumbnails/26.jpg)
ReferencesReferencesPrinciple Component Analysis in
Wikipedia http://en.wikipedia.org/wiki/Principal_component_analysis
A tutorial on Principal Components Analysisby Lindsay I Smith http://www.sccg.sk/~haladova/principal_components.pdf
Principle Component Analysis in Image Processing by M. Mudrov´, A. Proch´zkahttp://dsp.vscht.cz/konference_matlab/matlab05/prispevky/mudrova/mudrova.pdf
26