data reduction for multivariate analysisise.tamu.edu/inen614/chapter 3_p3.pdf · data reduction for...

ISEN 614 Advanced Quality Control (Anomaly and Change Detection) Dr. Yu Ding

1

Data reduction for multivariate analysis

• Using T2, m-CUSUM, m-EWMA can help deal with the multivariate

detection cases. But when the characteristic vector x of interest is of

high dimension, it is difficult to perform a task of detection.

• In a high dimension, the noise components can add up to a great

magnitude, even if individual ones are relatively small. As a result, the

aggregated noise effect can overwhelm the signal effects and makes it

harder to reject the null hypothesis. This is known as "curse of

dimensionality."

• On the other hand, it may not be necessary to use the original high

dimensional data vector to perform a detection task. By the principle of

effect sparsity, it is always the "vital few" instead of the "trivial many"

that matters. if one can extract the so-called "vital few", then the task

of detection can be conducted in a (much) lower data dimension.

• This process of mapping a high-dimension data vector to a low-

dimension "vital few" is data reduction.


2

Data reduction: principal component analysis

• We will introduce the principal component analysis (PCA) as the data

reduction tool here.

• Basic idea: look at the 2-D data cloud, one can easily notice that in

certain direction, the variance is larger than those in the other

directions. If a method can help identify all the major directions where

most of the variability exist, those are the "vital few" effects to be

monitored. A set of transformed variables along the "vital few"

directions is called "principal components (PC)."


3


• Definition of principal component (PC)

- A short answer: a PC is a particular linear combination of elements

in the original vector xp1, which has the largest variation.

- A formal mathematical definition


4


• Definition of principal component (PC)

- PCA is to find ai's such that variances of y's are maximized, i.e.,

How to find the PCs?


5


• PCA: find the ai's that give us the PCs.

• Recall that Result 3.6, the spectral decomposition:

E, the eigenvector matrix, can transform a set of correlated variables

into a set of uncorrelated variables. As it turns out, the transformed

variables also have the largest variability along their corresponding

direction. So we should let ai = ei, the eigenvector.


6


• Result 3.7 (principal component analysis):


7


• Example 3.5:

• Find the eigenvalues/eigenvectors of x

- Use MATLAB function eig(.) but notice that the MATLAB function

arrange the eigenvalues in ascending order.


8


• Example 3.5: Principal components


9


• Example 3.5: a graphic illustration


10


• PCA can also be applied to a correlation matrix .


11


• Revisit Example 3.5.


12


• More remarks


13


• After applying PCA to the original data set, we will have the same

number of PCs as the number of elements in the original vector, In

order to reduce the data dimension, we can only retain the first few

principal components, corresponding to the largest values in

eigenvalue.

• So the question is how to decide the number of PCs to be kept?

- Pareto plot of eigenvalues: With the eigenvalues ordered from

largest to smallest, select the first m eigenvalues (and the

corresponding PCs) if their aggregated effects can explain more

than certain percentage (say, 85%) of the total variation in the data.

- Scree plot: With the eigenvalues ordered from largest to smallest,

a scree plot is a plot of i versus i. We look for an elbow (bend) in

the plot. The number of components is taken to be the point at

which the remaining eigenvalues are relatively small and all about

the same size.


14


• Pareto and scree plots

Pareto plot Scree plot


15


• So the question is how to decide the number of PCs to be kept?

- Minimum description length (MDL) criterion (a more objective

criterion):


16

Data reduction: Example 3.6

• Example 3.6: data reduction and detection in a forging process.

• Data are obtained by strain sensors mounted on the supporting pillars

of a forging press. They are in the form of profile signals (four of those

are displayed in the earlier slide of Chapter 3). Each profile is

digitalized into a vector of p=224 dimension. The historical dataset has

a total of n = 530 profile signals. The data set is denoted as

{xi} i =1 , …, 530 and each xi is a 2241 vector.

Tonnage

SensorsShut

Height

Punch

Speed

Die

Forging Press

Tonnage

Sensors

Gib

Bearing

Tie Rod

Linkage

Slide

Flywheel

Upright

Bolster

Crown

Bed


17


• Example 3.6: here the objective is to perform a Phase I analysis that

separate the in-control from the rest of the data.

• sample statistics

• Perform a PCA on S (substitute for )

- Use the MDL criterion

it will keep 33 eigenvalues

0 10 20 30 40 50 60 70 80 90 1000.8

1

1.2

1.4

1.6

1.8

2

2.2x 10

5

principal components

MDL

values


18


• Perform a PCA on S (substitute for )

- Scree plot

- Finally retain the first three PCs.

1 2 3 4 5 6 7 8 9 100

1000

2000

3000

4000

5000

6000

7000

8000

9000

10000

eig

en

va

lue

s

principal components


19


• We use a multiple univariate detection charts on the first three PCs.

One retionale that we can do this is because the PCs are uncorrelated

so monitoring the individual PCs will not miss out the change in

correlation in the original signals.

• We choose α = 0.0027 for individual charts so that the combined α for

the whole procedure is 1 - (1 - 0.0027)3 = 0.0081, three times higher

than the individual charts.

• Recall that this is a Phase I analysis. After the analysis, we need to

remove the out-of-control data points (seg # 1 and seg #3) and use the

"in-control" data to establish the baseline for future monitoring and

detection.


20


• The control charts: observe three major segments

0 100 200 300 400 500 600-500

0

500P

C1

0 100 200 300 400 500 600-200

0

200

PC

2

0 100 200 300 400 500 600-200

0

200

index of cycles

PC

3


21


• Average of the original signals corresponding to the three segments:

the different is much more subtle to notice than in the PCs.

0 50 100 150 200 250 300 350-200

0

200

400

600

800

1000

1200

1400

Crank Angle (degree)

To

nn

ag

e (

ton

)

Seg # 1

In-Control

Seg # 3

data reduction for multivariate analysisise.tamu.edu/inen614/chapter 3_p3.pdf · data reduction for...

Documents