stock market analysis and prediction

8/8/2019 Stock Market Analysis and Prediction

1/26

STOCK MARKET ANALYSIS And

PREDICTION

By:

Vivek Bhalgat

Vivek Bijlwan

(under Dr. Ratna Sanyal)


2/26

SATYAM happenedRs. 430

Rs. 6.30 Rs. 117

In a span of 9 months , one could have made his money 18 times!!

OR

One could have cashed in at 430 , when others would sell at Rs. 6.30


3/26

WhyWarren Buffett is the richest man

on Earth? In his own words

The basic ideas of investing are to look at

stocks as business, use the market'sfluctuations to your advantage..

So , what is a fluctuation ?

How to identify it?


4/26

But How?

VALUE

INTRINSIC EXTRINSIC

Intrinsic value, or sometimes known as "Fundamental Value", is the value that remains in

an option when all of its extrinsic value has diminished due to Time Decay. It is the actual

value of a stock that has been built into the price of the option.


5/26


6/26

ICAICA

Blind Signal Separation (BSS) or Independent Component Analysis (ICA) is theidentification & separation of mixtures of sources with little priorinformation.

Applications include:

Audio Processing

Medical data

Finance

Array processing (beamforming)

Coding

and most applications where Factor Analysis and PCA is currently used. While PCA seeks directions that represents data best in a |x0 - x|

2 sense,ICA seeks such directions that are most independent from each other.

We will concentrate on Time Series separation of Multiple Targets


7/26

The simple Cocktail Party ProblemThe simple Cocktail Party Problem

Sources

Observations

s1

s2

x1

x2

Mixing matrix A

x =As

n sources, m=n observations


8/26


9/26

MotivationMotivation

Get the Independent Signals out of the Mixture


10/26

ICA Model (Noise Free)ICA Model (Noise Free)

Use statistical latent variables system(IID)

Random variable sk instead of time signal

xj = aj1s1 + aj2s2 + .. + ajnsn, for all j

x =As

ICs s are latent variables & are unknown AND Mixing matrixA isalso unknown

Task: estimate A and s using only the observeable random vector x

Lets assume that no. of ICs = no of observable mixtures

andA

is square and invertible So after estimating A, we can compute W=A-1and hence

s = Wx = A-1x


11/26

IllustrationIllustration

2 ICs with distribution:

Zero mean and variance equal to 1

Mixing matrix A is

The edges of the parallelogram are in thedirection of the cols of A

So if we can Est joint pdf of x1 & x2 and then

locating the edges, we can Est A.

! 12

32

A

e

!otherwise

sifsp

i

i

0

3||32

1)(


12/26

RestrictionsRestrictions

si are statistically independent

p(s1,s2) = p(s1)p(s2)

Nongaussian distributions

The joint density of unit

variance s1 & s2 is symmetric.So it doesnt contain anyinformation about thedirections of the cols of themixing matrix A. So A canntbe estimated.

If only one IC is gaussian, theestimation is still possible.

!2

exp2

1),(22

21

21

xxxxpT


13/26

AmbiguitiesAmbiguities

Cant determine the variances (energies)of the ICs Both s & A are unknowns, any scalar multiple in one of the

sources can always be cancelled by dividing the correspondingcol of A by it.

Fix magnitudes of ICs assuming unit variance: E{si2}=1

Only ambiguity of sign remains

Cant determine the order of the ICs Terms can be freely changed, because both s and A areunknown. So we can call any IC as the first one.


14/26

ICA Principal (NonICA Principal (Non--Gaussian is Independent)Gaussian is Independent)

Key to estimating A is non-gaussianity

The distribution of a sum of independent random variables tends toward a Gaussiandistribution. (By CLT)

f(s1) f(s2) f(x1) = f(s1 +s2)

Where w is one of the rows of matrix W.

y is a linear combination of si, with weights given by zi. Since sum of two indep r.v. is more gaussian than individual r.v., so zTs is moregaussian than either of si. AND becomes least gaussian when its equal to one of si.

So we could take w as a vector which maximizes the non-gaussianity of wTx.

Such a w would correspond to a z with only one non zero comp. So we get back the si.

szAswxwy TTT !!!


15/26

Measures of NonMeasures of Non--GaussianityGaussianity

We need to have a quantitative measure of non-gaussianity for ICAEstimation.

Kurtotis : gauss=0 (sensitive to outliers)

Entropy : gauss=largest

Neg-entropy : gauss =0 (difficult to estimate)

Approximations

where v is a standard gaussian random variable and :

224 }){(3}{)( yEyEykurt !

! dyyfyfyH )(log)()(

)()()( yyyJ gauss !

_ a222 )(

48

1

12

1)( ykurtyEyJ !

_ a _ a? A2)()()( vGEyGEyJ }

)2/.exp()(

).cosh(log1)(

2uayG

yaa

yG

!

!


16/26

Data Centering &WhiteningData Centering &Whitening

Centeringx =x E{x}

But this doesnt mean that ICA cannt estimate the mean, but it just simplifies theAlg.

ICs are also zero mean because of:E{s}= WE{x}

After ICA, add W.E{x} to zero mean ICs Whitening

We transform the xs linearly so that the x~ are white. Its done by EVD.x~= (ED-1/2ET)x = ED-1/2ET Ax = A~s

where E{xx~}= EDET

So we have to Estimate Orthonormal Matrix A~

An orthonormal matrix has n(n-1)/2 degrees of freedom. So for large dim A wehave to est only half as much parameters. This greatly simplifies ICA.

Reducing dim of data (choosing dominant Eig) while doing whitening alsohelp.


17/26

RESULTS

Data taken

TCS at BSE for the past 400 days.


18/26

Our Data sources : BSE and NSE

BSE

NSE

TCS


19/26

Intrinsic & Extrinsic

TCS


20/26


21/26

Their ICs

HCL Infosys

Wipro


22/26

Correlation(TCS , Infosys)


23/26

And the others

Correlation(TCS,Wipro) Correlation(Infosys,Wipro)


24/26

With other sectors

Correlation(TCS, JK Cement) Correlation(TCS, Reliance)


25/26

Work after Mid-Sem

Wavelet Transform :

Why Wavelet Transform?

Why not Fourier ? Time invariant

Why not Short term Fourier transform ? Heisenbergs Uncertainty

PrincipleWavelet Transform : Multi Resolution Signal Analysis

Unlike the STFT which has a constant resolution at all times and

frequencies, the WT has a good time and poor frequency resolution at

high frequencies, and good frequency and poor time resolution at low

frequencies

Analysis and Evaluation of Results


26/26

http://users.rowan.edu/~polikar/WAVELETS/WTtutorial.html

http://www.cis.hut.fi/aapo/papers/IJCNN99_tutorialweb/

http://en.wikipedia.org/wiki/Independent_component_analy

sis Pierre Comon (1994): Independent Component Analysis: a

new concept?, Signal Processing, Elsevier, 36(3):287--314 (The

original paper describing the concept of ICA)

References:

stock market analysis and prediction

Documents