stock market analysis

1

2

• Almost all players on the market (brokerage firm, banks, wall street) use technical analysis for the study of stock price evolution, mostly as a complement for fundamental analysis.

• Even an expert human has a limited ability in analyzing large quantities of correlations between useful data's without the help of a computer.

• Increasing computational power makes way for a new breed of expert analysts: Machine Learning algorithms.

• Of course, the human still plays an important role in valuing, guiding and interpreting the results.

3

• The human is not a good analyzing machine by default:

•People have no conscious experience of most of what is happening in their minds.

• We tend to perceive what we expect to perceive.

• Mind sets for quickly and are resisting to change.

• New information is assimilated to existing images.

• “When the facts change, I change. And what do you do, sir?” Keynes

4

To offer you an image of what the stock market is

To answer the question: Can I predict the price of a stock and if yes, to what extent?

To provide knowledge of some of the latest machine learning methods you can use in order to make profits on the stock market

5

The Stock Market

Is it possible to predict the future price of a stock?

6

The Stock Market

Is it possible to predict the future price of a stock?

Stock Market Analysis: Schools

Fundamental Analysis

Technical Analysis

7

The Stock Market

Is it possible to predict the future price of a stock?

Stock Market Analysis: Schools

Fundamental Analysis

Technical Analysis

Designing a Hidden Markov Model for this problem

8

The Stock Market

Is it possible to predict the future price of a stock?

Stock Market Analysis: Schools

Fundamental Analysis

Technical Analysis

Designing a Hidden Markov Model for this problem

SQL Server Analysis Services

9

The Stock Market

Is it possible to predict the future price of a stock?

Stock Market Analysis: Schools

Fundamental Analysis

Technical Analysis

Designing a Hidden Markov Model for this problem

SQL Server Analysis Services

What’s next?

Review

10

• Earning Per Share

• (Free) Cash Flow per Share

• P/E Ratio

• Trends

• Inflation/Deflation

• Incidental transactions

• Demography

• Liquidity

• Volume

• Market sentiment

• Industry economic strength

11

• Efficient Market Hypothesis

• Dow Theory

• Analysis

12

• If the value of a firm can be related to its financial characteristics (growth prospects, risk profile and cash flows) then we can see if the stock is under or overvalued.

• Assumptions:

• The relationship between value and underlying financial factors can be measured

• The relationship is stable over time

• Deviations from the relationship are corrected in a reasonable time period

13

"We try to stick to businesses we believe we understand. That means they must be relatively simple and stable in character. If a business is complex and subject to constant change, we're not smart enough to predict future cash flows.“ W. B.

14

• Assumptions

• Stock price reflects everything that has or could affect a the company (including fundamental factors and market psychology), removing the need to consider these factors separately.

• Price movements follow established trends.

• History repeat itself.

15

• Problems

• Patterns of financial time series are dynamic and complex.

• We need to balance between long-term trends and short-term sideways.

• We need to determine usefulness of information.

16

•We need:

• A financial expert could apply different unknown strategies. We can model this by considering a set of states where each state is a unknown strategy

• We can make different observations regarding the time series evolution with each strategy. We will need set of observations K. Observations could be Rise, Drop, Large Rise …

• Which strategy we will first apply? A vector of the initial state distribution probability.

• Which strategy is most likely to be applied next? A matrix that contain transition probabilities between states (strategies).

17

18

• Problems

• One observation like the return rate, or LR is not enough. We need to feed our machine with vectors of observations containing for example price and volume.

•Also how do we model data more accurately? For each state we could build more probability functions that describe the data (from different views); also we need to quantify all of this probability distributions and to estimate each distribution’s weight into the final quantity.

• How to smooth the time series, do we eliminate noise than can confuse the interpretation?

• More recent data should be more valuable, that is more sensitive. But if the recent data is in highly volatile periodhow do we treat this situation so the model wouldn’t be affected in the training process?

19

• To model data we will use a Gaussian mixturemodel at each state. A mixture model combines more probability density functions that have the form of a Gaussian distribution.

• For each adjustable distribution we will assign adjustable weights to give more credit to some distribution, that, at the given time period, predicts more accurately.

20

21

22

• The Gaussian mixture must be multivariate in order to accept a vector of observation with a length higher the 1.

• We will use the Gaussian Mixture as a forecast generator.

•What are the parameters for re-estimation?

• For each Gaussian distribution used in each mixture we need to re-estimate the mean, covariance matrix and the weight attached that describes it best, based on the training cases

• A transition probability matrix between states

23

• Now that we found a way to model more accurately the data, we ask:

• At training, how do we capture recent trends?

• We use a training pool that drops the oldest data when a new one is available

• In financial time series recent data has more value than old data. How do we add more value to the recent data?

• We give her more weight.

• Stock market time series are highly volatile. How do we smooth the series?

• Using an exponential moving average method.

24

• Exponential Moving Average

• Solves the problem of the importance of recent data and the problem of needing to smooth the time series in order to eliminate misleading information and concentrate on trends.

• The sensitivity of recent data can be controlled using the Dow Jones Composite Index, which is a less volatile index.

25

• Re-estimation of parameters

26

•

Analysis Services

ServerMining Model

Data Mining Algorithm Data

Source

Your Application

OLE DB/ ADOMD/ XMLA

Deploy

BI Dev

Studio

(Visual

Studio)App

Data

27

• Try a different approach other than Gaussian Mixture

• More work on the EM algorithm

• HMM versus Microsoft Time Series in SQL 2008 (ARIMA + ART)

28

Review

Questions