machine learning prediction of stock markets

32
Machine learning aided prediction of financial markets Nikola Milošević Email: [email protected] Blog: http://inspiratron.org Twitter: @dreadknight011

Upload: nikola-milosevic

Post on 22-Jan-2018

1.150 views

Category:

Economy & Finance


0 download

TRANSCRIPT

Page 1: Machine learning prediction of stock markets

Machine learning aided prediction of financial markets

Nikola Milošević

Email: [email protected] Blog: http://inspiratron.orgTwitter: @dreadknight011

Page 2: Machine learning prediction of stock markets

Who am I?

• BSc and MSc in EE and CS, University of Belgrade, Serbia

• PhD student University of Manchester

• OWASP (Open Web Application Security Project) local and project leader

• Mentor at Google Summer of Code

• Open Source contributor

• About 4 years experience in industry

(Software testing and development – Banking, Telco, SaaS, Mobile)

• Head of Technology at MUTIS Finance Society

Page 3: Machine learning prediction of stock markets

Agenda

• Machine learning introduction

• Finance and its resistance to machine learning

• Approaches to investing• Fundamental

• Behavioral

• Research examples• Predicting long term stock prices using machine learning

• Predicting FOREX using social media

• Conclusion

Page 4: Machine learning prediction of stock markets

What is Machine learning

Page 5: Machine learning prediction of stock markets

How does it work?

• Machine learning is the subfield of computer science that

"gives computers the ability to learn without being explicitly programmed" (Arthur Samuel, 1959)

Page 6: Machine learning prediction of stock markets

Types of Machine learning

Page 7: Machine learning prediction of stock markets

Achievements of ML

Page 8: Machine learning prediction of stock markets

What about handling money?

• Not that much

• Fund manager often can’t understand the model

• Who is accountable for losses?• Data scientist/Quant/Software engineer?

• Fund manager?

• Data?

• Algorithm?

• Computer?

Page 9: Machine learning prediction of stock markets

Machine learning as a black box or black magic

Page 10: Machine learning prediction of stock markets

Understanding models

• Machine learning ≠ black box

• Machine learning ≠ black magic

• Complexity matches the problem• Rules are not simple!

• Complex problems – complex solutions (models)

• Shortcut understanding does not work

• Right cross validation, right dataset

• Machines make as many mistakes as human do

(in many cases even less)

Page 11: Machine learning prediction of stock markets

Approaches in finance

• H: Machine can learn same things human can

Behavioural finance

Long-term investing Short-term investing

Page 12: Machine learning prediction of stock markets

Technical analysis - traditionally

• Graham criteria (1949)• Stock Selection for the Defensive Investor:

• 1. Not less than $100 million of annual sales.

• [Note: This works out to $500 million today based on the difference in CPI/Inflation from 1971]

• 2-A. Current assets should be at least twice current liabilities.

• 2-B. Long-term debt should not exceed the net current assets.

• 3. Some earnings for the common stock in each of the past 10 years.

• 4. Uninterrupted [dividend] payments for at least the past 20 years.

• 5. A minimum increase of at least one-third in per-share earnings in the past 10 years.

• 6. Current price should not be more than 15 times average earnings.

• 7. Current price should not be more than 1-1⁄2 times the book value.

• Graham number = 22.5 ∗ 𝐸𝑃𝑆 ∗ 𝐵𝑉

Page 13: Machine learning prediction of stock markets

Technical analysis

• Analyse technical indicators and ratios• Over time

• Dependencies

• Graham model too strict

• It is hard to develop new models • Mathematically demanding

• People are not good with numbers

• Time consuming

• Can machines help?

Page 14: Machine learning prediction of stock markets

Predicting long term movement of stock price

• Use machine learning on past 2-3 year data

• Data obtained using Bloomberg terminal

• Data include 28 indicators

• Book value, Market capitalization, Change of stock Net price over the one month period,

Percentage change of Net price over the one month period, Dividend yield, Earnings per share,

Earnings per share growth, Sales revenue turnover, Net revenue, Net revenue growth, Sales

growth, Price to earnings ratio, Price to earnings ratio -five years average, Price to book ratio,

Price to sales ratio, Dividend per share, Current ratio, Quick ratio, Total debt to equity, margins,

asset turnover…

Page 15: Machine learning prediction of stock markets

Predicting long term movement of stock price (2)

• Selected 1739 stocks from different indexes (S&P 1000, FTSE 100

and S&P Europe 350…)

• Calculated which ones price grew more than 10%

• Used different Machine learning algorithms and 10 fold cross validation for evaluation

• Used Python for scripting and Weka toolkit for machine learning

Page 16: Machine learning prediction of stock markets

Results

• Trial with all financial indicators as a features

Page 17: Machine learning prediction of stock markets

Results (2)

• We performed feature selection

• Experiment with only 11 indicators

Page 18: Machine learning prediction of stock markets

11 indicators

• The performance turned out not to be significantly different, but it showed that only 11 indicators are enough

Page 19: Machine learning prediction of stock markets

FinAnalyzer

• Tool that uses a model and Yahoo hidden API

• https://github.com/nikolamilosevic86/FinAnalyzer

• Open source (GPLv3)

• Decision support

• Not liable of any loses

• Join development

(contact me)

Page 20: Machine learning prediction of stock markets

Behavioural finance

• Psychology and emotion influence decision

• People follow their peers

• Sentiment of the information is shaping the decision

Page 21: Machine learning prediction of stock markets

Social media and news

Page 22: Machine learning prediction of stock markets

Sentiment and FOREX

Page 23: Machine learning prediction of stock markets

Major forex pairs

Page 24: Machine learning prediction of stock markets

System overview

Page 25: Machine learning prediction of stock markets

Sentiment classification

Page 26: Machine learning prediction of stock markets

Twitter sentiment - challenges

• Short text – dense information

• Ungrammatical language

• Abbreviations

• Typos

• Emoticons

• Links, hashtags, mentions

Page 27: Machine learning prediction of stock markets

Evaluation - sentiment

Page 28: Machine learning prediction of stock markets

Correlations – sentiment vs pair

Page 29: Machine learning prediction of stock markets

Correlations with sentiment (What we found?)

• There is correlation between sentiment and market movement

• In time it is variable (some times 5 minutes, some times 5 hours)

• Some times ambiguous

• More research necessary

• When to enter/exit?

Page 30: Machine learning prediction of stock markets

Conclusion

• Finance is all about information

• Information flooded world

• Machine learning, data science, text mining• Here to HELP! Try them

• Predicting without emotional influence

• Accountability

• Consider all information for better prediction• More than human can digest

Page 31: Machine learning prediction of stock markets

Reference

• Milosevic, Nikola. “Equity forecast: Predicting long term stock price

movement using machine learning. ” arXiv preprint arXiv:1603.00751 (2016).

• https://arxiv.org/ftp/arxiv/papers/1603/1603.00751.pdf

• Martic, Miljan. “Twitter sentiment analysis for foreign exchange market

movement orediction.” (2014).

Page 32: Machine learning prediction of stock markets

Email: [email protected]: http://inspiratron.orgTwitter: @dreadknight011