deeplearning in finance

Download Deeplearning in finance

Post on 17-Aug-2015

176 views

Category:

Software

2 download

Embed Size (px)

TRANSCRIPT

  1. 1. Deep learning in finance Data science in finance 28 / 07 Sbastien Jehan
  2. 2. CAPITAL FUND MANAGEMENT Fundamental research applied to financial markets CFM invests in Science, Technology and Finance 23 years of experience in managing trading systems A rational process that produces robust Trading Algorithms Proprietary software able to execute & control large volumes Opportunities : Software Engineers mastering C++, Python System, Network & Database Administrators PhDs in theoretical physics, applied mathematics, informatics PROPRIETARY AND CONFIDENTIAL - NOT FOR REDISTRIBUTION
  3. 3. Artificial Neural Networks - History of Artificial Neural Networks - Recurrent Neural Networks - Applications in Finance AI for the enterprise 2015 $220 millions 2025 $11.1 billions (+56% / year)
  4. 4. Current applications to neural networks Medical image processing: mostly feed forward neural network Robokinetics and robovision: smooth moves and object detection Military: DARPA Synapse, objective 10 billion neurons (86 billions human brain) in 2 liters space for end of 2016. Email spam detection, Image classification Text recognition
  5. 5. 10/2013 90-99/100 ON CAPTCHAS
  6. 6. 1957: The perceptron - d D0 D1 D2 Input Layer Output Layer Destinations
  7. 7. 1957: The perceptron D0 D1 D2 Input Layer Output Layer Destinations FEED FORWARD
  8. 8. 1957: The perceptron D0 D1 D2 Input Layer Output Layer Destinations SINGLE LAYER
  9. 9. Teaching - d D0 D1 D2 Input Layer Output Layer Theorical Y0 Y1 Y2 - - - Supervised objective function
  10. 10. Application .NET
  11. 11. Applications Is A or B Linear problem
  12. 12. Applications Is A or B Not linearly separable => no convergence
  13. 13. 1986: Multilayers perceptron input vector hidden layers outputs
  14. 14. 1986: Multilayers perceptron input vector hidden layers outputs +29 ANS
  15. 15. 1986: Multilayers perceptron input vector hidden layers outputs Back- propagate error signal
  16. 16. Back propagation Activations The error: Update Weights: 0 1 0 .5 -5 5 Slide credit : Geoffrey H
  17. 17. Back propagation Activations The error: Update Weights: 0 1 0 .5 -5 5 Slide credit : Geoffrey Hinton errors
  18. 18. Drawbacks GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING
  19. 19. Drawbacks GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING LEARNING DECREASE WITH HIDDEN LAYERS NUMBERS
  20. 20. Drawbacks GRADIENT NATURE => SLOW CONVERGENCE => SLOW LEARNING LEARNING DECREASE WITH HIDDEN LAYERS NUMBERS CAN FIND ONLY LOCAL MINIMA OF ERRORS
  21. 21. Before 2006: The deapest, the worst DEEP NETWORKS (LOT OF HIDDEN LAYERS) ARE WORSE THAN WITH ONE OR TWO LAYERS ! SLOWER AND LESS ACCURATE
  22. 22. Deep ? How deep ? DEEP IS >=5 LAYERS
  23. 23. 2006: breakthrough
  24. 24. Who cares ? 2013: Director of Facebook AI research Google distinguished researcher Montreal University
  25. 25. The effect of unsupervised learning WITHOUT UNSUPERVISED LEARNING (top quartiles) WITH UNSUPERVISED LEARNING
  26. 26. Becomes more non-linear, and this is good: it prevents the gradient learning to be transferred to previous layers for local optima The first layer should react to input changes.
  27. 27. Becomes more non-linear, and this is good: it prevents the gradient learning to be Yet we dont know how. Just represent the dominant factors of variation of the input.
  28. 28. Size matters
  29. 29. Connectivity cost
  30. 30. Connectivity cost
  31. 31. Infrastructure costs The bad news In 2012, It took Google 16.000 CPU to have a single process real time cat face identifier http://hexus.net/tech/news/software/41 537-googles-16000-cpu-neural-network- can-identify-cat/ The good news In 2017, public Beta testing of HP The machine, based on Memristors replacing transistors for some parts of the chip. http://insidehpc.com/2015/01/video- memristor-research-at-hp-labs/
  32. 32. Imagenet classification results 2012 2014 Deep Learning GoogleNet 6.66% 2015 Microsoft Research 4.94 % ai is learned http://arxiv.org/pdf/1502.01852v1.pdf
  33. 33. Reverse engineering deep learning results(2012) SUPERVISED IMAGE CLASSIFICATIONS + OTHER TRAININGS First layer: always Gabor Filters like or Color Blob
  34. 34. Reverse engineering deep learning results(Nov 2014) SUPERVISED IMAGE CLASSIFICATIONS + OTHER TRAININGS General or Specific Layer ?? Transfer ANN layers among trained models
  35. 35. Static deep learning in finance + Person detection PMFG Performance heat map Trade opportunity detection Market States Current markets Reduce information redundancy for a goal
  36. 36. PMFG: Planar Maximally Filtered Graph of the correlation matrix Correlation, hierarchies and Networks in Financial Markets
  37. 37. market states Cluster time periods by correlation matrix similarity
  38. 38. Heat map 1 year perf, 23/07
  39. 39. 1990: RNN nonstationary I/O mapping, Y(t)f(X(t)), X(t) and Y(t) are timevarying patterns
  40. 40. 1990: RNN nonstationary I/O mapping, Y(t)f(X(t)), X(t) and Y(t) are timevarying patterns The closest thing to computer dreams
  41. 41. RNN detailed
  42. 42. Problems in RNN Time lag > 5 => Difficult to learn The error either vanishes or explodes in the learning process Divergent behavior
  43. 43. RNN for GARCH(1,1) predictions IN SAMPLE: 665 observations 01/2011 09/2013 OUT OF SAMPLE: 252 observations 09/2013 09/2014 LAG=1, NO PROBLEM
  44. 44. A new approach: RNN with wavelet sigmoid (2D, time and frequency) April 2015 PREDICT SEMI-CHAOTIC TIME SERIE (MACKEY-GLASS), SOLUTION OF Rapidly vanishing property of wavelet function => No divergent behavior
  45. 45. Particle swarm optimization Concepts Applications: Training a neural network using PSO instead of backpropagation
  46. 46. PSO principles Here I am! The best perf. of my neighbours Mybest perf. x pg pi v Collective Intelligence: Particles Adjust their positions according to a ``Psychosocial compromise between what an individual is comfortable with, and what society reckons => Solve continuous optimization problems
  47. 47. Training neural network with PSO (started in 2006) Unsupervised training with PSO: Not trapped in local minima Faster than back propagation
  48. 48. PSO ANN accuracy (July 2014) Levenberg-Marquadt: second order in BP error evaluation Credit Approval dataset, UCI
  49. 49. The challenge of multidimensionality - Dimensionality reduction techniques
  50. 50. Dimension reduction techniques PCA using Neural Networks(2012) PCA with Gaussian assumption: Training set: 1/16 compression, 4x4 blocks Result
  51. 51. Radial Basis Function Neural Network with (2D)^2 PCA (April 2015, Shangai index) TRAIN TEST
  52. 52. Radial Basis Function Neural Network with (2D)^2 PCA (April 2015, Shangai index) TRAIN TEST Exercise: find the issue
  53. 53. Thank you International workshop on Deep Learning, Lille, France (July 10/11 2015): https://sites.google.com/site/deeplearning2015/accepted- papers

View more