origins, evolution, and future directions of matlab€¦ · statistics optimization financial...
TRANSCRIPT
1© 2015 The MathWorks, Inc.
Origins, Evolution, and Future Directions of
MATLAB®
Loren Shure
2
-3-2
-10
12
3
-2
0
2
-5
0
5
x
Peaks
y
Agenda
Origins
Evolution
Tomorrow
3
Computational Finance Workflow
Research and Quantify
Data Analysis
& Visualization
Financial
Modeling
Application
Development
Reporting
Applications
Production
Share
Automate
Files
Databases
Datafeeds
Access
4
MATLAB – The Financial Development Platform
Financial
Statistics Optimization
Financial Instruments Econometrics
MATLAB
Parallel Computing MATLAB Distributed Computing Server
Files
Databases
Datafeeds
Access
Reporting
Applications
Production
Share
Data Analysis and Visualization
Financial Modeling
Application Development
Research and Quantify
Datafeed
Database
Spreadsheet Link EX Builder NE
Builder EX
Builder JA
Compiler
Rep
ort G
en
era
tor
Pro
du
ctio
n S
erv
er
Trading Toolbox
Research and Quantify
Data Analysis
& Visualization
Financial
Modeling
Application
Development
Reporting
Applications
Production
Share
Automate
Files
Databases
Datafeeds
Access
5
Origins of MATLAB
6
1965 Birth of L-Shaped MembraneOrigin of MathWorks Corporate Logo
Cleve Moler’s Stanford Ph.D. Thesis
Finite Difference Methods for Eigenvalues of Laplace’s
Operator
Blog post
7
1965 Birth of L-Shaped MembraneEvolution of MathWorks Corporate Logo
vibes
Vibrating L-shaped membrane 1984 1987
1992 1994
1997
8
1977 MATLAB Created
Cleve’s Work in Computational Mathematics
1967 1971 1976 19791977
Idea formed: Matrix Laboratory
FORTAN Program
Matrix as a Data Type
9
1981 MATLAB 0.0
MATLAB Command Line Program Debuts as a Teaching Tool
10
1979 MATLAB 0.0
MATLAB Command Line Debuts as a Teaching Tool
Stanford: 1979-80 Winter Quarter
CS237 - Numerical Analysis
Math & CS Students
Not impressed
Engineering Students
Love MATLAB
Interactive text-based interpreter – No need to compile!
11
Evolution of MATLAB
-3-2
-10
12
3
-2
0
2
-5
0
5
x
Peaks
y
12
1983-1984 Jack Little’s Vision
MathWorks Founded to Commercialize MATLAB
MATLAB becomes a language for Technical Computing
M-file scripts and
functions
Toolboxes
13
1987 Loren Shure Joins MathWorks as Hire #1
MATLAB Developer
1988/9
14
1992 MATLAB 4
Command Line, Graphics, and Help Browser (helpdesk)
MATLAB is more than a language:
• Library of mathematical functions
• Customizable by toolboxes
• M-file functions and scripts
• 2d/3d Graphics
• Matrix data type (2-d complex array)
15
1996 MATLAB 5.0
MATLAB Language Evolves Beyond the MATRIX Data Type
16
1996 MATLAB 5.0
Financial Toolbox Launched
17
1998-1999 MATLAB 5.2 (R10) and 5.3 (R11)
Desktop Evolves, Debugger, Enhanced 3D Graphics
18
2000-2003 MATLAB 6.0-6.5 (R12-R13)
New Dock-able Desktop, Command History, Function Handles
fhandle = @humps;
x = fminbnd(fhandle, 0.3, 1)
x =
0.6370
MATLAB JIT Introduced
19
2000-2003 MATLAB 6.0-6.5 (R12-R13)
Start Menu, More GUI Tools (Apps), and More Finance Products
JAVA Interface
20
2004 MATLAB 7 (R14)
Evolving Desktop, Distributed Computing, Market Datafeeds
Anonymous functions:
sqr = @(x) x.^2;
Nested functions:
function x = A(p1, p2)
...
function y = B(p3)
...
end
...
end
21
2004 MATLAB 7 (R14)
And Publishing of Scripts
Published ReportMATLAB scripts
22
2006 MATLAB 7.2 (R2006a)
Move to Twice a Year Release, M-Lint Integrated into Editor
23
2007 MATLAB 7.5 (R2007a/b)
Dataset and Categorical Arrays
Statistics Toolbox (future MATLAB data types)
• Categorical variables
• Dataset variable
24
2008 MATLAB 7.8 (R2008a)
Econometrics Toolbox Launched, Performance Improvements
25
2008 MATLAB 7.8 (R2008a)
Modernized Object-Oriented Programming
26
2008 MATLAB 7.9 (R2008b)
New Symbolic Engine
27
2009 MATLAB 7.8/7.9 (R2009a/b)
Protected Code, .NET Interface
MATLAB User MATLAB User
Visible IP Sharing
MATLAB User
Protected IP Sharing
p-code
MATLAB Code
MATLAB Apps
28
2010 MATLAB 7.10/7.11 (R2010a/b)
GPU Support
Additional Code
Additional Code
29
2011 MATLAB 7.13/7.14 (R2011a/b)
Big .MAT Files – Partial Read/Write, 80+ Tools (aka Apps)
30
31
2012 MATLAB 7.13 (R2012b)
New Desktop Launched, Improved Discoverability, Apps
32
2012 MATLAB 7.13 (R2012b)
New Desktop Launched, Improved Discoverability, Apps
34
Web
Server
Application
Server
Database Server
Pricing
Risk
Analytics
Portfolio
Optimization
MATLAB Production ServerMATLAB
Compiler
Web Applications
Desktop Applications
Batch Applications
2012 MATLAB 7.13 (R2012b)
MATLAB Production Server Launched
35
2012 MATLAB 7.13 (R2012b)
MATLAB Production Server Example: Web Asset Allocation
http://54.208.30.187/mwa/
36
2012 MATLAB 7.13 (R2012b)
MATLAB Production Server Example: Predictive Modeling
http://is.gd/mwdfw
37
2013 MATLAB 7.14/15 (R2013a/b)
Unit Testing and Datasets become Tables in MATLAB
38
Future Directions of MATLAB
2014 - onwards
39
MATLAB – The Financial Development Platform
Financial
Statistics Optimization
Financial Instruments Econometrics
MATLAB
Parallel Computing MATLAB Distributed Computing Server
Files
Databases
Datafeeds
Access
Reporting
Applications
Production
Share
Data Analysis and Visualization
Financial Modeling
Application Development
Research and Quantify
Datafeed
Database
Spreadsheet Link EX Builder NE
Builder EX
Builder JA
Compiler
Rep
ort G
en
era
tor
Pro
du
ctio
n S
erv
er
Trading Toolbox
40
MATLAB – The Financial Development Platform
Financial
Statistics Optimization
Financial Instruments Econometrics
MATLAB
Parallel Computing MATLAB Distributed Computing Server
Files
Databases
Datafeeds
Access
Reporting
Applications
Production
Share
Data Analysis and Visualization
Financial Modeling
Application Development
Research and Quantify
Datafeed
Database
Spreadsheet Link EX
MATLAB
Compiler
SDK
MATLAB
Compiler
Rep
ort G
en
era
tor
Pro
du
ctio
n S
erv
er
Trading Toolbox
41
1,000s of applicationsMILLIONS of users
Terminal - mainframe, mini
HUNDREDS OF MILLIONS
of users10,000s of applications
PC - LAN, Internet
BILLIONS of users
Cloud – mobile, browser, social, big data
MILLIONS of apps
Source: IDC, 2013
Cloud as a New Platform
42
MATLAB Mobile
Support for iPhone, iPad, and Android
43
Cloud Enhancing Your MATLAB Desktop
MATLAB Discributed Computing Server on Amazon EC2
44
The 4 V’s of Big Data
Volume
Velocity
Variety
Veracity
45
Big Data Capabilities in MATLAB
Memory and Data Access
64-bit processors
Memory Mapped Variables
Disk Variables
Databases
Datastores
Platforms
Desktop (Multicore, GPU)
Clusters
Cloud Computing (MDCS on EC2)
Hadoop
Programming Constructs
Streaming
Block Processing
Parallel-for loops
GPU Arrays
SPMD and Distributed Arrays
MapReduce
46
New Big Data Capabilities in MATLAB
Memory and Data Access
64-bit processors
Memory Mapped Variables
Disk Variables
Databases
Datastores
Platforms
Desktop (Multicore, GPU)
Clusters
Cloud Computing (MDCS on EC2)
Hadoop
Programming Constructs
Streaming
Block Processing
Parallel-for loops
GPU Arrays
SPMD and Distributed Arrays
MapReduce
47
Analyze Big Datamapreduce
Use the powerful MapReduce programming
technique to analyze big data
– Multiple items (keys) to organize and process
– Intermediate results do not fit in memory
On the desktop
– Analyze big database tables (Database Toolbox)
– Increase compute capacity (Parallel Computing Toolbox)
– Access data on HDFS to develop algorithms for use on Hadoop
With Hadoop
– Run on Hadoop using MATLAB Distributed Computing Server
– Deploy applications and libraries for Hadoop using MATLAB Compiler
********************************
* MAPREDUCE PROGRESS *
********************************
Map 0% Reduce 0%
Map 20% Reduce 0%
Map 40% Reduce 0%
Map 60% Reduce 0%
Map 80% Reduce 0%
Map 100% Reduce 25%
Map 100% Reduce 50%
Map 100% Reduce 75%
Map 100% Reduce 100%
48
1503 UA LAX -5 -10 2356
540 PS BUR 13 5 186
1920 DL BOS 10 32 1876
1840 DL SFO 0 13 568
272 US BWI 4 -2 359
784 PS SEA 7 3 176
796 PS LAX -2 2 237
1525 UA SFO 3 -5 1867
632 PS SJC 2 -4 245
1610 UA MIA 60 34 1365
2032 DL EWR 10 16 789
2134 DL DFW -2 6 914
1503 UA LAX -5 -10 2356
540 PS BUR 13 5 186
1920 DL BOS 10 32 1876
1840 DL SFO 0 13 568
272 US BWI 4 -2 359
784 PS SEA 7 3 176
796 PS LAX -2 2 237
1525 UA SFO 3 -5 1867
632 US SJC 2 -4 245
1610 UA MIA 60 34 1365
2032 DL EWR 10 16 789
2134 DL DFW -2 6 914
UA
PS
DL
DL
2356
186
1876
568
US
PS
PS
UA
US
UA
DL
DL
245
1365
789
914
359
176
237
1867
UA 2356
PS 186
PS 237
UA 1867
UA 1365
DL 1876
DL 914
US 359
US 245
Data Store Map Reduce
Mapreduce
49
Statistics and Machine LearningWhat’s New
Classification Learner
New app to train models and classify data
using supervised machine learning
Features
Import and interactively explore data
Choose kfold or holdout validation
Train SVM, kNN, bagged trees and other algorithms
Assess results using classification accuracy, ROC curves and Confusion Matrices
Export models to the MATLAB or generate MATLAB code
50
Statistics and Machine Learning
What’s New?
New:
Classification Learner app
Multiclass SVM
Statistical tests for comparing classifiers
Kmediods Clustering (robust to outliers)
C Code Generation for PCA
Enhancements:
Speedup of the kmeans and gmdistribution using the kmeans++
Performance enhancements for decision trees and performance curves
Requires MATLAB Coder
51
Datastore
MATLAB Distributed Computing Serverwith Hadoop
MATLAB
MapReduce
Code
HDFS
Node Data
MATLAB
Distributed
Computing
Server
Node Data
Node Data
Map Reduce
Map Reduce
Map Reduce
Hadoop
52
MATLAB Compilerwith Hadoop
MATLAB
MapReduce
Code
Datastore
HDFS
Node Data
Node Data
Node Data
Map Reduce
Map Reduce
Map Reduce
MATLAB
runtime
Hadoop
53
…and much more to come
54
MATLAB – The Financial Development Platform
Financial
Statistics Optimization
Financial Instruments Econometrics
MATLAB
Parallel Computing MATLAB Distributed Computing Server
Files
Databases
Datafeeds
Access
Reporting
Applications
Production
Share
Data Analysis and Visualization
Financial Modeling
Application Development
Research and Quantify
Datafeed
Database
Spreadsheet Link EX
MATLAB
Compiler
SDK
MATLAB
Compiler
Rep
ort G
en
era
tor
Pro
du
ctio
n S
erv
er
Trading Toolbox
55
© 2015 The MathWorks, Inc. MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a list of
additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective holders.