high performance quantum inspired models in machine learning
DESCRIPTION
Machine learning is a field of artificial intelligence that seeks to identify patterns in empirical data. Applications range from biomedical imaging to financial forecasting, but scalability remains an issue: sophisticated algorithms do not scale well and they are hard to parallelize in a cluster or using GPUs. Quantum mechanics, on the other hand, is a traditional area for high-performance computing, and the underlying mathematical framework has attracted attention outside the domain of physics. Quantum inspired learning methods are at the confluence of the two respective fields, promising highly scalable algorithms that are also able to capture patterns where traditional approaches were less successful.TRANSCRIPT
High Performance Quantum Inspired Models in Machine Learning
High Performance Quantum Inspired Modelsin Machine Learning
Peter Wittek
Swedish School of Library and Information ScienceUniversity of Boras
11/05/12
High Performance Quantum Inspired Models in Machine Learning
Outline
1 Introducing Machine Learning
2 Scalability in Machine Learning
3 The emergence of quantum inspired methods
4 Digression
5 Experimental Results
6 Conclusions
High Performance Quantum Inspired Models in Machine Learning
Introducing Machine Learning
Problem Statement
Patterns in dataFundamentally data-driven, not model-drivenStatistical and non-statistical methodsOf particular interest: kernel density estimators
0
0.05
0.1
0.15
0.2
-4 -2 0 2 4 6 8
Den
sity
func
tion
x
High Performance Quantum Inspired Models in Machine Learning
Introducing Machine Learning
Methods
Countless: every data set has its own methodSupervised versus unsupervisedClustering, neural networks, support vector machines,genetic algorithms, etc.
a) b)
High Performance Quantum Inspired Models in Machine Learning
Scalability in Machine Learning
Scaling out
Data-intensive processing: MapReduceSimple operations on a large volumeTypically not the most sophisticated algorithmsMore interesting algorithms do not scale well
High Performance Quantum Inspired Models in Machine Learning
The emergence of quantum inspired methods
Some concepts
Classical systems Quantum mechanicsHamiltonian HamiltonianPoisson bracket CommutatorNewton’s law of motion Ehrenfest’s theoremProbability distribution Density operatorRandom variable Self-adjoint matricesFormula of total probability FTP with interference
High Performance Quantum Inspired Models in Machine Learning
The emergence of quantum inspired methods
A brief history
1920s: Quantum mechanics and quantum probabilitytheory1933: “Classical” probability theory1957: QM is in fact a probability theory1990s: Nature/physics inspired learning methods2000s: Quantum inspired learning
High Performance Quantum Inspired Models in Machine Learning
The emergence of quantum inspired methods
Why quantum?
Fundamentally non-commutativeContextuality is everything (compare it to Bayes’ rule)EntanglementSuperposition
High Performance Quantum Inspired Models in Machine Learning
The emergence of quantum inspired methods
Application fields
A growing range of machine learning methods: particleswarm optimization, evolutionary algorithms, neuralnetworks, etc.Particularly popular: language technology
The meaning of a term is in superposition: the contextcollapses it to one sense.Entanglement: pet-fish problemThe measurement is specific to the individual, the context isnon-classical.
High Performance Quantum Inspired Models in Machine Learning
The emergence of quantum inspired methods
A bonus
QM is quintessentially linearBLAS is available on every hardwareCombine ML with QM to arrive at scalable models
High Performance Quantum Inspired Models in Machine Learning
Digression
Quantum Computing and Computational Intelligence
Quantum information theoryLevel of abstraction: qubitsExponential explosion in representative powerGood match for computational intelligence
High Performance Quantum Inspired Models in Machine Learning
Digression
Cloud computing
Cloud cluster instancesAssembling a GPU cluster in a matter of minutesFreedom of choice in softwareAvailable to anyone
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Hardware and software
AWS Cluster Instances2x4 CPU cores per instance2x Tesla C205024Gbyte of RAMLinux environmentOpenMPI, CUDA
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Dynamic Quantum Clustering
Assign wave function to data points
Initialize Gram matrix Nij = 〈ψi |ψj〉 = e−(xi−xj )
2
4σ2
Calculate Hamiltonian: Hij = 〈ψi |H|ψj〉 = 〈ψi |(T + V (x))|ψj〉.Calculate position operator Xij = 〈ψi |x |ψj〉Compute eigendecomposition of NCompute square root of NBasis transformation of HamiltonianBasis transformation of position operatorrepeat
Compute matrix exponential of transformed Hamiltonian attime tnCompute expectation of value of position operator at timetn
until
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Dynamic Quantum Clustering
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Dynamic Quantum Clustering
0
1
2
3
4
5
6
64 128 256 512 1024 2048 4096 8192
Tim
e (
s)
Matrix size
CPUGPU w/o memory transferGPU with memory transfer
(c) Execution time, sin-gle precision
0
50
100
150
200
250
64 128 256 512 1024 2048 4096 8192
Sp
ee
du
p
Matrix size
Without Memory TransferWith Memory Transfer
(d) Speedup, singleprecision
0
1
2
3
4
5
6
64 128 256 512 1024 2048 4096 8192
Tim
e (
s)
Matrix size
CPUGPU w/o memory transferGPU with memory transfer
(e) Execution time,double precision
0
20
40
60
80
100
120
140
160
180
64 128 256 512 1024 2048 4096 8192
Sp
ee
du
p
Matrix size
Without Memory TransferWith Memory Transfer
(f) Speedup, doubleprecision
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Self-organizing maps
2D neural network: M = n1, . . . ,nk
Associated weight vectors: W = w1(t), ...,wk (t) at time tSeek best matching neuronsAdjust weights: wj(t + 1) = wj(t) + αhbj(t)[x(t)− wj(t)]
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Self-organizing maps
High Performance Quantum Inspired Models in Machine Learning
Experimental Results
Self-organizing maps
Method Execution Speeduptime over CPU
CPU (64 cores) 2042s -GPU (16 Tesla) 433s 4.71xCPU (64 cores) 1882s -(One epoch)GPU (16 Tesla) 194s 9.68x(One epoch)
Table: Execution time of self-organizing maps
High Performance Quantum Inspired Models in Machine Learning
Conclusions
Ongoing work
Extending the range of algorithmsLarge-scale experimentsAttempts at unification
High Performance Quantum Inspired Models in Machine Learning
Conclusions
Summary
http://squalar.org/bsc_talk.pdf
Parallel code, GPU computing, HPC in general are fairlynew in machine learningQL methods solve the problem in a spectacular wayAdditional advantages: contextuality, entanglement,superposition