practical heirarchical temporal memory for time series prediction author: nicholas hainsey faculty...
TRANSCRIPT
Practical Heirarchical Temporal Memory for Time Series Prediction
Author: Nicholas HainseyFaculty Advisor: Dr. C. David Shaffer
Heirarchical Temporal Memory
• Neural network• Created by Jeff Hawkins• Designed to mimic the human neocortex
INPUT
Prediction
Network
ABCABC
Time
Input Encoding0 1 1 1 0
0 1 0 0 0
0 1 0 0 0
0 1 0 0 0
0 1 1 1 0
ABCABC
Time 01110 01000 01000 01000 01110
0 0 1 0 0
0 1 0 1 0
0 1 1 1 0
0 1 0 1 0
0 1 0 1 0
ABCABC
Time 00100 01010 01110 01010 01010
Spatial Pooler
0 1 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0
HTM Region
Input bits
Spatial Pooler
Proximal dendrite
0 1 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0Input bits
Spatial Pooler
0 1 1 1 0 0 1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0Input bits
3 0 2 1 0 2 0 3 0 1 Overlap Score
Spatial Pooler
• Goal:– Each input will activate a small percentage of the
columns– Similar inputs will activate overlapping sets of
columns
Temporal Pooler
InactiveActivePredicted
Temporal Pooler
InactiveActivePredicted
HTM Implementations
• HTMCLA– A C++ implementation based off Numenta’s CLA
white paper• HTM-CLA-Visualizer– Java interface for visualization of HTMs
• NuPIC– Created by Numenta, used in Nustudio
Nustudio
Nustudio
Nustudio
Run Simulation Stop
Nustudio
Connect to server
Connect to server
Nustudio
Nustudio
Run simulation from server
Nustudio
Pos (z): 2Was Predicted: TrueIs Active: TrueActivation Rate: .050Prediction Rate: .500
Nustudio
Nustudio
P1: MeanP2: Standard Deviation
Nustudio
Predictions with noise
Learning SD: 5.0 SD: 10.0 SD: 20.0
Noise with learningNo Noise
5.0 SD
10.0 SD
20.0 SD
Conclusions
• Additions to Nustudio– Constant simulation from file– Live simulation from server– Adding noise to incoming data– Viewing individual regions of the HTM at any step
• Noise Comparison– Still seems stable under varying levels of noise
Future Work
• More robust test of noisy data• More customization of noise– More distributions to choose from– More control over where noise is applied
• Ability to export prediction data