an artificial brain mechanism to develop a learning ... artificial brain mechanism to develop a...
TRANSCRIPT
An Artificial Brain Mechanism
to Develop a Learning
Paradigm for RobotsManish Kumar1, Rashmi Jha2
1Associate Professor, Department of Mechanical Engineering,
University of Cincinnati, Cincinnati, Ohio
[email protected], 513-556-53112Associate Professor, Dept. of Electrical Engineering and
Computing Systems, University of Cincinnati, Cincinnati, Ohio
[email protected], 513-556-1361
Need for Biological Brain Inspired Ultra-Low
Energy Computing
Human vs. Robot for Space Exploration
Humans hold a number of advantages over robots. They
can make quick decisions in response to changing
conditions or new discoveries, rather than waiting for time-
delayed instructions from Earth.
http://www.wired.com/2012/04/space-humans-vs-robots/
Limitations of Currently Available
Machine Learning: Deep Neural Network
(DNN)
DNN on conventional computing architecture are
compute intensive, power hungry, need a large set
of training data , and are trained to solve just some
specific sets of problems.
Limitations of Traditional CMOS
Transistor Scaling and Computing
http://epc-co.com/epc/EventsandNews/FastJustGotFasterBlog/Issue11.aspx
Cost of CMOS
transistor is rising at
20 nm node and
beyond for the first
time in history.
http://www.extremetech.com/computing/116561-the-death-of-cpu-scaling-from-
one-core-to-many-and-why-were-still-stuck
Motivation for Artificial Brain
“Taking advantage of the almost 83,000
processors of one of the world's most
powerful supercomputers, the team was able to
mimic just one percent of one second's worth
of human brain activity—and even that took 40
minutes.” – Gizmodo, 2013
“Challenge is to
create an exascale
computing system
by 2018 that
consumes only 20
megawatts (MW) of
power.”- DOE grand
challenge.
Supercomputers
“1014 Neurons, 1015
Synapses, 1013 to 1016
Instructions per sec, 10 W
of Power (e.g. retinal
operation).”
BrainScalable
Ultra Low-
Energy
Motivation for Artificial Brain
“Taking advantage of the almost 83,000
processors of one of the world's most
powerful supercomputers, the team was able to
mimic just one percent of one second's worth
of human brain activity—and even that took 40
minutes.” – Gizmodo, 2013
“Challenge is to
create an exascale
computing system
by 2018 that
consumes only 20
megawatts (MW) of
power.”- DOE grand
challenge.
Supercomputers
“1014 Neurons, 1015
Synapses, 1013 to 1016
Instructions per sec, 10 W
of Power (e.g. retinal
operation).”
BrainScalable
Ultra Low-
Energy• How does a biological “Brain” work ??
• How can we make an artificial brain on
chip???
How Does a Biological Brain
Process Information?
https://computing.llnl.gov/tutorials/parallel_comp/
Von-Neumann
Architecture
Brain-Inspired Paradigm of Computing
Components: Neurons,
Reconfigurable Synapses,
Interconnects
STDP
Syn
aptic E
ffic
acy
Today’s Computing
Markram et. al., Front. Synp. NeurosSc. 2011
Neuron Operation and Action Potential Firing
Synapse
Sensory Signal Processing
Temperature,
odor etc.
(Effector Cells)
Central Nervous
System
What does it mean from neuro-
inspired device perspective?
• High fan-out spiking device
• Ultra-low energy consumption ~10 fJ/spike
• Scalable
• High reliability and endurance
• Reconfigurable
• Ultra Low-power
• Scalable
• High endurance and reliability
• Minimal Variability
Neuron
Synapse
Sensory Information Coding
Spike Coding of Odor
Mainland et. al., Trends in Neurosciences August 2014, Vol. 37, No. 8
Sensory Neurons in Silicon
Axon-Hillock Circuit, proposed by Prof. Carver Mead,
1980’sIndiveri et. al., Frontiers in Neuroscience, 2011
Learning Algorithms• Supervised Learning
– Feed-Forward
– Back-Propagation
– Gradient-Descent
• Unsupervised Learning for Spiking Neural Network
– Hebbian Learning (Spike Timing Dependent Plasticity)
• Neurons that fire together, wire together
• Basis of Associative Memory
Spike Timing Dependent
Plasticity
Which synapses are
strengthened? Which ones are
depressed?
Bi et. al. J. of NeuroSci, 1998
17
Doped Oxide Dynamics for Synaptic Memory
10-13
10-12
10-11
10-10
10-9
-0.6 -0.4 -0.2 0 0.2 0.4 0.6
Initial IV
Device 1Device 2Device 3Device 4Device 5Device 6Device 7Device 8Device 9Device 10
Cu
rre
nt
(A)
Voltage (V)
0
2 10-6
4 10-6
6 10-6
8 10-6
1 10-5
1.2 10-5
0 0.5 1 1.5 2 2.5 3
Positive Hysteresis
Cycle1Cycle2Cycle3Cycle4Cycle5Cycle6Cycle7Cycle8Cycle9Cycle10Cycle11Cycle12Cycle13Cycle14Cycle15Cycle16
Cu
rre
nt
(A)
Voltage (V)
-1.4 10-7
-1.2 10-7
-1 10-7
-8 10-8
-6 10-8
-4 10-8
-2 10-8
0
-1.6 -1.4 -1.2 -1 -0.8 -0.6 -0.4 -0.2 0
Negative Hysteresis
Cycle1
Cycle2
Cycle3
Cycle4
Cycle5
Cu
rre
nt
(A)
Voltage (V)
100µm x 100 µm
Mandal/Jha et. al., Nature Sci. Rep, 2014
1
1.5
2
2.5
3
3.5
4
4.5
0 100 200 300 400 500 600 700
Conductance vs Pulse 1ms
Excitation (5Hz)Depression (5Hz)Excitation (15Hz)Depression (15Hz)
Cu
rre
nt
@0
.5V
rea
d (
nA
)
Pulse #10 Potentiating Pulses 2.5V/1ms applied at given frequency and Current measured at 0.5V read after excitation.
Potentiation was repeated 30 times (total 300 pulses) with conductance measuring intervals of 10 pulses. After
potentiation for 300 pulses, 300 depression pulses at -1.5V/1ms were applied at the given frequency and
measurement of current at 0.5V read was done in intervals of 10 pulses.
18
Potentiation and Depression with Pulses
1
2
3
4
5
6
0 100 200 300 400 500 600 700
Conductance vs Pulse 10ms
Excitation (5Hz)Depression (5Hz)Excitation (15Hz)Depression (15Hz)
Cu
rre
nt
@0
.5V
rea
d (
nA
)
Pulse #
19
Endurance with Pulses of Different Pulse-Widths
10
15
20
25
30
35
40
0 200 400 600 800 1000 1200
1 ms Pot.1 ms Dep.10ms Pot.10 ms Dep.20 ms Pot.20 ms Dep.50 ms Pot.50 ms Dep.100 ms Pot.100 ms Dep.200 ms Pot.200 ms Dep.
Cu
rren
t @
0.5
V r
ea
d (
nA
)
Cycle #
20
Distribution over Cycle to Cycle
0
50
100
150
200
250
300
14 16.8 19.6 22.4 25.2 28
Pot. (50ms)Dep. (50ms)
Co
un
t
Range (nA)
dep.
: 18.53
dep.
: 0.356 pot.
: 25.303
pot.
: 0.896
0
50
100
150
200
16 20 25 30 35 40
Pot. (200ms)Dep. (200 ms)
Co
un
tRange (nA)
dep.
: 19.7548
dep.
: 1.493
pot.
: 33.6524
pot.
: 1.3408
0
50
100
150
200
12 14 17 19 22 24
Pot. (10 ms)Dep. (10 ms)
Co
un
t
Range (nA)
dep.
: 16.116
dep.
: 0.586
pot.
: 20.3828
pot.
: 0.8272
0
50
100
150
200
250
10.8 12.8 14.8 16.8 18.8 20.8
Pot.1msDep. 1ms
Co
un
t
Range (nA)
dep.
: 12.936
dep.
: 0.2416
pot.
: 14.766
pot.
: 0.57
0
50
100
150
200
12 14 16 18 20 22 24
Pot. (20ms)Dep. (20ms)
Co
un
t
Range (nA)
dep.
: 16.394
dep.
: 1.55
pot.
: 21.478
pot.
: 2.876
0
50
100
150
200
12 16 20 24 28 32
Pot. (100 ms)Dep. (100 ms)
Co
un
t
Range (nA)
dep.
: 18.7035
dep.
: 0.982
pot.
: 27.9565
pot.
: 0.9163
21
Spike Timing Dependent Plasticity
-60
-40
-20
0
20
40
60
80
-60 -40 -20 0 20 40 60
%a
ge c
han
ge
t (ms)
(t)=83.03*exp(-t/41.3)
(t)=-47.03*exp(t/59.03)
=1.3713
=0.9934
Δt (+/-) Feedback (2.5V/-1.5V)
10 ms 200 ms
20 ms 100 ms
30 ms 50 ms
40 ms 20 ms
50 ms 10 ms
Synaptic Memory Device Model
Ru
Mn:HfO2
TiN/W
Ru
Mn:HfO2
TiN/
W
Ru
Mn:HfO2
TiN/
W
Initial State Potentiation (LTP) Depression (LTD)
-
+
𝐼 = 𝑞𝜇𝐸𝐴𝑛0 exp −𝜙𝐵 −
𝑞𝐸𝜋𝜀
𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛2 exp −
𝜙𝐵 −𝑞𝐸𝜋𝜀
𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛1 exp −𝜙𝐵 −
𝑞𝐸𝜋𝜀
𝑘𝑇
-
+ +
-
Mandal/Jha et. al., Nature Sci. Rep, 2014
Sarim/Kumar/Jha et. al., NAECON, 2016
Neuromorphic Platform
• A neuromorphic platform isconfigured as an array ofseveral Synaptic Memorydevices arranged as shown.
• The arrays are connected toproximity sensors that sendin the information about thevicinity of the robot. Themotor neuron circuits movethe robot wheels.
Simulation Framework• Two different models, viz., mathematical device model and
experimentally derived device model, for synaptic memory
devices with the neuromorphic platform were implemented
to demonstrate unsupervised learning in a robot.
• This approach was validated by simulating the robot to
navigate in an unknown environment with randomly placed
obstacles.
• The commercially available Khepera III robot [4] is modeled
with a two-wheeled differential drive robot kinematics. The
robot consists of five ultrasonic sensors that give the
information about the vicinity of the robot.
[4] http://www.k-team.com/mobile-robotics-products/old-products/khepera-iii
𝜈 =𝑟
2𝜔𝑅 + 𝜔𝐿
𝜔 =𝑟
𝑏𝜔𝑅 − 𝜔𝐿
𝑥 = 𝜈 cos 𝜃 𝑦 = 𝜈 sin 𝜃 𝜃 = 𝜔
Robot Kinematics
,Gianluca et. al., IEEE Transactions on Robotics 21.5 (2005): 994-1004.
Learning Scheme
target
Device Model:
𝑤 = 𝑓 𝜈𝑀𝑅
𝑓 𝜈𝑀𝑅 = 𝐼0𝑠𝑖𝑔𝑛 𝜈𝑀𝑅 𝑒 𝜈𝑀𝑅 /𝜈0 − 𝑒𝜈𝑡ℎ/𝜈0
𝜈𝑀𝑅 𝑡, Δ𝑇 = 𝛼𝑝𝑜𝑠𝑠𝑝𝑘 𝑡 − 𝛼𝑝𝑟𝑒𝑠𝑝𝑘 𝑡 + Δ𝑇
Δ𝑤 Δ𝑇 = 𝑓 𝜈𝑀𝑅 𝑡, Δ𝑇 𝑑𝑡 = 𝜉 Δ𝑇
STDP learning function:
𝜉 Δ𝑇 = 𝑎+𝑒− Δ𝑇 𝜏+ 𝑖𝑓 Δ𝑇 > 0
−𝑎−𝑒− Δ𝑇 𝜏− 𝑖𝑓 Δ𝑇 < 0
Mathematical Model
change in structural parameter of the device
memristor voltage
where
Δ𝑇 is the difference in the spike times of pre-
and post-synaptic neurons
𝑠𝑝𝑘 𝑡 is the spike shape
𝐼0 , 𝑣0 are device parameters, 𝑣𝑡ℎ is the
threshold voltage of the device above which it
spikes, 𝛼 are attenuation parameters in pre- and
post-synaptic neurons. 𝜉 is the change in the
synaptic weight that is used to implement STDP.
Robot Navigation Results
1 2 3
4 5 6
o : start | × : target
Navigation Results with Synaptic Memory
Device Model
Ru
Mn:HfO2
TiN/W
Ru
Mn:HfO2
TiN/
W
Ru
Mn:HfO2
TiN/
W
Initial State Potentiation (LTP) Depression (LTD)
-
+
𝐼 = 𝑞𝜇𝐸𝐴𝑛0 exp −𝜙𝐵 −
𝑞𝐸𝜋𝜀
𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛2 exp −
𝜙𝐵 −𝑞𝐸𝜋𝜀
𝑘𝑇𝐼 = 𝑞𝜇𝐸𝐴𝑛1 exp −𝜙𝐵 −
𝑞𝐸𝜋𝜀
𝑘𝑇
-
+ +
-
Mandal/Jha et. al., Nature Sci. Rep, 2014
Sarim/Kumar/Jha et. al., NAECON, 2016
Robot Navigation Results
1 2 3
4 5 6
o : start | × : target
Conclusions• We demonstrated the potential for having an onboard
“artificial brain” for Robots based on emerging
neuromorphic devices.
• Using artificial brain architecture, a successful Robotic
navigation was demonstrated using unsupervised learning
scheme to guide the robot in complex environments using
the local knowledge of obstacles only.
– Our approach overcomes the issue of local minima
which is a challenge for other navigation algorithms.
• Our approach is projected to be highly energy-efficient and
scalable for implementation on any robot.
• Future work is targeted towards the actual implementation
of these neuromorphic devices based artificial brain on
Robots and field verification of the navigation.
Student Contributors
1. Mohammad Sarim
Robotics Lab, Department of Mechanical and Materials
Engineering, University of Cincinnati
2. Thomas Schultz
EDACS Lab, Department of Electrical Engineering and
Computing Systems , University of Cincinnati
3. Saptarshi Mandal (Now at Arizona State University)
Acknowledgement• This project is currently supported by National Science
Foundation under CAREER (Award # 1556294), and SaTC (Award # 1556301).
• We would like to thank Dr. Mark Ritter and his group at IBM TJ Watson Research Center.
• We would like to thank our collaborators Dr. GennadiBersuker (Sematech), Dr. David Gilmer (Sematech), Dr. Prashant Majhi (Intel), Dr. Kevin Leedy (AFRL), Dr. Marc Cahay (U. of Cincinnati), Dr. Ali Minai ( U. of Cincinnati), Dr. Swaroop Ghosh (USF), Dr. Scott Molitor (U.Toledo).
Thank You!
Questions and Suggestions?