carlsim 3: concepts, tools, and applications

PowerPoint Presentation

Cognitive Anteater Robotics Lab (CARL)University of California, Irvine, CA, USAMichael BeyelerCARLsim 3

Concepts, Tools, and Applications

1

November 4, 20152Brain architecture =/= conventional computer architecture

Massive parallelism (1011 neurons)Massive connectivity (1015 synapses)Excellent power-efficiency ~ 20W for 1016 flopsProbabilistic responses and fault-tolerant Autonomous, on-line learningLow-performance components (~100 Hz)Low-speed comm. (~meters/sec)Low-precision synaptic connectionsBrain Computations

http://www.socsci.uci.edu/~jkrichma/CARLsim

Maybe mention efficiency of IBM chip as well ().2

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim3Spiking Neural Network (SNN) models describe key aspects of neural function and network dynamicsabstract away many molecular and cellular detailsable to capture neuronal, synaptic, and network dynamicscan incorporate on-line learning that depends on millisecond time resolution

SNN models are composed of:spiking point-neurons for computationIzhikevich spiking neurons: 20 different neuron typesdynamic synapses for learning and memory storageSynaptic receptors for AMPA, NMDA, and GABAvariable-delay axons for communicationneuromodulatory systems to control action selection and learning

Spiking Neural Networks

Izhikevich, 2003, 2004

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim4Constructing Functional SNN ModelsCARLsim 3

neural circuits:createtune/optimizeexplore

TheoreticalmodelExperimentaldataFunctionalapplication

TheoreticalneuroscienceReal-timeapplicationsNeuromorphicengineering

stop to think about what people are looking for in a simulator: needs to fit your purposes. neuron model, connectivity, plasticity, visualization, tuning.4

GPU-accelerated spiking neural network simulator

User-friendly, well documented.Runs on Linux, Mac OS X, Windows systems with CUDA SDK.Scalable, extendable.PyNN-like interface.Highly optimized for NVIDIA GPUs.Capable of simulating biological detailed neural models.

Freely available at:http://www.socsci.uci.edu/~jkrichma/CARLsim/

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim5CARLsim in a Nutshell

5

Provides a PyNN-like user interfaceLets you configure networks, apply input stimuli, and monitor network activityNovember 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim6CARLsim API

6

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim7Easily create complex network topographies:Pre-defined connection typesfull, random, one-to-one, gaussianCustom connection typesarbitrary connectivity

1D/2D/3D topography:ConnectivityNeurons organized on a grid1D/2D/3D receptive fieldsarbitrary topographic connections

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim8Short-term plasticity (STP)Homeostatic synaptic scalingSpike-timing dependent plasticity (STDP)Dopamine-modulated STDPSynaptic Plasticity

8

Versatile toolbox for the visualization and analysis of neuronal, synaptic, and network information.Generates raster plots, histograms, heat maps, flow fields.Plot, record, load, save.November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim9MATLAB Offline Analysis Toolbox

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim10capable of quickly and efficiently tuning large-scale SNNsarbitrary fitness functionmulti-objective optimization

utilizes Evolutionary Computations in Java (ECJ)fully integrated with CARLsim

transparent use of a GPUup to 60x speedups compared to single-threaded CPU simulation

Emily will tell you all about it.Automated Parameter Tuning Interface

Benchmark: 80-20 network with E-STDP (Vogels & Abbott, 2005)CPU: Intel Core i7 CPU 920 @ 2.67 GHzGPU: NVIDIA GTX 780 (3 GB of GDDR5, 2304 CUDA cores)

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim11Performance Benchmarks

PyNN-like user interfaceNovember 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim12CARLsim 3 Code Sample

Neuron modelSynapse modelSynaptic plasticityInputToolsIntegration methodsFront-endsBack-endsPlatformsLeaky integrate-and-fire (LIF)Izhikevich 4-paramHodgkin-HuxleyCurrent-bsaed (CUBA)Conductance-based (COBA)AMPA, NMDA, GABANeuromodulationShort-term plasticity (STP)E-STDPI-STDPDA-STDPSynaptic scaling / homeostasisCurrent injectionSpike injectionParameter tuningAnalysis and visualizationRegression suiteForward / exponential EulerExact integrationRunge-KuttaPython / PyNNC / C++JavaSingle-threadedMulti-threadeddistributedSingle GPUMulti-GPULinuxMac OS XWindowsCARLsimXXXX/XXXXXXXXXXXXXX/XXXBrianXXXXXX/XXX/XXX//XXXXXXXX/XXXGeNNXXXXX//X/X/XXXXXXXNCSXXXXX/XXXX/XXXXXXXXXNeMoXX/XXXXXXXXXXXXXXXXNengoXXXXXXXX/XXXXXXXXXXXXXXNESTXXXXXX/XXXXXXXXXXXXXXXXPCSIMXXXXXX/XXXXXX/XXX/XXXXXX/

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim13Comparison of SNN Simulators

Visual Cortical ProcessingM. Beyeler, N. Oros, N. Dutt, and J.L. Krichmar (2015). A GPU-accelerated cortical neural network model for visually guided robot navigation. Neural Networks (in press).M. Beyeler, M. Richert, N.D. Dutt, and J.L. Krichmar (2014). Efficient spiking neural network model of pattern motion selectivity in visual cortex. Neuroinformatics 12, 435-454.M. Richert, J.M. Nageswaran, N. Dutt, and J.L. Krichmar (2011). An efficient simulation environment for modeling large-scale cortical processing. Frontiers in Neuroinformatics 5.

Tactile Sensory ProcessingL.D. Bucci, T.S. Chou, and J.L. Krichmar (2014). Tactile Sensory Decoding in a Neuromorphic Interactive Robot. IEEE Conference on Robotics & Automation (ICRA).

Neuromodulation, Attention, and Working MemoryM.C. Avery, N. Dutt, and J.L. Krichmar (2014). Mechanisms underlying the basal forebrain enhancement of top-down and bottom-up attention. European Journal of Neuroscience 39.M. Avery, N. Dutt, and J.L. Krichmar (2013). A large-scale neural network model of the influence of neuromodulatory levels on working memory and behavior. Frontiers in Computational Neuroscience.

Object Categorization and PlasticityM. Beyeler, N.D. Dutt, and J.L. Krichmar (2013). Categorization and decision-making in a neurobiologically plausible spiking network using a STDP-like learning rule. Neural Networks 48.K.D. Carlson, M. Richert, N. Dutt, and J.L. Krichmar (2013). Biologically Plausible Models of Homeostasis and STDP: Stability and Learning in Spiking Neural Networks. Paper presented at: International Joint Conference on Neural NetworksNovember 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim14Recent CARLsim Models

The bottom left video was not run in real time.14

November 4, 201515Visually guided robot navigation:SNN model of motion processing in cortical areas V1 and MT80k neurons and 4 million synapsescontrolled an autonomous robot in real-timereplicated human behavioral paths for obstacle avoidance (Fajen & Warren, 2003)CARLsim in Real-Time Applications(Beyeler et al., 2015)


November 4, 201516Primary visual cortex (V1)tuned to simple attributes of shape, motion, color, texture, depth

Middle temporal (MT) areatuned to coherent local motion (retinal flow)

Posterior parietal cortex (PPC)Polysensory areas (VIP, MST, 7a, etc.)Tuned to global, complex motionSelf-motion and object motionSpatial reference framesPath integration (?)Visual Motion Pathway

(Britten, 2008)


November 4, 201517Problem: Local-velocity sample is different from object velocityGoal: Disambiguate local-velocity samples and integrate them into an accurate estimate of the global (object) velocity

Intersection-of-constraints (IOC):Each local velocity sample constrains the global object velocityFind object velocity by integrating local samplesThere is evidence that MT firing rates represent the velocity of moving objects using IOC

Aperture Problem(Bradley & Goyal, 2008)


Motion is an orientation in space-timeV1 simple cells: space-time oriented receptive fields

Spatiotemporal energy model:Linear filtering / motion energy / opponent energyAdelson & Bergen (1985), Simoncelli & Heeger (1998)November 4, 201518Primary Visual Cortex (V1)

(Simoncelli & Heeger, 1998)(Bradley & Goyal, 2008)


November 4, 201519Middle Temporal Area (MT)ExcitationFeedforward and localSpecific InhibitionCross-direction inhibitionUnspecific InhibitionDivisive normalization

MT CDSMT PDShttp://www.socsci.uci.edu/~jkrichma/CARLsim(Beyeler et al., 2015)

Model Response to Motion Patterns

V1MT CDSMT PDS(Beyeler et al., 2014)(CDS: component-direction-selective, PDS: pattern-direction-selective)November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim20

November 4, 201521Simple control law for obstacle avoidanceused by honeybees for visual control of flight (Srinivasan & Zhang, 1997)try to balance flow magnitude in left and right hemisphereturn away from the side with greater flow magnitudeHumans might make use of similar strategy (Kountouriotis et al., 2013)Posterior Parietal Cortex (PPC)


Introducing Le Carl: The French RobotNovember 4, 201522Android Based Robotics


Server / client communicationABR server:monitors video / sensory information of ABR clientsremotely starts / stops camera, sensors, GPS, and IOIO (TCP)hosts spiking neural network model

November 4, 201523Android Based Robotics (ABR)

(Oros & Krichmar, 2014)http://www.socsci.uci.edu/~jkrichma/CARLsim

November 4, 201524Technical Setup and Typical Workflow(Beyeler et al., 2015)http://www.socsci.uci.edu/~jkrichma/CARLsim

November 4, 201525Simulated Neuronal Responses During Visual Navigation(Beyeler et al., 2015)http://www.socsci.uci.edu/~jkrichma/CARLsim

November 4, 201526Behavioral Results


Comparison to Psychophysical Data

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim27

CARLsim highlights:Runs on Linux, Mac OS X, Windows.Highly optimized for NVIDIA GPUs: CUDA 5.0, device capability 2.0User-friendly, scalable, extendableDownload: www.socsci.uci.edu/~jkrichma/CARLsim

Presented a large-scale spiking neural network thatis biologically inspiredsolves the aperture problem via cortical mechanismsis integrated with a real-time, real-world robotics platformAndroid Based Robotics combined with CARLsim is the first step toward a complete robot navigation system.November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim28Conclusions

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim29Team CARL (2015)Front row Ian Schweer, Emily Rounds, Jeff Krichmar, Alexis Craig, Sean CampbellBack row Nik Dutt, Georgina Lean, Ting-Shuo Chou, Kris Carlson, Tiffany Hwu, Michael BeyelerSupported by the National Science Foundation and Qualcomm Technologies Incorporated.

November 4, 2015http://www.socsci.uci.edu/~jkrichma/CARLsim30