![Page 1: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/1.jpg)
Jimmy Abbas
Neuromorphic Design of Smart Prosthetic and
Therapeutic Systems
![Page 2: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/2.jpg)
Complex visual fields
Control of locomotion
Hippocampus - path planning dynamics at a fixed point?
Olfaction
Reward systems
Risk aversion ?
Auditory cues
Attention
Social behavior
Color
Path planning
System state performance
Sensorimotor integration
![Page 3: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/3.jpg)
Opportunities for Neuromorphic Engineers in Medical Rehabilitation
• smart prostheses• artificial limbs• electrical stimulation systems
![Page 4: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/4.jpg)
Smart ProstheticsNovember 9-11, 2006 - Irvine, CA
Goals:
• Enhance the climate for conducting interdisciplinary research and break down related institutional and systemic barriers
• Stimulate new modes of scientific inquiry
• Encourage communication between scientists and public
NAKFI - Themes to date:2008 - Complexity (Nov. 13-15, 2008) 2007 - Aging / Longevity (Nov. 15-18, 2007)
2006 - Smart Prosthetics2005 - Genomics2004 - Designing Nanostructures2003 - Signaling
http://www.keckfutures.org/
![Page 5: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/5.jpg)
Medtronic Corp.
Cochlear Corp.
Alfred Mann Institute,
Cleveland FES Center
Victhom Human Bionics
Motion Control Corp.
Cyberkinetics, Inc.
Prostheses of Today and Tomorrow
![Page 6: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/6.jpg)
Prostheses of the Future
Intelligent
Responsive
Biomimetic
![Page 7: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/7.jpg)
Mechanical Prostheses:Artificial limbsHeart valvesCerebral shunts
Neural prostheses:Hand graspStanding/steppingBladder/bowel controlExerciseMemory/cognitionNeuromotor therapyVisionHearing
Limb lossSpinal cord injuryStrokeBrain injuryParkinson’s DiseaseDeafnessBlindnessMemory impairmentsALSCerebral palsy
Why?
What?
![Page 8: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/8.jpg)
Task Group Topics• Describe a framework for replacing damaged cortical tissue and
fostering circuit integration to restore neurological function.• Build a prosthesis that will grow with a child (such as a heart valve or
cerebral shunt, or a self healing prosthesis).• Develop a smart prosthetic that can learn better and/or faster.• Brain interfacing with materials: recording and stimulation
electrodes.• Refine technologies to create active orthotic devices.• Structural tissue interfaces: enabling and enhancing continual
maintenance and adaptation to mechanical and biologic factors.• Sensory restoration of perception of limb movement and contact.• Design a functional tissue prosthesis.• Create hybrid prostheses that exploit activity-dependent processes.• Can brain control guide or refine limb control?
![Page 9: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/9.jpg)
Task Group 3: Develop a smart prosthesis that can learn better and/or faster.
client prosthesis
facilitator
![Page 10: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/10.jpg)
Task Group 3: Develop a smart prosthesis that can learn better and/or faster.
client prosthesis
facilitatorclient:• representations used in the brain• how to transmit information• mechanisms of plasticity• how to maximally exploit plasticity
Knowledge/technology gaps:
![Page 11: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/11.jpg)
Task Group 3: Develop a smart prosthesis that can learn better and/or faster.
client prosthesis
facilitator
facilitator:• identification of intermediate milestones
(performance and neurophysiological)• individuality of minimal detectable differences• customizing for specific user groups• engage the user
Knowledge/technology gaps:
![Page 12: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/12.jpg)
Task Group 3: Develop a smart prosthetic that can learn better and/or faster.
client prosthesis
facilitator
Knowledge/technology gaps:
prostheses:• biomorphic sensors/actuators for
compatibility w/ neural reps• detecting and communicating user
intent and motor commands• maintain/improve performance
based on dynamic interface• real-time machine learning • redundancy for versatility, efficiency
![Page 13: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/13.jpg)
seamless integration
where are the ‘smarts’?
cooperative learning
co-adaptationmaximizing plasticity
versatility
automated fitting
minimal cognitive demandsease-of-use
neuromorphic systemsbiomimicry
coordination
sensorimotor integration
information transmission/coding
utilizing plasticity autonomous - direct control
self-repairing
high throughput, high fidelity
amazondotcomificationredundancy
Smart ProstheticsNovember 9-11, 2006 - Irvine, CA
![Page 14: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/14.jpg)
![Page 15: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/15.jpg)
Top five activities (from a list of 34) amputees would like to be able to perform with their electric prostheses
1) Type/use a word processor2) Open a door with a knob3) Tie shoelaces4) Use a spoon or fork5) Drink from a glass.
Transradial Users
1) Type/use a word processor2) Cut meat3) Tie shoelaces4) Open a door with a knob5) Use a spoon or fork
Transhumeral Users
Atkins et al., 1996
![Page 16: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/16.jpg)
Surface EMG/ EEG
Implanted EMG
Direct peripheral nerve interface
Electrocorticogram
Recording w/ penetrating spinal/brain electrodes
Non-Invasive
Centrally Invasive
Command Input Source to Prosthesis
Sensory Feedback to User
Visual
Auditory
Cutaneous Stimulation
Peripheral Nerve Stimulation
Recording w/ penetrating spinal/brain electrodes
Non-Invasive
Centrally Invasive
![Page 17: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/17.jpg)
If everyone is thinking alike then someone isn't thinking. - General George S. Patton
![Page 18: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/18.jpg)
Peripheral Nerve
Horch; Utah, ASU
Electrocorticogram
Moran & Leuthardt; Wash. U.Williams; U. Wisconsin
Nicolelis; Duke U.
Donoghue; Brown U.Cyberkinetics, Inc.
Intracortical arrays
Muscle re-innervation
Kuiken; RIC
![Page 19: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/19.jpg)
![Page 20: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/20.jpg)
Where to put the ‘smarts’in smart prostheses?
![Page 21: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/21.jpg)
Endogenoussystem:
Exogenous system:
neural control
neuromorphic control
biological sensors
engineered sensors
muscles/ tendons
biomorphic actuators
Integrating Engineered and Physiological Systems
• biomimicry• physiological engineered• adaptive systems• system integration
![Page 22: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/22.jpg)
Oscar Pistorius
Hugh Herr, MIT
![Page 23: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/23.jpg)
Spinal cord
Control of locomotion
Spinal cord injury
Locomotor retraining
![Page 24: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/24.jpg)
The spinal cord is a conduit for information transfer from the brain.
The spinal cord is a highly sophisticated computational structure that mediates a wide range of physiological functions.
![Page 25: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/25.jpg)
Recruitment modulation
Rate coding
Co-contraction
StiffnessForce-velocity relationship
Reflexes
Contraction dynamicsFatigue
Length-tension
Biarticular muscleSecondary action of a muscle
Length
tension
Key features of neuromotor control that are either mediated by or have implications for spinal cord organization
![Page 26: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/26.jpg)
Pattern Generator (PG)
Spinal Segmental Circuitry (SSC)
SupraspinalCenters
motoneuronoutput
PGmotorcommands
PG reflexmodulation
output
Musculoskeletal System
ascendingoscillatory
drive
descendingPG inputs
descendingmotor
commands
spinalreflexpaths
PGreflexpathways
supraspinalreflex
pathways
Neural organization for locomotor control:
• Anatomical organization
• Reflexes (feedback)
• Pattern generation
• Multi-level structure
![Page 27: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/27.jpg)
most inputs to motoneurons from the brain are indirect (via spinal interneurons)
![Page 28: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/28.jpg)
• spinal cord mediated reflexes
• supraspinal modulation
... ...Musclespindle Muscle
spindleExtensormuscle
Flexormuscle
Descendingpathway
Motor neurons
Ia inhibitoryinterneuron
Ia afferent
Ia afferent
...... ......Musclespindle Muscle
spindleExtensormuscle
Flexormuscle
Descendingpathway
Motor neurons
Ia inhibitoryinterneuron
Ia afferent
Ia afferent
![Page 29: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/29.jpg)
Spinal Central Pattern Generators (CPGs)
• CPG produces basic pattern of locomotion• neuromodulators (5-HT, NE, DA, NMDA)• sensory feedback integrated into CPG operation• CPG activity is affected by training
From Barbeau et al. Brain Res. Rev. 30: 27, 1999
Brain Spinal Cord
Intracellular/extracellular recordings
Brain Spinal Cord
Intracellular/extracellular recordings
![Page 30: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/30.jpg)
The spinal cord is a conduit for information transfer from the brain.
The spinal cord is a highly sophisticated computational structure that mediates a wide range of physiological functions.
![Page 31: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/31.jpg)
Using spinal cord models for locomotor control
Woergoetter
![Page 32: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/32.jpg)
Hybrid Carbon:Silicon system
• neuromorphic aVLSI circuit • real-time closed loop • 1:1 phase coupling
5 sec
2 V VR
C
E
5 sec
2 V VR
C
E
Brain Spinal Cord
Ventral RootRecordings
C
E
L L
E
C
Brain Spinal Cord
Ventral RootRecordings
Brain Spinal Cord
Ventral RootRecordingsVentral RootRecordings
C
E
L L
E
CC
E
L L
E
C
Using spinal cord models for locomotor control
Jung, et al. 2001
![Page 33: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/33.jpg)
250,000 people with SCI in US
‘complete’ vs. ‘incomplete’
cervical - thoracic
Spinal Cord Injury (SCI)
Personal impact:• sensorimotor function• mobility and transfers• cardiovascular function• bladder/bowel• respiration• sexual function• exercise and recreation
Costs of Spinal Cord Injury:• health care costs in $billions/yr• lost productivity• reduced quality of life
Kentucky Agrability
![Page 34: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/34.jpg)
Spinal Cord Injury
![Page 35: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/35.jpg)
Causes of SCI:
(250,000 people with SCI in US)
motor vehicle crashes
falls
acts of violence
sports
other
Spinal Cord Injury: Clinical Outcomes from the Model Systems, Stover, DeLisa, & Whiteneck, Aspen Publishers,1995
Spinal Cord Injury
At the time of injury:• 63% are < 30 years old• 80% are employed or in
school• 90% have not gone beyond
high school• 54% are unmarried
Neurologic level of injury => degree of impairment• thoracic level lesion => paraplegia (46.2%)• cervical level lesion => tetraplegia (52.9%)
![Page 36: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/36.jpg)
Kwon et al.; The Spine Journal 4:451-464, 2004
Primary injury due to mechanical impact
Secondary injury due to inflammation and response cascade
A window of opportunity for neuroprotective interventions
In: Neurobiology of Spinal Cord Injury, Edts.R.J. Kalb and S.M. Strittmatter, Humana Press, Totowa, N.J.
rat thoracic injury
human cervical injury
![Page 37: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/37.jpg)
Intrinsic properties of motoneurons change after SCI
From Bennett et al. J. Neurophysiol. 86: 1955, 2001
![Page 38: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/38.jpg)
Motoneurons atrophy after spinal injury; exercise preserves structure
From Gazula et al. J. Comp. Neurol. 476: 130, 2004
![Page 39: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/39.jpg)
... ...Musclespindle Muscle
spindleExtensormuscle
Flexormuscle
Descendingpathway
Motor neurons
Ia inhibitoryinterneuron
Ia afferent
Ia afferent
...... ......Musclespindle Muscle
spindleExtensormuscle
Flexormuscle
Descendingpathway
Motor neurons
Ia inhibitoryinterneuron
Ia afferent
Ia afferent
SCI affects reflex modulation spasticity
Bladder-sphincter dyssynergybladder hyperreflexia
Yoshimura, Prog. Neurobiol. 1999
![Page 40: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/40.jpg)
www.robomedica.com
Promoting Locomotor Recovery after Incomplete SCI
www.litegait.com
Partial Weight Bearing Therapy (PWBT)
![Page 41: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/41.jpg)
If everyone is thinking alike then someone isn't thinking. - General George S. Patton
![Page 42: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/42.jpg)
Robot-assistedColumbo; (Lokomat)
IntraspinalMicrostimulation
Mushahwar, Horch, Prochazka
Epidural Spinal Cord Stimulation
Herman, HeDimitrievic
Therapist-assistedHarkema
Reflex stimulationField-Fote
![Page 43: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/43.jpg)
Promoting Functional Recovery After SCI
• overhead harness reduces load• treadmill assists with movement• cyclic loading pattern• exploits activity-dependent plasticity
Supplement with Neuromuscular Electrical Stimulationmore complete sensory pattern to spinal cord?
increased (or more functional) plasticity?
Treadmill Training
Neuromuscular Stimulation
Neuromorphic Adaptive Control
![Page 44: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/44.jpg)
surfaceintramuscularintraneuralcuffepimysial
Case Western Reserve Univ.
Neuromuscular Stimulation: electrode types
NeuroTECH
U. Utah
![Page 45: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/45.jpg)
Muscle nonlinearities are exacerbated by recruitment properties and fatigue
• limited input range• steep-slope regions• changes with fatigue
0
10
20
30
40
50
60
70
0 20 40 60 80 100
stimulation level (%)
angl
e (d
eg) pre-trial
after 12 min(300 cycles)
![Page 46: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/46.jpg)
Muscles SkeletonPatternGenerator
adaptation
PatternShaper
feedback
PG/PS Control System
PG/PS control: PG generates cyclic pattern
- structure based on neural modelPS fine tunes the pattern
- single layer adaptive neural networkObjectives:
customize stimulationaccount for nonlinearities, dynamics, etc.adjust for fatigue
![Page 47: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/47.jpg)
d V (t)/dt = (I (V , t) + I (V , t) + I - V (t) / R ) / Ci i i i i i im syn m PM m inj m m m
)(t)-(-i VVm+1
1 = (t)yoimii
2e
i ij ij isyn j syn syn mI (t) = y (t) g (E - V (t))∑
i i iPM NMDA KCaI (t) = I (t)+ I (t)
Membrane dynamics:
Output function (firing rate):
Membrane currents:
PG Neuron Equations
![Page 48: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/48.jpg)
i i i i iNMDA NMDA NMDA i NMDA mI (t) = K g p (t)(E - V (t))
(t)p CVE A - (t))p-(1 CEV A=dt
(t)pdi
))/-((
i
))/-((i imir
iirmi
i
βαβα ee
i i i i iKCa KCa NMDA KCa mI (t) = g [Ca (t)](E - V (t))
](t)Ca[-(t))V-E(K(t)p=dt
](t)Cad[iNMDANMDAmCaNMDANMDANMDAi
iNMDAiiiii
δρ
Burst initiation:
PG Pacemaker Currents
i i iPM NMDA KCaI (t) = I (t)+ I (t)
Burst termination:
Adapted from Brodin, 1991
![Page 49: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/49.jpg)
Es2 Is2 Ia2
I a1 I s1 Es1
Burst initiationMutual inhibitionReciprocal inhibition
Es Exciter of synergistsIs Inhibitor of synergistsIa Inhibitor of antagonists
Inhibitory synapseExcitatory synapse
___
__
PG Circuit
![Page 50: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/50.jpg)
actualdesired
flexorsextensors
time (sec)
angle (deg)
torque(Nm)
PS z(t)
PG y(t)
![Page 51: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/51.jpg)
![Page 52: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/52.jpg)
actualdesired
time (sec)
angle (deg)
torque(Nm)
PS z(t)
PG y(t)
flexorsExtensorsR1 neuron
Reflexes with phase-resetting
![Page 53: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/53.jpg)
Phase-dependent reflexes
![Page 54: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/54.jpg)
actualdesired
flexorsExtensorsR1 neuron
time (sec)
angle (deg)
torque(Nm)
PS z(t)
PG y(t)
Reflexes without phase-resetting
![Page 55: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/55.jpg)
time of disturbance application (% cycle)
Percent reduction
Peak muscle torque
RMS muscle torque
Phase-dependent reflexes
![Page 56: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/56.jpg)
PG model provides:basic movement rhythmphase-resettingphase-dependent reflexescycle period adjustments
Needs:localized learningmore complex movement patternsadditional reflexes
![Page 57: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/57.jpg)
Muscles SkeletonPatternGenerator
adaptation
PatternShaper
feedback
PG/PS Control System
PG/PS control: PG generates cyclic pattern
- structure based on neural modelPS fine tunes the pattern
- single layer adaptive neural networkObjectives:
customize stimulationaccount for nonlinearities, dynamics, etc.adjust for fatigue
![Page 58: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/58.jpg)
PS Neuron Equations
∑=
=m
jtkjytkjwtzk
1)()()(
muscle/skeleton
PG
PS
zy
jx Y
Ydese
Σ
Σ
0 0.2 0.4 0.6 0.8time (sec)
individual PS neuron activity levels, y1j
0
0.5
1
![Page 59: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/59.jpg)
PS Learning
∑ −=Δn
nTtkjyn
tetkjw )(1)()( η
)()()( tytytw prepostη=Δ
)()()( tytytw preaη=Δ
Hebbian learning algorithm:
Heterosynaptic Hebbian learning algorithm:
Time-averaged learning algorithm:
![Page 60: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/60.jpg)
system output (degrees)
0 0.2 0.4 0.6 0.8 1-0.2-0.1
00.10.2weighted PS
neuron activity levels, w1jy1j
0 5 10 15 20-0.2neuron index, j
20 40 60 80 1000
5
10
15RMS error per cycle (degrees)
0 0.2 0.4 0.6 0.8 1-20-10
01020
0 0.2 0.4 0.6 0.8 1-0.4-0.2
00.20.4
PS output, z1(t)
20 40 60 80 1000
5
10
150 0.2 0.4 0.6 0.8 1
-20-10
01020
0 0.2 0.4 0.6 0.8 1-0.4-0.2
00.20.4
0.2
0 0.2 0.4 0.6 0.8 1-0.2-0.1
00.1
0 5 10 15 20-0.2neuron index, j
time (sec) time (sec)
time (sec)
time (sec)
time (sec)
time (sec)
time (sec)
time (sec)
c5%=14 cycles c5%=4 cycles
RMS10=.57° RMS10=.50°
-0.10
0.10.2
PS neuron weights, w1j
0.2
-0.10
0.1
0 0.2 0.4 0.6 0.8 1time (sec)
0 0.2 0.4 0.6 0.8 1time (sec)
individual PS neuron activity levels, y1j
n=10, na=5, η=.002 n=10, na=15, η=.002
0
0.5
1
n(k)
0
0.5
1
![Page 61: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/61.jpg)
Angle (deg)
0 5 10
0
20
40
Time (sec)
90 95 100
0
20
40
Time (sec)
PG/PS Desired
0 10 20 30 40 500
20
40
60
Cycle #
RMS (%)
N=32
PG/PS PD +/-STD
PG/PS Control
![Page 62: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/62.jpg)
Adapting to account for fatigue
0 50 100 150 200 250 3000
10
20
30
40RMS error (%)
OFFOFF OFFON ON ON
Cycle #
Adaptation:
Mean ± 1 STD (n=16)
![Page 63: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/63.jpg)
Thigh Segment
(Hip Angle)(Deg) -15
0
15
-10
10
30Shank
SegmentAngle (Deg)
0102030
Knee JointAngle (Deg)
Fixed Stim01
01
0Stim by PGPS 0
11
47 48 49 50Cycle Number
-15
0
15
-10
10
30
0102030
0 1 2 3Cycle Number
actual
desired
FWR
gluts
first 3 cycles last 3 cycles
quadshams
control of multi-segment movements
![Page 64: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/64.jpg)
-20
-10
0
10
20First 10 cycles
Hip angle (Deg)
-20
-10
0
10
20Last 10 cycles
20 40 60 80 1000
0.2
0.4
0.6
0.8
1
% of cycle
Normalized hams stim
20 40 60 80 1000
0.2
0.4
0.6
0.8
1
% of cycle0 0
20 40 60 80 1000 20 40 60 80 1000
![Page 65: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/65.jpg)
BF
IL
VL
ST
TA
GM
HumanPre-clinical
![Page 66: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/66.jpg)
80
100
120 Hip
60
90
120 Knee
40
85
130 Ankle
40
65
90 Shoulder
80
115
150 Elbow
Fore Limb
1 1.5 2 2.5 3
Foot Fall
time (sec)
H
G
(deg
) F
(deg
)
E
(deg
)
D
(deg
)
C
(deg
)
B
A Hind Limb
HLLFLLHLRFLR
Pre-clinical model development (Jung, et al., ASU)
BFIL
VL
ST
TA
GM
Thota et al. J. Neurotrauma 22(4): 442-465, 2005
![Page 67: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/67.jpg)
Can we control the stepping movement?
Can we promote functional plasticity?
![Page 68: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/68.jpg)
CK200: FES Exercise In Clinic or Home
These projects are being supported by the NIH-NCMRR: 1R43-HD050006-01, 5R44-HD041820-04 to customKYnetics, Inc.
Disclosure: J. Abbas is part-owner and co-founder of customKYnetics, Inc.
Commercial partnership: Commercial partnership: customKYneticscustomKYnetics, Inc. , Inc.
Adaptive Stimulator for Exercise and Rehabilitation
Stimulation-Augmented Exercise and Neuromotor Therapy
![Page 69: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/69.jpg)
Integrating physiological and artificial systems
Endogenoussystem:
Exogenous system:
neural control
neuromorphic control
biological sensors
engineered sensors
muscles/ tendons
biomorphic actuators
• biomimicry
• intact artificial
• system integration
![Page 70: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/70.jpg)
Center for Adaptive Neural Systems
neural prostheses and advanced prosthetic systems
adaptation in neural systems
neuromorphiccontrol systems
biological inspiration
technology to replace lost or missing functionality
technology to promote learning or adaptation
neuro-inspired system design
![Page 71: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/71.jpg)
Jimmy Abbas
Neuromorphic Design of Smart Prosthetic and Therapeutic Systems
FULTONs c h o o l o f e n g i n e e r i n g s c h o o l o f e n g i n e e r i n g
Ira A.
Acknowledgements:NIHNSFFDAUS ArmyASUBarrow Neurological InstituteBanner Good Samaritan Medical Center
JoAnne RiessPankaj KatariaXia ZhangEric HartmanJunli OuJason Gillette, PhDEd Stites, MD, PhDSusan McDowell, MD
Manoshi BhowmikAndrea DowningAlison SitekRanu Jung, PhDNarayanan Krishnamurthi, PhDKen Horch, PhDDevin Jindrich, PhDSharon Crook, PhDBertan Bakkaloglu, PhDBrian Smith, PhDSteve Philips, PhDRichard Herman, MDJohann Samanta, MDPadma Mahant, MDDenise Campagnolo, MDDan Lieberman, MD
![Page 72: Neuromorphic Design of Smart Prosthetic and Therapeutic ...papers/neuromorphic2008/lectures...Neuromorphic Design of Smart Prosthetic and Therapeutic Systems. Complex visual fields](https://reader035.vdocuments.net/reader035/viewer/2022081405/5f09c8277e708231d428783d/html5/thumbnails/72.jpg)