PROJECT: Neural Recording Laboratory PRODUCT: Research Laboratory ROLE: Architect/Doctoral Researcher OVERVIEW In January 2006 I joined the laboratory of Dr. Stephen Helms Tillery, who only months before had accepted a tenure track position in the department of Bioengineering at Arizona State University. Our goal was to build a laboratory to conduct world-class research into the neural basis of sensor and motor function in the brain. The opportunity was rare and exciting, but we truly started from scratch. My first glimpse of the lab was of completely empty room. The Boss told me what he wanted and said, “Build it.” MY ROLE This was my most complex and comprehensive accomplishment to date. I was responsible for procurement, installation, configuration, software development, and hardware development for everything in the lab. I was given broad budgetary discretion for purchases. If something was required and it didn’t exist, I designed it in Solidworks then fabricated it with machine tools or rapid prototyping machines. If software was required, I developed it in C/C++, Python, Matlab, Simulink, etc... Over the next 2 years the following core pieces of the laboratory setup were developed:

1. 6-AXIS INDUSTRIAL ROBOT: A robot arm was required to present objects to research subjects. Developed custom real-time control routines in C++ using manufacturer SDK. Integrated pneumatic end effector tool changer system with 6-DoF Force/Torque sensor. Developed custom front end GUI using LabView.

2. VIRTUAL REALITY SIMULATION: Our experiments required subjects to execute tasks in a virtual reality environment. This was developed in the Python programming language in using Vizard development software. VR control was integrated into overall system using LabView.

3. 3-D MOTION CAPTURE: We tracked and analyzed the detailed kinematics of our subjects’ hand during the experimental task using an active marker (LED) system from Phasespace. Developed custom marker arrays and core software routines to acquire maker data in real time to drive animations in the VR simulation.

4. NEUROPHYSIOLOGICAL RECORDING SETUP: We recorded the activity of single neurons in the sensory cortex of the brain during our experiments. The Plexon system was integrated in the overall system using LabView software

5. SYSTEM CONTROL SOFTWARE: Developed a hybrid PC/Real-Time application in LabView to unify all components of the system, including sensory I/O, robot commands and VR control. The end product was a unified control GUI from which all aspects of the experiment could be monitored and controlled.

6. MONKEYS!!: Truly the most unpredictable and challenging aspect of the this work; an

education within an education. Learned to handle, train and work with two male Rhesus macaque monkeys to obtain neurophysiological data.

OUTCOME After nearly four and a half years of continuous work, I completed my experimental work and doctoral dissertation, graduating in May of 2010. My legacy is a neurophysiological recording laboratory that is uniquely capable, flexible and reconfigurable for many kinds of neural experimentation protocols. Accomplishing the PhD made me a better engineer and taught me how to carry out research. What I now seek is the opportunity to unify the myriad skills acquired in my work and education so far into a single goal requiring multidisciplinary skills.

EXPERIMENTAL SETUP

1. Monkeys were trained in a novel Reach-to-Grasp task in which all visual cues were presented in a Virtual Reality simulation.

2. Grasp object of different sizes were presented in the workspace behind the VR

presentation and hand position and digit kinematics were tracked using an active marker motion capture system.

3. The firing activity of single neurons in sensory cortex was recorded while the monkeys

completed either a physical task (object present in the workspace) or a randomly inserted virtual task (object presented just out of reach)

4. This approach permitted manipulation of the actual and expected sensory outcome of

the task.

SMORG ROBOT AND ASSOCIATED HARDWARE.

A. THE 6-AXIS INDUSTRIAL ROBOT. Mounted on a custom platform and controlled using custom software. Dedicated signal and air channels routed through the robot enabled feedback from a 6-DOF F/T sensor, object touch sensors and control of a pneumatic tool changer. B. THE ROBOT END EFFECTOR. The F/T sensor (b2) was mounted directly to the robot end effector (b1). The master plate of the tool changer (b3) was mounted to the F/T sensor using a custom interface plate. Air lines originating from ports on the robot controlled the locking mechanism of the master plate. C. GRASP OBJECT ASSEMBLY. The object was mounted to a six-inch standoff post that mounted to a tool plate. Touch sensors were mounted flush with the object surface and wires were routed to the object interior for protection. Power and signal lines were routed through a pass-through connector (not visible), through the robot interior to an external connector on the robot base.

A B

C

b1

b2

b3

CAPTURING HAND KINEMATICS.

Our experiments required knowledge of detailed hand kinematics – of monkeys! This figure highlights just some of the hardware and electronics development I conducted to accomplish this task. A. COMMERCIALLY AVAILABLE DATA GLOVES feature numerous integrated bend sensors to capture the posture of the digits and palm but were prohibitively expensive and difficult to customize to the monkey hand. B. EARLY PROTOTYPE OF THE CUSTOM MONKEY GLOVE. Bend sensors and electronics were removed from a gaming glove and reconfigured to the monkey hand. C. A WIRELESS VERSION OF THE MONKEY GLOVE. This more advanced version featured 5 bend sensors, 2-axis roll/pitch sensing, wireless bluetooth transmisstion and a rechargeable battery. Electronics were encased in epoxy for protection. D. AN ALTERNATIVE STRATEGY FOR PASSIVE MOTION CAPTURE. Cube markers with finger attachment clips were developed to utilize larger markers for a novel camera sensing technique. This approach captured only crude measures of hand posture ACTIVE MARKER LEDS. We eventually settled on an active marker LED system from Phasespace, show at right.

A CB

D

GENERAL SMORG LAB PICTURES

TESTING ROOM Motion capture cameras surround the subject seating area. A 3D monitor placed overhead reflects the VR environment into a mirror directly ahead. The robot presented grasp objects in the workspace

HARDWARE AND ELECTRONICS These experiments required the integration of much hardware and electronics CUSTOM HARDWARE These experiments also required the development of custom hardware, such as the grasp objects seen here.

PROJECT: Experimental Device Development PRODUCT: Actuated Ankle Foot Orthosis (AAFO) ROLE: Project Manager OVERVIEW Advensys, LLC is an early stage startup company developing adaptive neuromorphic systems for “advancing human mobility.” In 2004 the company was awarded a competitive Phase I contract by the US Army to develop a neuromorphically controlled lower leg orthosis capable of providing ambulatory support for soldiers with injuries sustained in combat. The company was tasked with developing a prototype electronic control system based on neural pattern networks, management of AAFO hardware development and experimental assessment of the integrated system. The duration of Phase I prototype development, integration and testing was just 6 months. MY ROLE I was hired by the President and co-founder of the company as the Program Manager. My duties included several significant subtasks of the overall project:

7. NUMERICAL MODELING OF NEURAL PATTERN GENERATOR NETWORK: a biologically inspired oscillating signal generator patterned after spinal locomotor circuitry of the lamprey. The activity of this network could be entrained by an external periodic signal and would form the basis of our real-time control system.

8. PROTOTYPE ELECTRONICS DEVELOPMENT: Following numerical simulation and characterization, the neural pattern generator was implemented in breadboard electronics using RC circuit models of neural membrane dynamics.

9. REAL-TIME CONTROL SYSTEM DEVELOPMENT: A real-time numerical controller was developed using Matlab, Simulink and Real-Time Workshop. The controller was driven by a hip angle sensor and drove the ankle “push-off” of the AAFO.

OUTCOME Phase I was completed on time and under budget. The device was demonstrated to senior Army program officials in a live demonstration and feedback was overwhelmingly positive. Advensys was invited to apply for Phase II funding, and was subsequently awarded approximately $1.5 million over 2 years.

NUMERICAL MODELING OF NEURAL PATTERN GENERATOR NETWORK THE UNIT PATTERN GENERATOR (UPG) CONCEPT: A bi-laterally symmetric network whose connectivity is based on the known architecture of the lamprey spinal cord. The three spinal neural classes that form the kernel of the uPG are: E- excitatory, L-inhibitory, and C-crossed inhibitory. The E, L, and C neurons represent the core of the network and the M neurons are the “output” units of the network.

SIMULATION RESULTS OF NEURAL NETWORK ENTRAINMENT IN RESPONSE TO INJECTED CURRENT INPUT. Both sides of the network oscillate at default frequency before entrainment, and then assume the frequency of the injected current.

PROTOTYPE ELECTRONICS DEVELOPMENT

PROTOTYPE UNIT PATTERN GENERATOR (UPG) ELECTRONICS: Unit Pattern Generator network showing neurons implemented in analog hardware.

ELECTRONICS TESTING: uPG network testing

REAL-TIME CONTROL SYSTEM DEVELOPMENT

CONTROL SYSTEM DEVELOPMENT: Simulation and controller development were in Matlab, Simulink and Real-Time Workshop (RTW). Compiled, executable RTW code was ported to the Hardware Computer via an ethernet link. The Hardware Computer executed the real-time control system code, including sampling analog sensor input (analog-to-digital) from external hardware and asserting all analog commands (digital-to-analog) to hardware through the PCI DAS1200 A/D, D/A card. Two Break-Out Boards (BOB) provided numbered conductor connection points for system I/O. Motor commands, were sent from the BOB to the Amplifier Board, which directly drove the motor.

!