neuromorphic and biomorphic engineering systems 3d interconnect technology that allows three or- ......

250 Neuromorphic and biomorphic engineering systems

311:47-52, 2006; C. E Camerer, G. Loewenstein, and D. Prelec, Neuroeconomics: how neuroscience can inform economics, J Econ. Lit., 439-64, 2005; M. Kosfeld et al., Oxytocin increases trust in humans, Nature, 435(2):673-676, 2005; F? J. Zak, Neu- roeconomics, Philos. 2: Rqy. SOC. B, 359:1737-1748, 2004; P. J. Zak, The neurobiology of trust, Sci. Amez, pp. 88-95, June 2008.

Neuromorphic and biomorphic engineering systems Many biological systems, from the molecular scale to the macroscale and from the body to the brain, display remarkable efficiency and robustness. For example, a single mammalian cell, approximately 10 micrometers in size, performs complex biochemical signal processing on its mechanical and chemical input signals with highly noisy and imprecise parts, using approximately 1 picowatt (lo-'' W) of power. Such signal processing enables the cell to sense and amplify minute changes in the concentrations of spe- cific molecules amid a background of confoundiigly similar molecules, to harvest and metabolize energy contained in molecules in its environment, to detox- ify poisonous molecules, toLsense if it has been in- fected by a virus, to communicate with other cells in its neighborhood, to move, to maintain its struc- ture, to regulate its growth in response to signals in its surroundings, to speed up chemical reactions via sophisticated enzymes, and to replicate itself when it is appropriate to do so. The approximately 20,000- node gene-protein and protein-protein molecular network within a cell makes even the most advanced nano-engineering of today look crude and primitive.

The brain is made of approximately 22 x lo9 neurons that form a densely connected network of approximately 240 x 10'' synaptic connections. This network performs approximately 1015 synaptic operations per second at approximately 14 W of power, several orders of magnitude more energy efficient than the most advanced computers. The brain can perform real-time, reliable, complex tasks with un- reliable and noisy devices. It uses remarkably com- pact hardware built with a rich array of biochemical and biophysical devices and is architected with a 3D interconnect technology that allows three orders of magnitude more connectivity than the most advanced engineering systems of today. The brain is adaptive and plastic with rapid learning and general- ization capabilities that outperform the most sophisticated machine-learning algorithms.

Can we learn from nature to build better engineering systems that are equally impressive, robust, and efficient? The goal of neuromorphic engineering, a term coined by Carver Mead, is to take inspiration from neurobiological architectures to build better engineering systems, "morphing" them with insight from their natural neurobiological domains to be useful in artificial engineering domains. More generally, we can define a biomorphic system as one that takes

inspiration from any architecture in biology, for ex. ample, the architecture of cells, to create a mOrphed version that is useful in an engineering context. Thus, airplanes are biomorphic architectures that are in. spired by the winged flight of birds. A neuromorphic silicon cochlea or silicon retina is inspired by the architecture of the ear or the eye and performs highly parallel nonlinear filtering, gain control, and corn- pressive computations on an audio or image input respectively.

Relation of engineering to biological systems, Biomorphic solutions have sometimes been rein- vented by engineers without their even knowing that they are biomorphic or that they already exist in nature: The use of chirp signals for accurate range sensing in radars was invented by engineers around World War 11, but bats had already been using ul- trasonic chirps for range sensing in their biosonar systems for millions of years. Positive-feedback circuits were invented about 100 years ago but have been present in sodium ion channels for more than 100 million years. Thus, knowledge of systems in nature can provide useful ideas for engineering. Several biomorphic architectures, such as machine-learning and pattern-recognition systems inspired by the o p eration of neurons in the brain, are already widely used in artificial systems.

In biomorphic systems, it is important to keep the insightful "babyn and throw out the cluttering "bath- water" details. Certain architectures in biology may be accidents of evolution, may be more suited to the constraints of a biological organism, and may serve or may have served a purpose that we do not yet understand. Consequently, their relevance to a different engineering context where the constraints are diifer- ent may be questionable. Birds are not airplanes and airplanes are not birds, although the study of one can shed insight into the study of the other. Hence, it is important to evaluate a biomorphic engineering system by traditional engineering metrics to insightfully understand where value can be added.

Types of biomorphic systems. From an engineering point of view, where do biomorphic systems add value? They clearly have the potential to shine in the following kinds of systems:

1. Ultralow-power and highly energy efficient sensing, actuating, and information-processing SYs- tems.

2. Signal processing and pattern-recognition sY* tems that need to operate in noisy environments and over a wide dynamic range of inputs.

3. Robust and efficient computation with noisy and unpredictable devices.

4. Systems with feedback, adaptation, and learning at multiple spatial and temporal scales.

5. Systems that integrate technologies from diverse domains.

6. Self-repairing systems. 7. Self-assembling systems. 8. Energy-harvesting systems. 9. Robotic systems. Features of biornorphic systems. How do biomoP

phic systems appear to accomplish these feats?

Sarpeshkar, R. "Neuromorphic and biomorphic engineering systems."McGraw-Hill Yearbook of Science & Technology 2009. New York: McGraw-Hill, 2009. pp. 250-252.

Neuromorphic and biomorphic engineering systems 251

These systems have many common features, some of which will be discussed.

1. The exploitation of analog physical basis functions for computation in biochemical, biomechani- cal, or bioelectronic technologies, rather than mere digital logic functions, as in traditional computation. The increase in efficiency then arises out of exploiting the graded analog degrees of freedom ifl each signal and device rather than treating signals as being merely "on" or "off' or treating devices as just being switches, as in digital systems. However, the robustness of such analog computing

is established in a different manner from digital systems. Clever feedback-and-regulation systems and inherently noise-robust topologies are built to ensure that the overall output and overall system is robust even though each device and signal in it is not. It is this combination of feedback and analog system design that allows biological systems to operate both robustly and efficiently. Rather than operate in a collective fashion, with several low-precision digital elements that collectively interact to implement a high-precision or complex operation, biological systems operate with many low-precision analog elements that collectively interact to implement a high-pre~ision or complex operation. The interactions and processing can have a hybrid analogdigital nature with both all-or-none digital and graded analog processes being present. AU-or-none interactions are often useful in performing digital signal restora- tion in analog systems that are prone to the effects of noise and in making decisions. For example, after extensive analog processing has been performed on the multiple inputs of a neuron, the final output of a neuron is often an all-or-none "spike" or voltage pulse. This spike is fired by the neuron as soon as the voltage near a particular somatic region of the neuron exceeds a threshold.

Biomorphic systems have used programmable analog processing with feedback-and-feedforward calibration to construct ultralow-power cochlear- implant processors for the deaf. Such processors have lowered processing power by more than an order of magnitude while being robust to several sources of noise and while maintaining high levels of flexibility.

2. The use of highly sophisticated technologies: The ear is an example of a highly sophisticated technology that integrates microfluidics, micromechan- its, piezoelectrics, and microelectronics into a system to perform more than lo9 arithmetic operations Per second of spectrum-analysis computations with 14 FW of power in a volume not much larger than the size of a pea. Biomorphic systems have mimicked the architecture of the cochlea in radio-frequency (RF) technologies to construct efficient ultrawide- band spectrum analyzers, that is, RF cochleas. As another example, the brain's 3D interconnect technology allows a fanout of approximately 10,000 per

rather than just approximately 5 per logic gate in electronic technologies today.

3. The use of nonlinear and adaptive processing: During development, cells implement ingenious

nonlinear diffusion-anddegradation partial differen- tial equations that ensure that cell differentiation is robust to variations in the concentrations of "morphogen" molecules. These concentration val- ues are important in deciding whether the cell de- velops or "differentiatesn into one cell type versus another, for example, a liver cell versus a kidney cell. Learning is an adaptive or feedback process that alters the parameters or topology of a system over slow time scales such that it more efficiently processes signals in its environment. Learning is ubiquitous in biological systems. Nonlinear and adap tive signal processing in the ear has led to a bio- inspired companding algorithm for improving the perception of speech recognition in noise in both deaf patients and in artificial speech-recognition systems.

4. The use of scalable cellular architectures: The use of scalable cellular architectures, with strong integration of processing, memory, sensing, actua- tion, and communication functions in each local cellular unit, rather than only in specialized regions as in traditional architectures, is common in several systems in biology. Such architectures range from a network of skin cells to a network of neurons in the brain. John von Neumann, the inventor of traditional digital architectures named after him, was himself aware of the limitations of his inventions. He engaged in research on cellular automata and analog computation in his later years, inspired by computation in the brain.

5. The use of ingeniously clever algorithms: The control of eye movement appears to use sophisticated feedback-control loops that function with good speed, accuracy, and stability in spite of large delays in the system, a challenging task in engineering. Biological systems appear to use knowledge and learning to constantly tune parameters in predictive architectures that can compensate for such delays. In general, biological systems are adept at tuning them- selves to respond optimally to the signal statistics in their environment such that they are both robust and efficient. Increasingly, advanced circuits in engineering are using predictive digital compensation of errors in analog systems to enable better robustness and efficiency.

Historical and evolutionary perspectives. The use of nature to inspire better engineering is as old as it is new. The most advanced aircraft are beginning to explore the use of turbulence just as birds do, even though the Wright brothers gave up on such strate- gies because they were too difficult to implement at the time. Nevertheless, airplanes are still very far from achieving energy efficiencies that can compete with that of a bird.

Humans have a long way to go before their architectures will successfully compete with those in nature, especially in situations where ultra- energy-efficient or ultralow-power operation are paramount. In evolutionary environments, food was always a scarce resource, so biological systems that needed to harvest their energy and raw materials from food needed to be very efficient. The result


252 Neutron scattering

has been the creation of incredibly energy-efficient L. Turicchia et al., A bio-inspired companding strat. architectures. egy for spectral enhancement, IEEE Trans. Speech

Biomedical applications. Several biomorphic and Audio Process., 13:243-253, 2005.

neuromorphic architectures have led to useful circuits for biomedical applications, where learning from biology is only natural in helping to lix systems when they do not work. Ultralow-power operation is also important for these applications. For example, ultra-low-power analog-to-digital conversion inspired by the operation of spiking neurons in biology has led to a very energy efficient converter for biomedical applications. Another bio-inspired algorithm has helped create an algorithm that has bene- fits for helping the deaf perceive music.

Learning architectures that mimic the connectivity and adaptation in connection strengths of synapses, that is, the junctions that connect neurons to each other, have been very important in artificial machine learning systems. Many learning systems exploit local learning rules at artificial synapses to automatically create self-organizing systems at a more global network level. Such systems automatically teach them- selves to function from mere examples or by ex- tracting patterns in their input data. Ultralow-power analog learning circuits are being applied to create architectures for learning and decoding the movement intentions of paralyzed patients from their brain signals.

The field of biomorphic design suggests that we can mine the intellectual resources of nature to create devices useful to humans, just as we have mined her physical resources in the past. Such mining will require us to combine inspiration with perspiration and to understand how nature works with insight.

For background information see ADAPTIVE CON- TROL; ADAPTIVE WINGS; ANALOG COMPUTER; AR- TIFICIAL INTELLIGENCE; BIOMEDICAL ENGINEERING; BRAIN; CELL ORGANIZATION; CELLULAR AUTOMATA; CONTROL SYSTEMS; CYBERNETICS; HEARING IMPAIR- MENT; NEURAL NETWORK; NEUROBIOLOGY; NONLIN- EAR CONTROL THEORY; SIGNAL. PROCESSING in the McGraw-Hill Encyclopedia of Science &Technology.

Rahul Sarpeshkar Bibliography. L. C. Aiello et al., The expensive-

tissue hypothesis: the brain and digestive system in human and primate evolution, Cuw Anthro- poL, 36(2): 199-22 1, 1995; G. Cauwenberghs and A. Bayoumi, Learning in Silicon: Adaptive VLSINeu- ral Systems, Kluwer Academic Publishers, 1999; A. Eldar et al., Robustness of the BMP morphogen gra- dient in Drosophila embryonic patterning, Nature., 419:304-308, 2002; C . Mead, Neuromorphic electronic systems, Proc. IEEE, 78: 1629- 1636, 1990; D. A. Robinson et al, A model of the smooth-pursuit eye-movement system, Biol. Cybern., 55(1):43-57, 1986; R. Sarpeshkar, Analog versus digital, Neu- ral Comput., 10: 1601-1638, 1998; R. Sarpeshkar et al., An ultra-low-power programmable analog bionic ear processor, IEEE Trans. Biomed. Circuits Syst., 52(4):711-727, 2005; R. Sarpeshkar et al., Low-power circuits for brain-machine interfaces, IEEE Trans. Biomed. Circuits Syst., in press, 2008;


neuromorphic and biomorphic engineering systems 3d interconnect technology that allows three or- ......

Documents