Digital Signal Digital Signal Processing, Processing,

Cellular Cellular Automata, and Automata, and

ParallelismParallelismKarl SchrammKarl SchrammSan José State UniversitySan José State University

Computer Science DepartmentComputer Science DepartmentCS 240CS 240

Spring 2003Spring 2003

OutlineOutline Introduction to digital signal Introduction to digital signal

processingprocessing Relationship of DSPs and CAs Relationship of DSPs and CAs

(Hint: parallelism) (Hint: parallelism) An overview of parallel computingAn overview of parallel computing The future?The future?

Introduction to Digital Introduction to Digital Signal ProcessingSignal Processing

Manipulation of a signalManipulation of a signal Signals are waveformsSignals are waveforms Typically carried out in realtimeTypically carried out in realtime Calculations are handled by Calculations are handled by DDigital igital

SSignal ignal PProcessors (DSP)rocessors (DSP)

DSP UsesDSP Uses Telecom: Telecom:

MultiplexingMultiplexing Signal encoding/decodingSignal encoding/decoding CompressionCompression

MilitaryMilitary Sonar & Radar processingSonar & Radar processing Visual tracking systems for laser-guided weaponsVisual tracking systems for laser-guided weapons Communication encryption/decryptionCommunication encryption/decryption

MedicalMedical CAT & MRI imagingCAT & MRI imaging

EntertainmentEntertainment Audio & visual analysis and effectsAudio & visual analysis and effects

DSP OverviewDSP Overview Receives a signal, processes it, and outputs resultReceives a signal, processes it, and outputs result Several DSPs can be linked together in a serial Several DSPs can be linked together in a serial

chain chain Can be implemented in software or hardwareCan be implemented in software or hardware

Hardware is best for realtime performanceHardware is best for realtime performance

[6: page 37]

Analog to Digital Analog to Digital ConversionConversion

Converts a continuous, analog signal into a Converts a continuous, analog signal into a discrete, digital signaldiscrete, digital signal

Analog signal is sampled at a regular intervalAnalog signal is sampled at a regular interval Sampled values are stored as a data streamSampled values are stored as a data stream

1D array1D array

[6: page 41]

SamplingSampling Shannon’s sampling theoremShannon’s sampling theorem

A signal must be sampled at a rate at least 2x the of the maximum frequency component A signal must be sampled at a rate at least 2x the of the maximum frequency component (Nyquist frequency) or else aliasing will occur [Nyquist (1928), Shannon (1949)](Nyquist frequency) or else aliasing will occur [Nyquist (1928), Shannon (1949)]

Aliasing is the inability to accurately reconstruct a signal from sampled dataAliasing is the inability to accurately reconstruct a signal from sampled data Low-pass analog filter (anti-aliasing filter) applied before samplingLow-pass analog filter (anti-aliasing filter) applied before sampling

DSPs vs. CAsDSPs vs. CAs Both perform the same function:Both perform the same function:

Receive data & process itReceive data & process it DSP calculations are usually complexDSP calculations are usually complex

A DSP chain is similar to the successive A DSP chain is similar to the successive generations of a CAgenerations of a CA Calculations of CAs don’t usually change over Calculations of CAs don’t usually change over

timetime A DSP can be viewed as a 1D Continuous A DSP can be viewed as a 1D Continuous

CACA Signal is a continuous valued arraySignal is a continuous valued array

top: [3]; bottom: [5]

CAs as DSPsCAs as DSPs CAs can be used in CAs can be used in

sound synthesissound synthesis Quantized CA Quantized CA

values can be used values can be used to play a tone or to play a tone or trigger a MIDI trigger a MIDI note (CAMUS)note (CAMUS)

Continuous CA Continuous CA values can be used values can be used to generate a to generate a signal (CASound)signal (CASound)

CAs and ParallelismCAs and Parallelism Parallelism is an important aspect of CAsParallelism is an important aspect of CAs

Cells are independent entitiesCells are independent entities Parallelism allows CAs to be used to model:Parallelism allows CAs to be used to model:

Fluid dynamicsFluid dynamics Cell & crystal growthCell & crystal growth Etc.Etc.

CAs are parallel machinesCAs are parallel machines SIMDSIMD Shared MemoryShared Memory

DSPs must exhibit parallelism to be like a DSPs must exhibit parallelism to be like a CACA

Flynn’s TaxonomyFlynn’s Taxonomy SISDSISD

SSingle ingle IInstruction stream nstruction stream SSingle ingle DData streamata stream von Neumann architecturevon Neumann architecture

SIMDSIMD SSingle ingle IInstruction stream nstruction stream MMultiple ultiple DData streamata stream

MIMDMIMD MMultiple ultiple IInstruction stream nstruction stream MMultiple ultiple DData ata

streamstream MISDMISD

MMultiple ultiple IInstruction stream nstruction stream SSingle ingle DData Streamata Stream Pipelined architecturePipelined architecture

Parallel Computing Parallel Computing Memory ConfigurationsMemory Configurations

Shared MemoryShared Memory Processors share a single address spaceProcessors share a single address space Processors communicate via memoryProcessors communicate via memory

MulticomputerMulticomputer Name is derived from the term Name is derived from the term message passing message passing

multiprocessorsmultiprocessors Processors don’t share a single address spaceProcessors don’t share a single address space Processors communicate by passing messages via an Processors communicate by passing messages via an

interconnection networkinterconnection network Difficult to program Difficult to program

can’t share memorycan’t share memory message passing is error pronemessage passing is error prone

Typically scales better than a shared memory systemTypically scales better than a shared memory system

Requirements for Requirements for ParallelismParallelism

Data must be parallelizableData must be parallelizable Divide and conquerDivide and conquer

Must have a parallel hardware Must have a parallel hardware architecturearchitecture

DSP Data ParallelismDSP Data Parallelism Digital signal is an array of valuesDigital signal is an array of values

Arrays can be accessed in parallelArrays can be accessed in parallel Used in smoothing and simple low-pass filtersUsed in smoothing and simple low-pass filters Otherwise limited value in DSPOtherwise limited value in DSP

DecompositionDecomposition Divide a signal into many smaller, simpler signalsDivide a signal into many smaller, simpler signals Smaller, simpler signals can processed in parallelSmaller, simpler signals can processed in parallel Used in many crucial DSP applications:Used in many crucial DSP applications:

Encoding/decoding information on a carrier signalEncoding/decoding information on a carrier signal Frequency analysis Frequency analysis Advanced filteringAdvanced filtering Multi-band equalizationMulti-band equalization Etc.Etc.

[6: page 98]

Superpostion and Superpostion and DecompositionDecomposition

A signal can be A signal can be decomposed into several decomposed into several smaller, simpler signalssmaller, simpler signals

Decomposed signals can Decomposed signals can added together to form added together to form the original signalthe original signal

Processing decomposed Processing decomposed signals has the same signals has the same effect as processing the effect as processing the original signal original signal if the if the system is linearsystem is linear

Linear SystemsLinear Systems HomogeneityHomogeneity

Modification in the input signal’s amplitude results Modification in the input signal’s amplitude results in a corresponding modification in the output.in a corresponding modification in the output.

f(x)t = tyf(x)t = ty AdditivityAdditivity

For any two inputs, the sum of the outputs is the For any two inputs, the sum of the outputs is the same as if the sum of the two inputs had been same as if the sum of the two inputs had been processed together.processed together.

f(xf(x11) = ky) = ky11

f(xf(x22) = ky) = ky22

f(xf(x11) + f(x) + f(x22) = ky) = ky11 + ky + ky22

f(xf(x11 + x + x22) = ky) = ky11 + ky + ky22

DSPs and LinearityDSPs and Linearity Most DSP operations are linearMost DSP operations are linear Nonlinear DSP operations are Nonlinear DSP operations are

parallelizable as wellparallelizable as well Subdivide the data arraySubdivide the data array Emulate the nonlinear operation with a Emulate the nonlinear operation with a

linear operationlinear operation

[6: page 105]

Fourier DecompositionFourier Decomposition Decomposes signal into Decomposes signal into

sine & cosine signalssine & cosine signals Sinusoidal waves are Sinusoidal waves are

easy to processeasy to process UniformityUniformity Simple representation Simple representation

(amplitude & phase)(amplitude & phase) Sinusoidal fidelitySinusoidal fidelity

If a sine wave is input to If a sine wave is input to a linear system, the a linear system, the output will be a sine output will be a sine wave at the same wave at the same frequencyfrequency

Fourier EquationsFourier Equations Continuous Fourier Transformation:

X(f)= ∫ x(t)e-i2ft dt Discrete Fourier Transformation (DFT):

X(m) = ∑x(n)e-i2nm / N

Fast Fourier Transformation (FFT): Optimized DFT algorithm

Euler’s relationship:e-i = cos() – isin()

[6: page 511]

DSPs and Hardware DSPs and Hardware ParallelismParallelism

Minor hardware parallelism is achieved with Minor hardware parallelism is achieved with Harvard architectureHarvard architecture Adds a separate memory store and bus for instruction Adds a separate memory store and bus for instruction

datadata Allows for concurrent retrieval of instructions and dataAllows for concurrent retrieval of instructions and data

DSPs and Hardware DSPs and Hardware ParallelismParallelism

Several of the same type of DSP can be linked Several of the same type of DSP can be linked together into a multiprocessing arraytogether into a multiprocessing array SIMD & MIMDSIMD & MIMD Shared & multicomputer memory architecturesShared & multicomputer memory architectures

Linking processors requires “glue” logicLinking processors requires “glue” logic ““Glue” logic controls data flow and task distributionGlue” logic controls data flow and task distribution Some DSPs are designed with “glue” logic built inSome DSPs are designed with “glue” logic built in

Analog Devices SHARC familyAnalog Devices SHARC family Texas Instruments TMS320 familyTexas Instruments TMS320 family

SummarySummary CAs are parallel computational CAs are parallel computational

devicesdevices SIMD, Shared Memory deviceSIMD, Shared Memory device

DSPs can be parallel computational DSPs can be parallel computational devicesdevices SIMD, Shared Memory deviceSIMD, Shared Memory device MIMD, MulticomputerMIMD, Multicomputer

Parallelism is the tie that bindsParallelism is the tie that binds

The Future?The Future? Sony, IBM, and Toshiba’s “Cell Sony, IBM, and Toshiba’s “Cell

Microprocessor” technology Microprocessor” technology A single processor has 4 - 16 general-purpose A single processor has 4 - 16 general-purpose

processor cores, known as “cells”processor cores, known as “cells” ““Cell” processors are linked together via high Cell” processors are linked together via high

speed communication busspeed communication bus Devices with “Cell” chips can be connected over a Devices with “Cell” chips can be connected over a

high-speed network to perform distributed high-speed network to perform distributed computing computing

May be first used in the Sony Playstation 3 video May be first used in the Sony Playstation 3 video game consolegame console

Process streaming video data over a networkProcess streaming video data over a network Earliest expected rollout date is 2005Earliest expected rollout date is 2005

ReferencesReferences[1] Lyons, R. (2001). Understanding Digital Signal Processing.

Upper Saddle River, NJ: Prentice Hall.[2] Margolus, N. (1993). “CAM-8: A Computer Architecture Based on Cellular Automata”.

Cambridge, MA: MIT Laboratory of Computer Science.Available: http://www.ai.mit.edu/people/nhm/cam8.pdf

[3] Miranda, E. (2002). “CAMUS: A Cellular Automata Music Generator”.Available: http://website.lineone.net/~edandalex/camus.htm

[4] Rorabaugh, C. (1998). DSP Primer.New York: McGraw-Hill.

[5] Schramm, K. (2003). “CASound: Sound Synthesizing 1D CA”.Available: http://sjsu.rudyrucker.com/~karl.schramm/applet/

[6] Smith, S. (1997). The Scientist and Engineer's Guide to Digital Signal Processing.San Diego, CA: California Technical Publishing.

[7] Spooner, J., (August 6, 2002). “Chip trio allows glimpse into ‘Cell’”.CNET News.comAvailable: http://news.com.com/2100-1001-948493.html?tag=fd_top

[8] Wilkinson, B. & Allen, M. (1998). Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers.Upper Saddle River, NJ: Prentice Hall.

[9] Wolfram, S. (2001). A New Kind of Science.Champaign, IL: Wolfram Media, Inc.