senses, sensing, and sensors

with applications toICT for

sustainable developmentmel siegel

robotics instituteschool of computer science

starting points• sensing is often the problematic element in

the sense-think-act-communicate paradigm– it is all well-and-good to say, e.g., “what we need

to halt the spread of AIDS is a 50-cent battery operated pen-sized sensor for HIV infection ...”, but where is that sensor?

• the unavoidable tradeoff between sensitivity and specificity threatens system robustness

• sensor fabrication technology lags – so can ride the coat tails of – VLSI technology

what senses do we want?at least the human ones ...

• vision: eyes (optics, light)• hearing: ears (acoustics, sound)• touch: skin (mechanics, temperature)• odor: nose (vapor-phase chemistry)• taste: tongue (liquid-phase chemistry)• sixth sense (proprioception): joint

angles

human senses are dual-use• balance

– ears (in addition to hearing)• acceleration

– stomach (in addition to digestive senses)• sound

– chest cavity (in addition to breathing senses)

• touch– touch: tongue (in addition to taste)– temperature: skin (in addition to force)

a blessing and a curse• artificial sensors also respond to

multiple stimuli– almost all respond to temperature– many respond to acceleration and pressure– many respond to light and other radiation– sensors are “physics experiments that failed”

• these can often be “compensated”using differential techniques

• however noise is always present– environmental stimuli of same sort as signal– thermal and other noise internally generated

we can’t escape it ...

• it’s the Joe BFSTPLK effect …

but with clever techniques ...

• ... we can extract very small signalsfrom overwhelming quantities of noise

military communications hidesmall signals in overwhelmingenvironmental noise, so your adversarydoesn’t even know you are talking – encryption is a backup in case hesomehow manages to actually hear you

we would also like to haveextended ranges and

modalities• vision outside the RGB spectrum• active vision

– radar and laser range measurements

• hearing outside 20 Hz – 20 kHz range– ultrasonic range measurement

• chemical senses beyond taste and smell

• radiation: , , -rays, neutrons, etc

and sensors for modalities that humans might have ...

• electric fields?• magnetic fields?• radio (electromagnetic) waves?• pheromones? (probably yes)• “weather”?• what else??

we need artificial sensing for …

• everything the human senses can do …• without handicaps (specs, hearing aids,

…)• able to use human sensory aids

– telescopes, microscopes, etc– stethoscopes, sound amplifiers, etc– micromanipulators w/ haptic feedback, etc

• plus all the senses that we don’t have(or we don’t have with good cognition)but which our applications demand

here’s the textbook I recommend

•Handbook of Modern SensorsPhysics, Designs, and ApplicationsJacob Fraden3rd ed., 2004, XVII, 589 p. 403 illus., HardcoverISBN: 0-387-00750-4$89.95

• Ch1 Data Acquisition (3) digital interface• Ch2 Sensor Characteristics (1) raw signals• Ch3 Physical Principles of Sensing (0) materials science• Ch4 Optical Components of Sensors (0) optical science• Ch5 Interface Electronic Circuits (2) analog interface• Ch6 Occupancy and Motion Detectors is someone there?• Ch7 Position, Displacement, and Level where is something?• Ch8 Velocity and Acceleration how fast?• Ch9 Force, Strain and Tactile Sensors how heavy?• Ch10 Pressure Sensors how heavy (fluid)?• Ch11 Flow Sensors how fast (fluid)?• Ch12 Acoustic Sensors how loud?• Ch13 Humidity and Moisture Sensors how wet?• Ch14 Light Detectors how bright?• Ch15 Radiation Detectors how radioactive?• Ch16 Temperature Sensors how hot or cold?• Ch17 Chemical Sensors what is it made of?• Ch18 Sensor Technologies (0) sensor fabrication

old: transduce to human vision

• thermometers: temperature-to-length• barometers: air pressure-to-length• scales: weight-to-angle• humidity: hair curl-to-angle• indicator dyes: chemistry-to-color• photo film: light/radiation-to-silver

density• speedometers: velocity-to-angle

new: transduce to electronics

• thermistor: temperature-to-resistance• electrochemical: chemistry-to-voltage• photocurrent: light intensity-to-current• pyroelectric: thermal radiation-to-voltage• humidity: absorbed water-to-capacitance• length (LVDT): displacement-to-inductance• microphone: sound pressure-to-<anything>

sense-think-act loop

measurand

transduce perception to electrical signal

measure involts, amps, ohms,henrys, farads, etc.

convert fromsignal to symbol

compute control actiontransduce signal toheat, displacement,

illumination, etcconvert from

symbol to signal

SENSOR

ACTUATOR

ADC

DAC

sense-think-act loop

transduce perception to electrical signal

measure involts, amps, ohms,henrys, farads, etc.

convert fromsignal to symbol

compute control actiontransduce signal toheat, displacement,

illumination, etcconvert from

symbol to signal

SENSOR

ACTUATOR

ADC

DAC

measurand

e n v i r o n m e n t

measurand & measurement system immersed in

environment• nature confounds measurement with:

– temperature– quantization– chaos

• environment confounds measurement with:– complex universe: 3 K radiation, weather, ...– interfering signals: power lines, radio/TV, ...– engineering and materials limitations, ...

distribution of replicated measurements

0

1

2

3

4

5

6

7

8

9

998.0 998.5 999.0 999.5 1000.0 1000.5 1001.0 1001.5 1002.0

Measured Height [mm]

Nu

mb

er

of

Me

as

ure

me

nts

sensor fusion: people & robotics• Barfogenesis : Does Virtual Reality Make You Sick?

– ... I was getting nauseous from the film. My eyes wanted me to "stop flying," while my body said, "you're not moving at all" ...

– http://serendip.brynmawr.edu/bb/neuro/neuro00/web3/Pili.html

• ... the only things consistently real about Virtual Reality were headaches and motion sickness ... (Steve Ellis)– http://www.firstscience.com/site/articles/virtual_reality.asp

• some corresponding problems in “artificial” sensing:– multi-modality image registration (e.g., light + ultrasonic)

– multi-media synchronization (e.g., sound + video)

– one modality, several measurements, how to combine them?

– several measurements contribute to a calculated result; given component confidences, how to calculate overall confidence?

combiningmultiple measurements

• three thermometers give measurements T1 = 19.1 ± 2.0 CT2 = 18.6 ± 2.5 C T3 = 19.3 ± 1.0 C

• what is your best estimate of the actual temperature, and what is your estimate of theerror in your estimate of the actual temperature?

• weight by reciprocals of respective uncertainties(actually weight by reciprocals squared)

• this is what we mean by “sensor fusion”

combiningmultiple errors

• you compute the volume of a box by multiplying together measurements of its height, width, and depth: V = (h ± Δh) (w ± Δw) (d ± Δd)

• what is your estimate of the error ΔV?• it is the (quadrature) sum of

{∂V/∂h,∂V/∂w,∂V/∂d}weighted by {Δh, Δw, Δd}

• this is what we mean by “error propagation”

electronic sensors• many sensors are just resistors, capacitors,

or inductors (sometimes containing “unusual” materials) whose parameters depend on some feature of the environment:– thermistors: R (temperature T)– humidity sensors: C (absorbed water vapor)– proximity sensors: L (distance to a surface)– magneto-resistive sensors: R (magnetic field B)– photo-conductors: R (incoming light intensity)

• other sensors are fundamentally voltage sources:– electrochemical sensors: V

(chemistry)– photovoltaic sensors: V (light

intensity)– magnetic pickup loops: V (B(t))

• still other sensors are fundamentally current or charge sources:– Faraday cup (e.g., solar wind collector)– photocell (e.g., “electric eye”)– CCD camera sensor– many kinds of radiation detectors

• for example, smoke detectors in which vapors accompanying the smoke affect the charge collected in a radioactive environment

• and other sensors fundamentally extract power from radiated fields:– antennas: directed radio frequency

energy– microphones: directed acoustic

energy

• many kinds of sensors are essentially perturbed communication devices:– a signal transmitter– a signal receiver – the nature of the path between them

disturbs (or sometimes enhances) the communication between them

– from which disturbance an interesting property of the medium is deduced

• for example, smoke detectors in which the smoke just attenuates a light beam

instruments vs. sensors• many devices thought of as “sensors” at

an applications level are very complex instrument systems at an engineering level

analog-to-digital conversionscan generation background subtractioncurve fittinglibrary searchingpattern recognition

sensing needs in contextof the WEHAB agenda

• Water and Sanitation• Energy• Health and Environment• Agriculture• Biodiversity and Ecosystem

Management

water & sanitation• good clean water essential to all WEHAB:

energy, health, agriculture, biodiversity• humans not good sensors of water quality:

awful seeming can be safe, & vice versa• complex instruments, e.g., GC-MS, too big,

expensive, hard to use and maintain• semiconductor, MEMS, organic polymer

sensors all promising– but require IT support for linearization, library

searching, pattern recognition, alarms, etc

• see:

http://www.geocities.com/RainForest/5161/lab2.htm– separation techniques– measurement techniques

• gravimetric• electrochemical• colorimetry / spectrophotometry• titration• chromatrography• mass spectrometry

alternative 3rd world appropriate water quality

sensing • arrays of discrete sensors • integrated sensor arrays / chemical imagers • hand-held smart instruments

(Cyrano Inc.)

energy (& off-grid capability)

• reliable IT requires reliable hardware even in the absence of a reliable power grid– sensors needed for controls that provide

local backup and stability to compensatefor global unreliability and instability

• IT can ensure reliability of the power grid and power generation infrastructures– sensors needed to detect faults (and theft!),

compensate for environmental perturbations, monitor and correct power quality, etc

• see: http://www.electrotek.com/seminars/hirel.htm– living with an unreliable power grid

• better batteries: modern batteries include complex circuitryto sense and regulate changing and discharging

• local generation means: a lot of low-grade energy is available,but it takes smart adaptable systems to utilize it

• uninterruptible power supplies: batteries + inverters + meansto intelligently shut down when batteries run down and intelligently re-start when power becomes available again

– making the power grid reliable• rapidly sensing fault, diagnosing problem, taking corrective

action• remote sensing to detect illegitimate usage (e.g., theft)• measuring, assessing, and improving power quality• integrating local and national generating capacity by allowing

users to become suppliers when they have excess local capacity

power availability and quality

http://portland.indymedia.org/en/2004/08/294881.shtml

health & environment• instruments for rapid, reliable, sensitive

detection of illness – or impending illness– diabetes: disease of modern diet and lifestyle– identification of infectious agents (viruses etc)

• protecting employees, residents, and their environments from industrial pollution

• detecting bad side effects of good projects, e.g., arsenic in ground water

• detecting side effects of war, e.g., mines

• see: http://www.sciencedaily.com/releases/2003/09/030924054754.htm

Livestock Health Sensors And Wireless Data Storage In The Works

• see: http://www.cis.rit.edu/~rlkpci/urssra_Kremens.pdf

Low Cost Autonomous Field-Deployable Environment Sensors

health & environmental sensors

http://www.yenra.com/glucose-monitor/glucose monitor +insulin pump

http://www.weathertools.com/oregon4.htmlweather station w/ wireless sensor moduletransmitters

agriculture• many sensing requirements intimately

tied to health (e.g., animal health), weather sensing and prediction, water availability

• many chemical sensing requirements: soil and water pH and nutrient content, oxygen availability and demand

• many physical sensing requirements: water availability, retention, evaporation

• for advanced technology, need the full robotic arsenal for navigation and work

• see http://www.agrotechnology.kvl.dk/teaching/phddanetpft/pdf/04_sensingsystems.pdf

Sensor Technology in Agriculture• see

http://www.kuleuven.ac.be/onderwijs/aanbod/syllabi/I0F33AE.htm

syllabus of a course (in Dutch) on physical sensors, chemical sensors and bio-sensors for agricultural and food applications

sensors for agriculture

http://www.sentek.com.au/products/products.asp?lang=enportable soil monitoring system

http://www.ntechindustries.com/mapping.htmlgreenseeker optical plant health monitor

http://www.trimble.com/AgGps_autopilot.htmlautomatically steerstractor perfectly straight, center pivot, curves or headlands

biodiversity & ecosystems

• large- and small-scale characterization and monitoring of animal and plant species

• evaluation and exploitation of medicinals, both unknown and used traditionally

• control of poaching, encroachment, etc,while preserving indigenous cultures

• tracking animal movements• tagging legitimately collected specimens• enabling transport of more fragile species

• seehttp://www.digitalgovernment.org/search/projects/project.jsp?ID=108Biodiversity and Ecosystem Informatics - Wireless Sensor Networks for Dense Spatio-Temporal Environmental Monitoring

• seehttp://investigate.conservation.org/xp/IB/expeditions/pantanal/day8/day8_tools.xml

neat night pictures of animals in jungle

sensors for biodiversity & ecosystems

http://www.esa.int/export/esaEO/SEMPMB0XDYD_environment_1.htmlpressure map above hurricane Frances

http://www.landcareresearch.co.nz/services/ecosat/presentations/wetland.pps#7wetland mapping project in New Zealand

parting points• sensing is often the problematic element in

the sense-think-act-communicate paradigm– it is all well-and-good to say, e.g., “what we need

to halt the spread of AIDS is a 50-cent battery operated pen-sized sensor for HIV infection ...”, but where is that sensor?

• the unavoidable tradeoff between sensitivity and specificity threatens system robustness

• sensor fabrication technology lags – so can ride the coat tails of – VLSI technology