blind assistive technology bill reading device (batbrd) professor aura ganz ian mcalister

1Department of Electrical and Computer Engineering

Blind Assistive Technology Bill Reading Device (BATBRD) Professor Aura Ganz

Ian McAlisterColin Smith

Chris NeylandErick Drummond

TEAM GANZ Final Project Review


OUTLINE Motivation Product Requirements Product Design Manufacturing Image Processing Experimental Results Conclusions SDP Demonstration Day Tasks Future Enhancements


Motivation Population• 1.8 million legally blind individuals in the U.S. • 21.2 million reported experiencing vision loss

US Currency Problem• No current identification methods• Current technology is very expensive

Goals Develop low cost, easy-to-use, portable

electronic bill reader for the blind community


Product Requirements Product Requirements

Low Cost Intuitive Interface Design – 2 to 3

buttons max Small Form Factor - <50in3

Performance - <10 sec, >90% Accuracy Upgradable Bill Library Battery Life – One Hour Continuous Use


Final Product Design – HardwareSystem Block Diagram: Production Model:

Beagleboard

Audio Device

Camera

SD Card

Audio Library

Image

Capture

Image Library

Batteryw/ 5V Regulator

LEDs

Image Recognition Algorithm

Push Buttons AVR Microcontroller

Colin Smith

Update this slide to just the picture on the right and then include list of stuff plus associated cost


Cost list here with total Costs

Beagleboard: $160 Enclosure: $14.60 Microcontroller and Misc Hardware: $15 PS3 Eye: $35 Battery/Charger $25 SD Memory Card: $12 Total: 246.60


Manufacturing of PCB Custom PCB needed for User Interface Designed Schematic Using Eagle

Photo Paper used for layout relative to Beagleboard Top/Bottom layers printed separately on glossy paper

Bottom layer taped to copper board and holes drilled Top layer mirrored, placed on reverse side, and lined up

with holes Hot iron used to transfer toner to copper board

Paper rinsed in water bath PCB place in Ferric Chloride bath to dissolve unwanted copper Toner washed off with acetone and through holes drilled PCB fit and tested


Manufacturing of Project Enclosure Polycarbonate Case – Best for BATBRD Project Dimensions in inches

4.53L X 3.54W X 2.17H = 34.8 in3

Meets size requirements Painted for lighting/cosmetic purposes Holes drilled for switches and power connections Rails installed for user interface


CAMERAIMAGEDATA

CAPTURE IMAGE COMMAND

BUTTON DEPRESSED?

CHECK IMAGE AGAINST $1 TEMPLATES

HIGEST CORRELATION >

X?

TEMPLATE IMAGEDATA(ONE)

RETRIEVE AND OUTPUT ERROR

MESSAGE

START(AWAITING INPUT)

YES

NO

TIMEOUT

CHECK IMAGE AGAINST $10 TEMPLATES

TEMPLATE IMAGEDATA(TEN)

RETRIEVE AND OUTPUT MESSAGE CORRESPONDING

TO TEN

YES


Image Processing Find Best Match by Using Normalized Cross-Correlation

Coefficients Function (NCC)

C = Correlation Matrix I = Captured Image, = Sample Image Mean at location: u,v T = Template Image, = Template Mean u,v = Indices at Point of Correlation

Best Match Retrieved by OpenCV Function: cvMinMaxLoc( image, &minval, &maxval, &minloc, &maxloc, 0 );

Where maxval is a number from -1 to +1 corresponding to best match. This Value Can then Be Used to Determine if There is a Match

yx yxvu

vuyx

TyxTIyvxuI

TyxTIyvxuIvuC

, ,22

,

,,

]),([]),([

]),(][),([),(


Experimental Design Experiment 1

Timing/Accuracy with Ideal Images Experiment 2

Thresholds with Ideal Images and Second ID Experiment 3

Thresholds/Accuracy with degraded images Experiment 4

Thresholds/Accuracy with further degraded images Experiment 5

Project Enclosure Threshold/Accuracy worst case degraded image.


Experiment 1 – Timing/Accuracy with Ideal Images Pixel Size

360 x 240 Templates under complete darkness (no backlighting) Results

Significant time delay Reduced Accuracy

Thresholds too low New generation bill Rail Structure

Bill Type Time (s)Maximum

Threshold ValueOverall

Accuracy

Reg $1 1.902 0.936 100.0001gen $5 12.793 0.965 100.0002gen $5 6.610 0.911 87.5003gen $5 4.963 0.908 75.000

1gen $10 9.601 0.905 100.0002gen $10 3.890 0.879 83.3333gen $10 3.794 0.888 75.0001gen $20 10.253 0.950 100.0002gen $20 4.534 0.933 100.0003gen $20 2.860 0.957 100.000

Overall 6.120 0.923 92.083

Data includes averages of 12 tests conducted for each bill*


Exp. 2 – Thresholds with Ideal Images and 2nd ID Changes from Experiment 1

Pixels reduced to 160x120 Templates retaken with tinted backlighting Camera moved Rails more securely fixed Max Threshold .85

Results Worst case time < 4 seconds All corners identifying well Backlighting no longer a problem

Averages of 12 tests conducted for each bill

Bill TypeMax Threshold

Value2nd Highest Threshold

Reg $1 0.917 0.6351gen $5 0.898 0.5622gen $5 0.947 0.7893gen $5 0.916 0.829

1gen $10 0.945 0.6762gen $10 0.942 0.8583gen $10 0.936 0.8611gen $20 0.924 0.5902gen $20 0.931 0.7573gen $20 0.939 0.842


Exp 3 Threshold/Accuracy non-ideal images

Bill TypeMaximum Threshold

Value

2nd Highest Threshold different

denominationOverall Accuracy

Reg $1 0.814 0.532 100.0001gen $5 0.817 0.548 100.0002gen $5 0.858 0.584 100.0003gen $5 0.809 0.574 100.0001gen $10 0.855 0.540 100.0002gen $10 0.881 0.534 100.0003gen $10 0.842 0.515 100.0001gen $20 0.841 0.507 100.0002gen $20 0.830 0.495 100.0003gen $20 0.845 0.531 100.000

Test 3 Accuracy with non-ideal images (1 diagonal Stripe)Data includes averages of 8 tests conducted for each bil l

1 Diagonal Stripe added to image capture area to simulated degraded/damaged bills


Exp. 4 – Continued Bill Degradation Tests Simulated Degradation 2 Stripes

3 Stripes

4 Stripes

Bill Type Max Threshold 2 Stripes

Max Threshold 3 Stripes

Max Threshold 4 Stripes

Reg $1 0.762 0.746 0.6501gen $5 0.747 0.782 0.6972gen $5 0.805 0.803 0.7253gen $5 0.795 0.774 0.708

1gen $10 0.754 0.709 0.6412gen $10 0.845 0.834 0.7443gen $10 0.790 0.804 0.7151gen $20 0.763 0.744 0.6782gen $20 0.800 0.782 0.7213gen $20 0.808 0.809 0.735

Data includes averages of 8 tests conducted for each bil l


Exp. 5 – Project Enclosure, Non Ideal Images Maximum Threshold Identification over 11% wrong identification Using Greater than .65 for threshold Zero Wrong IDs

High rescan rate no incorrect identification

% ID % Rescan Request

% Wrong ID

Max Threshold Only 88.75 0 11.25Greater than .6 53.75 43.75 2.5

Greater than .625 40 58.75 1.25Greater than .65 32.5 67.5 0

Statistics all Bills under "worst case" degredation"


Power Usage Battery Power Supply:

The Battery Provides 7.4V and 2.2Ahr 7.4V * 2.2A hr = 16.28W hr

BATBRD Power Usage Beagleboard and supporting hardware

operates at 5V and draws 600 – 800mA The BATBRD draws a maximum of 4W. We can safely operate the board for at

least ~4hrs


Product Requirements - Conclusion Product Requirements





TBD for SDP Demonstration Day

Work out minor issues with user interface, power-up sequence Meet with Jenny again: let her use it, get her input Audio adjustments: improve playback, eliminate ‘pops’ Possibly remake templates, modify lighting Continue testing for accuracy as changes are made


Future Outlook Jenny’s requests

Audio notification of battery life/charging status Prerecorded audio user’s manual Physically unique charging plug shape

Beyond SDP 2010 Tailored processing hardware (FPGA, ASIC) Application-specific DSP techniques in IP algorithm Smaller, low-profile imaging device & battery


Questions ?

blind assistive technology bill reading device (batbrd) professor aura ganz ian mcalister

Documents

department of electrical

blind community

blind individuals

expensivegoalsdevelop

buttons maxsmall form