1

Finger detection evaluation system

Su-ting, Chuang

2

OutlineIntroductionRelated workHardware configurationFinger Detection systemOptimal parameter estimation frameworkConclusion

3

OutlineIntroductionRelated workHardware configurationFinger Detection systemOptimal parameter estimation frameworkConclusion

4

IntroductionMotivation

Evaluate components in finger detection systems

Verify and improve performance of finger detection systems

MethodDevelop an optimal parameter estimation

frameworkUse most prevalent finger detection system as

testbed Touchlib

5

OutlineIntroductionRelated workHardware configurationFinger Detection systemOptimal parameter estimation frameworkConclusion

6

Related WorkFTIR (Frustrated Total Internal Reflection)

J. Y. Han, “Low-cost multi-touch sensing through frustrated total internal reflection," in Proceedings of the 18th annual ACM symposium on User interface software and technology (UIST '05). New York, NY, USA: ACM Press, 2005, pp. 115-118.

7

Related WorkDI (Diffused Illumination)

J. Rekimoto and N. Matsushita, “Perceptual surfaces: Towards a human and object sensitive interactive display," Workshop on Perceptural User Interfaces (PUI'97), 1997.

8

Related WorkTouchLib

A multi-touch development kit

Finger detection processing flow chartBackgroundSubtraction

SimpleHighpass ScaleMono Threshold

9

OutlineIntroductionRelated workHardware configurationFinger Detection systemOptimal parameter estimation frameworkConclusion

10

Hardware configurationTable setup

11

Hardware configurationOrder of diffuser layer and touch-glass layer

Diffuser layer

IR illuminatorIR camera

spot

IR illuminatorIR camera

Touch-glass layer

IR camera

spot

IR camera

12

Hardware configurationProblem:

IR rays will be reflected by the touch-glass and resulting hot spot regions in camera views

Solution:Use other cameras to recover the regions

which are sheltered by IR spots

13

OutlineIntroductionRelated workHardware configurationFinger Detection systemOptimal parameter estimation frameworkConclusion

14

Detection system

IR cam

Pre-processing

Imageprocessin

g

FingerAnalyzing

DataAssociatio

n

DataTransmissio

n

IR cam GPU

CPU

15

Detection systemPre-processing

ImageFusion(Blend)

IR Camera

IR camera

Undistortion

Undistortion

HomoWarp

HomoWarp

16

Pre-processingUndistortion

Undistort foreground objectsWarp

Unify finger size among different position of table

Image fusionMask hot spots and recover them from the

other camera imageFinger at border won’t be discard

17

Pre-processingAdvantage of implementing on GPU

Increase performance High frame rate

Preserve CPU for application computation Enable detection system and interactive application

on the same computer Reduce unsynchronized problem among different

computers

18

Detection systemImage processing

BackgroundSubtraction Normalization Simple

HighpassMono Threshold

19

Image processingNormalization

Motivation Eliminate influence due to non-uniform lighting

condition Various finger touch response

Hard to decide a good thresholdMethod

Model distribution of IR illumination Use specific material to simulate foreground Calculate each pixel’s dynamic range

Stretch dynamic range to 0-255

20

Finger AnalyzingConnected componentfinger size evaluation

21

Data associationFingertip matching

Matching fingertips among framesUsing bipartite algorithm

Fingertip trackingSmooth detected results and fix lost resultsUsing Kalman filter

22

OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion

23

Optimal parameters estimation frameworkMotivation

Find optimal parameters for finger detection system

24

Optimal parameters estimation frameworkProcedure

Define parameters for finger detection systemCollect samples

Various finger size Various hand gesture

Search optimal parameters Verify performance of all possible parameter

combinations

25

Optimal parameters estimation frameworkCollect samples

Task Soft /Hard touch Vertical/Oblique touch Various fingers

Sample set Each task has 2x2x5 samples

Sample collection Step-by-step instruction Straightforward UI design

Finger touch position

5timer

Instructions….

26

Optimal parameters estimation frameworkSearch optimal parameters

Exhaustive search Test various parameter combination in each set

Step Each parameter combination

Detect finger touch Verify detection result

Calculate error rate

27

Optimal parameters estimation frameworkDetection systemframe

Optimal parameter finder

Parameter Set

Detection Result

GroundTruth

OptimalParameter Set

VerifyNext Parameter Set

Generator

DetectionResult

GroundTruth

Error Rate

Parameter Set

Sampleset

28

OutlineIntroductionRelated workHardware configurationDetection systemOptimal parameter estimation frameworkConclusion

29

Thanks for your attention