christian holz patrick baudisch
DESCRIPTION
christian holz patrick baudisch. high-precision touch input based on fingerprint recognition. fachgebiet human-computer interaction. occlusion. fat finger. so touch is inaccurate or is it?. could it be that it is not the fingers but our touch devices that are wrong?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/1.jpg)
christian holzpatrick baudisch
high-precision touch inputbased on fingerprint recognition
![Page 2: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/2.jpg)
fachgebiet human-computer interaction
![Page 3: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/3.jpg)
occlusion
![Page 4: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/4.jpg)
fat finger
![Page 5: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/5.jpg)
so touch is inaccurate
or is it?
![Page 6: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/6.jpg)
could it bethat it is not the fingers but our touch devices that are wrong?
![Page 7: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/7.jpg)
Part 1 (science):even though screens are 2D, pointing is not
Part 2 (engineering):sensing fingers in 3D highly accurate touch
![Page 8: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/8.jpg)
no fat fingerwe claim there is
problem
![Page 9: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/9.jpg)
perceivedinstead, almost all observed targeting error comes from
probleminput point
![Page 10: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/10.jpg)
perceived input point problem
target
[Benko, Wilson, & Baudisch 2006]
touch device perceives
![Page 11: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/11.jpg)
offset
why we hope it’s the perceived input point problem?
the fat finger problem, in contrast is always noise = error
we can correct for it
![Page 12: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/12.jpg)
offset
why we hope it’s the perceived input point problem?
the fat finger problem, in contrast is always noise = error
we can correct for it
![Page 13: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/13.jpg)
while there is always an offset, we hypothesize thatthe offset depends on the pointing situation
our main hypothesis
![Page 14: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/14.jpg)
so what does “pointing situation” mean?
![Page 15: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/15.jpg)
!= [iPhone, Wang et al.]
1yaw
![Page 16: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/16.jpg)
!=[Forlines et al., CHI’07]
2pitch
![Page 17: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/17.jpg)
!=3roll
![Page 18: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/18.jpg)
!=
4 finger shape
![Page 19: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/19.jpg)
!=
4mental model
![Page 20: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/20.jpg)
(… and there might be more e.g., head position/parallax…)
![Page 21: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/21.jpg)
we ran auser studya non 2D-model
![Page 22: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/22.jpg)
xy xy
current model
touch padscreen
![Page 23: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/23.jpg)
nD xy
proposed model
touch padscreen
![Page 24: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/24.jpg)
we ran auser studyuser study 1
![Page 25: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/25.jpg)
task
![Page 26: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/26.jpg)
1. target here
2. hit okay
task
![Page 27: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/27.jpg)
1pad rotation (yaw)
![Page 28: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/28.jpg)
90° 45° 15° 0° -15°
roll2roll
![Page 29: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/29.jpg)
3pitch
90° 65° 45° 25° 15°
![Page 30: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/30.jpg)
4user12 participants
(all students, so differencesamong them will be lower bound)
![Page 31: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/31.jpg)
footswitch
on-screeninstructions
controlledhead position parallax
capacitivetouch pad
![Page 32: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/32.jpg)
every trial recorded as a dot at the touch location
dependent
![Page 33: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/33.jpg)
we measure targeting accuracy assuming perfect calibration size of ellipse that contains 95% of all samples.
example
7.5 m
m1.5 cm
![Page 34: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/34.jpg)
main effects forroll, pitch, yaw, & participantID
hypotheses
![Page 35: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/35.jpg)
2 pad rotations× 2 sessions (pitch, roll)× 5 angles× 6 repetitions per angle× 5 blocks
= 600 trials / participant
12 participants design
![Page 36: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/36.jpg)
1 2 3 4 5 6
results
![Page 37: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/37.jpg)
if the additional IVs had no impact,we would expect to see something like this
![Page 38: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/38.jpg)
-15°0°
15°45°90°
rotate condition
no-rotate condition
but touch locations do indeed fall into clusters…
![Page 39: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/39.jpg)
know
not
...
know
use
r
know
yaw
know
use
r...
know
3DO
F
know
use
r...0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
error bars are standard deviation
butto
n siz
e in
cm
for 9
5% a
ccur
acy results
requires 5.2mm button
~three times more accurate allow three times smaller device
trad
ition
alca
paci
tive
requires 15mm button
![Page 40: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/40.jpg)
(participant #4, roll varied)
target
1cm
1pad rotation (yaw)
![Page 41: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/41.jpg)
1pad rotation (yaw)
![Page 42: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/42.jpg)
-15°0°
15°45°90°
rotate condition
no-rotate condition
(participant #4, roll only)2roll
![Page 43: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/43.jpg)
1cm
3pitch 1025456590
![Page 44: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/44.jpg)
all data by participant #1-6
1 2 3 4 5 6
tilt
roll4users
![Page 45: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/45.jpg)
7 8 9 10 11 12
tilt
roll
all data by participant #7-124users
![Page 46: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/46.jpg)
know
not
...
know
use
r
know
yaw
know
use
r...
know
3DO
F
know
use
r...0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
error bars are standard deviation
spre
ad in
cm
resultsrequires 5.2mm button
trad
ition
alca
paci
tive
requires 15mm button
![Page 47: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/47.jpg)
how (in)accurate current devices are (button must be that big)
if we knew thepad orientation
if we knewfinger angles
![Page 48: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/48.jpg)
also need to know user ID, or we will overcompensate for people like this one
shouldn’t we be able to make such a device?
![Page 49: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/49.jpg)
Part 1 (science):even though screens are 2D, pointing is not
Part 2 (engineering):sensing fingers in 3D highly accurate touch
![Page 50: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/50.jpg)
optical tracker
what do you mean: “not very practical”? retro reflective markers on finger… 6-16 camera setup…
makes a great “gold standard” implementation to test the concept
![Page 51: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/51.jpg)
mobileok, maybe something a bit more
![Page 52: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/52.jpg)
![Page 53: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/53.jpg)
![Page 54: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/54.jpg)
![Page 55: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/55.jpg)
![Page 56: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/56.jpg)
gets everything a traditional touchpad gets+ roll, pitch, yaw, & participantID
![Page 57: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/57.jpg)
devices that sense touch and fingerprint already exist
![Page 58: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/58.jpg)
![Page 59: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/59.jpg)
this is very different from micro rolls [CHI 2009]
![Page 60: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/60.jpg)
calibrationhave user touch a known target repeatedly and with different finger postures create database of (fingerprint, target offset)
useobtain fingerprint as user touches the devicelook up similar fingerprints in the databaseaggregate associated offsets (k nearest neighbor) and apply it
algorithm
![Page 61: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/61.jpg)
user study 2
![Page 62: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/62.jpg)
1 tracking device
fingerprintoptical tracker
“simulated capacitive” (just contact area)
![Page 63: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/63.jpg)
2rotation
![Page 64: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/64.jpg)
3 roll & pitch
roll -15° 0° 15° 45° 90°
pitch
15°
25°
45°
65°
90°
![Page 65: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/65.jpg)
optical beats simulated capacitive by ~3x(based on user study 1)
fingerprint beats simulated capacitive(let’s find out by how much)
hypotheses
![Page 66: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/66.jpg)
2 rotations× 13 angles× 5 repetitions per angle× 5 blocks
= 650 trials / participant
12 participants design
![Page 67: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/67.jpg)
results
![Page 68: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/68.jpg)
5.00
4.00
3.00
2.00
1.00
0.00
Error bars: +/- 1 SE
Mea
n spr
ead i
n mm
rawcapacitive
rotation-aware
capacitive
fingerprint-based
correction
tracker-based
correction
5.00
4.00
3.00
2.00
1.00
0.00
Error bars: +/- 1 SEM
ean s
prea
d in m
m
rawcapacitive
rotation-aware
capacitive
fingerprint-based
correction
tracker-based
correctionerror bars are standard deviation
spre
ad in
cm results
simulatedcapacitive
5.00
4.00
3.00
2.00
1.00
0.00
Error bars: +/- 1 SEM
ean
spre
ad in
mm
rawcapacitive
rotation-aware
capacitive
fingerprint-based
correction
tracker-based
correctionfingerprint optical
as expected a factor of 3x
works!
potential for improvement
![Page 69: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/69.jpg)
conclusions
![Page 70: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/70.jpg)
benefits1. make more reliable touch input devicesenter text on mobile touch device with high accuracy
2. avoid need for targeting aidssuch as offset cursor, shift, zooming,as they cost time and make touch less “direct”
3. make smaller mobile touch devicesbring touch input to watch-size mobile devices
use roll/pitch/yaw/userID touch device to
![Page 71: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/71.jpg)
2/3 (7/8 of surface) of “fat finger problem”really stem from an oversimplified model of touch
touch is not 2D
model
![Page 72: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/72.jpg)
find a closed representation of user profile speed up learning
combine with in-cell touch screens make small
next steps
![Page 73: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/73.jpg)
fachgebiet human-computer interaction
thanks to my new group athasso plattner institutein berlin/potsdam
![Page 74: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/74.jpg)
Christian HolzPh.D. Student, masters from Hasso Plattner InstituteMasters project with Steve Feiner at Columbia University, New York
.
![Page 75: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/75.jpg)
Gerry Chuintern at Hasso Plattner InstituteMasters from U of Toronto
.
![Page 76: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/76.jpg)
![Page 77: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/77.jpg)
joe konstan: university of minnesotadaniel fisher: microsoft researchgary marsden: south africa, capetownmichael rohs: telekom labsscott klemmer: stanfordmark billinghurst: hitlab new zealandlucia terrenghi: vodaphone
come visit
![Page 78: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/78.jpg)
fachgebiet human-computer interaction
open Ph.D./post doc position
![Page 79: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/79.jpg)
![Page 80: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/80.jpg)
all people
10 22 45 60 90
-10
0
10
45
90
without sense of rotation
![Page 81: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/81.jpg)
10 22 45 60 90
-10
0
10
45
90
all people
with sense of rotation
![Page 82: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/82.jpg)
0.40
0.30
0.20
0.10
0.00
Error bars: +/- 1 SE
raw capacitive rotation-awarecapacitive
per-anglecapacitive
Mea
n sp
read
in c
m
per user spread
![Page 83: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/83.jpg)
Professor in computer science at Hasso Plattner Institute2002- research scientist at Microsoft Research, Redmond, WA2003- affiliate professor at University of Washington Seattle, WA2000-2002 research scientist at Xerox PARC2000 Ph.D. in computer science from TU Darmstadt
patrickbaudisch
![Page 84: christian holz patrick baudisch](https://reader035.vdocuments.net/reader035/viewer/2022062218/56816380550346895dd46260/html5/thumbnails/84.jpg)
Sean GustafsonPh.D. StudentMasters University of Manitoba, Canada on visualization, off-screen pointing
.
spatial cognitionon mobile