visual screen: transforming an ordinary screen into a touch screen zhengyou zhang & ying shan...
Post on 22-Dec-2015
227 views
TRANSCRIPT
![Page 1: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/1.jpg)
Visual Screen: Transforming an OrdinaryScreen into a Touch Screen
Zhengyou Zhang & Ying Shan
Vision Technology GroupMicrosoft Research
![Page 2: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/2.jpg)
Motivation
Transform an ordinary screen into a touch
screen using an ordinary camera
![Page 3: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/3.jpg)
Configuration
Position a camera so it can see the whole screen
![Page 4: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/4.jpg)
Fingertip Tracker
Action
Detector
&
Event
Generator
Mouse EventsVideo
Input
Visual Screen
FingertipDetector
HomographyMapping
Screen Detection
Virtual Touch Screen
Non-flatness correctionCalibration
The System
![Page 5: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/5.jpg)
System Diagram
![Page 6: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/6.jpg)
Calibration
• Mapping from image coordinate to the screen coordinate
• Homography if the screen is flat• More accurate method required when the
screen is curved
![Page 7: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/7.jpg)
Plane Plus Residue Flow
H
![Page 8: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/8.jpg)
The actual screen coordinate can be found by the homography corrected by an interpolated residue vector
Mapping
![Page 9: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/9.jpg)
Plane Plus Residual Flow
Before correction After correctionOriginal calibration points Reprojected image points
![Page 10: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/10.jpg)
Segmentation
• Images of screen pixels have some degrees of invariance in the color space
• Compute a color model for the screen without the indicator
• Compute a color model for the indicator • Use standard Bayes classifier to segment the
indicator from the screen background
![Page 11: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/11.jpg)
Segmentation
Before segmentation After segmentation
![Page 12: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/12.jpg)
Locate the Finger Tip
•Initial location from horizontal histogram•Fit the centerline of the finger•Finger tip is the intersection of the centerline and the segment boundary
![Page 13: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/13.jpg)
Experiments: Draw Bubbles for Fish
![Page 14: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/14.jpg)
Experiments: Barney Under the Sea
![Page 15: Visual Screen: Transforming an Ordinary Screen into a Touch Screen Zhengyou Zhang & Ying Shan Vision Technology Group Microsoft Research zhang@microsoft.com](https://reader030.vdocuments.net/reader030/viewer/2022032523/56649d7b5503460f94a5f824/html5/thumbnails/15.jpg)
Thank You