projector calibration of interactive multi-resolution display systems

Projector Calibration of Interactive Multi-Resolution Display Systems

互動式多重解析度顯示系統之投影機校正

Presenter: 邱柏訊Advisor: 洪一平教授

Outline

• Introduction• Related Work• Calibration of Interactive Tabletop Displays• Calibration of Interactive Cylindrical Displays• Experiments and Error Analysis• Conclusion

2

Introduction

• Multi-Resolution Display System• Features

• A future personal desk• Cost-effective large high-res display

3

• i-m-Top

Fovea Projector

Mirror & PTU

Peripheral ProjectorIR Camera

IR LEDs

4

Introduction

Introduction

5

• Motivation• Disadvantages of Manual Calibration

• Tedious and inefficient• Technician dependent accuracy

• Goal• Automatic Calibration

• Fast and accurate

Outline


6

Related Work

• Automatic Projector Calibration• Smarter Presentations

• Camera-assisted approach

R. Sukthankar, R. Stockton, and M. Mullin, “Smarter Presentations: Exploiting Homography in Camera-Projector Systems,” In Proceedings of International Conference on Computer Vision (ICCV), 2001.

7

C

T

P = CT-1

Related Work

• Automatic Projector Calibration• Lee et al.

• Optical sensors embedded• Gray code patterns

Lee, J., Dietz, P., Aminzade, D., Raskar, R., and Hudson, S. "Automatic Projector Calibration using Embedded Light Sensors", Proceedings of the ACM Symposiumon User Interface Software and Technology, October 2004.

8

Gray Code Pattern

Outline

• Introduction• Related Work• Calibration of Interactive Tabletop Displays

• Hardware Configuration• Implementation

• Calibration of Interactive Cylindrical Displays• Experiments and Error Analysis• Conclusion

9

• Hardware Configuration

Color Camera

10

Calibration of Interactive Tabletop Displays

• Implementation• Step1. Peripheral Projector Calibration• Step2. Fovea Projector Calibration• Step3. Infrared Camera Calibration


11

• Peripheral Projector Calibration

Implementation

Surface( S )

Peripheral Projector ( PP )

Our goalHPP S

12

Camera( C )


Implementation

13

White Pattern

HC S


Implementation

14

HC S

HC PP

Visual Pattern

Option 1: Circle Pattern

Option 2: Concentric Circle Pattern

Option 3: Middle Line Pattern


Implementation

15

HC S

Gray Code Pattern

HC PP

HPP S HC

S= HCPP

-1

Implementation


16


17


Implementation

18

• Fovea Projector Calibration

Surface( S )

Fovea Projector ( FP )

Cam( C )

Our goal

HFP(θ, ϕ) S

Implementation

19


HC S

HCFP(θ, ϕ)

Visual Pattern

Option 1: Circle Pattern

Option 2: Concentric Circle Pattern

Option 3: Middle Line Pattern

Implementation

20


HC S

HCFP(θ, ϕ) HFP(θ, ϕ)

S HC S= HC

FP(θ, ϕ)-1

Gray Code Pattern

Implementation

21

• Fovea Projector Calibration• Problem : Project to anywhere desired

• Range of PTU angle : pan(-3087~3087), tilt(-908~604)• Calibration for all PTU angles is impractical

• Solution• Sample PTU angles with fixed interval• Interpolate mapping function ( f1 and f2 )

• •

1 : ( , ), where = ( , ) is one surface coordinates s s sf p PTU p x y

2 ( , ): ( , ) SFPf PTU H

Implementation

22

• Mapping Function Interpolation• Record trajectories of fovea projection

Implementation

23

• Mapping Function Interpolation• Record trajectories of fovea projection

Implementation

• Mapping Function Interpolation• Interpolate f1 for one given surface coordinate ps

0Y

X

Surface

1

10

0d

2d

3d1d

1 00 0

1 0

1 02 0

3 2

( )( )( )( )

N

N

dd d

dd d

24

1 : ( , )sf p PTU

Y

X

Surface

Implementation

• Mapping Function Interpolation• Interpolate f2 for one given PTU angle (θ,ϕ)

0

1

10

N

N

2 ( , ): ( , ) SFPf PTU H

Implementation


26


27


Implementation

• Infrared Camera Calibration

Surface( S )

0s1s3s

2s5s4s

HIRC S

HS PP

Source Image

IR Camera(IRC)

Warped Image

28

Implementation

• Infrared Camera Calibration

29

• Summary

Calibrate peripheral projector

Calibrate IR cameras

Calibrate fovea projector

Obtain valid projector angle

Interpolate mapping function

Repe

at fo

r all

sam

pled

PTU

ang

les

30

: Peripheral Projector Calibration

: Fovea Projector Calibration

: Infrared Camera Calibration


Outline

• Introduction• Related Work• Calibration of Interactive Tabletop Displays• Calibration of Interactive Cylindrical Displays

• Hardware Configuration• Projector Calibration

• Experiments and Error Analysis• Conclusion

31

• Hardware Configuration

90 cm

85 cm

120 cmSurface

ProjectorProjector

Surface

Front View Back View

Hardware Component

32

Calibration of Interactive Cylindrical Displays

33

• Projector Calibration• Piecewise Planar Mapping

• Texture mapping• Using corresponding points between surface &

projector• Problem

• Projector intrinsics estimation


34

• Projector Calibration• Projector Intrinsics Estimation

• Step1. Initial guess• Step2. Measure corresponding points m and M

P0P1P2P3P4

P9P8P7

P6P5

Projection

Surface

Marker

: m: M

a = [fx, fy, cx, cy]


35

• Projector Calibration• Projector Intrinsics Estimation

• Step1. Initial guess • Step2. Measure corresponding points m and M• Step3. Estimate extrinsics E with a, m, M• Step4. Update with E, m, M• Step5. Repeat step 3, 4 until

a = [fx, fy, cx, cy]

a = a + △a||△a || < ε


• Projector Calibration

Pre-Warping

Source Image Warped Image

36


Outline


37

Mode PP FP (1 set) FP (36 sets) FP (300 sets)Manual 1~2 1~2 40~60 X

Auto 0.1 0.1 ~4.8 ~40

Unit: minuteFP : Fovea Projector

PP : Peripheral Projector

38

Experiments and Error Analysis- Calibration of Tabletop Displays

• Comparison of Calibration Time• Manual vs. Auto

• Error Analysis• Calibration Error of Peripheral Projector

39

Circle Pattern Concentric Circle Pattern Middle Line Pattern0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

10.892

0.703000000000001

0.867000000000003

0.4550.327000000

000002

0.155

Avg.S.D.

Erro

r (pi

xel)


Circle Pattern Concentric Circle Pattern Middle Line Pattern0

0.1

0.2

0.3

0.4

0.5

0.6

0.381000000000002

0.5070.57

0.136 0.125

0.265Avg.S.D.Er

ror (

pixe

l)

• Error Analysis• Calibration Error of Fovea Projector

• Without mapping function interpolation

40


• Error Analysis• Calibration Error of Fovea Projector

• With mapping function interpolation

16 32 64 1280

0.5

1

1.5

2

2.5

1.41399999999998

1.44699999999999

1.653

1.95800000000001

0.408 0.481 0.326000000000002 0.241

Avg. S.D.

Sampling Interval (PTU angles)

Erro

r (pi

xel)

16 32 64 1280

500

1000

1500

2000

2500

3000 2660

682172 45


# of

cal

ibra

tion

pose

s

16 32 64 1280

50100150200250300350400 355

9123 6


Tim

e (m

in)

41


42


• Applications are run after Calibration

Outline


43

44

Conclusion

• We propose an automatic projector calibration method of interactive tabletop displays, which is both efficient and accurate

• The fovea projector is able to correctly project over the whole tabletop surface

• Calibration of cylindrical displays is achieved with semi-automatic approach

Thanks for your listening

45

projector calibration of interactive multi-resolution display systems

Documents