real-time phase-stamp range finder with improved accuracy akira kimachi osaka electro-communication...
TRANSCRIPT
![Page 1: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/1.jpg)
Real-Time Phase-Stamp Range Finder with Improved Accuracy
Akira KimachiOsaka Electro-Communication University
Neyagawa, Osaka 572-8530, Japan
1August 2, 2009 Optical Engineering + Applications, San Diego Convention Center
![Page 2: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/2.jpg)
Outline
• Introduction• Phase-stamp range finder (PSRF)
– Time-domain correlation image sensor (CIS)– Phase-stamp imaging (PSI)– Problem of artifacts
• Accuracy improvement by calibrating the CIS for PSI– Experiment #1: CIS output behavior in PSI– CIS output model for PSI– Compensation for phase stamp errors– Experiment #2: accuracy evaluation
• Conclusions
2
![Page 3: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/3.jpg)
Real-time range imaging
• Demands– Assembly/inspection of industrial products– Environment recognition for robots/vehicles– Observation of mobile objects– Assistance in human workspace
• Active vs. passive range finders– Active methods are preferable in terms of accuracy/reliability
3
# of sensors
Active illumination
Accuracy/reliability
Methods
Active 1 Yes High
Light sectioningStructured lightTime-of-flightTime-stampPhase-stamp
Passive > 1 No LowBinocular stereoMultiple-camera
![Page 4: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/4.jpg)
Real-time active range finders• Robustness vs. depth resolution
– Difficult to establish both
4
Method SensorDepth
resolution
Robustness to disturbance
Ambient illuminationSurface textureTemporal
variationSpatial
variation
Light sectioning
High-speed camera
< 1 mm Low Low Low
VLSI sensor < 1 mm Low High Low
Structured lightHigh-speed camera
< 1 mm Low High High
Time-of-flight (TOF)
VLSI sensor > 1 mm High High High
Axi-Vision Camera
> 1 mm High High High
Time-stamp VLSI sensor < 1 mm Low Low Low
Phase-stampCorrelation image sensor
< 1 mm High High High
![Page 5: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/5.jpg)
flat board @ 400 mm
Objective
• Phase-stamp range finder (PSRF)Kimachi and Ando, Electronic Imaging (2007)
– Time-domain correlation image sensor (CIS)– Frame-rate 3D capture based on “phase stamp” imaging (PSI)
• Problem of artifacts
• Solution– Analyze the behavior of CIS outputs in PSI– Model the CIS outputs for PSI– Calibrate the PSRF by compensating for CIS output errors
5
undulation
~ 3.5 mm rmsrandomnoisepattern
~ 1 mm rms
![Page 6: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/6.jpg)
Correlation image sensor (CIS)
6
Ando and Kimachi, Trans. IEEE ED (2003)
Output images
Average intensity
200x200-pixel CMOS camera
Temporal correlation
: frame integral
![Page 7: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/7.jpg)
frame
Phase-stamp imaging (PSI)
7
Light pulseenergy image
Phase stampimage
Correlation images
uncorrelated
noiseHigher temporal resolutionthan the frame period T
![Page 8: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/8.jpg)
Phase-stamp range finder (PSRF)
8
SOLenergy
ambientillumination
surfacereflectance
(SOL)
Incident lightintensity
Referencesignals
SOLangle
Phasestamp
Correlation images
Range image
: pixel spacing in x
•One SOL scan in one frame•Based on PSI
Kimachi and Ando,Electronic Imaging (2007)
•Range image from single frame •Ambient illumination removed•Surface reflectane canceled
( )ijB tijR
ijz ( , )kQ i j
![Page 9: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/9.jpg)
Experimental PSRF system
9
CIS 200x200-pixel
Frame rate 12.5 fps
Reference signal frequency
50 Hz
Mirror scanning rate
25 Hz
Laser DPSS , 40 mW , 658 nm
Camera lens 25 mm F1.4
![Page 10: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/10.jpg)
PSRF artifacts
• Artifacts in range images– Undulation– Random noise pattern
• Considered to be caused by errors in detected phase stamps
• Temporal correlations may not follow the ideal characteristics with respect to SOL incidence time
10
ijz
ij
( , )kQ i jijt
phase stamp range mapaverage intensity SOL energy
(flat board @ 400 mm)
0
/2
/2
![Page 11: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/11.jpg)
Investigating CIS outputs in PSI
• Capture a sequence of correlation images while shifting pulse occurrence time
11
( , , )kQ i j 0 50 Hz ( 20 ms)f T
Pulse occurrence time 0~20 ms (1 ms step)
Pulse height 0~8, 9 levels
Pulse width 0.2~2 ms (0.2 ms step)
Reference signal amplitude 0.05~0.5 V (0.05 step)0V
ht
0t0t
![Page 12: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/12.jpg)
Results: image average behavior
12
Distortions in from sine functions become severer as increases( )kQ 0V
Image averages of temporal correlations increase monotonicallywith , , and
( )kQ 0Vt h
( , , )kQ i j
Pulse width variedt Pulse height variedh Amplitude varied0V
Undulations in the phase stamp become severer as increases0V
![Page 13: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/13.jpg)
Results: pixel-wise deviation behavior
13
0
/2
/2
Pulse width variedt Pulse height variedh Amplitude varied0V
Pixel-wise deviations of temporal correlations increasemonotonically as , , and 0Vt h
( , , )k i j ( , , )kQ i j
Pixel-wise deviations of computed phase stamps oscillate with 0tij
( , ) (100,100)i j
oscillate with in arbitrary waveforms, randomly with respect to pixel and channel
( , , )k i j 0tk( , )i j
![Page 14: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/14.jpg)
• Ideal characteristic of CIS temporal correlation outputs
• Experiment-based CIS output model for PSI
– Undulation/distortion→ Harmonics , ,
– Pixel-/channel-wise random deviation→ Coefficients , , , ,
– Dependence on light pulse energy → Multiplication by
• Coefficients are estimated from a sequence of PSI images by least-squares fitting
CIS output model for PSI
14
( , )ka i j ( , )kb i j ( , )kc i j ( , )kd i j ( , )ke i j
sin k cos 2 k sin 2 k
,ijA t h
noise
(for fixed )0V
![Page 15: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/15.jpg)
Compensation for phase stamp errorsFor a single-frame set of temporal correlation images
, , and ,
1. Compute the phase stamps
2. Approximate the PSI CIS model by regarding as containing a small error
3. Estimate and pixel-wise by least-squares fitting to the model with and the pre-estimated coefficients , , , ,
4. Obtain the phase stamp estimates15
1( , )Q i j 3( , )Q i j
( , )kQ i j
2 ( , )Q i j
ij1ij
ijij
ij ijA
( , )ka i j ( , )kb i j ( , )kc i j ( , )kd i j ( , )ke i j
![Page 16: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/16.jpg)
Results: CIS calibration for PSI (1)
16
( , ) (100,108)i j
test data = calibration data
•Fitting — over a sequence of •Compensation — on a single frame of
( , , )kQ i j kQ
![Page 17: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/17.jpg)
Results: CIS calibration for PSI (2)
17
108j
test data = calibration data
0 4 mst
0 10 mst
0 17 mst
test data ≠ calibration data
![Page 18: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/18.jpg)
Results: CIS calibration in PSI (3)
18
test data ≠ calibration data
Before compensation
After compensation
![Page 19: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/19.jpg)
Results: artifacts removal in PSRF
19
SOL intensity phase stamp
@ 400 mm
compensateduncompensated
flat board
paper box
can bottle
Compensation Before After
Offsetphase [rad] −2.50 0.00
range [mm] 5.37 0.70
Undulation (rms)
phase [rad] 7.44 1.16
range [mm] 3.45 0.48
Random noise pattern (rms)
phase [rad] 1.81 0.92
range [mm] 1.09 0.55
![Page 20: Real-Time Phase-Stamp Range Finder with Improved Accuracy Akira Kimachi Osaka Electro-Communication University Neyagawa, Osaka 572-8530, Japan 1August](https://reader035.vdocuments.net/reader035/viewer/2022062300/56649e545503460f94b4b5da/html5/thumbnails/20.jpg)
Conclusions• Artifacts in PSRF outputs have been removed
– CIS outputs were modeled based on PSI experiments– A method for compensating for phase stamp errors in CIS
outputs was proposed– Confirmed in experiments
• Accuracy improved on a PSRF system– Undulation — 3.45 mm → 0.48 mm– Random noise pattern — 1.09 mm → 0.55 mm
20