jason rebellowavelab.uwaterloo.ca/slam/2017-slam/lecture12-calibration...camera calibration using a...
TRANSCRIPT
![Page 1: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/1.jpg)
Jason Rebello | Waterloo Autonomous Vehicles Lab
Calibration Overview
Jason Rebello10/07/2017
No moose was hurt in the making of this presentation
![Page 2: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/2.jpg)
Calibration | Why do we need calibration ?
Jason Rebello | Waterloo Autonomous Vehicles Lab
- Data from multiple complementary sensors leads to increased accuracy and robustness
- Sensors need to be spatially and temporally calibrated
- Calibration helps transform measurements from different sensors into a common reference frame
Source: Arun Das
![Page 3: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/3.jpg)
Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Source: Arun Das
Required Calibrations● Camera intrinsic and extrinsic● IMU to Camera● Lidar to GPS● Lidar to Camera
![Page 4: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/4.jpg)
Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Source: Arun Das
Required Calibrations● Camera intrinsic and extrinsic● IMU to Camera● Lidar to GPS● Lidar to Camera
![Page 5: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/5.jpg)
Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Source: Arun Das
Required Calibrations● Camera intrinsic and extrinsic● IMU to Camera● Lidar to GPS● Lidar to Camera
![Page 6: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/6.jpg)
Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Source: Arun Das
Required Calibrations● Camera intrinsic and extrinsic● IMU to Camera● Lidar to GPS● Lidar to Camera
![Page 7: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/7.jpg)
Calibration Camera Intrinsics |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Camera Intrinsics
![Page 8: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/8.jpg)
Image Formation | Projection pipeline
Jason Rebello | Waterloo Autonomous Vehicles Lab
Forward Projection
Source: Robert Collins, CSE486
(a) (b) (c)
Image to Pixel CoordinatesExtrinsics R,t Intrinsics focal length, ox, oy
![Page 9: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/9.jpg)
Image Formation | Camera Parameters
Jason Rebello | Waterloo Autonomous Vehicles Lab
(a) Extrinsic Transformation (Rotation + Translation)
- Transforms points from World to Camera coordinate Frame
- Homogeneous coordinates allow for easy matrix multiplication
(b),(c) Perspective Projection
Camera Coordinates Pixel CoordinatesSource: Robert Collins, CSE486
![Page 10: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/10.jpg)
Image Formation | Distortions
Jason Rebello | Waterloo Autonomous Vehicles Lab
Radial DistortionSource: Scaramuzza
Tangential Distortion Source: MathWorks
Nikon Lens SystemSource: Aaron Bobik
![Page 11: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/11.jpg)
Intrinsic Parameter Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Camera Calibration using a 2D Checkerboard
Known size (30 mm) and structure (6 x 9 )
Identify corners easily. Distorted Corrected
Where do I set my world coordinate frame ?
![Page 12: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/12.jpg)
Why use a checkerboard ? |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Can set the world coordinate system to one corner of the checkerboard
All points lie in the X,Y plane with Z=0
![Page 13: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/13.jpg)
Projection |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 14: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/14.jpg)
Homography |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 15: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/15.jpg)
Homography Solution |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 16: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/16.jpg)
Homography constraints |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 17: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/17.jpg)
Intrinsic Parameters |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Find B , recover K through the Cholesky decomposition chol (B) = AAT
A = K-T
![Page 18: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/18.jpg)
Solve for b |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Construct a system of linear Equations Vb = 0
V = [ vT12 v
T11- v
T22 ]
T … 2x6 matrix for 1 image
Where
vij = [ h1ih1j , h1ih2j+h2ih1j , h3ih1j+h1ih3j , h2ih2j , h3ih2j+h2ih3j , h3ih3j ]
For n images
V = 2n x 6 matrix
![Page 19: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/19.jpg)
Intrinsic Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 20: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/20.jpg)
Intrinsic Parameters |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 21: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/21.jpg)
Intrinsic Parameters |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 22: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/22.jpg)
Intrinsic Parameters |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 23: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/23.jpg)
Intrinsic Parameters |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 24: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/24.jpg)
How do I calibrate a camera ? |
Jason Rebello | Waterloo Autonomous Vehicles Lab
1) Print a target checkerboard (Several available online)
2) Note dimensions of target, size of square
3) Take multiple images by moving target to different locations on the image
4) Calibrate using OpenCV, Matlab
5) Reprojection error below 0.2 is good
Ximea camera: 5.3 ㎛ / pixEdmund Optic lens: 3.5 mm
Focal length in pixel = 3.5 / (5.3x10-3) = 660.37
Question: What is the problem with this sort of calibration ?
![Page 25: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/25.jpg)
Calibration Camera Extrinsics |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Camera Extrinsics
![Page 26: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/26.jpg)
Extrinsic Calibration | Stereo Vision
Jason Rebello | Waterloo Autonomous Vehicles Lab
Rotation and Translation
![Page 27: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/27.jpg)
Advantages of Stereo Vision |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Stereo Vision:
1) Recover Depth
2) Limit the search space in the second image for point correspondence
![Page 28: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/28.jpg)
PnP Algorithm |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Given:
1) Known 3D points in world frame ( Checkerboard Points)
2) Pixel Locations on image plane
3) Intrinsic parameters of Camera
Estimate:
Transformation from world to camera
![Page 29: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/29.jpg)
Extrinsic Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
1) Given 3D points in world frame Pw and corresponding 2D pixel locations in both cameras zs and zm
2) Estimate Transformations from world to both cameras using PnP algorithm Ts:w and Tm:w
3) Transform 3D points from world to Fs camera. Ps = Ts:w Pw
4) Transform 3D points from camera Fs to camera Fm via Extrinsic transformation.Pm = Tm:s Ps
5) Project points onto Fm. camera. z’m = ⌅(Pm)
Reprojection Error:
∑images ∑points d( z’m , zm )2 + d( z’
s , zs )2
![Page 30: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/30.jpg)
How should I calibrate the extrinsics ? |
Jason Rebello | Waterloo Autonomous Vehicles Lab
1) Print a target checkerboard (Several available online)
2) Note dimensions of target, size of square
3) Take multiple images in both cameras at same time by moving target to different locations on the image
4) Calibrate using OpenCV, Matlab
5) Reprojection error below 0.4 is good
![Page 31: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/31.jpg)
End of Calibration I |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 32: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/32.jpg)
Lidar to Camera Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Target
Where does this calibration differ from the extrinsic camera calibration ?What sort of target should be used ?
![Page 33: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/33.jpg)
HDL 32 vs HDL 64 |
Jason Rebello | Waterloo Autonomous Vehicles Lab
32 beam64 beam
64 beam
32 beam
![Page 34: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/34.jpg)
Lidar response problems |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 35: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/35.jpg)
Lidar Camera Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
1) Given 3D points in world frame Pw and corresponding 2D pixel locations zc in the camera
2) Estimate Transformations from world to the camera using PnP algorithm Tc:w.
3) Input initial transformation from lidar to camera (approximate). Tc:l
4) Transform 3D points from Lidar Fl to camera Fw via Extrinsic transformation.Pw = inv( Tc:w )Tc:l Pl
5) Determine which points Pl lie on target within some threshold. 6) Project points on plane
7) Fit ‘generated’ point cloud to detected point cloud and get end-points.
8) Determine end points of target on image
Reprojection Error: ∑images ∑points d( ⌅(Tc:l(Pl )) , zc )2
![Page 36: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/36.jpg)
Calibration Results |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Before Calibration After Calibration
![Page 37: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/37.jpg)
Alternative methods |
Jason Rebello | Waterloo Autonomous Vehicles Lab
a) Martin Velas et.al Calibration of RGB Camera With Velodyne LiDAR
b) Gaurav Pandey et.al Automatic Extrinsic Calibration of Vision and Lidar by Maximizing Mutual Information
c) Jesse Levinson et.al Automatic Online Calibration of Cameras and Lasers
![Page 38: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/38.jpg)
Questions |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 39: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/39.jpg)
Lidar to GPS |
Jason Rebello | Waterloo Autonomous Vehicles Lab

![Page 40: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/40.jpg)
Hand-Eye Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Static Static
Motion Capture System
Fiducial Target
Reflective Marker
Camera
Estimate
Estimate
![Page 41: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/41.jpg)
Hand-Eye Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
- TVM : Marker to Vicon Motion Tracking
- TMC : Camera to Marker
- TVF : Fiducial Target to Vicon
- TFC : Camera to Fiducial Target
![Page 42: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/42.jpg)
Hand-Eye calibration on Car |
Jason Rebello | Waterloo Autonomous Vehicles Lab
- Calibration with motion capture system and fiducial target similar to Lidar (SLAM) and GPS
- Planar motion leads to observability issues (Z, roll pitch)
- Need to find area that allows for sufficient excitation (car motion).
![Page 43: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/43.jpg)
IMU to Camera Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Camera
World
IMU
![Page 44: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/44.jpg)
IMU to Camera Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Quantities Estimated:- Gravity direction expressed in World Frame- Transformation between Camera and IMU- Offset between Camera time and IMU time- Pose of IMU- Accelerometer and gyroscope biases
Assumptions Made:- Camera Intrinsics are known- IMU noise and bias models are known- Have a guess for gravity in world frame- Have a guess for the calibration matrix- Geometry of calibration pattern is known- Data association between image point and
world point known
![Page 45: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/45.jpg)
Questions |
Jason Rebello | Waterloo Autonomous Vehicles Lab
![Page 46: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/46.jpg)
IMU to Camera Calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
- Time varying states are represented as weighted sum of a finite number of known basis functions. phi(t) assumed to be known
- yj is a measurement that arrived with timestamp tj. h(.) is a measurement model that produces a predicted measurement from x(.) and d is unknown time offset
- The analytical Jacobian of the error term, needed for nonlinear least squares is derived by linearizing about a nominal value d, with respect to small changes in d.
![Page 47: Jason Rebellowavelab.uwaterloo.ca/slam/2017-SLAM/Lecture12-Calibration...Camera Calibration using a 2D Checkerboard Known size (30 mm) and structure (6 x 9 ) Identify corners easily](https://reader034.vdocuments.net/reader034/viewer/2022050517/5fa0f0e0c8ee260c8e61afd7/html5/thumbnails/47.jpg)
IMU Camera calibration |
Jason Rebello | Waterloo Autonomous Vehicles Lab
Accelerometer measurement, gyroscope measurement (at time k) pixel location (time t+d), h(.) projection model, n is noise. J is number of images
IMU biases
Error terms are associated with the measurements constructed as the difference between the measurement and the predicted measurement for current state estimate