by: navid einackchi
DESCRIPTION
به نام خدا. Isfahan University of Technology Electrical and Computer Department. Master Thesis of Computer Engineering- Artificial Intelligence and Robotic. Image alignment and stitching using Object Recognition Methods. By: Navid Einackchi. Supervisors: Dr Rasoul Amirfattahi - PowerPoint PPT PresentationTRANSCRIPT
1
By: Navid EinackchiBy: Navid Einackchi
Isfahan University of Isfahan University of TechnologyTechnology
Electrical and Computer Department
Spring 2007Spring 2007
Image alignment and stitching using Object Recognition MethodsImage alignment and stitching using Object Recognition Methods
Supervisors: Dr Rasoul Amirfattahi
Dr javad Askari
Advisor: Dr M. Saraee
Master Thesis of Computer Engineering- Artificial Intelligence and Robotic
به نام خدا
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
Some slides are fro
m other
Some slides are fro
m other
sources
sources
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image Alignment and StitchingImage Alignment and Stitching
• DefinitionDefinition • ApplicationsApplications
– Mosaic Image
– Panorama
– Virtual Environment
– Robotic
تصوير حاصل
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image Alignment and StitchingImage Alignment and Stitching
• DefinitionDefinition • ApplicationsApplications
– Mosaic Image
– Panorama
– Virtual Environment
– Robotic
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image Alignment and StitchingImage Alignment and Stitching
• DefinitionDefinition• ApplicationsApplications
– Mosaic Image
– Panorama
– Virtual Environment
– Robotic
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Aligning ConceptsAligning Concepts
• Image TransformationsImage Transformations– Applicable Transformations
• Find transformationsFind transformations– How many parameters?
– Calculating Parameters
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image TransformationsImage Transformations
• TransitionTransition– Number of Parameters: 2
– Number of Points: 1
11
y
x
ihg
fed
cba
y
x
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image TransformationsImage Transformations
• EuclideanEuclidean– Number of Parameters: 3
– Number of Points: 2
11
y
x
ihg
fed
cba
y
x
cossin
sincosR
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image TransformationsImage Transformations
• SimilaritySimilarity– Number of Parameters: 4
– Number of Points: 2
11
y
x
ihg
fed
cba
y
x
cos.sin.
sin.cos.
ss
sssR
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image TransformationsImage Transformations
• AffineAffine– Number of Parameters: 6– Number of Points: 3
11
y
x
ihg
fed
cba
y
x
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image TransformationsImage Transformations
• PerspectivePerspective– Number of Parameters: 8– Number of Points: 4
11
y
x
ihg
fed
cba
y
x
1
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Image Transformation ComputationImage Transformation Computation
• DirectDirect– Mapping one image into another using different parameters– Define an Error Function based on pixel intensity difference
• Using Corresponding pointsUsing Corresponding points– Obtaining Corresponding points– Computing Transformation parameters using corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Direct MethodDirect Method
Error Function
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Direct MethodDirect Method
Error Function
؟؟
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Direct MethodDirect Method
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Using Corresponding pointsUsing Corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Using Corresponding pointsUsing Corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Using Corresponding pointsUsing Corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
• How many points?How many points?
• How to select?How to select?– Using human
– Automatically
Using Corresponding pointsUsing Corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector
–Using SIFT Detector–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
StitchingStitching
• StitchingStitching• ChoicesChoices
– Final Plane
– Pixel weighting
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector
–Using SIFT Detector–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Object Recognition MethodsObject Recognition Methods
• Local Feature DetectionLocal Feature Detection– Edge– Corner– Hole
• Interest Regions MatchingInterest Regions Matching– Appearance Matching– Geometric Matching
• Finding Object Relations in ImageFinding Object Relations in Image
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector
–Using SIFT Detector–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Salient FeaturesSalient Features
• Features of Feature!Features of Feature!– Invariants against transformations– Ability to explain the image– Ability to being Matched (Repeatability)
• Selected FeaturesSelected Features– Corners– Holes
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector
PlainNo changes in every direction
EdgeHigh changes in Edge
Direction
CornerHigh changes in every
direction
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector(Continued)(Continued)
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I
Slow change
direction
Fast change Direction
(max)-1/2
(min)-1/2
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector(Continued)(Continued)
1
2
“Corner” 1, 2 both bigboth big
1 ~ 2
Edge1 >> 2
Edge2 >> 1
Plain
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector(Continued)(Continued)
•Harris measurementHarris measurement
2det traceR M k M
1 2
1 2
det
trace
M
M
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector(Continued)(Continued)
•Harris measurement responseHarris measurement response
1
2
“Corner”
“Edge”
“Edge”
“Plain”
R > 0
R < 0
R < 0|R| small
Local Maxim
um bigger than Threshold
Local Maxim
um bigger than Threshold
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector(Continued)(Continued)
• FeaturesFeatures– Invariant against rotation
– Invariant against intensity
R
x
Threshold
R
x
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Harris DetectorHarris Detector(Continued)(Continued)
•ExampleExample
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
• Scale ProblemScale Problem
Scale InvariantScale Invariant
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Scale InvariantScale Invariant
Finding Appropriate Window Size
Finding Appropriate Window Size
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Scale SelectionScale Selection
• Using Differential of Gaussian FiltersUsing Differential of Gaussian Filters– Laplacian of Gaussian (LoG)– Difference of Gaussian (DoG)
- =Window thst m
aximize fil
ter
Window thst m
aximize fil
ter
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Automatic Scale SelectionAutomatic Scale Selection
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Automatic Scale SelectionAutomatic Scale Selection
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
SIFT Detector SIFT Detector
• Bulb DetectorBulb Detector• Invariant against scaleInvariant against scale• Using DoGUsing DoG
– Bulb detection
– Scale detection
مقیاس
x
y
DoG
DoG
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
SIFT DetectorSIFT Detector
• Detecting Bulb and Scale SimultaneouslyDetecting Bulb and Scale Simultaneously– Bulb position = maximum in plain
– Scale = maximum in third dimension
Bulb position
Scale position
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector
–Using SIFT Detector–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
DescriptorDescriptor
• Vector which describes region of interest pointVector which describes region of interest point– Pixels– Histogram– Differential
• Feature of DescriptorFeature of Descriptor– Invariant against image distortion (view, angle, intensity)– Able to express similarities and differences
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
SIFT DescriptorSIFT Descriptor
• Based on value and direction of gradientsBased on value and direction of gradients• 128 dimensional Vector128 dimensional Vector• ConstructionConstruction
1. Calculation of value and direction of gradient of each pixel
2. Making direction histogram of gradients
3. Rotating interest region based on dominant direction
4. Dividing the region into 16 region
5. Constructing direction histogram for each region
0
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Construction ExampleConstruction Example
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
DescriptorDescriptor FeaturesFeatures
• Invariant againstInvariant against– Rotation– Position– Affine changes in intensity (I Ia + b)
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
MatchingMatching
• Appearance MatchingAppearance Matching– Comparing distance of each two descriptors– Euclidean Distance– K nearest neighbor
• Geometrical MatchingGeometrical Matching– Based on geometrical relations between points– Estimation of image transformation– Rejecting false corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Example of Geometrical MatchingExample of Geometrical Matching
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Automatically image alignmentAutomatically image alignment
• Direct MethodDirect Method
• Based on interest pointsBased on interest points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
CompareCompare
Interest pointsInterest points Direct MethodDirect Method
LowLow (Order of number of interest (Order of number of interest
points)points)
HighHigh((order of number of pixelsorder of number of pixels))
Time and Memory Time and Memory orderorder
HighHighLowLow
((Based on pixel intensityBased on pixel intensity))Resisting against Resisting against intensity changesintensity changes
YesYes NoNoAbility to recognize Ability to recognize
corresponding corresponding imageimage
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector
–Using SIFT Detector–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Brown and Lowe AlgorithmBrown and Lowe Algorithm
O(nO(n44 m m44))
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Brown and Lowe AlgorithmBrown and Lowe Algorithm
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Brown and Lowe AlgorithmBrown and Lowe Algorithm
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Similarity Transform rather than perspectiveSimilarity Transform rather than perspective
• WhyWhy– Detector are unable to handle perspective – Small portion of each images is overlapped– Increasing the tolerance of true corresponding
• So We use Similarity TransformationSo We use Similarity Transformation
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
First Proposed AlgorithmFirst Proposed Algorithm
1.1. Detecting Interest points using Harris detectorDetecting Interest points using Harris detector
2.2. Defining descriptors for each pointDefining descriptors for each point
3.3. Finding corresponding points using appearanceFinding corresponding points using appearance• Comparing interest points with each other
• K Nearest Neighbor
4.4. Finding relation between 2 imagesFinding relation between 2 images• Using Similarity
• Based on voting
5.5. Rejecting wrong corresponding points based on geometrical relationsRejecting wrong corresponding points based on geometrical relations
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
First Proposed AlgorithmFirst Proposed Algorithm
1.1. Detecting Interest points using Harris detectorDetecting Interest points using Harris detector
2.2. Defining descriptors for each pointDefining descriptors for each point
3.3. Finding corresponding points using appearanceFinding corresponding points using appearance• Comparing interest points with each other
• K Nearest Neighbor
4.4. Finding relation between 2 imagesFinding relation between 2 images• Using Similarity
• Based on voting
5.5. Rejecting wrong corresponding points based on geometrical relationsRejecting wrong corresponding points based on geometrical relations
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Voting MechanismVoting Mechanism
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
100
0cossin
0sincos
ss
ss
similarityT
Voting MechanismVoting Mechanism
Sc
ale
Sc
ale
Angle
O(nO(n22 m m22)) از مرتبهاز مرتبه
3469
2255
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Applied Rotation
Estimated Rotation
Applied Scale
Estimated Scale
EvaluationEvaluation
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
ConcludingConcluding
• Very good for rotationVery good for rotation• Good for little scalingGood for little scaling• Weak for big scalingWeak for big scaling
– Due to Harris detector weakness
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector–Using SIFT Detector
–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
ImprovementImprovement
• Using less time order algorithmUsing less time order algorithm
• Using Scale Invariant Feature TransformUsing Scale Invariant Feature Transform
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Second proposed AlgorithmSecond proposed Algorithm
1.1. Detecting Interest pointsDetecting Interest points• SIFT Detector
2.2. Finding Corresponding pointsFinding Corresponding points• Based on appearance• Using k-d tree• Distance ratio threshold
3.3. Finding relation between imagesFinding relation between images• Voting on step 2 based on Appearance corresponding
4.4. Rejecting wrong corresponding pointsRejecting wrong corresponding points• Based on geometric relations between points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Distance Ration ThresholdDistance Ration Threshold
• Find 1NN and its distance = D1Find 1NN and its distance = D1• Find 2NN and its distance = D2Find 2NN and its distance = D2• Find ratio of these distances (D1/D2 = ratio)Find ratio of these distances (D1/D2 = ratio)
– If Lower than threshold -> reject
– Else accept
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Distance Ration ThresholdDistance Ration ThresholdMy ImprovementMy Improvement
• Find 1NN and its distance = D1Find 1NN and its distance = D1• Multiply D1 by ratio = rangeMultiply D1 by ratio = range• Look for range in k-d treeLook for range in k-d tree
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Results of threshold on ratioResults of threshold on ratio
• Threshold Ratio = 0.5Threshold Ratio = 0.5
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Results of threshold on ratioResults of threshold on ratio
• Threshold Ratio = 0.7Threshold Ratio = 0.7
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Second proposed AlgorithmSecond proposed Algorithm
1.1. Detecting Interest pointsDetecting Interest points• SIFT Detector
2.2. Finding Corresponding pointsFinding Corresponding points• Based on appearance• Using k-d tree• Distance ratio threshold
3.3. Finding relation between imagesFinding relation between images• Voting on step 2 based on Appearance corresponding
4.4. Rejecting wrong corresponding pointsRejecting wrong corresponding points• Based on geometric relations between points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
• Each interest point hasEach interest point has– Dominant Direction
– Scale
• Use this information for votingUse this information for voting
Estimating Geometrical Estimating Geometrical RelationRelation
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Estimation HistogramEstimation Histogram
Using threshold methodUsing 3NN method
Sca
leS
cale
Direction
Sca
leS
cale
Direction
O(nm)O(nm)
O(n)O(n)
Maximum Maximum
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Percent of True Corresponding
True Corresponding Points
Potential Corresponding points
Method
487% 19 39 Distance Ratio t=0.5
46% 23 50 Distance Ratio t=0.6
333% 27 81 Distance Ratio t=0.7
17% 44 259 Distance Ratio t=0.9
0.06% 49 710 Distance Ratio t=1
0.07% 52 710*2→710 2 Nearest Neighbor
0.07% 53 710*3→710 3 Nearest Neighbor
0.08% 57 710*4→710 4 Nearest Neighbor
Comparing Knn and Ratio Threshold settingComparing Knn and Ratio Threshold settingBased on Number of true corresponding pointsBased on Number of true corresponding points
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Comparing Knn and Ratio Threshold settingComparing Knn and Ratio Threshold settingBased on Number of true corresponding pointsBased on Number of true corresponding points
Percent of True Corresponding
True Corresponding Points
Potential Corresponding points
Method
487% 19 39 Distance Ratio t=0.5
46% 23 50 Distance Ratio t=0.6
333% 27 81 Distance Ratio t=0.7
17% 44 259 Distance Ratio t=0.9
0.06% 49 710 Distance Ratio t=1
0.07% 52 710*2→710 2 Nearest Neighbor
0.07% 53 710*3→710 3 Nearest Neighbor
0.08% 57 710*4→710 4 Nearest Neighbor
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Ratio Difference 2nd Maximum 1st Maximum Method
8.57 26.5 3.5 30 Distance Ratio t=0.5
7.75 27 4 31 Distance Ratio t=0.6
6.85 30.75 5.25 36 Distance Ratio t=0.7
1.95 46 48 94 2 Nearest Neighbor
1.46 36 78 114 3 Nearest Neighbor
1.35 36 101 137 4 Nearest Neighbor
Comparing Knn and Ratio Threshold settingComparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)(In Tolerating False Corresponding)
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Comparing Knn and Ratio Threshold settingComparing Knn and Ratio Threshold setting(In Tolerating False Corresponding)(In Tolerating False Corresponding)
Ratio Difference 2nd Maximum 1st Maximum Method
8.57 26.5 3.5 30 Distance Ratio t=0.5
7.75 27 4 31 Distance Ratio t=0.6
6.85 30.75 5.25 36 Distance Ratio t=0.7
1.95 46 48 94 2 Nearest Neighbor
1.46 36 78 114 3 Nearest Neighbor
1.35 36 101 137 4 Nearest Neighbor
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Applied Rotation
Estimated Rotation
Applied Scale
Estimated Scale
EvaluationEvaluation
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
• IntroductionIntroduction– Definition– Aligning Concepts– Stitching Concepts
• Object Recognition ProblemObject Recognition Problem– Salient Feature– What is Descriptor– Matching
• Proposed Method for AligningProposed Method for Aligning– Using Harris Detector– Using SIFT Detector– Stitching
• Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
StitchingStitching
• Pixel Weights based on their distancePixel Weights based on their distance
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
StitchingStitching
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
StitchingStitching
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Other examplesOther examples
• Adobe Photoshop could not stitch these imagesAdobe Photoshop could not stitch these images
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Other ExamplesOther Examples
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Other ExamplesOther Examples
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
Presentation ProcessPresentation Process
•IntroductionIntroduction–Definition
–Aligning Concepts–Stitching Concepts
•Object Recognition ProblemObject Recognition Problem–Salient Feature
–What is Descriptor–Matching
•Proposed Method for AligningProposed Method for Aligning–Using Harris Detector
–Using SIFT Detector–Stitching
•Concluding and SuggestionsConcluding and Suggestions
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
ConcludingConcluding
• IdeasIdeas– Using Harris and SIFT as a detector– Using Similarity transform as a general image transform– Using voting based on direction and scale of each interest points– Introducing Evaluation Method
• ResultsResults– Harris is more reliable but SIFT is much better in presence of scale transform– Similarity can handle prospective transform for human eyes– Voting can obtain similarity transform with much lower time order– Threshold ratio has a better behavior than Knn
By: Navid EinackchiBy: Navid Einackchi Spring 2007Spring 2007
Image alignment and stitching using object recognition methodsImage alignment and stitching using object recognition methods
ThanksThanksQuestions?Questions?