![Page 1: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/1.jpg)
1
Difference of Gaussian ScaleSpace Difference of Gaussian ScaleSpace Pyramids for SIFTPyramids for SIFT[1][1] Feature Detection Feature Detection
Chris MurphyChris MurphyBallard BlairBallard Blair 6.3756.375[1] Lowe, David G.
Distinctive Image Features from Scale-Invariant Keypoints,International Journal of Computer Vision, 2004.
![Page 2: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/2.jpg)
2
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
`
Two images of a Coral Reef near Puerto Rico.How do we align, or register, them?
![Page 3: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/3.jpg)
3
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
We have to figure out which points in one image, or features match which features in the other image.
![Page 4: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/4.jpg)
4
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
Then, we can calculate the best coordinate transformfrom one image to the other image, and blend them.
![Page 5: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/5.jpg)
5
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
How do we find these features?
![Page 6: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/6.jpg)
6
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007* “Good” being defined as relatively robust to change in lighting, viewpoint, or occlusions.
How do we find these features automatically? SIFT!SIFT finds good* features, consisting of:
An X, Y location A scale An orientation A Feature vector; a 128 bin Histogram of gradient values used to identify and match features between images.
![Page 7: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/7.jpg)
7
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
SIFT looks for extrema in Difference of Gaussian filtered versions of an image. This computation is done for many image sizes, or
Octaves, and with a variety of different strength blurs, or Scales.
Blurred Version:
More Blurred Version:
-
Calculating these Difference of Gaussian imagesin hardware was the focus of our project.
![Page 8: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/8.jpg)
8
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
High level architecture:●Input Multiplexer●Blur Units●Down-sample feedback●Difference Units
![Page 9: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/9.jpg)
9
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
Gaussian blur is achieved through Convolution
Hardware:● Shift Register● Multiply● Accumulator
![Page 10: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/10.jpg)
10
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
![Page 11: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/11.jpg)
11
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
Architecture decisions:● Self contained blur units
● Separable Convolution
● Separate Image Buffers
Implications:● Lots of Memory (~1MB)
● Parallel execution
● Distributed control
![Page 12: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/12.jpg)
12
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
![Page 13: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/13.jpg)
13
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
Memory Management● 6 memory units in design
● Address Generation
● Down-sampling
● Extra pixel reads
Hardware:● 43 generated RAMs per unit
● Write / Read timing
● RAM generator
![Page 14: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/14.jpg)
14
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
Size:
w/o MEM: 1058500 um^2w/ MEM: ~10000000 um^2
Speed (w/o MEM):
Clock: 155 MHzCycles per image: 750,000Images per Sec: 200
MEMORY
LOG
IC
![Page 15: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/15.jpg)
15
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
![Page 16: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/16.jpg)
16
Difference of Gaussian ScaleSpace PyramidsDifference of Gaussian ScaleSpace Pyramids
Chris Murphy and Ballard Blair – 6.375 – May 2007Chris Murphy and Ballard Blair – 6.375 – May 2007
Error
![Page 17: Difference of Gaussian ScaleSpace Pyramids for SIFT ...csg.csail.mit.edu/6.375/6_375_2007_www/projects/group5_final_presentation.pdfChris Murphy and Ballard Blair – 6.375 – May](https://reader033.vdocuments.net/reader033/viewer/2022050502/5f944ff0e43a7654e7441e65/html5/thumbnails/17.jpg)
17
Difference of Gaussian ScaleSpace Difference of Gaussian ScaleSpace Pyramids for SIFTPyramids for SIFT[1][1] Feature Detection Feature Detection
Chris MurphyChris MurphyBallard BlairBallard Blair 6.3756.375[1] Lowe, David G.
Distinctive Image Features from Scale-Invariant Keypoints,International Journal of Computer Vision, 2004.