tattoo based identification: sketch to image...
TRANSCRIPT
![Page 1: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/1.jpg)
Tattoo Based Identification: Sketch to Image Matching
Hu Han and Anil K. JainDept. of Computer Science & Eng.
Michigan State Universityhttp://www.cse.msu.edu/biometrics/
1
ICB‐20136th IAPR International Conference on Biometrics
Technical Co-Sponsors
![Page 2: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/2.jpg)
Outline
• Background• Proposed approach• Tattoo sketch‐image database• Experimental results• Summary and future work
2
![Page 3: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/3.jpg)
Background
• Harris Interactive: One in five (21%) U.S. adults have at least one tattoo[1]
– Most gang members have tattoos in their bodies
3
Some examples of gang tattoos (http://www.gangink.com/index.php?pr=GANG_LIST)
[1] Harris Interactive: http://www.harrisinteractive.com/NewsRoom/HarrisPolls/tabid/447/mid/1508/articleId/970/ctl/ReadCustom%20Default/Default.aspx
![Page 4: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/4.jpg)
Background
• Tattoos on human body provide important clues to the identity of a suspect
4
Tattoos assist in arrest of suspects[1,2] ![1] http://www.tampabay.com/news/publicsafety/crime/article1249486.ece[2] http://portcitydaily.com/2012/09/26/tattoos‐aid‐in‐arrest/
![Page 5: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/5.jpg)
Background
• Tattoo image‐to‐image matching [Jain’07, Acton and Rossi'08, Lee’12]
5
The query image is a “near duplicate” of the true tattooimage if present in the database
![Page 6: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/6.jpg)
Background
• Tattoo image of the suspects may not be available ‐ Draw a tattoo sketch following the verbal description of the eyewitness
6Tattoo and face sketches of a suspect who attempted an abduction of a 13‐year‐old girl,released by the Royal Canadian Mounted Police (RCMP) in Chilliwack in January 2011
![Page 7: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/7.jpg)
Tattoo sketch to image matching
• Challenges
7
Tattoo sketch Tattoo image(i) Different modalities (sketch vs.photo)
(ii) Inaccurate description ordrawing of a tattoo sketch w.r.t.suspect’s true tattoo image
(iii) Deformations between tattoosketch & image
![Page 8: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/8.jpg)
Proposed Approach
• Motivation– Deformations between holistic tattoo patterns
8
![Page 9: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/9.jpg)
Proposed Approach
• Motivation– Similarities between local patterns
9
![Page 10: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/10.jpg)
Proposed Approach
• Detecting local patterns from tattoo shape (Canny edge detector) images using SIFT
10
![Page 11: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/11.jpg)
Proposed Approach
• Some tattoo shape extractions may not be good– We should also use the tattoo appearance image
11
Tattooimage
Local patterns detected inshape image
Local patterns detected in
appearance image
![Page 12: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/12.jpg)
Proposed Approach
• The larger the number of common local patterns between tattoo sketch andtattoo image, the higher their similarity– For each local pattern: SRC
12
+ e
Dictionary
Sparse coefficients
= X
![Page 13: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/13.jpg)
Tattoo sketch‐image database
13
• There is no operational tattoosketch data set that we could findfrom law enforcement agencies
• we construct a data set consistingof 100 tattoo sketches drawn bytwo different subjects, eachsketch corresponding to a tattooimage
• 10,000 tattoo images provided bythe Michigan State Police are usedto populate the gallery– Probe: 100 tattoo sketches– Gallery: 10,100 tattoo images
We have released this database in our website:http://biometrics.cse.msu.edu/pubs/databases.html
![Page 14: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/14.jpg)
Experimental results
• Robustness to deformations
14
![Page 15: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/15.jpg)
Experimental results
• Robustness to deformations
15
Rotation Shear Twirl
![Page 16: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/16.jpg)
Experimental results
• Local pattern detection from shape images vs. appearance images
16
50 100 150 2000
10
20
30
40
50
60
Rank
Iden
tific
atio
n R
ate
(%)
Local pattern from shapeLocal pattern from appearance
![Page 17: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/17.jpg)
Experimental results
• Comparison with a state‐of‐the‐art matcher –Tattoo‐ID [Jain’07, Lee’12]
17A. K. Jain, J.E. Lee, R. Jin: Tattoo‐ID: Automatic Tattoo Image Retrieval for Suspect and Victim Identification. PCM 2007.J.E. Lee, W. Tong, R. Jin, and A. K. Jain, "Image Retrieval in Forensics: Tattoo Image Database Application", IEEE Multimedia, Vol. 19, No. 1, pp. 40‐49, 2012.
0 50 100 150 2000
10
20
30
40
50
60
Rank
Iden
tific
atio
n R
ate
(%)
ProposedTattoo-ID [Jain'07, Lee'12]
![Page 18: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/18.jpg)
Experimental results
• Examples of tattoo sketch to image matching
18
![Page 19: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/19.jpg)
Experimental results
• Examples of tattoo sketch to image matching
19
![Page 20: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/20.jpg)
Experimental results
• Computation cost– Tattoo‐ID: 30 Secs. per retrieval with C++ implementation
– Proposed: 55 Secs. per retrieval with Matlabimplementation
20
Testing platform: WIN7, Intel Core2 2.4G, 4G RAMGallery size: 10,100 tattoo images
![Page 21: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/21.jpg)
Summary and future work
• Summary– We constructed a tattoo sketch database, and proposed a scheme for matching tattoo sketches to tattoo images
– Our method outperforms a state‐of‐the‐art matcher• Future work
– Increase the size of tattoo sketch database– Improve the performance by integrating human annotations of tattoos, and by introducing structural constraints between various local patterns
21
![Page 22: Tattoo Based Identification: Sketch to Image Matchingbiometrics.cse.msu.edu/Presentations/HuHan_ICB13...Tattoo sketch‐image database 13 • There is no operational tattoo sketch](https://reader035.vdocuments.net/reader035/viewer/2022063023/5ffebd1e74d839666576e541/html5/thumbnails/22.jpg)
Thank you!
Hu Han and Anil K. JainDept. of Computer Science & Eng.
Michigan State Universityhttp://www.cse.msu.edu/biometrics/
22