Confocal Microscopy and Striated Tool Marks: A Statistical Study and

Potential Software Tools For Practitioners

3 2 1 0 1 2 3

Carol Gambino, Patrick McLaughlin, Loretta Kuo, Peter Diaczuk, Gerard Petillo, Frani Kammerman, Lauren Claytor, Peter Shenkin, Nicholas

Petraco, James Hamby and Nicholas D. K. Petraco

Outline• Introduction

• Details of Our Approach• Data acquisition• Confocal Microscopy for Surfaces

• Surface/Profile Pre-processing

• Results of Statistical Discrimination/Error Rate Estimates for Primer Shears

• Suggested Operating Guidelines

Introduction• DNA profiling the most successful application of

statistics in forensic science.• Responsible for current interest in “raising standards” of

other branches in forensics…??

• No protocols for the application of statistics to comparison of tool marks.• Our goal: application of objective, numerical pattern

comparison to tool marks

Caution: Statistics is not a panacea!!!!

• All impressions made by tools and firearms can be viewed as numerical patterns– Machine learning trains a computer to recognize

patterns • Can give “…the quantitative difference between an

identification and non-identification”Moran • Can yield identification error rate estimates

Background Information

• Obtain striation/impression patterns from 3D confocal microscopy

• Store files in ever expanding database• Data acquisition is labor

intensive and time consuming

• Data files will be made available to practitioner community through web interface

Data Acquisition

Increasing surface height

All-in-Focus 2D Image

Overlay confocal “z-stack”

Confocal Microscopy

• 3D confocal image of portion of chisel striation pattern

Glock 19 fired cartridge cases

Bottom ofFiring pin imp.

Glock Fired Cartridges

Five Consecutively Manufactured Chisels

Lead impression media

Striation patterns generated at 32o

70 striation patterns total:• 20 for traditional comparison• 50 for confocal microscopy

G. Petillo

G. Petillo

Craftsman Screwdrivers

Striation patterns in lead

Striation patterns in wax

Experimental Research Design

Extended ImageJ Functionality

2D profiles

3D surfaces(interactive)

Experimental Research Design

• Software can detect edges of significant “lines”:

• Or software can turn any profile into a “barcode”:

profile

barcode

Experimental Research Design

Website

• 3D surfaces– ImageJ visualization

• 2D and 3D

– ImageJ measurements

• R scripts/programs for statistical analysis

• Preprints of papers

Downloadable

• Form removal

• Register and optionally shift skewed profiles

• Use max CCF

• Optional filter surface into waviness and roughness components

• Cubic spline filter:

• Striation pattern processing: Hamby 4

Claytor 1

Primer shear from the same Glock 19

Primer Shear

Primer shear from two different Glock 19s

Research Results

Mean total profile:

Mean “waviness” profile:

Mean “roughness” profile:

Experimental Research Design

• Statistical pattern comparison!

• Modern algorithms are called machine learning

• Idea is to measure features that characterize physical evidence

• Train algorithm to recognize “major” differences between groups of featureswhile taking into account

natural variation and measurement error.

What Statistics Can Be Used?

• Primer shears (82-91 profiles)– PCA-SVM, CPT at the 95% level of confidence

• Empirical error rate was 4.7%• No “uninformative” intervals were returned

– PCA-SVM, HOO-CV• Error rate estimate is 0.0%-4.4%, depending on the number of replicates

– PLS-DA, Bootstrap (>10 replicates only)• 95% confidence interval for error rate: [0%, 0%]

– PLS-DA, HOO-CV• Error rate estimate is 0.0%-4.3%, depending on the number of replicates

• Results so far are on par with expectations• More samples are being prepared for analysis

Primer Shear

Preliminary suggested operating guidelines• Visualization is MOST important

– Trained examiner assessment– 3D microscopy and visualization

• For statistical analysis:– # of replicates VERY important. – Train “machine learning method” on suspect tool and

tools producing “similar marks” (close in data space)• SVM, PLS-DA

– Get I.D. error rate estimates from various methods• Large test sets, bootstrap, cross-validation

– Classify an unknown form a crime scene• Use CPT for a level of confidence for the “association”

References• Biasotti AA. A statistical study of the individual characteristics of fired bullets. J Forensic Sci

1959;4(1):34-50.

• Efron B, Tibshirani RJ. An introduction to the bootstrap. 1st ed. Boca Raton: Chapman & Hall/CRC, 1993.

• Geradts Z, Keijzer J, Keereweer I. A new approach to automatic comparison of striation marks. J Forensic Sci 1994;39(4):974-80.

• AFTE. Theory of Identification as it Relates to Toolmarks. AFTE J. 1998;30(1):89-8.

• Moran B. A report on the AFTE theory of identification and range of conclusions for tool mark identification and resulting approaches to casework. AFTE J 2002;34(2):227-35.

• Vovk V, Gammerman A, Shafer G. Algorithmic learning in a random world. 1 ed. New York: Springer, 2005.

• Neel M, Wells M. A comprehensive statistical analysis of striated tool mark examinations. Part 1: Comparing known matches to known non -matches. AFTE J 2007;39(3):176-98.

• Gammerman A, Vovk V. Hedging predictions in machine learning. The Computer J 2007;50(7):151-77.

References• Schafer G, Vovk V. A tutorial on conformal prediction. J Machine Learning Research 2008;9:371-

421.

• Howitt D, Tulleners F, Cebra K, Chen S. A calculation of the theoretical significance of matched bullets. J Forensic Sci 2008;53(4):868-75.

• Chumbley LS, Morris MD, Kreiser MJ, Fisher C, Craft J, Genalo LJ Davis S, Faden D, Kidd J. 2010. Validation of Tool Mark Comparisons Obtained Using a Quantitative, Comparative, Statistical Algorithm. J Forensic Sci 2010;55(4):953-961.

• Bachrach B, Jain A, Jung S, Koons RD. A statistical validation of the individuality and repeatability of striated tool marks: Screwdrivers and tongue and groove pliers. J Forensic Sci 2010;55(1):348-57.

• Chu W, Song J, Vorburger T, Yen J, Ballou S, Bachrach B. Pilot study of automated bullet signature identification based on topography measurements and correlations. J Forensic Sci 2010;55(2):341-7.

• Petraco N. Color Atlas of Forensic Toolmark Identification. 1st ed. Boca Raton: Chapman & Hall/CRC, 2010.

• Petraco NDK, Shenkin P, Speir J, Diaczuk P, Pizzola PA, Gambino C, Petraco N. Addressing the National Academy of Sciences’ Challenge: A Method for Statistical Pattern Comparison of Striated Tool Marks. J Forensic Sci 2011, (accepted).

• Gambino C, McLaughlin P, Kuo L, Kammerman F, Shenkin P, Diaczuk P, Petraco N, Hamby J, Petraco NDK. Forensic Surface Metrology: Tool Mark Evidence. Scanning, 2011 (accepted).

Acknowledgements

• Research Team:

• Mr. Peter Diaczuk

• Ms. Carol Gambino

• Dr. James Hamby

• Dr. Thomas Kubic, Esq.

• Off. Patrick McLaughlin

• Mr. Jerry Petillo

• Mr. Nicholas Petraco

• Dr. Peter A. Pizzola

• Dr. Jacqueline Speir

• Dr. Peter Shenkin

• Mr. Peter Tytell

• National Institute of Justice

• New York City Police Department Crime Lab

• John Jay College of Criminal Justice

• Ms. Alison Hartwell, Esq.

• Ms. Lauren Claytor

• Helen Chan

• Manny Chaparro

• Aurora Ghita

• Eric Gosslin

• Frani Kammerman

• Brooke Kammrath

• Loretta Kuo

• Dale Purcel

• Stephanie Pollut

• Rebecca Smith

• Elizabeth Willie

• Chris Singh

• Melodie Yu

• Greg Frasier

Website Information and Reprints/Preprints:

npetraco@gmail.com

npetraco@jjay.cuny.edu