Strengthening the Science in Forensic Science: An Update on Research and Development

5th Annual Prescription for Criminal Justice Forensics (June 6, 2014)

Gerry LaPorteActing DirectorNational Institute of JusticeOffice of Investigative and Forensic Sciences

What has been happening since 2009?

13 RecommendationsRecommendation #3• Research is needed to address issues of accuracy,

reliability, and validity in the forensic science disciplines.

Recommendation #5• Encourage research programs on human observer

bias and sources of human error in forensic examinations.

Focus of Discussion: Impression and Pattern Evidence Disciplines• Friction Ridge Analysis (Latent Prints) • Firearm & Tool Mark Examinations (FTE)• Shoeprint and Tire Tread Evidence• Questioned Documents/Handwriting Examinations• Bloodstain Pattern Analysis (BPA)

This discussion does not focus on Bite Mark Evidence, Detector Dog Teams, Forensic Audio and Video Analysis, or Microscopic Hair Analysis

What is your perception of forensic science R&D in the areas of impression and pattern evidence? What ‘grade’ would you give the efforts?

• Grade: F – There has been insufficient progress since 2009 to respond to the research and development recommendations in the NAS Report.

• Grade: C – There has been some progress since 2009. There are still many unanswered questions with respect to accuracy, reliability, and error rates.

• Grade: B – There has been significant progress since 2009. There have been multiple studies to address accuracy, reliability, and error rates.

• Grade: A - There has been overwhelming progress since 2009. There is no longer a need for significant investment in additional strategies to address accuracy, reliability, and error rates.

Lack of Commitment to R&D Will Impede Knowledge and Innovation

If the law has made you a witness, remain a man of science. You have no victim to avenge, no guilty or innocent person to convict or save — you must bear testimony within the limits of science.1

1. Dr. P.C.H. Brouardel, 19th Century French Medico-legalism.

DOJ Issues Scientific and Research Integrity Policy

http://www.justice.gov/open/doj-scientific-integrity-policy.pdf

DOJ Asserts NIJ Independence

“Across-the-board culture of scientific, technological and research validity, reliability, accuracy, objectivity

and integrity”

“Director of the National Institute of Justice shall have final authority over all grants”

“NIJ retains control over the timing and content of research reports”

NIJ REACTS IMMEDIATELY TO THE 2009 NAS REPORT WITH RESPECT TO THE CALL FOR MORE SCIENTIFIC RESEARCH

NIJ Issues ‘First of its Kind’ Solicitation Immediately Following the NAS Report

NIJ Establishes the Office of Investigative and Forensic Sciences (OIFS) in 2010• The mission of OIFS is to improve the quality and

practice of forensic science through innovative solutions that support research and development, testing and evaluation, technology, information exchange and the development of training resources for the criminal justice community.

• Scientific staff include physical scientists with varying backgrounds and experience

NIJ Leadership: Establishing NIJ as the Nation's Leader in Scientific Research on Crime and Justice

• “NIJ Director John H. Laub, Ph.D., and his long-time research partner, Robert J. Sampson, Ph.D., of Harvard University, are joint recipients of the 2011 Stockholm Prize in Criminology”

• “Upon the departure of Director John H. Laub, Deputy Director Greg Ridgeway, who received his doctorate in statistics, became Acting Director”

NIJ Has a Substantial Investment in Forensic Science R&D

• Since 2009, NIJ has– Provided more than $100 million in R&D awards– Launching more than 250 R&D projects– Producing more than 600 scientific publications,

presentations, and final technical reports

FY 2011-2013 NIJ R&D Solicitations

Applied Research & Development Basic Research

NIJ Collaborates with NIST by Providing More than $19 Million since 2009• 2009 to 2014 - NIST’s Applied Genetics Human Identity Team • 2009 to 2014 - NIST’s Forensic Toolmark Analysis Project

Team • 2011 - NIST conducted a review and evaluation of the

accuracy of commercially available photo scales • 2012 - Expert Working Group on Human Factors in Latent

Print Analysis • 2013 - Latent Print AFIS Interoperability Working Group • 2013 - Working Group for Biological Evidence Preservation• 2013 - Working Group on Forensic Science Laboratories• Present - Working Group on Presenting Forensic Science

Evidence Using Quantitative and Qualitative Terms

Overall Distribution of Funds Between Applied and Basic/Fundamental

67%

33%

Overall NIJ Distribution of Funding: Applied vs. Basic/Fundamental

Applied Basic/Fundamental

Forensic Science R&D Funding for FY2009-2013

Discipline Funding Awarded % Awards

Anthropology $4,541,571 4.49% 19

Controlled Substances $2,659,868 2.63% 9

Crime Scene Investigation $4,229,401 4.18% 14

Digital Forensics[1] $2,821,191 2.79% 5

Forensic Entomology $483,323 0.48% 1

Fire and Arson Investigation$2,848,270 2.82% 7

Forensic DNA $35,960,168 35.56% 74

Friction Ridge $8,543,027 8.45% 20

General Forensics[2] $2,170,237 2.15% 6

Impression Evidence $10,936,182 10.81% 28

Forensic Pathology $6,912,372 6.83% 12

Questioned Documents $1,989,618 1.97% 5

Forensic Toxicology $5,345,849 5.29% 14

Trace Evidence $11,697,765 11.57% 37

$101,138,842100.00% 251

Overall NIJ Distribution of Funding: Forensic Discipline Categories

AnthropologyControlled SubstancesCrime Scene InvestigationDigital ForensicsForensic EntomologyFire and Arson InvestigationForensic DNAFriction RidgeGeneral ForensicsImpression EvidenceForensic PathologyQuestioned DocumentsForensic ToxicologyTrace Evidence

Office of Science and Technology Policy (OSTP) Releases Report in February 2014

http://www.whitehouse.gov/sites/default/files/microsites/ostp/forensicscience_progressreport_feb-2014.pdf

Snapshot of Other Efforts• From 2011 through 2014, the Annual Meeting of the

American Academy of Forensic Sciences (AAFS) has included over 4,000 oral/poster presentations and 137 workshops.

• From 2009-2013, the Journal of Forensic Identification (JFI) has received 341 manuscripts and published 231 peer reviewed articles.

• From 2009-2013, the Association of Firearm and Tool Mark Examiners (AFTE) Journal has published 255 articles. The AFTE Journal was accepted to SCOPUS in 2013.

Other Journals Focused on Forensic Science R&D• Journal of Forensic Sciences• Forensic Science International• Journal of the Canadian Society of Forensic Science• Journal of Forensic and Legal Medicine• Journal of the American Society of Questioned

Document Examiners• Many open access online Journals

WHAT WAS HAPPENING PRE-2009?

Pre-2009?• NIJ R&D responded to the needs of the forensic

science community. i.e., NIJ can only support proposals submitted for consideration!

• Heavy investment in DNA R&D• Investments continued in innovation and new

technologies• Limited successful challenges to admissibility of

impression and pattern evidence – over a decade passed since Daubert v. Merrell Dow Pharmaceuticals (1993)

Pre-2009?• No evidence that impression and pattern evidence

disciplines were not accurate and reliable: Karl Popper philosophy – science must be falsifiable.

All swans are white ... until you see a black swan• Research was being conducted throughout

laboratories for years, but submitting manuscripts for peer review/publication was considered burdensome – there is anecdotal evidence of a culture change.

NIJ Research is Based on the Needs of the Forensic Science Community: Use ofTechnology Working Groups (TWGs)• DNA Forensics

DNA & Forensic Biology

• General ForensicsCrime Scene & Medicolegal Death Investigation Controlled Substances & Toxicology Impression and Pattern Evidence Instrumental Analysis/Chemistry

NIJ Research is Based on the Needs of the Forensic Science Community

http://nij.gov/topics/forensics/Documents/fy14-forensic-twg-table.pdf

Fiscal Year

Total Investment*

Number of Projects*

Average Award Size

1995 $378,294.00 4 $94,573.501996 $715,857.00 5 $143,171.401997 $1,495,436.00 6 $249,239.331998 $1,312,240.00 5 $262,448.001999 $3,112,895.00 10 $311,289.502000 $3,953,664.00 7 $564,809.142001 $2,979,398.00 5 $595,879.602002 $3,948,253.00 10 $394,825.302003 $4,256,720.00 11 $386,974.552004 $5,258,507.00 13 $404,500.54

2005 $10,246,365.00 17 $602,727.35

2006 $2,164,109.00 9 $240,456.562007 $4,048,563.00 10 $404,856.302008 $8,377,722.00 17 $492,807.182009 $5,894,294.00 18 $451,868.772010 $9,370,401.00 18 $520,577.832011 $5,592,829.00 14 $399,487.792012 $4,947,153.00 13 $380,550.23

19951996

19971998

19992000

20012002

20032004

20052006

20072008

20092010

20112012

$0

$2,000,000

$4,000,000

$6,000,000

$8,000,000

$10,000,000

$12,000,000

R&D Investment in Forensic Biology/DNA

1985Discovery of VNTR probes

1st paper on PCR

201319911st paper on STR

2005DNA Fingerprint Act

2000Backlog

Elimination Act

1998FBI Launches CODIS

2004DNA Initiative Started1999

NIJ Report -- FS: Review of Status and Needs

1997NIJ/NIST/ASCLD: Forensic

Science Summit

Investment in DNA Research and Development

Number of Wrongful Convictions Attributed to Unvalidated/Improper Forensic Science

1970 1975 1980 1985 1990 1995 2000 2005 20100

2

4

6

8

10

12

14

16

18

20

Lack of evidence that invalid impression and pattern evidence science is a root cause to wrongful convictions• Not a single exoneration has been attributed to the

accuracy and reliability of a method* with respect to:• Friction Ridge Analysis• Firearm & Tool Mark Examinations • Shoeprint and Tire Tread Evidence• Questioned Documents/Handwriting Examinations• Bloodstain Pattern AnalysisNote: Negligent analysis, fabrications/alterations of reports, withholding data, exaggerated testimony, and individual examiner errors are not the basis of unvalidated science.

Laboratory Operations: Perception that there are problems with the foundational science as it relates to accuracy and reliability (Not True)

EXAMPLES OF RESEARCH

NAS #3: Research to address issues of accuracy, reliability, and validity in the forensic science disciplines

Improving the Understanding and the Reliability of "Sufficiency" in Friction Ridge Examination• Pennsylvania State University• Award: 2010-DN-BX-K267

– This study aimed to understand how latent print examiners determine the value of a latent print by testing the ability of practitioners to annotate the quality, number, and spatial relationships of features in experimental latent print trials.

– Main empirical findings show that examiner decision-making is related to the number and spatial relationships of fingerprint minutiae, and factors like experience and workload has no clear relationship with the reliability of examiners’ conclusions.

NOBLIS-FBI Study: False Positive Rate of 0.1%

http://www.pnas.org/content/early/2011/04/18/1018707108.full.pdf

TWG Need: Establishment of best practices in trace evidence analysis to improve efficienciesSignificance of Elemental Analysis from Trace Evidence• Florida International University• Award: 2009-DN-BX-K252

Glass is one of the most common types of trace evidence found in case

scenarios, such as hit-and-run accidents, burglaries,

kidnappings, homicides and shootings.

The goal of this research was to develop analytical protocols and to assess the utility of

glass source comparisons by way of several interlaboratory studies conducted by 31 forensic examiners representing 23 different laboratories

in the United States, Mexico, Canada, and Germany.

Link: https://ncjrs.gov/pdffiles1/nij/grants/242325.pdf

NAS #3: Research to address issues of accuracy, reliability, and validity in the forensic science disciplines

Improve the Scientific Foundation of Forensic Firearm and Toolmark Identification• Miami-Dade Police Department Crime Laboratory• Award: 2009-DN-BX-K230

– This study analyzed the repeatability and uniqueness of striations and impressions on cartridge cases fired in 10 consecutively manufactured slides by evaluating the ability of firearms and tool mark examiners to correctly identify test fired cartridge casings fired from the same slides at different intervals (i.e., the identification of same gun evidence).

Link: https://www.ncjrs.gov/pdffiles1/nij/grants/237960.pdf

Just Released

Using Quantitative Image Comparisons for Estimating Error Rates• The model developed in the paper was able to

account for 64% of the variation in print comparison accuracy on a novel set of fingerprint images. “While some of the predictors we found likely comport with what fingerprint examiners would have intuited, being able to demonstrate their role scientifically, and measuring, rather than just assuming, their importance is a significant step forward,” Mnookin stated.

Summary of Forensic Science Research and Development Awards

http://www.nij.gov/topics/forensics/exhibits/pages/forensic-awards.aspx

Examples of Latent Print R&D

Examples of Impression and Pattern

Gerry LaPorte, M.S.F.S.Acting DirectorNational Institute of JusticeOffice of Investigative & Forensic SciencesGerald.Laporte@usdoj.gov

Important Links:

Forensic Science R&D Awards: http://nij.gov/topics/forensics/pages/forensic-awards.aspx

Fiscal Year 2012 Funding for DNA Analysis, Crime Laboratory Capacity Enhancement and Other Forensic Activities:http://ncjrs.gov/pdffiles1/nij/244196.pdf