USING CORRECTION FILES, USING CORRECTION FILES, AUDIT REPORTS, AND AUDIT REPORTS, AND PROFICIENCY TESTS.PROFICIENCY TESTS.

Jennifer FriedmanJennifer Friedman

Office of the Los Angeles County Public Office of the Los Angeles County Public DefenderDefender

213-974-2922213-974-2922

jfriedma@co.la.ca.usjfriedma@co.la.ca.us

Dayton August 22, 2004Dayton August 22, 2004

TOPICS TO BE DISCUSSEDTOPICS TO BE DISCUSSED

• What are correction files, audit What are correction files, audit reports and proficiency tests?reports and proficiency tests?

• How do we obtain them?How do we obtain them?

• Once we obtain them, what do we Once we obtain them, what do we do with them?do with them?

Correction FilesCorrection Files

Alternative NamesAlternative Names

• Anomalous Result Anomalous Result LogsLogs

• Error LogsError Logs• Contamination Logs Contamination Logs • Logs of Unexpected Logs of Unexpected

ResultsResults• Variance ReportsVariance Reports• Corrective Action Corrective Action

FilesFiles

• Report of Quality Report of Quality ProblemsProblems

• Quality Control Quality Control DiscrepanciesDiscrepancies

• Instances of Instances of ContaminationContamination

• Control Control Discrepancy LogsDiscrepancy Logs

Quality Assurance Standards for Quality Assurance Standards for Forensic DNA Testing LaboratoriesForensic DNA Testing Laboratories

• Standard 14.1-Standard 14.1-

““The laboratory shall establish and The laboratory shall establish and follow procedures for corrective action follow procedures for corrective action whenever proficiency testing whenever proficiency testing discrepancies and/or case work errors discrepancies and/or case work errors are detected.”are detected.”

www.fbi.gov/programs/labwww.fbi.gov/programs/lab

Corrective Action Corrective Action DocumentationDocumentation

• Standard 14.1.1-Standard 14.1.1-

““The laboratory shall maintain The laboratory shall maintain documentation for the corrective documentation for the corrective action. Such documentation shall be action. Such documentation shall be retained in accordance with retained in accordance with applicable federal or state law.”applicable federal or state law.”

Example #1Example #1

• Description of Problem: Analyst’s Description of Problem: Analyst’s profile observed in profiler plus profile observed in profiler plus sample of subject reference.sample of subject reference.

• Change in Procedure: Better care Change in Procedure: Better care must be taken in working with must be taken in working with samples. When working with samples. When working with samples, masks and/or bio-safety samples, masks and/or bio-safety cabinet must be used.cabinet must be used.

Example #2Example #2

• Description of Problem: Apparent amount Description of Problem: Apparent amount of reference FID-11 contamination in of reference FID-11 contamination in referenceFID-12. Samples was re-amplified referenceFID-12. Samples was re-amplified and contamination was still present.and contamination was still present.

• Re-extracted reference samples. No Re-extracted reference samples. No contamination present.contamination present.

• Change in Procedure: Contamination must Change in Procedure: Contamination must have occurred at extraction or slot blot. have occurred at extraction or slot blot. Analyst will make sure to use new pipette Analyst will make sure to use new pipette with every sample.with every sample.

Audit ReportsAudit Reports

• American Society of Crime Lab American Society of Crime Lab Directors- Laboratory Assurance Directors- Laboratory Assurance Board (ASCLD-LAB)Board (ASCLD-LAB)

• FBI DNA Quality Assurance Audit –FBI DNA Quality Assurance Audit –

External AuditExternal Audit

• Internal AuditsInternal Audits

ASCLD-LAB AuditASCLD-LAB Audit

• Management and OperationManagement and Operation

• Personnel StandardsPersonnel Standards

• Physical Plant StandardsPhysical Plant Standards

ASCLD-LAB Grading SystemASCLD-LAB Grading System

• EssentialEssential

• Important Important

• DesirableDesirable

A laboratory must achieve not less A laboratory must achieve not less than 100% of the essential, 75% of the than 100% of the essential, 75% of the important and 50% of the desirable important and 50% of the desirable criteria.criteria.

Examples of “Essential”Examples of “Essential”

• Written procedures Written procedures for handling for handling evidenceevidence

• Written or Written or electronic chain of electronic chain of custody custody documentationdocumentation

• Evidence protected Evidence protected from loss, cross from loss, cross transfer, transfer, contamination.contamination.

• Secure over night Secure over night and long term and long term storagestorage

• Comprehensive Comprehensive quality assurance quality assurance manualmanual

• Annual internal audit Annual internal audit with documentationwith documentation

• Routine checks of Routine checks of reliability of reliability of reagentsreagents

Examples of “Important”Examples of “Important”

• Instruments/Instruments/equipment in proper equipment in proper working orderworking order

• Adequate work Adequate work spacespace

• Proper ventilationProper ventilation

• Supervisors review Supervisors review lab activities and lab activities and personnelpersonnel

• Instruments/ Instruments/ equipment equipment adequate for the adequate for the procedures usedprocedures used

• Internal blind Internal blind proficiency testingproficiency testing

• Adequate space for Adequate space for instrumentation instrumentation

Examples of “Desirable”Examples of “Desirable”

• Sufficient space for Sufficient space for storage of suppliesstorage of supplies

• General cleanliness General cleanliness and “good and “good housekeeping”housekeeping”

• Lab staff Lab staff understand and understand and support objectivessupport objectives

• Written procedures Written procedures for control of for control of supplies and supplies and equipmentequipment

• Procedures for Procedures for inventory of supplies inventory of supplies and equipmentand equipment

• Constructive Constructive discussion between discussion between supervisors and supervisors and subordinatessubordinates

Preliminary Audit ReportPreliminary Audit Report

• ““Areas for trace, serology and DNA Areas for trace, serology and DNA are totally inadequate….Laboratories are totally inadequate….Laboratories in which useable space falls below in which useable space falls below adequate levels may adequate levels may experience…..an increased risk of experience…..an increased risk of mishandled or contaminated mishandled or contaminated evidence.”evidence.”

Final Audit ReportFinal Audit Report

• ““No change, if anything, the room is No change, if anything, the room is even more crowded than even more crowded than previously…”previously…”

• Improvement- “The post PCR room is Improvement- “The post PCR room is now dedicated to this purpose. The now dedicated to this purpose. The laundry washer and dryer have been laundry washer and dryer have been relocated.”relocated.”

Proficiency TestsProficiency Tests

• Internal openInternal open

• Internal blindInternal blind

• External openExternal open

• External blindExternal blind

ObservationsObservations

• Deviating from rfu Deviating from rfu cutoffcutoff

• Changing calls after Changing calls after “second read”“second read”

• Notes indicating Notes indicating that 2 test tubes that 2 test tubes were both labeled were both labeled “item #1”“item #1”

• Deviation from SOPDeviation from SOP

• Fabric had stain on Fabric had stain on both sides, “called both sides, “called Heather to find out Heather to find out if they were trying if they were trying to trick him”to trick him”

• Mistakes in Mistakes in reporting the resultsreporting the results

• Neg/Pos control Neg/Pos control problemsproblems

How Do We Obtain These How Do We Obtain These Documents?Documents?

Case AuthorityCase Authority

Brady v. MarylandBrady v. Maryland, (1963) 373 U.S. 83, (1963) 373 U.S. 83Kyles v. WhitleyKyles v. Whitley, (1995) 514 U.S. 419, (1995) 514 U.S. 419United States v. BagleyUnited States v. Bagley, (1985) 473 , (1985) 473

U.S. 667U.S. 667

The prosecution must disclose to the defense any The prosecution must disclose to the defense any evidence that is both favorable to the defendant evidence that is both favorable to the defendant and material either to guilt or to punishmentand material either to guilt or to punishment. . Evidence that is favorable to the defendant Evidence that is favorable to the defendant encompasses impeachment evidence as well as encompasses impeachment evidence as well as exculpatory evidenceexculpatory evidence . .

Case Authority-Proficiency Case Authority-Proficiency TestsTests• State v. ProctorState v. Proctor, (S.C.App.2001), 559 S.E.2d , (S.C.App.2001), 559 S.E.2d

318318

overruled on other grounds.overruled on other grounds.

““We conclude the trial court erred in denying We conclude the trial court erred in denying Proctor’s motion to have proficiency test Proctor’s motion to have proficiency test records of the DNA expert disclosed.” Courts records of the DNA expert disclosed.” Courts and defense are not required to accept self-and defense are not required to accept self-serving assertions. A DNA expert like all other serving assertions. A DNA expert like all other experts is subject to “scrutiny and query in experts is subject to “scrutiny and query in regard to qualifications and competency.”regard to qualifications and competency.”

Proficiency Tests ContinuedProficiency Tests Continued

Keen v. CommonwealthKeen v. Commonwealth, (1997) 24 Va. , (1997) 24 Va. App. 795, 485 S.E.2d 659App. 795, 485 S.E.2d 659

D entitled not only to results of particular D entitled not only to results of particular analyst but also to collaborative testing id analyst but also to collaborative testing id number for lab in order to determine number for lab in order to determine whether anyone in the lab had failed a whether anyone in the lab had failed a proficiency testproficiency test. . Note this was the Virginia Note this was the Virginia State crime lab.State crime lab.

Case Authority- Error Logs and Case Authority- Error Logs and Audit ReportsAudit Reports

• United States v. Cedano-ArellanoUnited States v. Cedano-Arellano, , (2003) 332 F.3d 586(2003) 332 F.3d 586

Error to deny “a broad range of Error to deny “a broad range of materials” including handler’s log, all materials” including handler’s log, all training records and score sheets, training records and score sheets, certification records, training certification records, training standards and manuals.standards and manuals.

Character Trait EvidenceCharacter Trait Evidence

• In some jurisdictions, evidence in the In some jurisdictions, evidence in the form of opinion, reputation or specific form of opinion, reputation or specific instances of conduct may be admissible instances of conduct may be admissible to show that an individual was acting in to show that an individual was acting in conformity with the trait or conduct.conformity with the trait or conduct.

• Habit and custom evidence may also be Habit and custom evidence may also be admissible to prove an individual acted admissible to prove an individual acted in conformity with his habit or custom.in conformity with his habit or custom.

State Supreme Court RulesState Supreme Court Rules

• Illinois Supreme Court Rule 417 Illinois Supreme Court Rule 417 requires disclosure of all relevant requires disclosure of all relevant materials pertaining to DNA materials pertaining to DNA including, “all reports, memoranda, including, “all reports, memoranda, notes, phone logs, notes, phone logs, contamination contamination recordsrecords, and data related to the , and data related to the testing performed in the case.”testing performed in the case.”

State and Federal Freedom State and Federal Freedom of Information Act of Information Act

RequestsRequests

PrivilegePrivilege

• Official information. Acquired in Official information. Acquired in confidence.confidence.

• Public interest in not disclosing Public interest in not disclosing outweighs disclosure.outweighs disclosure.

Public Interest Public Interest

• Release of crime lab’s audit reports to the Release of crime lab’s audit reports to the public defender, who will use the report public defender, who will use the report to attempt to exploit any deficiency the to attempt to exploit any deficiency the lab may have made, may have a chilling lab may have made, may have a chilling effect on the candor and thoroughness of effect on the candor and thoroughness of the participants, and the the participants, and the lab’s lab’s corresponding ability to maintain its corresponding ability to maintain its high level of accuracy and continued high level of accuracy and continued improvement in the future.”improvement in the future.”

But……..But……..

• As Barry Fisher, Director of the Sheriff’s Scientific Services Bureau wrote in the prologue to his book Techniques of Crime Scene Investigation, "A better‑informed defense bar will also raise questions about the quality of forensic science laboratories and their forensic practitioners. One way to evaluate a laboratory’s quality is to inquire if it is accredited....[I]t is only a matter of time before accreditation will be a defacto requirement for evidence admissibility." Barry A.J. Fisher, Techniques of Crime Scene Investigation, CRC Press 7th Ed., (2004). Mr. Fisher also states, "[l]aboratory records must be open for reasonable access when legitimate requests are made by officers of the court." Id. at 19.

Additional SourcesAdditional Sources

• www.collaborativetesting.comwww.collaborativetesting.com• www.chron.com/cs/CDA/ssistory.mwww.chron.com/cs/CDA/ssistory.m

pl/topstory/2470016pl/topstory/2470016Media taking the lead in publishing Media taking the lead in publishing contents of audit reports.contents of audit reports.

• www.darwin.bio.uci.edu/~mueller/www.darwin.bio.uci.edu/~mueller/Information regarding errors in Information regarding errors in proficiency tests.proficiency tests.

• www.scientific.orgwww.scientific.org

What Do We Do With What Do We Do With Them Once We Have Them Once We Have

Them?Them?

Pretrial Pretrial Daubert/FryeDaubert/FryeAudit reports may demonstrate that a Audit reports may demonstrate that a laboratory has failed to adhere to generally laboratory has failed to adhere to generally accepted standards.accepted standards.Proficiency tests may demonstrate that a Proficiency tests may demonstrate that a particular analyst fails to follow generally particular analyst fails to follow generally accepted procedures.accepted procedures.Corrective action files, audit reports and Corrective action files, audit reports and proficiency tests may also demonstrate a proficiency tests may also demonstrate a failure to adhere to procedures set out in a failure to adhere to procedures set out in a standard operating procedures manual. standard operating procedures manual. See United States v. SheaSee United States v. Shea, (1997 E. District , (1997 E. District New Hampshire) 957 F. Supp. 331, 340 n.27New Hampshire) 957 F. Supp. 331, 340 n.27

Trial: Case Law Supporting Trial: Case Law Supporting of Admissibility of Audit of Admissibility of Audit ReportsReports• Error to preclude cross-examination of Error to preclude cross-examination of

analyst regarding protocol (required analyst regarding protocol (required by ASCLD/FBI audits). Relevant to by ASCLD/FBI audits). Relevant to weight and credibility of the evidence. weight and credibility of the evidence. “Science is often accepted in our “Science is often accepted in our society as synonymous with truth.”society as synonymous with truth.”

State v.LehrState v.Lehr, (Az. 2002) 38 P.3d 1172, , (Az. 2002) 38 P.3d 1172, 1180-11811180-1181

Proficiency Test Results Proficiency Test Results and Underlying Data are and Underlying Data are

Relevant to Dispel the Relevant to Dispel the “Myth” of the Zero Error “Myth” of the Zero Error

Rate.Rate.

Case Law Supporting Case Law Supporting Admissibility of Proficiency Admissibility of Proficiency TestsTests

Failed proficiency tests and/or errors or Failed proficiency tests and/or errors or deviations from protocols on such tests deviations from protocols on such tests are arguably relevant to lab error rates.are arguably relevant to lab error rates.DNA test too flawed to be admitted where DNA test too flawed to be admitted where jury was denied the opportunity to hear jury was denied the opportunity to hear testimony regarding an attempt to testimony regarding an attempt to influence a defense expert, disagreement influence a defense expert, disagreement between analyst and supervisor regarding between analyst and supervisor regarding interpretation and inadequate interpretation and inadequate documentation.documentation.Murray v. StateMurray v. State, (Fla. 2002) 838 So.2d , (Fla. 2002) 838 So.2d 1073, 10811073, 1081

Case Law Supporting Case Law Supporting Admissibility Corrective Action Admissibility Corrective Action InformationInformation

• It was error to preclude the defense from It was error to preclude the defense from cross-examining the expert concerning cross-examining the expert concerning incidents of contamination within the lab. “As incidents of contamination within the lab. “As a general matter, when DNA evidence is a general matter, when DNA evidence is admitted against an accused in a criminal trial, admitted against an accused in a criminal trial, questions on cross-examination regarding how questions on cross-examination regarding how that specific DNA evidence was obtained, and that specific DNA evidence was obtained, and the the laboratory conditions under which the laboratory conditions under which the DNA tests in that case were conductedDNA tests in that case were conducted, , should be allowed.should be allowed.Williams v. StateWilliams v. State, (Md.1996), 679 A.2d 1106 , (Md.1996), 679 A.2d 1106 overruled on other grounds.overruled on other grounds.

Concluding RemarksConcluding Remarks

• Audit reports, corrective action files Audit reports, corrective action files and proficiency tests provide a wealth and proficiency tests provide a wealth of information about a laboratory and of information about a laboratory and the analysts who work in the lab. This the analysts who work in the lab. This information is essential to the effective information is essential to the effective cross-examination of the witnesses cross-examination of the witnesses who testify at admissibility hearings who testify at admissibility hearings and at trial.and at trial.

THE END.THE END.