validity, reliability, and quality assurance

Chapter 3

Validity, Reliability, andQuality Assurance

‘‘Science is one of the very few human activities in which errors are systematically criticized andfairly often, in time, corrected. ”

Sir Karl R. Popper1902-

“In most cases, reasonable prudence is in fact common prudence; but strictly it is never itsmeasure; a whole calling may have unduly lagged in the adoption of new and available devices. It nevermay set its own tests, however persuasive be its usages. Courts must in the end say what is required;there are precautions so imperative that even their universal disregard will not excuse their omission. ”

Justice HolmesThe T.J. Hooper, 60 F.3d 737 (2d Cir. 1932)

“The right to search for truth implies also a duty; one must not conceal any part of what one hasrecognized to be true. ’

Albert Einstein1879-1955

CONTENTSPage

VALIDITY AND RELIABILITY OF DNA TECHNOLOGIES . . . . . . . . . . . . . . . . . . . . . .Single-locus Probe Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Multilocus Probe Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Polymerase Chain Reaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

QUALITY CONTROL AND QUALITY ASSURANCE . . . . . . . . . . . . . . . . . . . . . . . . . . . . .The Role of Professional Societies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Nonregulatory Mechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .State Authority To Regulate Crime Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Federal Authority To Regulate Crime Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

SETTING STANDARDS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .STANDARDIZATION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .FINDINGS AND SUMMARY . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .CHAPTER PREFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

59606869717273758082838385

BoxesPage

3-A. Considerations for Choosing Forensic Single-locus Probes . . . . . . . . . . . . . . . . . . . . . . 623-B. Scientific Controls for RFLP Analysis. . . . . . . . . . . . . . . . ,......+ . . . . . . . . . . . . . . . . . 643-C. Statistics and RFLP Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673-D. Quality Assurance and Drug Testing Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 713-E. Quality Assurance and the FBI TechnicalWorking Group on

DNA Analysis Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 743-F. Negligence Litigation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 763-G. Quality Assurance and Clinical Laboratories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

FiguresPage

3-1. Example of One DNA Pattern in a Rape Case . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 603-2. DNA Typing in Two Paternity Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 13-3. DNA Typing and Murder: A Less Than Ideal First Analysis and a Solution.. . . . . . 65

Chapter 3

Validity, Reliability, and Quality Assurance

Given the variation in DNA sequence amongindividuals (see ch. 2), no scientific doubt exists thattechnologies available today accurately detect ge-netic differences. Properly performed and inter-preted, a sufficiently detailed examination of twosamples of DNA can determine if DNA patternsmatch, and, if they do, the likelihood that a singlesource is responsible for both samples (except in thecase of identical twins).

Nevertheless, it is generally agreed that applyingDNA tests to forensic samples, especially criminalevidence, potentially presents more difficulties thananalyzing samples in basic research or clinicaldiagnosis. Samples from crime scenes are frequentlysmall and might be of poor quality because ofexposure to a spectrum of environmental onslaughts.And unlike paternity samples, where each sample isfrom an identified source, the contributor to evi-dence taken from a crime scene is often unknown. Todate, several studies have elucidated ways to over-come some of the demands of using DNA typing onforensic samples. Other efforts are under way tofurther refine and develop strategies that adhere togenerally accepted practices in the scientific com-munity.

What, then, constitutes a sufficient examinationof a forensic sample? When does “sufficientlydetailed” become unduly burdensome? What crite-ria are necessary for valid and reliable DNA typingof forensic samples? Does consensus exist for somescientific issues and not others? If so, what can beresolved? Is resolution necessary for all scientificissues? If not, what areas can and should be covered,and what areas are best left to the discretion of theforensic analyst? What are the best mechanisms forsettling differences of opinion? And finally, whodecides? As the U.S. criminal justice system increas-ingly turns to DNA tests, these questions, some ofthem pressing, must be addressed.

This chapter identifies and analyzes several keyissues that bear on the validity and reliability ofDNA tests for forensic uses, including:

●

●

technical advantages and limitations of therestriction figment length polymorphism (RFLP)technique,technical advantages and limitations of thepolymerase chain reaction (PCR) technology,

●

●

standards-laboratory and personnel-for en-suring accurate DNA typing in the forensiccontext, andmechanisms for quality assurance.

For some issues, agreement or near agreement hasbeen reached; the chapter describes these areas. Italso examines how consensus has been achieved inother applications of DNA techniques or in newmedical technologies, and analyzes how such proc-esses could pertain to forensic applications of DNAtechnologies. Finally, the chapter discusses congres-sional, Federal, and State interest in quality assur-ance.

VALIDITY AND RELIABILITY OFDNA TECHNOLOGIES

An important matter in the use of DNA asevidence (see ch. 4) is whether the detection meth-ods are scientifically valid. Validity centers onwhether a test will correctly identify true matchesand true nonmatches. For RFLP analysis, a valid testor set of tests would not falsely classify or excludea subject by yielding a profile not true to type, i.e.,a spurious pattern would not randomly arise. TheOffice of Technology Assessment (OTA) findsthat molecular and genetic principles underlyingDNA techniques are solid and can be successfullyapplied to forensic casework. Forensic uses ofDNA tests are valid.

Initial concerns about the validity of DNA typingfor forensic applications focused on the nature of thesamples. Casework samples are obtained from avariety of less-than-sterile materials (e.g., glass,wood, dirt, and fabric) that are often subjected tosunlight, moisture, or desiccation. Samples can alsobe contaminated with unknown genetic materialsuch as bacteria, plant, or animal secretions. Valida-tion studies, however, have established that generalDNA techniques are applicable for the breadth ofconditions likely to be encountered in forensiccasework, and have dispelled notions that RFLPanalysis is invalid because of such conditions. ForRFLP analysis, these studies confirmed that forensicsamples in and of themselves are not barriers toapplying DNA technologies that use single-locusand multilocus probes (2,19,32,34,35,50,52,66,67).

–59–

60 ● Genetic Witness: Forensic Uses of DNA Tests

Similar validation studies for PCR are being per-formed (18,25,82).

A second aspect of DNA testing of forensicsamples is reliability. Any test must be reliablei.e., it must measure reproducibly that which it iscapable of measuring under routine conditions ofuse. Reliable tests must perform reproducibly withina laboratory, across many laboratories, and in thehands of different practitioners. Thus, reliabilityinvolves several factors, including the proceduresused, laboratory performance, laboratory recordkeep-ing, quality control, and quality assurance. OTAfinds that, properly performed, DNA technolo-gies per se are reliable.

A reliable procedure, used carelessly, does notrender the test unreliable-the particular test resultwould be in question. For forensic DNA analysis,questions exist about the appropriateness of usingcertain procedures over others, how data areinterpreted, or about the extent or type of qualitycontrol and quality assurance necessary to mini-mize human factors and ensure that a particulartest result is reliable (62,70,86). As described laterin this chapter, some argue that because greaterexperience exists for RFLP analysis, it is currentlymore reliable than PCR.

Finally, although forensic uses of DNA tests arevalid and reliable when performed properly, manyharbor the misconception that DNA typing appliedto forensic samples always yields a ‘‘yes’ or “no”answer. A test that does not give a ‘‘yes’ or ‘‘no’each time is not incorrect, nor unreliable. An impor-tant and often overlooked result of an analysis couldbe “inconclusive”—a result that should not bemisconstrued as either a match or exclusion.

Single-locus Probe Analysis

Several acceptable protocols exist to determinewhether two samples yield similar or different DNApatterns at various loci. In forensic applications, themethod most commonly employed in the UnitedStates involves using DNA probes that detect sizedissimilarities among individuals at certain loci. Byusing several probes in combination or sequentially,an exam]nier can determine whether DNA patternsfrom questioned samples are consistent with a sus-pect’s pattern (figure 3-1) or, in paternity cases,whether an alleged father’s pattern is consistent witha child’s (figure 3-2).

Figure 3-l—Example of One DNA Patternin a Rape Case

Biological evidence from this rape case was separated by labora-tory techniques into separate male and female fractions. Aftersingle-locus RFLP analysis of these fractions and known samplesobtained from the victim and suspect, the results reveal that—forthis particular probe~the DNA pattern of the male fraction matchesthe pattern of the suspect.SOURCE: Federal Bureau of Investigation, 1989.

The basic tool used in single-locus and multilocusprobe analysis is called Southern blotting, or South-ern transfer (84) (see ch. 2). One of the workhorsesof molecular biology, the process has been in dailyuse in thousands of laboratories for over a decade.Thus, extensive experience with Southern blottingin research and clinical testing supports the consen-sus that the technology itself is both a valid andreliable method to examine DNA.

The widespread use of Southern blotting hasdemonstrated that accurate and reliable single-locusanalysis can be obtained across a broad spectrum ofconditions. It has also defined the range of artifactsand errors that can occur, and led to solutions toavoid or minimize problems. For example, althoughpartial digestion, differential electrophoresis, cross-hybridization, background hybridization, loadingerrors, probe contamination, incomplete stripping ofmembrane before rehybridization, and loss of small

Chapter 3—Validity, Reliability, and Quality Assurance . 61

Figure 3-2—DNA Typing in Two Paternity Cases

Ml Cl AF1 M2 C2a C2b AF2

DNA typing in two different paternity disputes revealed that thealleged father (AF) is the biological father in case 1, but not in case2. Note that all bands present in the child in case 1 (Cl) can beaccounted for in either the mother (Ml) or alleged father (AF1 ), Incase 2, however, no bands from the alleged father (AF2) appearin either child (C2a and C2b), nor do bands that the children do notshare with their mother (M2) match any present in the allegedfather (A2). This analysis involved a “cocktail” of four single-locusprobes.SOURCE: Cellmark Diagnostics, 1989.

Photo credit: Federal Bureau of Investigation

“Southern blotting”: DNA is being transferred from gelsto nylon membranes. The gel is placed on a sponge, whichsits in a salt solution. The nylon membrane is placed on thegel, and a stack of filter papers on top serves as a wickthat draws the salt solution up through the sponge, gel, andmembrane~a process that results in the transfer of DNA

from the gel to the membrane. (The glass plateserves as a weight.)

fragments during electrophoresis are all problemsthat have occurred atone time or another in laborato-ries, a variety of scientific controls make it possibleto recognize, or account for, such situations. Further,most artifacts generally will lead to false non-matches, or exclusions (although false inclusionscould occur, e.g., by incorrectly placing DNA sam-ples on a gel or by loss of bands due to sampledegradation) (63).

Nevertheless, forensic applications of single-10CUS analysis make somewhat different demands onthe method than research and clinical applications.Whereas diagnostic testing usually consumes only afraction of a typical sample, from which a test can berepeated, forensic case samples can be degraded,contaminated, or in limited supply. DNA diagnos-tics generally involves determining which RFLP achild has inherited from each parent. Since a total ofat most four possibilities exists for the child (twoalternatives from the mother multiplied by twoalternatives from the father), the system has built-inconsistency checks that alert scientists to errors,such as extra or missing bands; because choices atdiagnostic loci are discrete, the results can typicallybe scored by visual examination alone. In contrast,a continuum of possible fragment sizes exists at lociscreened in forensic DNA testing; precise measure-ments of the fragment lengths and objective stan-dards for deciding whether DNA patterns match are


essential. Finally, detailed population genetics anal-yses are not required in diagnostic DNA testingbecause conclusions depend only on the samplestested. Forensic analysis, however, requires infor-mation about the frequency of DNA patterns in thegeneral population if conclusions are to be drawnabout the probability that matched patterns arosefrom one contributor, or whether many people in acommunity could have been the source.

Thus, it is important to identify potential sourcesof errors that could lead to false nonmatches andmatches when DNA technologies are used on foren-sic samples. Most would agree that avoiding falsematches in criminal cases is paramount, becausedecisions regarding liberty, and sometimes life, areat stake. Broadly speaking, three central issues mustbe considered in evaluating single-locus probe anal-ysis:

●

●

●

the artifacts that might appear, which mightlead to a false interpretation, match or non-match, of the samples;the accuracy involved in declaring that bandsare a match or nonmatch between two RFLPpatterns; andthe population characteristics of RFLP pat-terns. (Although the validity of forensic DNA

ics, when DNA typing results do not exclude anindividual, population genetics becomes essen-tial to the interpretation.)

Controls to Avoid Artifacts

Forensic DNA tests generally involve limitedquantities of samples that could have been exposedto centaminants or environmental insults. In someinstances, forensic samples might have marginallysufficient DNA to analyze; if too little DNA isobtained, less intense bands might not be detected,misleading the analyst. Some forensic DNA samplesmight be partially degraded, resulting in the isola-tion of an insufficient quantity of high molecularweight DNA; depending on the probe used, largefragments might not be detected. Finally, forensicsamples can contain chemicals, or DNA from addi-tional sources (a second person or bacteria), thatinterfere with complete DNA digestion by the re-striction enzyme or normal gel electrophoresis.

In different ways, each of these situations couldcreate problems that might interfere with analysis orinterpretation of forensic DNA test results. Severalfactors to minimize potential problems are impor-tant, including: quality control of reagents; choice ofenzyme and probe (box 3-A); and built-in scientific

tests per se does not involve population genet- flags, or controls. Controls are especially critical.

Box 3-A-Considerations for Choosing Forensic Single-locus Probes

Over 3,000 RFLP loci have been identified to date, including more than 100 highly polymorphic loci at whichmany alleles exist in the population. With such a wide range available, DNA probes that minimize possibleambiguities can be used. Most agree that, ideally, a DNA probe for forensic use should:

conditions;• produce well-characterized patterns (such as defined size range, number of bands, and relative band

intensities) so that unexpected patterns can be recognized; and● detect a single polymorphic fragment per allele so that each person’s test yields either one or two fragments,

depending on whether an individual is homozygous or heterozygous, respectively.

Avoiding probes that detect fragments of varying number or intensity minimizes the problem of identifyingthe true bands that comprise the RFLP pattern. Otherwise, the potential exists for uncertainty about whetherweak-intensity bands are part of the pattern. Furthermore, using a probe that identifies patterns with bands ofvariable number and intensity could make it difficult to identify bands on a dirty background.

Other considerations related to the use of a DNA probe for forensic casework include:

. using a series of probes from different chromosomes, or reasonably distant on the same chromosome, toensure that the regions sampled are independently segregating;

. availability of the probe for research purposes, which allows other scientists to confirm its properties; and● having the chromosomal position of the locus detected by the probe filed with the Human Gene Mapping

Workshop.SOURCE: Office of Technology Assessment 1990.


To prevent or minimize unexpected or uninter-pretable results, often referred to as artifacts oranomalies, scientists have developed several precau-tions based both on experience in basic research andon validation studies on simulated and actual foren-sic samples. Controls are steps built into any scien-tific analysis. They tell the scientist that the assay asa whole proceeded as expected, and that results fromunknown specimens are accurate and reliable. Fordespite vigilant quality control and rigorous adher-ence to an acceptable protocol, analyses can and dofail. Thus, controls tell a forensic examiner whetherthe overall analysis worked—i.e., whether expectedresults were obtained from standard samples (box3-B).

At present, scientists agree on the necessity forsome controls, but not others. Different levels ofsafeguards can be used in a scientific experiment indifferent laboratories and yield identical, accurateresults. Determining the type of controls neces-sary to ensure confidence in the results of anysingle DNA typing of a forensic specimen is of thehighest priority (see section on setting standards).

Match or Nonmatch?

As mentioned previously, clinical DNA analysisgenerally benefits from consistency checks providedby family relationships. Additionally, the types ofprobes usually used detect only a limited number ofwell-separated alleles, so that visual comparisonsuffices. In contrast, forensic uses of DNA testsinvolve determining whether one or more unknownsamples match samples collected from known indi-viduals, e.g., a suspect or victim. Probes that detecthighly polymorphic loci are used in forensic testingbecause they provide more information about iden-tity. In particular, because the trend in forensic caseshas been to use probes that involve as many as 50 to100 alleles that often involve fragments of similarlengths, comparing samples requires both visualcomparison and precise, quantitative measurementsof fragment position. What considerations are neces-sary in declaring that two or more DNA patternsmatch?

Controlling for Potential Problems—To demon-strate that two bands appearing to be the same lengthin a gel are in fact the same length, most molecularbiologists would perform a mixing experiment. Thatis, they would confirm that a 50-50 mixture of thetwo samples yields precisely the same RFLP patternas either sample alone. Mixing tests can often reveal

even small differences between samples, since co-electrophoresis allows a perfectly controlled compar-ison.

Mixing assays are generally not performed bycrime laboratories in any type of comparative case-work (e.g., drug analysis or protein electrophoresis)(59). And, at present, it appears that most laborato-ries do not routinely perform mixing tests on non-paternity DNA samples. One important concern isthe limited amount of DNA often obtained fromevidence. The difficulty of precisely measuring thequantity and quality of human DNA from forensicsamples so as to achieve an uniform mixture is alsoa consideration, as is the perceived difficulty inexplaining to a jury why an examiner would deliber-ately mix two samples. While the latter two issuesseem surmountable, most agree that the question ofrequiring mixing assays as a matter of routine shouldremain open on a case-by-case basis because, insome instances, not enough DNA will be available.These voices strongly argue that a failure to performthem is not grounds for a priori invalidating results.Nevertheless, many consider mixing tests as the“gold standard” for DNA typing of forensic speci-mens—a test that should always be performed inaccordance with standards linked to DNA quantity,not strictly case-by-case.

When a mixing test is not performed, identity isinferred by comparing the positions of bands, andhence their size, in two separate lanes. Becauselane-to-lane differences in electrophoresis can occurwithin a gel, resulting in ‘‘band shifts, ” the mostaccurate way to ascertain the size of a fragment is tomeasure its position relative to a set of internal lanecontrols, called monomorphic markers. That is, theunknown band should ideally be measured againstone or more bands of known size in the lane itself.Although monomorphic markers were not employedinitially, their use in forensic DNA analysis nowseems to be generally accepted.

In forensic casework, it might be necessary tomeasure the sizes of bands on different gels. Foren-sic laboratories performing DNA analysis currentlyrun evidence and suspect samples on the same gelwhenever possible. But occasions can arise when,for example, an evidence sample is exhausted on gelA during an analysis that excludes a suspect. Follow-ing a period of time, DNA from a new suspect mightneed to be tested. Again, using proper controls—monomorphic probes and known size markers—


Box 3-B—Scientific Controls for RFLP Analysis

Despite vigilant quality control and rigorous adherence to protocol, analyses can fail. Every valid and reliablescientific test, therefore, includes an appropriate set of scientific controls designed to demonstrate that the procedureworked correctly. When the test works properly, these controls yield certain expected results. If the observed resultsfor the controls deviate from what is expected, then the results for the case samples cannot be considered reliable.A variety of simple and widely accepted scientific procedures are available to detect errors and artifacts that canarise in forensic applications of RFLP tests. Such controls include, but are not limited to the following:

Control human DNA. Together with actual case samples on a gel, one lane should include a known humancontrol DNA that yields a known pattern. If the expected pattern is obtained, it verifies that the hybridizationproceeded as expected. Failure to obtain the expected pattern indicates that the hybridization went awry and shouldbe repeated. If a Y-chromosome specific probe is used to recognize male DNA, the blot should contain both controlmale and control female DNA samples. (The former is a “positive control” to prove that the hybridization woulddetect male DNA—if present—in the forensic case sample, while the latter is a “negative control” to demonstratethat the hybridization would not yield a spurious positive even if male DNA was not present.)

Molecular size markers. To provide a “molecular ruler” against which fragments sizes can be measured,several lanes should contain discrete DNA fragments of known size. Such ladders of standard molecular sizemarkers provide an initial test of whether the electrophoresis was uniform. By comparing the positions of fragmentsin the forensic samples to the markers’ positions, the approximate molecular size of the unknown fragments canbe calculated.

Internal lane controls. Even if the size markers appear to be distributed evenly, the analytical lanes might nothave run uniformly; differences in DNA concentration (figure 3-3) or other conditions within a sample (e.g., saltconcentration) can contribute to electrophoretic differences between lanes. Therefore, to account for such“band-shifting,” monomorphic DNA probe controls—i.e., probes to detect bands that are not polymorphic, but offixed size in the human population—should be used. If, in fact, lanes ran at different speeds, fragment sizes in eachlane can then be more accurately computed by using the internal controls as lane-specific molecular rulers. Morethan one internal standard, or monomorphic probe, should be used so that there are enough reference points to allowan accurate measurement.

Internal controls can also verify the presence of adequate quantities of high molecular weight DNA. If a DNAsample is partially degraded, the quantity of DNA above a certain size might be insufficient for detection; animportant consideration if the probe used detects alleles that are of high molecular weight. In this case, degradedDNA could lead to the conclusion that an individual is an apparent homozygote, rather than a heterozygote whoseupper band goes undetected due to DNA degradation. Such situations could give rise either to a false match ornonmatch. A monomorphic probe that detects a high molecular weight band could be used as a control.

Test for incomplete stripping. If a membrane is not stripped completely between sequential applications ofprobes, radioactively labelled probe can remain attached and produce a signal in a subsequent test. Performingautoradiography on the stripped membrane can demonstrate if a signal results from residual probe. If a pattern isseen, the membrane should be re-stipped. Even if autoradiography is not performed after stripping, extra bands canbe accounted for by superimposing consecutive x-ray films,

Control probing with plasmid DNA. Since plasmid vector DNA is a potential contaminant in samples andprobes, a band observed on an x-ray film might not be human DNA, but plasmid DNA in the sample that is beingrecognized by plasmid DNA in the probe. To rule this out, some forensic labs probe the Southern blots with plasrnidDNA to identify the location of any such bands, or use synthesized DNA probes lacking plasrnid sequences.SOURCE: OffIce of Technology Assessment 1990.

DNA from the new suspect can be tested on gel B ing rule is thus required, reflecting empirical meas-and accurately compared with results from gel A. urement variation observed to occur when known

samples are repeatedly analyzed.In any case, unless discrete allele systems are used

or mixing tests performed, determining whether two Reporting a Match—The systems currently usedsamples in different lanes match (on the same or to report a match vary. While some consensus existsdifferent gels) can require fine discrimination. In on what broad steps constitute an appropriate methodaddition to visual comparison, an objective match- for determining whether two samples match when

Chapter 3—Validity, Reliability, and Quality Assurance ● 65

Figure 3-3—DNA Typing and Murder: A Less Than Ideal First Analysis and a Solution

A. M K562 K1 Q1 M Q2 Q3 Q4 M Q5 Q6 M B. M K562 K1 K1 M Q3 M

In this murder case, six separate pieces of evidence were obtained from the crime scene and a suspect identified. RFLP analysis wasperformed (M=markers; K1 =suspect; Q1 -6=evidence; K562=standard), but the results revealed that too much of the suspect sample (Kl)had been placed on the gel, which led to distortion in the K1 lane, as well as the lanes next to it (K562 and Ql) (panel A).

Even though the suspect’s pattern for this probe is an extremely rare and unusual three-band pattern that is similar to the six questionedsamples, the forensic analyst cannot call a “match,” but must report the test “inconclusive” because the alignment is unacceptable. This,despite knowing from experience that if less suspect sample had been used, the patterns likely would have aligned and been called a match.

Fortunately, not all of evidence sample Q3 had been used in the test in panel A (although QI ,2,4-6 had been exhausted). The case wasrepeated (panel B) with diluted amounts of the suspect sample (Kl), which now clearly align with the evidence pattern (Q3). Had noevidence sample been available for an additional try, however, DNA analysis would have reported “inconclusive,” and could not have beenused as evidence to prove guilt or innocence.SOURCE: Federal Bureau of Investigation, 1989.

highly polymorphic loci are analyzed, scientistsdisagree to a certain extent on the details. Thissection describes the general sense of how a matchshould be reported when radioactive DNA probesare used, which is generally the case for criminalcasework.

The initial step in examining the results of hy-bridizing with a single-locus probe involves iden-tifying the bands in each lane. In all cases, each lanemust be evaluated independently-the presence of aband in one lane must not influence whether aquestionable signal in another lane should be identi-fied as a band. Ideally, the x-ray film would showonly RFLP bands from the test, and would showthem distinctly. However, even if there is some

degree of background hybridization or the bands arefaint, an exami ner can often reliably identify thepattern. For a probe that identifies a single polymor-phic band per allele, the task should be relativelyeasy: the lane should contain one (homozygote) ortwo (heterozygote) bands that are much more intensethan anything else in the lane. On the other hand,evidence samples might be an unequal mixture fromtwo or more contributors, so an additional faint band(or bands) should not just be discounted.

After the bands have been identified, the examinermust then determine whether they are in matchingpositions. Accurate identification of a band’s posi-tion depends on a properly exposed piece of x-rayfilm-one that yields thin, sharp bands. Overexpo-


sure (or too much DNA on a gel) results in broadbands that cover a significant size range-creatingthe risk of spuriously suggesting matches betweendistinct alleles and potentially masking other bands.A consensus exists that visual inspection must beperformed, as well as objective measurement ofband position with an appropriate measuring device,such as a computer digitizing pad or a computer-coupled camera. Opinion varies, however, over theextent and weight of operator involvement (seech. 5).

Once the size of the fragments is calculated (bycomparing the positions of the bands to the positionsof the size standards), how close must two bands beto declare a match? Identical measurements are notto be expected, nor are they achievable. Determiningan acceptable threshold of measurement imprecisionis the key. In fact, because gel electrophoresisconditions vary from laboratory to laboratory, whatis acceptable for one might not be appropriate foranother. What is agreed on is that an objectivematching rule should be used and adhered to (e.g.,fragment lengths within “x” percent based on thevariability observed empirically when known foren-sic samples are repeatedly tested).

Population Genetics

Finding that two samples have the same DNApatterns does not necessarily mean they came fromthe same individual, just as finding two specimenswith the same blood type does not mean they camefrom the same person. RFLP patterns represent onlya snapshot of the unique DNA sequence of eachindividual. Thus, in the absence of a result thatexcludes an individual, population genetics is anessential element in forensic uses of all genetictechniques, including DNA technologies.

The validity of forensic DNA tests does nothinge on population genetics. Interpreting testresults, however, depends on population frequenciesof the various DNA markers (1,9,10) (for RFLPanalysis, the size of the band in a particular test). Inother words, population genetics provides meaning—numerical weight—to DNA patterns obtained bymolecular genetics techniques.

Once a set of patterns (or just two patterns) match,population frequencies are used to report the fre-quency that such an event could arise randomly; theyare key to establishing confidence in associating anunknown evidence pattern with the pattern from a

suspect or victim. For example, whether 1 in 30billion, 1 in 2 million, 1 in 50, or 1 in 10 randomindividuals could be expected to be contributors toa specific piece of biological evidence. That basicscientific principles of population genetics can beapplied to forensic DNA analysis is not in ques-tion, but how best to apply which principles tosingle-locus RFLP analysis is under debate. Dis-agreement exists as to the extent such a debatecan or should be resolved.

Debate over population frequencies and RFLPanalysis takes several forms (16,17,29,57,69). Gen-eral agreement exists that any potential bias thatcould result from calculating population frequenciesbe conservative, i.e., favor a defendant. Neverthe-less, questions are raised about whether existingpopulation databases are properly applied, and whetherthey adequately support calculations of inclusion, ascurrently practiced. Some argue that the magnitudeof the number is not the issue, just that the analystassigns it with confidence that genetics principleshave been adhered to. Others argue that because ofthe pivotal role population frequencies can play inreporting results of forensic DNA tests, agreement isnecessary.

Calculating Population Frequencies of RFLPPatterns—After a laboratory has determined thatDNA patterns from forensic sample A match foren-sic sample B, an analyst needs to estimate howfrequently such a match might arise by chance in therelevant population. Calculating the population fre-quency of a DNA pattern consists of two steps. Thefirst step involves ascertainingg the frequency ofindividual bands by examining random populationsamples. The second step requires estimating thepopulation frequency of the overall DNA pattern(box 3-c).

Determining population frequencies of DNA frag-ments in a pattern, represented by bands on theautoradiograms, is a fundamentally empirical exer-cise. The size of the band in a pattern is compared toa database containing the distribution of fragmentsizes found in a previously studied group of individ-uals. In contrast, calculating the population fre-quency of the overall pattern is a fundamentallytheoretical exercise. Starting with the frequencies ofthe individual bands, an assumption must be madethat each represents statistically independent events.Using certain basic formulas from populations ge-netics, the probability that each of these independent

Chapter 3-Validity, Reliability, and Quality Assurance Ž 67

Box 3-C—Statistics and RFLP Analysis

Only a small fraction of DNA sequence differs between random individuals, except identical twins. UsingDNA probes allows scientists to detect some of those differences. For forensic analysis with the single-locus RFLPtechnique, using several single-locus probes in combination or serially allows a forensic examiner to concludewhether two different DNA samples came from the same person (or in paternity analysis, whether a man could havefathered a child) and report remarkable statistics that the event is more than a chance occurrence-sometimes morethan the number of people living on Earth.

Yet, one RFLP analysis can be thought of as only a snapshot of any given DNA sample. Thus, how manypictures need to be taken before scientists can be sure two unknown samples come from the same individual? Howdo they arrive at such a conclusion-expressed as a probability of the same event occurring in a random population?What is important: the number of snapshots taken? what information the snapshot reveals? or how often a certainpicture can be found in a population? In other words: How many probes? How many bands? How frequently dodifferent bands occur in the population?

Although actual calculations to assign a numerical value to RFLP analyses are far more complicated (andsomewhat controversial), the following exercise is designed to provide a sense of statistics and RFLP analysis. Thescenario assumes ideal genetic conditions, and eliminates one step in calculating statistics for RFLP tests byarbitrarily assigning frequencies to patterns, rather than bands.

Probe DetectsA 3 patterns; equally distributed in the population (33.3 percent each)B 2 patterns; equally distributed in the population (50 percent each)c 2 patterns; Cl is found in 90 percent of the population and C2 in 10 percent of the populationD 10 patterns; equally distributed in the population (10 percent each)E 20 patterns; equally distributed in the population (5 percent each)F 50 patterns; equally distributed in the population (2 percent each)

Suppose in case 1, probes A, B, C, and D were used, and pattern Cl was revealed in both a suspect and evidencesample. The frequency of this event—a suspect having the same composite profile as the evidence sample-wouldbe 0.333x 0.50x 0.90x 0.10= 0.014985, or about 1.5 percent, or every 3 in 200 people. On the other hand, if 3probes were used in case 2—D, E, and F—the frequency of a suspect matching this overall pattern would be 0.10x 0.05 x 0.02 = 0.000100, or 0.01 percent, or 1 in 10,000 persons. Even though fewer probes were used in case 2,a forensic analyst could declare a greater likelihood of inclusion. In fact, the absolute number of probes used insingle-locus RFLP analysis is less important than what information each probe reveals.

Similarly, suppose probes C and D were used in case 3, and patterns Cl and D revealed. The frequency thatthe same pattern would occur in a random sample of the population would be 0.90 x 0.10= 0.09, or 9 in 100 people.Yet, if the same two probes were used in case 4, but patterns C2 and D were revealed, the frequency of thiscombination would be 0.10 x 0.10 = 0.01, or 1 in 100 people. Even though identical probes were used, a forensicanalyst can report a lower chance that a random match had occurred in case 4 because the information provided bythe C probe for case 4 was more revealing—i.e., the analyst can declare that DNA testing in case 4 narrowed thenumber of individuals who could have contributed the sample more significantly than in case 3.SOURCE: OffIce of Technology Assessment 1990.

events would be observed is then computed to yield DNA fragments are statistically independent. Other-the frequency with which that particular pattern(genotype) occurs at that locus in the population.Finally, assuming that different single-locus probeshave been chosen because they are not correlated,the frequencies of the genotypes can be multiplied toachieve an estimate of how often that particularcomposite DNA profile could be expected in thepopulation.

One critical factor: These basic calculations areonly valid when applied to populations in which the

wise, the value calculated might greatly underesti-mate the true occurrence of the pattern in the generalpopulation—making a match seem rarer than itactually is. Essentially, the population must be onewhere individuals randomly marry and reproduce, sothat distinct subgroups are absent. In such freelymixed populations, there will be no correlationbetween the alleles on the maternal and paternalchromosomes (Hardy-Weinberg equilibrium) andno correlation between alleles at different loci (nolinkage disequilibrium).


If the population is not freely mixed, then correla-tions between alleles at two loci can exist, even ifthey lie on different chromosomes. In fact, alleles arenot randomly distributed among individuals. Certainalleles clearly concentrate within specific ethnicgroups, but where do the types of genetic markersused in forensic analysis fall? Consensus exists thatgenetic departures as extreme as those for raredisease alleles do not exist for alleles detected byforensic DNA probes (63). The best forensic DNAprobes would detect loci where the pattern of inheri-tance most closely resembles that expected in afreely mixed population.

However, no detailed population studies exist yet.Thus, individual population databases used to calcu-late probabilities of inclusion for DNA forensicanalysis should be examined for departures fromgenetic models. Even if significant departures arefound, some estimate of match probabilities is possi-ble. Precisely how statistical tests should be appliedto verify that significant deviations from randomexpectation do not occur is under debate, as is whatmathematical compensation can be made to accountfor any deviation.

Population Substructures-Because the UnitedStates is multiracial, with many ethnic subgroups,special care must be taken when determining t h elikelihood of obtaining a certain pattern withindistinct population subgroups. What effect does thispopulation substructuring have on the ability tocalculate genotype frequencies within racial sub-groups?

For example, several population genetics data-bases (see ch. 5) collect and classify information as‘‘ Hispanic,’ based on self-identification or sur-name. Hispanic as an ethnic subgroup, however, isextraordinarily broad in the United States. In partic-ular, it is possible that frequencies of certain RFLPpatterns in Mexican-Americans differ from citizensof Puerto Rican descent, which both could differfrom individuals of Cuban or El Salvadoran heri-tage. The Native American population in the South-western United States differs significantly fromNative Americans in Alaska or Hawaii. Similarly,race distinction as a division of population geneticscould be insufficient. RFLP patterns from persons ofJapanese, Chinese, Vietnamese, and Korean ances-try all might differ, yet be classified under thedesignation Asian or Oriental.

Yet how important are such subdivisions in calcu-lating the probability that a match is real or random?Most observers agree that, while a danger exists inoverplaying the “numbers game,” population fre-quencies based on existing databases can be obtained—although any reported identification frequency rep-resents an estimate to be used in conjunction withother evidence linking a defendant to a crime.Furthermore, the numerical significance of a matchthat is expressed needs to take into account thefrequently unknown ethnic association of the foren-sic specimen and details of the ethnic variation forthe population database used. On the other hand,many argue that while estimates can be made, a morerigorous and formal system for determiningg associa-tion probabilities is necessary-both because manyaspects of the genetics of RFLPs have yet to beelucidated, and because juries often place great storeon statistics (see ch. 4) (30).

Finally, what populations should be studied?Ideally, a random sample of the U.S. populationshould be tested. Since true random sampling isimpractical, as is sampling all ethnic subgroups inthe United States, human geneticists must determinestandards and criteria that will account for thestrengths and weaknesses of population geneticsdatabases. Although it might seem academic to someto know whether the frequency of random match is1 in 10 million or 1 in 10,000, others express concernabout population frequencies-especially in the con-text of any single individual on trial. Populationstudies and analyses of statistical reporting forsingle-locus RFLP tests are also relevant and criticalto the design and potential use of national databasesof DNA types (see ch. 5).

Multilocus Probe Analysis

As described in chapter 2, multilocus probessimultaneously detect a wide range of restrictionfragment length polymorphisms, thus yielding apattern of 30 or more bands per individual (33,48,49).Multilocus probes were the first probes used forDNA identification-an immigration case in theUnited Kingdom in 1985 (49). Initially, great excite-ment was generated about the use of multilocusprobes in forensic cases, since they could allowunique identification from a single hybridization.

Properly performed, multilocus probe analysis isreliable and valid, using the same basic techniquesas single-locus probe analysis. However, many be-


ing the issue of population genetics are the GeneticsSociety of America and the Society of Heredity andEvolution.

Lastly, since the technique of electrophoresis isthe basis in RFLP analysis for discruminating bandsizes among individuals, the expertise of membersof the Electrophoresis Society also could be broughtto bear on issues surrounding forensic applicationsof this technique. In particular, efforts of this profes-sional society to evaluate state-of-the-art and qualitycontrol considerations for electrophoretic methodscould be useful. Joint efforts involving scientistsfrom this society and forensic practitioners wouldthen be able to evaluate whether certain electro-phoretic methods were better suited to forensicwork, or if new developments in electrophoresiswould be adaptable to widespread use in forensiclaboratories.

Thus, several scientific professional societies thatrepresent stakeholders in forensic applications ofDNA identification exist. In addition, professionalorganizations devoted to interests of the legal com-munity, including the American Bar Association,the American Association of Trial Lawyers, theNational Association of Criminal Defense Lawyers,the National College of District Attorneys and theAmerican Civil Liberties Union, have an interest inresolving issues in DNA typing of forensic samples.Cooperation among professional organizations couldbe a powerful mechanism to ensure quality; in thearea of forensic DNA analysis, no single profes-sional society can claim sole, or even greatest,expertise. Although each group has specificstrengths and weaknesses, the collective wisdomand influence of professional groups on qualityassurance should not be underestimated or dis-counted. Nevertheless, professional society mem-bership or claims of adherence to different voluntaryprofessional guidelines can confuse the generalpublic, and should not be viewed as the ultimateimprimatur of quality.

Nonregulatory Mechanisms

Short of regulating crime laboratories and otherfacilities that perform DNA typing for nonmedicaluses, States and the Federal Government couldpromote quality assurance through nonregulatorymeans. Federal efforts, in particular, could facilitate

self-regulation. Nonregulatory action could alsotake the form of authorizing additional Federalresearch in forensic sciences—particularly cross-disciplinary projects that apply emerging basic re-search tools to real-world casework, or enhancedpopulation data collection for forensic DNA probes.Additional nonregulatory Federal efforts can en-courage the use of governmental, professional soci-ety, and industry resources to review forensic uses ofDNA tests,l or to hold consensus conferences thatmake recommendations for quality assurance offorensic DNA analysis.

A Federal role in a consensus conference processis not novel. For example, concern over costly andpossibly premature applications of medical innova-tions led to the 1977 Consensus Development Pro-gram (71,90,97). Part of the U.S. Department ofHealth and Human Services (DHHS), National Insti-tutes of Health (NIH), its purpose is to developconsensus on the clinical significance of new find-ings and the financial, ethical, and social impacts ofa procedure’s development and use. To that end, anOffice of Medical Applications of Research coordi-nates consensus conferences and other activities

Council, National Academy of Sciences, began a study of forensic DNA analysis.


In forensic analysis, the problem of cross-contamination could be particularly serious: Shoulda suspect’s sample accidentally contaminate a ques-tioned sample containing degraded (or no) DNA,subsequent PCR amplification of the questionedsample would show that it perfectly matched thesuspect. Thus, whereas cross-contamination in RFLPanalysis might more likely arise from mislabeling oftubes than physical cross-contamination of samples,forensic uses of PCR must stringently guard againstboth. Proper controls, including “no DNA con-trols, ’ ‘ are critical to interpretation of PCR-basedresults.

Proper controls and precautions for forensic usesof PCR have been proposed (4,61). Laboratoriesmust be forewarned, however, that extraordinarycare is needed in sample handling-greater than thelevel required in RFLP analysis. Some even arguethat it might be desirable if evidentiary and suspectsamples were not stored or amplified in the sameroom. In any case, even with carefully controlledtests, some argue that results in forensic caseworkshould probably be reconfined by an independentrepetition from the original sample. (Fortunately, theminimal sample requirements of PCR and the easeof the procedure make it practical, for the most part,to repeat the test multiple times.)

Misincorporation

Rarely-about once or twice every 20,000 to 1million bases—in the molecular copying process ofPCR is a nucleotide misincorporated (28,78). Canthis amount of misincorporation affect the validityof PCR for forensic uses? Theoretical modelingindicates that although a proportion of PCR productscan contain some misincorporation, such eventsoccur at random. Within an amplified sample, thechances of having a group of DNA molecules withthe exact same single base substitution would thenbe minuscule (80), unless the substitution occursearly in the reaction, when the effect could besignificant (63).

In fact, misincorporation does not create problemsin DNA sequencing analysis or probe typing ofPCR-amplified DNA (28), because the entire popu-lation of molecules is being ex amined, not a singlemolecule. Thus, misincorporation of nucleotidesmight not affect the ability of PCR to distinguishamong different DNA profiles in forensic samples.On the other hand, misincorporation of nucleotidescould be an issue if the initial amount of DNA is

minute, which is often the situation in a forensiccase. One component of standards for forensic use ofPCR might include the threshold quantity of DNAthat would be acceptable for valid and reliableexamination (25).

Differential Amplification

Differential amplification, i.e., preferential copy-ing of one allele over the other, is a concern in PCRtesting. During the course of an amplification, differ-ential amplification of one or the other of the twoalleles can occur due to variation in length (47),sequence difference, or contamination with non-DNA material in an evidence sample (63). Asmentioned previously, individuals are often hetero-zygous—i.e., have one band larger than the secondband in an RFLP pattern. Thus, if one allele ispreferentially amplified, one of the bands might notbe detected and the person mistakenly typed as ahomozygote.

Differential amplification can be addressed, inpart, by carefully characterizing the regions to beexamined and establishing standards of practice toavoid contamination. Thus, despite concerns aboutthis matter and the previously mentioned issues, fewdoubt that ongoing research will overcome ques-tions raised about PCR, with full technology transferof PCR in criminal investigations occurring in thenext few years.

Population Genetics

Population genetics considerations for PCR de-pend on the genetic locus amplified. At present, theonly genetic system generally employed in forensiccasework using PCR is the HLA DQx- 1 system—ahuman white blood cell antigen system. Results inthis system are scored through the “yes’ ’/’’no”assay described in chapter 2. HLA DQx-1 is a validand well-defined system (78), but it is not asdiscriminating as RFLP analysis. It can distinguishbetween two random individuals 93 percent of thetime (98). Such discriminating power has proveduseful in excluding or including suspects in criminalcases where conventional serological genetic analy-sis has failed, or where insufficient DNA wasavailable for single-locus analysis. And, althoughfew population studies involving PCR-based detec-tion systems have been defined to date, such infor-mation can be expected to accumulate rapidly asexperience in research laboratories increases. Onepopulation genetics issue that might need address-


ing: accounting for differential amplification ofalleles in a PCR-based analysis that reveals anapparent homozygote (63).

QUALITY CONTROL ANDQUALITY ASSURANCE

Laboratories use quality control to ensure that anassay’s quality achieves specified criteria. Qualitycontrol includes the steps taken by a laboratory toproduce consistent, interpretable results each timethe test is performed. A quality assurance programprovides evidence that quality control is beingsatisfactorily performed. Such documentation caninclude proficiency testing and external inspections(5,54-56,85). Quality control and quality assuranceare essential components of good laboratory prac-tice.

Congress has long had an interest in qualityassurance issues. Through its charge to protect thepublic welfare, Congress and the Federal Govern-ment have implemented an array of quality assur-ance programs-ranging from specific legislativeaction to encouraging voluntary mechanisms—in avariety of fields. In particular, quality assurance forboth drug testing (box 3-D) and clinical diagnosticlaboratories has been the focus of recent congres-sional and executive attention (53 FR 11970; PublicLaw 100-71; Public Law 100-578; 88,89,96).

The issue of quality assurance for the Nation’scrime laboratories is not novel. Scrutiny of crimelaboratories has been an issue since their prolifera-tion in the 1970s (68,73). Publicity surroundingDNA typing in criminal casework, coupled withthe fact that DNA technologies often capturegovernment and public interest (91,93-95), hassimply renewed interest in the performance offorensic facilities. Thus, while DNA testing servedas the catalyst for today’s debate about qualityassurance for crime laboratories, other tests previ-ously sparked attention about this subject (38,39,68).

In one respect, however, concern about forensicDNA analysis differs from previous attention toquality assurance of forensic services: Both publicand commercial private providers are involved.While some mechanisms to attain uniform, high-quality service can be the same for both sectors,other approaches might apply to only one.

This section concentrates on the role that can beplayed by professional societies, State and local

Box 3-D-Quality Assurance andDrug Testing Laboratories

Drug testing of employees and job applicants hasbecome increasingly commonplace. The dramaticincrease in testing facilities to handle samples hasspawned concern about ensuring that sufficient careis taken so that those tested are not harmed bypoor-quality tests or inadequate quality assurancepolicies or quality control procedures. In 1988, theGeneral Accounting Office surveyed all 50 Stateson the nature of laws, regulations, and other legallyenforceable provisions in effect that would governquality assurance of drug testing laboratories. Thesurvey revealed that no uniform system exists toregulate laboratories doing employee drug testing.Some States do have formal mechanisms specificfor quality assurance oversight of drug testingfacilities. Others regulate laboratories that performemployee drug analysis through general medical orclinical laboratory statutes. Still others voluntarilyadhere to standards prescribed by various profes-sional associations. Some do not control suchservices at all.

The Federal Government has moved to improveresults from laboratories providing employee drugtesting services (53 FR 11970; Public Law 100-71).Congress also is interested in ensuring quality inlaboratories that do employee drug testing. Legisla-tion considered during the 100th Congress wouldhave required proficiency testing and certificationby the U.S. Department of Health and HumanServices for all facilities engaged in urinalysis andblood analysis for employee drug testing. Similarlegislation is pending in the 101st Congress.SOURCE: Office of Technology Assessment 1990.

governments, and the Federal Government to ensurethat both private and public laboratories providehigh-quality forensic DNA typing. It discusses thestructure of professional societies and their potentialfor providing practitioner education, setting stan-dards, assuring adequate staffing and laboratoryfacilities, and developing a consensus among allparties who have an interest in high-quality forensicDNA analysis.

Further, Congress has declared that crime isessentially a local problem that must be dealt with byState and local governments (with Federal financialand technical support) if it is to be effectivelycontrolled (42 U.S.C. 3701). Crime laboratories are,in fact, public agencies, and State and local govern-ments play a key role in determining quality of


forensic services. Thus, this section examines therole of State and local governments in qualityassurance of DNA profiling.

In addition, the Federal role in quality assurancecan operate at both nonregulatory and regulatorylevels. The Federal Government can facilitate non-regulatory efforts to guarantee high quality. It canalso actively regulate standards of practice, proto-cols, and commerce in forensic services (especiallythose paid for by government programs). In particu-lar, this section describes quality assurance proto-cols implemented by the Federal Government inother laboratory testing areas, such as clinical diag-nostics laboratories and drug testing facilities. Fi-nally, the Federal Government has, over the last 15years, formed commissions that have recommendedaction on topics related to applications of the newDNA technologies. Federal powers to implement thesuggestions of these advisory groups also are ex-plored.

The Role of Professional Societies

Membership in professional societies is purelyvoluntary, as is members’ adherence to an organiza-tion’s code of conduct and standards. Professionalorganizations can set informal standards, make mem-bers undergo continuing professional education tomaintain active membership status, and requireperiodic examination. A professional organizationcan also survey its members and gather data on newtechniques. Again, taking part in such studies isvoluntary on the part of the membership.

In the forensic sciences, one of the many influen-tial societies is the American Academy of ForensicSciences (AAFS). A nonprofit professional societyorganized in 1948, AAFS is devoted to the improve-ment, administration, and achievement of justicethrough the application of science to the processes oflaw. The organization draws members from the 50States, all U.S. territories, and over 30 countries, andis the largest professional society devoted to forensicpractices. An ad hoc committee has been establishedto examine forensic applications of DNA tests and,as quality assurance mechanisms develop, AAFSmembers will play a key role in developing stan-dards and disseminating information.

Another group of forensic professionals is theAmerican Society of Crime Laboratory Directors(ASCLD). ASCLD guidelines do not bind a soci-ety’s members to a particular practice, but do serve

to develop some consensus among practitioners. Forexample, a DNA implementation committee hasbeen established (see ch. 6). ASCLD also encour-ages proficiency testing before an analyst is assignedcasework (6). In particular, ASCLD provides profes-sional advice to a proficiency testing program, andoffers a voluntary accreditation program (describedin following sections).

In addition to national organizations, forensicpractitioners in regional jurisdictions have pioneeredefforts to establish guidelines for quality assurance.For example, in 1987, the California Association ofCriminalists (CAC) and the California Departmentof Justice held a statewide symposium of serologiststo examine standards in quality assurance, training,record collection and evidence preservation, methodvalidation, and data interpretation. As a result of thesymposium, a document articulating the profes-sional consensus of serology practices within Cali-fornia was published (22). In addition, the CaliforniaAssociation of Crime Laboratory Directors (CACLD)endorsed a series of guidelines to evaluate DNAtesting by commercial services (21), which couldprovide criteria for measuring performance of publiccrime laboratories.

Other professional organizations, such as theCouncil on Forensic Science Education, the Ameri-can Association of Blood Banks (AABB), the Amer-ican Society for Histocompatibility and Immunogenetics(ASHI), and the International Society for ForensicHaemogenetics (ISFH), are likely to play an impor-tant role in debates surrounding quality assurancefor DNA analysis by crime laboratories. For exam-ple, AABB and ASHI have standards for using DNApolymorphisms in parentage testing (3,4). ASHIstandards address both RFLP analysis and PCR/HLA typing (4). ISFH recommendations encompassRFLP analysis for parentage and criminal samples(45).

Another important professional society, althoughnot directly involved in forensic sciences per se, isthe American Society of Human Genetics (ASHG).A society principally composed of scientific expertsin human genetics, ASHG could be useful in evalu-ating the utility, validity, and reliability of newlyemerging DNA technologies, as well as analyzingpopulation genetics data. A recent statement raisesseveral points that ASHG believes should be consid-ered in forensic DNA testing (7). Other professionalsocieties that could contribute to debates surround-


ing the issue of population genetics are the GeneticsSociety of America and the Society of Heredity andEvolution.

Lastly, since the technique of electrophoresis isthe basis in RFLP analysis for discruminating bandsizes among individuals, the expertise of membersof the Electrophoresis Society also could be broughtto bear on issues surrounding forensic applicationsof this technique. In particular, efforts of this profes-sional society to evaluate state-of-the-art and qualitycontrol considerations for electrophoretic methodscould be useful. Joint efforts involving scientistsfrom this society and forensic practitioners wouldthen be able to evaluate whether certain electro-phoretic methods were better suited to forensicwork, or if new developments in electrophoresiswould be adaptable to widespread use in forensiclaboratories.

Thus, several scientific professional societies thatrepresent stakeholders in forensic applications ofDNA identification exist. In addition, professionalorganizations devoted to interests of the legal com-munity, including the American Bar Association,the American Association of Trial Lawyers, theNational Association of Criminal Defense Lawyers,the National College of District Attorneys and theAmerican Civil Liberties Union, have an interest inresolving issues in DNA typing of forensic samples.Cooperation among professional organizations couldbe a powerful mechanism to ensure quality; in thearea of forensic DNA analysis, no single profes-sional society can claim sole, or even greatest,expertise. Although each group has specificstrengths and weaknesses, the collective wisdomand influence of professional groups on qualityassurance should not be underestimated or dis-counted. Nevertheless, professional society mem-bership or claims of adherence to different voluntaryprofessional guidelines can confuse the generalpublic, and should not be viewed as the ultimateimprimatur of quality.

Nonregulatory Mechanisms

Short of regulating crime laboratories and otherfacilities that perform DNA typing for nonmedicaluses, States and the Federal Government couldpromote quality assurance through nonregulatorymeans. Federal efforts, in particular, could facilitate

self-regulation. Nonregulatory action could alsotake the form of authorizing additional Federalresearch in forensic sciences—particularly cross-disciplinary projects that apply emerging basic re-search tools to real-world casework, or enhancedpopulation data collection for forensic DNA probes.Additional nonregulatory Federal efforts can en-courage the use of governmental, professional soci-ety, and industry resources to review forensic uses ofDNA tests,l or to hold consensus conferences thatmake recommendations for quality assurance offorensic DNA analysis.

A Federal role in a consensus conference processis not novel. For example, concern over costly andpossibly premature applications of medical innova-tions led to the 1977 Consensus Development Pro-gram (71,90,97). Part of the U.S. Department ofHealth and Human Services (DHHS), National Insti-tutes of Health (NIH), its purpose is to developconsensus on the clinical significance of new find-ings and the financial, ethical, and social impacts ofa procedure’s development and use. To that end, anOffice of Medical Applications of Research coordi-nates consensus conferences and other activities

Council, National Academy of Sciences, began a study of forensic DNA analysis.


with the NIH Bureaus, Institutes, and Divisions, andguides the appointment of expert advisory panels toreview and make recommendations on medical in-novations and their applications.

NIH consensus conferences are open to the publicand generally involve interdisciplinary panels drawnfrom a range of interests. Over 60 consensus confer-ences have been convened in the last decade, withnoticeable effects on the practice of medicine insome areas (44,71). The NIH process has served asa general model for consensus development andgroup judgment programs in the United States andabroad (8,44).

Consensus conferences on forensic DNA analy-sis, for example, could evaluate data on DNAprobes, including studies of the population geneticsof probes, and recommend protocols that list the bestmethods. Conferences and reports could also helpdefine a “successful” program, or distinguish ex-perimental from standard investigative techniques.

One important consideration in whether an NIH-like consensus process would be appropriate toforensic DNA testing is whether the questions areprimarily scientific, or primarily ethical or eco-nomic. The conferences are more effective whenthey are the former (58,92), although a recent exter-nal review of the program made recommendationsthat could strengthen its economic, social, andethical evaluations (44). Thus, the consensus confer-

ence process might be most amenable to resolvingdebates about appropriate probes, electrophoresisconditions, criteria for declaring a match, or calcu-lating population frequencies, but be less successfulin addressing a topic such as privacy of DNAdatabases. Nevertheless, an NIH-like consensus con-ference process could be effective and lead to greaterquality assurance in forensic practices using DNAtests.

One nonregulatory Federal initiative to examinequality control and quality assurance issues is beingspearheaded by the Federal Bureau of Investiga-tion’s (FBI) Technical Working Group on DNAAnalysis Methods (TWGDAM) (see ch. 6). Consist-ing of individuals representing forensic facilities ator near implementation of DNA profiling tech-niques, one TWGDAM document outlines a multi-faceted program to ensure quality RFLP analysis(box 3-E) (85).

Although some predict the TWGDAM guidelinesare likely to be the nucleus around which nationalconsensus on standards for quality assurance willevolve, others are less sanguine. Some critics objectto the closed nature of the initial decisionmaking orlack of representation in the group of certain inter-ested parties. A few argue that the FBI-largely aninvestigative and enforcement body—is an inappro-priate lead player, and thus they oppose any role forthe FBI in quality assurance mechanisms and stan-dards. On the other hand, the TWGDAM guidelines

Box 3-E--Quality Assurance and the FBI Technical Working Group on DNA Analysis Methods

From the outset, one goal of the FBI’s TWGDAM (see ch. 6) was to suggest guidelines that would assist a crimelaboratory in developing a quality assurance program for forensic RFLP analysis. Following review and revisionof proposed guidelines, the policies were published in 1989.

The TWGDAM guidelines are designed to encompass ‘all significant aspects of the laboratory process.” Theprogram includes considerations for personnel education and training, proper documentation of pertinent records,evidence handling, validation of analytical procedures, technical controls and standards, data analysis and reporting,proficiency testing, and independent auditing.

For proficiency testing, the TWGDAM guidelines state that open and blind proficiency tests must beperformed, and recommend that an analyst be subject to both types of proficiency testing annually. Yearlyindependent audits should also be conducted, and it is “highly desirable” that the inspection include at least oneperson from outside the agency.

Policies detailed in the TWGDAM program “represent the minimum quality assurance requirements for DNARFLP analysis. ” Although the guidelines are strictly voluntary, they could become the de facto standard for qualityassurance. For example, the Minnesota Supreme Court cited the TWGDAM guidelines in ruling on the admissibilityof DNA tests (see ch. 4).SOURCE: Office of Techmology Assessment 1990, based on Technical Working Group on DNA Analysis Methods (TWGDAM), “Guidelines

for a Quality Assurance Program for DNA Restriction Fragment Length Polymorphism Analysis,” Crime Laboratory Digest16(2):40-59, 1989.


Photo credit: National Institute of Stanckrds and Technology

U.S. Department of Commerce, National Institute of Standards and Technology, Gaithersburg, MD.

represent a first-step in a probable multistage proc-ess to achieve consensus on quality assurance pro-grams. In particular, because its members are foren-sic practitioners, TWGDAM proposals are likely toaddress the concerns and solutions of this stake-holder.

Finally, a significant part of quality assuranceinvolves confidence in measurement standards. TheU.S. Department of Commerce, National Institute ofStandards and Technology (NIST) (formerly knownas the National Bureau of Standards), established in1901 (15 U.S.C. 271), is a neutral, nonregulatoryagency that conducts research providing ground-work for the Nation’s measurement systems. Atpresent, NIST activities include evaluating sizemarkers, reagent quality, and electrophoresis condi-tions, so that DNA fragment sizes can be moreaccurately determined (76). Additionally, NIJ,through its Law Enforcement Standards Laboratoryat NIST, has initiated a program to examine stan-dards for DNA processing (75). As the only Federallaboratory with the explicit goal of researching andproviding reference standards, NIST proposals, asthey become available, will likely play an importantrole in quality assurance of forensic uses of DNAidentification.

State Authority To Regulate CrimeLaboratories

States individually make and enforce most crimi-nal laws. Inherent in this authority is the ability tomarshal the evidence required for conviction. Thus,each State controls how DNA evidence is analyzed—including setting standards for performance-andpresented in court (see ch. 4). Accordingly, Stateshave the authority to regulate forensic DNAtyping by both private laboratories and publiccrime laboratories. All State jurisdiction is limitedby the provisions of the U.S. Constitution regardingthe rights of individual citizens, but a State’s inher-ent powers to protect victims and suspects are broadand provide many potential avenues for regulation,even if parallel areas of Federal authority havedeveloped.

Regulation of medical facilities might provideguidance to the States. All licensing of medicalpersonnel and facilities is based on State law, andalmost all tort law is State-based, despite Federalactivity in all these areas (92). State authoritiesmost relevant to forensic uses of DNA tests arelicensing of laboratory personnel and monitoringfacilities. At least one State, Maryland, maintainsregulatory authority over one private forensic labo-


ratory through the issuance of a clinical laboratorylicense in the area of molecular biology (31).

To date, no State has enacted general licensingrequirements for crime laboratories. Several haverequirements in restricted areas such as forensicalcohol analysis, but no State has licensing require-ments for DNA typing in forensic casework. Nordoes any State have forensic licensing requirementsregulating DNA typing by private companies. Onenonregulatory means to regulate forensic uses ofDNA tests could be negligence litigation (box 3-F).

Crime Laboratory Personnel

Two general mechanisms to assure quality oflaboratory personnel exist: certification and licen-sure. Certification is a voluntary process, whilelicensing is government mandated. Licensing ofpersonnel is generally the domain of State govern-ments. It is a formal mechanism intended to protectboth the public and the profession, as well as provideguidance to the judicial system. For forensic DNAtesting, a State could specify particular qualifica-tions necessary for either public or private facilities.States could require their licensees to follow certainnationally recognized standards.

As well as requiring minimum standards, licens-ing provides States with the right to review an

with sanctions ranging from simple censure to li-cense revocation for failure to follow proper stan-dards in delivering services. On the other hand,without licensing, enforcement of honest practicemight be stronger, not weaker, because generalantifraud provisions might apply (36). In someinstances, possession of a license can provide apractitioner with a misleading imprimatur of exper-tise (36).

At present, no State requires licensing of crimelaboratory personnel or private practitioners per-forming DNA analysis on forensic samples. Incontrast, a majority of States regulate the qualifica-tions of clinical laboratory personnel (79). Althoughno State licenses criminalists or serologists, volun-tary certification programs are in place for someforensic fields, but not in criminalistics. As early as1979, proposals surfaced for certifying criminalists(26). At the time, a majority of professionals incriminalistics withheld support, and no nationalcertification program yet exists. In 1988, certifica-tion efforts for criminalistics were revived, and anAmerican Board of Criminalistics was incorporatedin August 1989 (27). At the State level, CAC begana certification test in May 1989 (11).

To set and implement licensing or certificationguidelines, however, the forensic science profes-

individual’s practice, and to discipline the person sional community must define the body of special-

Box 3-F—Negligence Litigation

Tort law is a nonregulatory means for social control of risks to health and safety. Permitting individuals to suethose who have wronged them through negligence serves as a mechanism for financial and emotional compensation,and for quality control. Theoretically, by making people responsible for their actions, individuals have an incentiveto act responsibly. In practice, negligence litigation involving DNA typing would probably suffer from the sameshortcoming found in medical malpractice litigation-a focus on past errors rather than future improvements.Nevertheless, as with medical malpractice, negligence litigation could have an effect on private entities that provideDNA typing for forensic purposes, particularly parentage testing.

To prove negligence, an individual would need to prove the commercial forensic practitioner breached a dutythrough neglect or lack of due care. Most likely, the plaintiff would have to show that the defendant did not adhereto “good accepted practice,” or that the industry-wide definition of such practice is so flawed that failure to gobeyond it constitutes negligence.

In the absence of a good-practice standard for forensic DNA testing, each party in a suit must look to otherfields to judge the defendant’s conduct. This problem complicates the presentation and evaluation of evidence. Ifone judge tries more than one case involving DNA testing, a de facto standard could develop for that courtroom,but would have little value as precedent outside that jurisdiction. Most courts also are not in a position to promulgatesuch standards outside the confines of a trial. Thus, while tort suits may remedy individual grievances, they do littleto force development of a nationwide, good-practice standard. Tort suits do a better job of enforcing standards afterthey have been developed. Without standards, the importance of negligence litigation on quality assurance forforensic DNA typing would appear to be minimal at present, and its future impact uncertain.SOURCE: Office of Technology Assessment 1990,

Chapter 3—Validity, Reliabiliity, and Quality Assurance ● 77

ized knowledge required, establish a system toidentify qualified persons who meet minimum stan-dards of practice, and agree to guidelines againstwhich courts can measure scientific evidence andperformance (12). In particular, minimum curricularrequirements, training, and continuing educationneed delineation. At present, professional training islargely through on-the-job apprenticeships, semi-nars, and workshops, including training by the FBI’sForensic Science Research and Trainin g Center(FSRTC) in an array of forensic specialties for about300 State and local crime laboratory personnelannually (51). Formal academic coursework in fo-rensic science at the undergraduate and graduatelevels is available in only a few institutions, withinternships not widely available.

Recently, however, progress in defining trainingand educational requirements has been made. Forexample, the TWGDAM quality assurance guide-lines address education and training of forensicpersonnel (85). The Council of Forensic ScienceEducators has been formed with the goal of develop-ing standards for forensic science education (12)0 In

the field of serology, that area where DNA typingexpertise is likely to fall, the Southern Associationof Forensic Scientists has drawn up a training outline(83). Serologists in California have proposed basiceducational requirements and trainin g needs forprofessionals in their State (22).

While some predict national consensus for train-ing and education requirements will be achieved,implementation of such a program does not seemimminent. Clearly, one of the best mechanisms toguarantee quality is providing adequate resourcesfor educating and training forensic laboratory per-sonnel. At present, such resources are woefullyinadequate, and most agree that increased State andFederal attention in this area is necessary.

In addition to adequate education and training offorensic laboratory personnel, quality control andquality assurance before evidence samples reach thelaboratory door has been underemphasized to date.Providing education for field personnel on how bestto gather and preserve evidence so that DNA identi-fication can be performed will aid and enhance theefforts of the forensic examiner.

Photo credit: Robyn Nishimi

Certificate of accreditation for voluntary program offered toforensic laboratories by the American Society of Crime

Laboratory Directors.

Crime Laboratory Facilities

As well as requiring licensing of personnel, Statescan mandate licensing of facilities or specific serv-ices within facilities. States could, for example,adopt laboratory licensing regulations aimed specif-ically at DNA typing programs, and not otherforensic technologies, in private and public laborato-ries. Supplemental to or in place of licensing can bean accreditation process offered by a neutral, exter-nal body, such as the College of American Patholo-gists for medical genetics or the Joint Commissionon Accreditation of Healthcare Organizations forvarious medical facilities. Accreditation can be strictlyvoluntary, and traditionally has been. But increas-ingly, as a condition of receipt of certain privilegesor in exchange for funding, State or Federal officialsrequire accreditation by a specific group or groups.

Licensing—As mentioned, no State currentlyregulates forensic service facilities in general, al-though States do have clear authority to oversee suchservices. In contrast, all 50 States and the District ofColumbia require that public and private hospitalsbe licensed, although the scope of the laws variesconsiderably (101). In 1988, in an effort to ensurequality services, Congress passed sweeping legisla-tion that subjects all clinical testing laboratories toa uniform standard of regulations (Public Law 100-578) (box 3-G).

Accreditation-Although not subject to manda-tory oversight, one voluntary accreditation program


Box 3-G-Quality Assurance and Clinical Laboratories

In October 1988, Congress passed sweeping legislation that subjects clinical laboratories to a number ofrequirements, including qualifications for the laboratory director, standards for the supervision of lab testing,qualifications for technical personnel, management requirements, and an acceptable quality control program. Priorto enacting the Clinical Laboratory Improvement Amendments of 1988 (CLIA) (Public Law 100-578), Federalregulations covered the approximately 13,000 labs that either transported samples between States or performed testsbilled to Medicaid and Medicare. Beginning in 1990, however, the Health Care Financing Administration (HCFA)of the U.S. Department of Health and Human Services (DHHS) will exercise sweeping regulatory authority overclinical laboratories. HCFA will set standards for staffing and maintaining all medical laboratories, includingphysician office testing. HCFA will also manage a comprehensive program to police the facilities and can imposesanctions.

CLIA is at once broad, encompassing the estimated 98,000 physician labs, and specific. For example, theSecretary of DHHS is to establish national standards for quality assurance in cytology services, including themaximum number of cytology slides that any individual may screen in a 24-hour period. The Secretary is alsorequired to determine and implement recordkeeping, inspection, and proficiency testing programs, and to study andreport to Congress on a range of issues gauging the impact of various quality assurance mechanisms.

CLIA expands DHKS’s regulatory authority over clinical laboratories, and grants HCFA the power to suspendor revoke a lab’s certificate for violation of the rules. Further, fines up to $10,000 for each violation or each dayof noncompliance can be levied, and jail sentences of 3 years can be imposed. The law continues to permit, subjectto approval by the Secretary, the involvement of State or private nonprofit associations (which at present includethe College of American Pathologists, the American Association of Bioanalysts, agencies in 3 States, the JointCommittee on Accreditation of Healthcare Organizations, and the American Osteopathic Association) to substitutefor the Federal regulatory process.

Prior to CLIA’s enactment, one issue of critical concern to Congress was proficiency testing programs. UntilCLIA, such programs varied broadly in testing criteria and in grading of test results. Moreover, uniform orminimally acceptable Federal standards did not exist. Now, except under certain circumstances, proficiency testingshall be conducted on a quarterly basis, with uniform criteria for all examinations and procedures. The Secretaryshall also establish a system for grading proficiency testing performance.SOURCE: Office of Technology Assessment 1990.

for crime laboratories does exist. Established in Others strongly argue that current voluntary pro-1981 by ASCLD, the program includes a self-evaluation, an inspection process, and requiredproficiency testing. As of December 1988, 58laboratories representing 15 Federal, State, and localagencies—about 20 percent of the Nation’s crimelaboratories-have been accredited during the pro-gram’s 7 years of operation. (An additional sevenlaboratories were accredited as of May 1989.) Incomparison, a 5-year-old AABB program to evalu-ate parentage testing laboratories had accredited 48labs representing about 39 percent of potentialfacilities as of December 1988 (1 1,12).

Many criticize current optional accreditation ef-forts as inadequate, and assert that mandatory ac-creditation by an external, neutral body is essential.2

grams improve- quality and are sufficient. Many feelthat action should focus on increasing participationin voluntary programs, rather than mandating ac-creditation.

Nevertheless, while simultaneously contributingto quality assurance, mandatory accreditation oflaboratories by independent, impartial organizations—in consultation with forensic practitioners andacademic forensic scientists-could be effective indispelling the notion of some that crime laboratoriesare not neutral bodies. Such programs could removethe perception of many defense attorneys and theirclients that crime laboratories are biased-workingprincipally toward conviction for prosecutors andpolice departments, and secondarily in defendants’

(36,37). It is unlikely, however, that the Antitrust Division of the U.S. Department of Justice (orpnvate individuals or corporations) will find any occasionto attack concerted action in forensic services per se, since these are largely public facilities, or generally run as small business. Additionally, it is difficultto imagine Federal Trade Commission scrutiny for potential ‘‘unfair practices” related to information disclosure: Individuals involved in commercialservices have routinely communicated their findings and procedures, in part to satisfy legal requirements of introducing DNA tests in court.


interest. On the other hand, accredi.and certification are costly and

tation, licensing,time-cons uming

endeavors that would likely place an additionalpersonnel and financial burden on public facilitiesalready overwhelmed with criminal casework andgenerally underfunded (74).

Proficiency Testing-Proficiency testing in crimelaboratories is currently offered through a programadministered by Collaborative Testing Services (CTS)in association with the Forensic Science Foundation(FSF) (the research arm of the AAFS). Participationis voluntary and anonymous, and more than half thecrime laboratories subscribed to the physiologicalfluids program in 1985 (65), which now includessamples for DNA testing. In place since the mid-1970s, the CTS-FSF program has supporters andcritics. Proponents point out that although not com-pulsory, the program provides a crime laboratory anopportunity to monitor the technical performance ofits employees and compare results with other labora-tories. Critics argue that results from the programmerely underscore the need for tighter control, evenmandatory regulation through legislation.

A 1978 study (73) found that an ‘appalling’ (68)number of participating laboratories reported erro-neous results in testing blind samples, with as manyas 94 out of 132 laboratories participating obtained‘‘unacceptable’ blood typing results. Another criticreports that from 1978 through June 1988, thenumber of errors for bloodstain or physiologicalstain proficiency tests varied from 7 percent for onetest to 77.7 percent for another, and that overall, anaverage of 25 percent of crime laboratories returningresults made errors (38). In one human blood test toevaluate genetic markers, 15 of 69 participatinglaboratories (21.7 percent) made at least 1 error (38).None of these tests involved DNA typing.

In contrast, another review of the CTS-FSF serol-ogy testing program reported strikingly differentfindings for 7,827 tests performed during 1978 tomid-1986 (81): an error rate of 2.4 percent (189errors). Further analysis revealed that 88 of theseerrors arose in laboratories that made three or moreerrors in the particular trial; which was acknowl-edged as an amount signifying serious problems.Subtracting the errors made by these laboratoriesreduced the rate to 1.3 percent. The study, whichincluded all but one proficiency trial during thatperiod, concluded that, on average, 79.1 percent ofreporting laboratories were error-free; 4.2 percent of


DNA samples for RFLP analysis.

laboratories reported three or more errors (81). Theauthor of this report, as well as many others, attributethe different findings to how ‘‘error’ was defined ineach study. The analysis reporting the greater errorrate counted ‘‘inconclusive’ results as errors, apractice with which the vast majority of scientistsdisagree. Similarly, “unacceptable” in the 1978study is attributed to laboratories lagging behind inemploying certain state-of-the-art tests, not to actualperformance (82).

In addition to the CTS-FSF program, some crimelaboratories subscribe to the voluntary proficiencyprogram sponsored by the AABB parentage testingcommittee, which also includes DNA typing. Boththe CTS-FSF and AABB voluntary programs, how-ever, are less rigorous than the comparable programin the United Kingdom. Not only is DNA profi-ciency testing already in place in the United King-dom, but the program includes blind tests slippedinto the flow of actual casework (100). Interspersingblind tests with case samples clearly yields the mostaccurate measure of a laboratory’s performance ona test.

With respect to blind trials of forensic DNAtesting in the United States, CACLD organized trialsusing case-simulated samples in 1987 and 1988. Thethree major commercial facilities then performingforensic DNA analysis participated in each trial. In


the first trial, out of 50 samples, 2 firms eachdeclared 1 false match (60) that could have resultedin the conviction of an innocent person. The errorsapparently arose from sample handling problems(11). The third company declared no false matches(60). In the second trial, one company again reportedan incorrect match (13).

To date, the FBI has not provided blind trials tocommercial laboratories, nor does it have plans to doso in the future. However, the FBI’s FSRTC willprovide initial open proficiency tests to those Stateand local laboratories that participate in the FBI’straining program. FSRTC also prepares proficiencysamples to monitor the performance of the FBI DNAAnalysis Unit. In addition, the FBI plans to adminis-ter a program that offers seed money to encouragecommercial ventures to develop proficiency samplesand testing (14,41). The FBI will not analyze DNAwork performed by State and local laboratories,however, having a longstanding policy not to reex-amine evidentiary materials previously examined byanother crime laboratory.

Some observers, generally not forensic analysts,suggest that a mandatory, independent process ofproficiency testing for public and private forensiclaboratories engaged in DNA testing should beestablished. Others, usually from crime laboratories,support open and blind proficiency tests per se, butcategorically oppose an independent, mandated pro-gram. What is clear is that proficiency testing haslong been recognized to be a key component ofquality assurance. Clinical laboratories, for exam-ple, are required by Federal law to meet acceptableperformance criteria under a proficiency testingprogram on a quarterly basis (Public Law 100-578).One administrator of a clinical laboratory profi-ciency testing program argues that such a program isthe best, economically feasible, external source fordetermining lab quality (53). In fact, the TWGDAMquality assurance guidelines-whose authors in-clude crime laboratory personnel-include require-ments for proficiency testing (85).

Although consensus exists that some sort of DNAproficiency testing program is desirable, disagree-ment arises over who shall administer it, who shalljudge what constitutes acceptable performance, andthe role of proficiency test results in court proceed-ings. Some argue that forensic practitioners aloneare best able to make such decisions, while othersmaintain that involvement of molecular biologists

and human geneticists is necessary. And, whilemany feel the present CTS-FSF program is well-placed to administer DNA proficiency testing, oth-ers believe a new system is necessary.

Finally, disagreement exists about the generalavailability of proficiency testing results for trialexamination. Some maintain that such testing isdesigned for internal quality assessment and feed-back, and should not be applied punitively againstall work performed by a particular examiner orlaboratory. Others strongly disagree, arguing thatproficiency testing data-especially in the absenceof standards-is the only way to determine whetherreliable findings were obtained for any case. And, asdemonstrated by the studies of the CTS-FSF pro-gram, how ‘‘inconclusive’ results are classified isimportant when error rates are reported.

Federal Authority To Regulate CrimeLaboratories

In theory, the Federal Government can only exer-cise those powers specifically granted to it in theConstitution. None of those powers relate directly toforensic practices in general, or to DNA typing bycrime laboratories in specific, Yet the Federal Gov-ernment is not powerless in this area. With respectto setting standards of practice for other types oflaboratories-most notably clinical diagnostics anddrug testing-Congress and the executive branchhave separately and together imposed requirementsdesigned to ensure consistently high quality. Thefollowing sections examine congressional authorityto regulate forensic uses of DNA technologies, andanalyze present Federal regulation of clinical anddrug testing laboratories.

Taxing and Spending Authority

Article I, Section 8 of the Constitution states thatCongress may spend money “for the commonDefense and general Welfare of the United States.’It is through the use of conditional appropri-ations—i.e., attaching strings to grants of money—that Congress derives its power to regulate throughspending (87). The Supreme Court has upheld con-gressional authority to impose conditions on the useof funds directly distributed to States by the FederalGovernment. States, to the extent they wish to availthemselves of such monies, must comply with thoseconditions (40).


Direct finding of crime laboratories would givethe Federal Government the authority to determinea wide variety of requirements for the delivery ofhigh-quality service. For example, the governmentcould attach certain conditions to funds earmarkedfor crime laboratories for DNA analysis, or attachconditions for general quality assurance to appropri-ations such as recent finding for drug analysis.Several models of this type exist, for examplereimbursement criteria under Medicare for an arrayof circumstances. One section of DHHS’s 1987‘‘Medicare Program Criteria for Medicare Coverageof Heart Transplants” (52 FR 10935) requires thateligibility for Medicare reimbursement for hearttransplants depends on a facility’s demonstratedexperience and survival rate. For DNA analysis offorensic samples, tying funding to actual perform-ance on proficiency tests could have a powerfulinfluence on the quality of services.

In addition to stipulations for direct funding tocrime laboratories, the Federal Government also hasthe power to condition the receipt of Federal moniesby a State (instead of by a single laboratory) on theState’s taking a specific regulatory action. Examplesof these types of stipulations include recent policiesthat tie State highway improvement grants to maxi-mum speed limits or the minimum drinking age.Thus, the Federal Government could link fundsprovided to State commissions or agencies to theadoption of certain quality assurance procedures orregulations that effect both State and local crimelaboratories. The power to apply such conditions islikely true even when the connection between theState program and DNA analysis is quite attenuated(92). Congress could mandate, for example, certainquality monitoring protocols for States acceptingfunds for prison construction or other non-DNA-related criminal justice uses.

Authority Over Interstate Commerce

The second major area over which Congress haswide authority to regulate forensic uses of DNAtechniques is through the commerce clause of Arti-cle I, Section 8, which provides the authority “Toregulate Commerce . . . among the severalStates. . . .“ Congressional authority to pass lawsrelating in any reasonable manner to interstate com-merce is such a broad power that judicial reviewaffirming the right is largely a formality (87). Mostjudicial review focuses instead on the intent of


A laboratory of the FBI DNA Analysis Unit, Washington, DC.

Congress to interpret the reach and scope of thelegislation.

Regulation of Products—The commerce powerprovides Congress the specific authority to regulatearticles of commerce that pass between two or moreStates. The Federal Government clearly could usethe commerce authority to require licensing offorensic facilities that solicit or provide forensicDNA typing services to out-of-State clients, as it hasfor medical laboratories engaged in interstate com-merce (42 U.S.C. 263). At the moment, only ahandful of facilities would be subject to regulationby Congress under this authority. Extremely broad,this authority also could be used to establish amechanism to regulate products used in DNA typingfor forensic applications.

Monitoring the Use of DNA Technologies-Con-gressional and executive interest and oversight ofDNA technologies is not unprecedented. Recom-binant DNA technologies have been subject toFederal scrutiny since the early 1970s. The NIHRecombinant DNA Advisory Committee, its Work-ing Group on Human Gene Therapy, and morerecently the Biotechnology Science CoordinatingCommittee have all been established to monitor orregulate various uses arising from the new genetictechnologies. Thus, the Federal Government couldestablish a committee or commission to monitor orregulate forensic applications of DNA tests.

As mentioned, the FBI is the principal investiga-tive arm of the Department of Justice, with no directauthority to regulate individual crime laboratories.The FBI is under the authority of the AttorneyGeneral and acts under the Attorney General’s


general statutory authority. Some suggest that theFBI should use its authority to issue official stan-dards for DNA analysis of forensic casework. On theother hand, others oppose an official role for the FBI,believing its laboratories and research facilitiesshould be subject to an independent commission orauthority established to provide guidance and over-sight of all private and public entities that do forensicDNA identification.

SETTING STANDARDSSetting standards for forensic applications of

DNA testing is the most controversial and unsettledissue, yet standards are the cornerstone of qualityassurance. Technical and operational standardsfor DNA typing in forensic casework are needed,and needed soon. The FBI (23), industry, molecularbiologists, biochemists, population geneticists, andforensic scientists all agree that standards are desira-ble. Agreement on what standards are appropri-ate, who should decide, how implementation ofstandards is best achieved, and whether theyshould be mandatory has not yet been reached.

Technical standards are needed to specify propergel controls, electrophoresis conditions, the extentthat computer-assisted matching should be permit-ted, population data to compute probabilities ofmatches, and many other parameters. It appears thatsetting technical standards-allowing flexibility forthe vagaries of forensic casework and emergingscientific developments—is within reach. A major-ity agree that such efforts should include balancedinput from all relevant scientific disciplines.

In contrast, operational standards, such as recordkeep-ing and proficiency testing, are likely to be morecontroversial, for attempts to regulate any sectorhave historically been met with resistance. Never-theless, such standards are necessary if full qualityassurance is to be achieved. Forensic scientists—practitioners and educators—argue that they aremost knowledgeable about how best to set opera-tional standards that achieve quality and meet theneeds of crime laboratories without being undulyburdensome. Some in the forensic community areprepared to meet this challenge.

Yet while many forensic scientists acknowledgethe need for standards in DNA typing, they resent theimposition of such standards by another scientificcommunity unfamiliar with the vagaries of forensic

casework—i.e., molecular geneticists. Some molec-ular geneticists, on the other hand, believe theirexperience over the past two decades with recombi-nant DNA technologies places them in a position todefine how DNA tests should be applied to forensiccasework. In fact, both communities can and shouldcontribute to standard setting. Forensic practitionersare most familiar with the practical problems andunique situations that can arise in the course of aninvestigation, which are situations not encounteredby molecular geneticists in laboratories. Likewise,research molecular biologists have knowledge aboutDNA tests on which forensic examiners can draw.Forensic academicians, who often are involved inearly stages of evaluating basic research tools beforea technology transfers into crime laboratories, areperhaps well placed to bridge the gap between crimelaboratory personnel and genetics researchers.

Many have expressed the opinion that an inde-pendent commission is the best mechanism to han-dle both technical and operational standards. Otherscall for a lead role for the FBI, which some reject asa conflict of interest. Still others seek Federal orState legislative solutions. In any case, crime labora-tories and forensic research have generally beenunderfunded, and new requirements will only in-crease financial difficulties. In addition, variousFederal grant assistance mechanisms have beenseverely reduced or eliminated in the past decade.Thus, while development of standards for recordkeep-ing and proficiency testing should be encouraged tomove forward, their costs should be recognized.Nevertheless, formalizing quality assurance mech-anisms, including standards, should proceed with-out delay. Such efforts will assist crime laboratoriesmaking decisions about using DNA profiling onsite(see ch. 6), private laboratories, the Federal Govern-ment, and the courts.

Some commentators contend that ultimately thejudicial process can provide a stringent test ofscientific evidence and the quality of work in aparticular case. Others strongly disagree, maintain-ing that it seldom does. The vast majority of scien-tific evidence introduced in criminal cases goesunchallenged by the opposition, usually the defense(72), which generally lacks sufficient resources todispute such evidence. Many argue that reliance onjudicial review for quality assurance has been anunfulfilled promise.


STANDARDIZATIONSetting standards to ensure quality is distinct from

developing a uniform, national system—i.e., stan-dardization-of DNA typing within the forensicscience community. Some contend that standardiz-ing the process is institutional insurance-an addi-tional layer of quality assurance. Others maintainthat while this step is a necessary component ofusable investigative DNA databanks, its role inensuring quality is minimal. Still others believestandardization could stifle rapid integration of fu-ture innovations.

No amount of standardization, especially of proce-dures, however, can be substituted for appropriatescientific analysis during the progress of an individ-ual case. By nature, nothing is routine in forensiccasework. The discretion of a qualified investigatorto evaluate a situation and implement appropriatemeasures-within standards that need to be estab-lished—is a fundamental component of quality as-surance.

Is standardization desirable, or even possible?Chapter 5 discusses standardization in greater detail.Nevertheless, achieving some modicum of standard-ization (e.g., for restriction enzyme-probe combina-tions used and data interpretation), appears neces-sary for an effective, national database.

FINDINGS AND SUMMARYPrior to the DNA era, the genetic analysis of

forensic samples was based strictly on a paradigm ofexclusion. Each genetic marker provided limitedinformation that eliminated a fraction of the generalpopulation as the originator of a sample (e.g.,excluding 30 percent, 67 percent, or 1.2 percent ofpersons as potential contributors), depending on themarker detected and the test result. Combiningresults for several different markers reduced the poolof persons who could have contributed to the biolog-ical sample. The objective, of course, was to excludeas many individuals as possible—i.e., to reduce thenumber of potential sources to the smallest possiblevalue.

DNA tests operate no differently. Yet their poten-tial power to discriminate has altered the perceptionthat genetics can be used only for positive exclusion,not positive identification. Among all humans ex-cept identical twins, no two share the same DNAsequence. Using single-locus probe analysis, foren-

sic examiners can accurately detect some of thesedifferences to the extent that examination of severalDNA markers can lead to a report that is, in effect,perceived to be a statistically positive associationbetween an individual and a piece of biologicalevidence. This change in perception, however, doesnot alter the fact that forensic uses of DNA tests—like traditional genetic marker analysis—are valid.

Are DNA tests reliable? Under routine conditionsof use, do they perform reproducibly within alaboratory, across many laboratories, and in thehands of different practitioners? OTA finds that,properly performed, DNA tests per se are relia-ble. Serious questions are raised, however, abouthow best to ensure that any particular test resultis reliable. These questions focus on data interpreta-tion, how to minimize realistic human error, and theappropriate level of monitoring to ensure quality.Such questions, which stem from actual court cases,underscore the need to develop both technical andoperational standards now.

Standards alone should not be construed as mak-ing evidence analysis absolutely reliable. Standardswould, however, provide a benchmark against whichall interested parties can judge a particular analysis.Undoubtedly some queries will still arise on acase-by-case basis. At such times, specific detailscan and should be evaluated in court. But, with timeand implementation of standards, such questionsshould decrease.

What standards are needed for private and publicfacilities doing forensic DNA testing, and whodecides? At present, only a vague consensus existsfor the first question, and none for the second. Nordoes consensus exist on how standards should beadministered. Professional societies are making ef-forts toward regularization of forensic uses of DNAtests, as is the Federal Government, especially theFBI and NIST. Some contend that such efforts areinsufficient because compliance is, or will be, en-tirely voluntary. These voices argue that qualityassurance lapses in both private and public facilitieswill persist with voluntary guidelines. Balancedagainst this is the belief of many that voluntarystandards are sufficient. Implicit in this point of viewis the conviction that consensus and implementationof technical guidelines and standards is imminent.

Yet while consensus has been achieved for someissues, other areas remain contentious. One area


needing particular attention is the population genet-ics of RFLP analysis. Controversy centers on thesize of the databases used and the precise approachthat should be used to calculate population frequen-cies. Some argue that the magnitude of the numberis not the issue, just that the analyst assigns it withconfidence that genetics principles have been ad-hered to. Others argue that because of the pivotalrole population frequencies can play in reportingresults of forensic DNA tests, agreement is neces-sary. Nevertheless, using certain conservative as-sumptions probably allows an analyst to assert alikelihood that matched samples came from thesame person. General agreement does exist that anypotential bias that could result from calculatingpopulation frequencies favor a defendant.

One area of population genetics of forensic DNAtyping might have an impact on both data analysis

Photo credit: Gary Ellis

and privacy considerations (see ch. 5). The dynamicand diverse nature of the U.S. population calls forspecial attention to collection and classification ofgenetic differences based on ethnic and racial sub-groups. For example, genetic data classified as‘‘Hispanic’ on the basis of self-identification orsurname could skew reported population frequen-cies, since DNA profiles for Mexican-Americans v.other Hispanic individuals, including those of PuertoRican, Cuban, or El Salvadoran descent, coulddiffer. Increased population data for RFLP analysiswould benefit both questions of population substruc-turing, as well as calculating population frequenciesin general.

Quality assurance mechanisms in forensic DNAprofiling encompass a range of options, includingcertification, licensure, accreditation, and proficiencytesting. Methods to implement these options exist,


such as efforts by professional societies, and formal,nonregulatory methods such as consensus buildingamong all interested stakeholders. States have au-thority to regulate DNA typing for forensic purposesby both private and public facilities, but to date noState has enacted general licensing requirements forprivate laboratories, crime laboratories, or person-nel. Likewise, the Federal Government has theauthority to direct that solutions be found for qualityassurance of forensic services.

Federal efforts toward quality assurance for labo-ratories doing forensic DNA profiling need notdevelop in a vacuum. Congress and the executivebranch have a longstanding interest in quality assur-ance for other laboratory services, most notablyclinical diagnostics and employee drug testing. Assuch, solutions for these sectors could prove usefulin evaluating quality assurance for laboratories per-forming DNA analysis in forensic casework.

Setting standards for quality assurance shouldproceed without delay. Such efforts will assist pri-vate laboratories, the Federal Government, thecourts, and public crime laboratories making deci-sions about implementing DNA profiling onsite.Such endeavors must also acknowledge that intro-ducing and maintaining formal quality assurancemechanisms can be costly and time-consuming, andwill place additional personnel and financial bur-dens on public facilities already overwhelmed withcasework and traditionally underfunded.

Finally, many questions surrounding forensicuses of DNA technologies are really questions ofpublic policy, as much as technical and opera-tional issues of forensic practice. As such, theinfluence and input of attorneys, businesses, govern-ment officials, and others in settling quality assur-ance issues is appropriate and important.

1.

2.

CHAPTER 3 REFERENCESActon, R. T., “Comparison of VNTR Allele Fre-quencies in White and Black Populations,” pro-ceedings, The International Symposium on HumanIdentification 1989: Data Acquisition and Statisti-cal Analysis for DNA Typing Laboratories, Madi-son, WI, 1990.Adams, D.E., Baechtel, F. S., and Budowle, B.,“DNA Analysis of Body Fluid Stains Subjected toEnvironmental, Chemical and Biological Insults,”abstract, American Academy of Forensic Sciences,41st Annual Meeting, Las Vegas, NV, February1989.

3.

4.

5.

6.

7.

8.

9.

10.

11,

12.

13,

14.

15.

16.

American Association of Blood Banks, AABBStandards Committee, “P.7000 DNA Polymor-phism Testing,” Standards for Parentage TestingLaboratories, 1989.American Society for Histocompatibility and Im-munogenetics, ASHI Quarterly 14(1), 1990.American Society for Quality Control, “QualitySystems Terminology,” American National Stan-dard ANSI/ASQC A3-1978, 1978.American Society of Crime Laboratory Directors—Laboratory Accreditation Board, Accreditation Man-ual (Lansing, MI: T.J. Nasser, Michigan StatePolice, 1985).American Society of Human Genetics, Ad HocCommittee on Individual Identification by DNAAnalysis, “Individual Identification by DNA Anal-ysis: Points to Consider,” American Journal ofHuman Genetics 46:631-634, 1990.Andreasen, P. B., “Consensus Conferences in Dif-ferent Countries: Aims and Perspectives,’ Interna-tional Journal of Technology Assessment in HealthCare 4:305-308, 1988.Baird, M., Balazs, I., Giusti, A., et al., “AlleleFrequency Distribution of Two Highly Polymor-phic DNA Sequences in Three Ethnic Groups andIts Application to the Determination of Paternity,American Journal of Human Genetics 39:489-501,1986.Balazs, I., Baird, M., Clyne, M., et al., “HumanPopulation Genetic Studies of Five HypervariableDNA Loci,’ American Journal of Human Genetics44:182-190, 1989.Bashinski, J. S., California DNA Lab, CaliforniaDepartment of Justice, Berkeley, CA, personalcommunication, August 1989.Bashinski, J. S., ‘‘Laboratory Standards: Accredita-tion, Training and Certification of Staff in theForensic Context,” DNA Technology and ForensicScience, J. Ballantine, G. Sensabaugh, J. Witkow-ski (eds.) (Cold Spring Harbor, NY: Cold SpringHarbor Laboratory Press, 1989).Bashinski, J.S. (chair), “California Association ofCrime Laboratory Directors: DNA Committee Re-port of CACLD Blind Trial #2,” California DNALab, California Department of Justice, Berkeley,CA, March 1990.Biotechnology Newswatch, “FBI To Offer SeedMoney for Probing Forensic DNA Proficiency,”Biotechnology Newswatch 9(18):5, 1989.Blake, E. T., ‘‘DNA Amplification and Typing FromAged Biological Evidence,” abstract, An Interna-tional Symposium on the Forensic Aspects of DNAAnalysis: U.S. Department of Justice, Federal Bu-reau of Investigation, Quantico, VA, June 1989.Brenner, C., and Morns, J., “Paternity Index Calcu-lations in Single Locus Hypervariable DNA Probes:


17.

18.

19.

20.

21.

22.

23.

24.

25.

26.

27.

28.

29.

Validation and Other Studies,” proceedings, TheInternational Symposium on Human Identification1989: Data Acquisition and Statistical Analysis forDNA Typing Laboratories, Madison, WI, 1990.Budowle, B., ‘‘Data for Forensic Matching Criteriafor VNTR Profiles,’ proceedings, The InternationalSymposium on Human Identification 1989: DataAcquisition and Statistical Analysis for DNA Typ-ing Laboratories, Madison, WI, 1990.Budowle, B., Federal Bureau of Investigation, Quan-tico, VA, personal communication, April 1989.Budowle, B., Shutler, G., Waye, J., et al., ‘‘Selectionof Restriction Endonuclease and Interprobe Com-parison for RFLP Technology,’ abstract, AmericanAcademy of Forensic Sciences, 41 st Annual Meet-ing, Las Vegas, NV, February 1989.Buoncristiani, M., von Beroldingen, C., and Sensa-baugh, G. F., “Effects of W Damage on DNAAmplification by the Polymerase Chain Reaction, ’abstract, An International Symposium on the Foren-sic Aspects of DNA Analysis: U.S. Department ofJustice, Federal Bureau of Investigation, Quantico,VA, June 1989.California Association of Crime Laboratory Direc-tors, “Position on DNA Typing of Forensic Sam-ples” (Oakland, CA: J. Bashinski, Oakland CrimeLaboratory, November 1987).California Association of Crirninalists, Bureau ofForensic Science and Unysis Corp., Report of aSymposium on the Practice of Forensic Serology,1987 (Sacramento, CA: F. Springer, Bureau ofForensic Services, 1987).Castonguay, R. T., Federal Bureau of Investigation,Washington, DC, personal communication, April1989.Cohen, J. E., “DNA Fingerprinting for ForensicIdentification: Potential Effects on Data Interpreta-tion of Subpopulation Heterogeneity and BandNumber Variability, ’ American Journal of HumanGenetics 46:358-368, 1990.Comey, C. T., Federal Bureau of Investigation, Quan-tico, VA, personal communication, April 1989.Criminalistics Certification Study Committee, Cer-tification in Criminalistics: A Find Report to theProfession (Los Angeles, CA: W.J. Cadman, Cali-fornia State University, Los Angeles, 1979).Criminalistics Certification Study Committee, Cer-

tification of Criminalists: A Report to the Criminal-istics Profession (LOS Angeles, CA: W.J. Cadman,California State University, Los Angeles, 1988).Erlich, H. A., Cetus Corp., Emeryville, CA, personalcommunications, May, August 1989.Evett, I. W., and Werrett, D. J., “Bayesian Analysisof Single Locus DNA Profiles, ’ proceedings, TheInternational Symposium on Human Identification

30.

31.

32.

33.

34.

35.

36.

37.

38.

39.

40.

41.

42.

43.

44.

45.

1989: Data Acquisition and Statistical Analysis forDNA Typing Laboratories, Madison, WI, 1990.Fienberg, S.E. (cd.), Panel on Statistical Assess-ments as Evidence in the Courts, National ResearchCouncil, The Evolving Role of Statistical Assess-ments as Evidence in the Courts (New York, NY:Springer-Verlag, 1989).Garner, D. D., Cellmark Diagnostics, Germantown,MD, personal communication, July 1989.Garner, D. D., Green, D. J., Herrin, G. L., et al.,“Validation Studies Utilizing the Jeffreys DNAProbes,’ abstract, American Academy of ForensicSciences, 40th Annual Meeting, Philadelphia, PA,February 1988.Gill, P., Jeffreys, A.J., and Werrett, D.J., “ForensicApplication of DNA ‘Fingerprints’,’ ‘Nature 318:577-579, 1985.Gill, P., Lygo, J.E., Fowler, S.J., et al., “An Evalua-tion of DNA Fingerprinting for Forensic Purposes,’Electrophoresis 8:38-44, 1986.Giusti, A., Baird, M., Shaler, R., et al., ‘Applicationof Deoxyribonucleic Acid (DNA) Polymorphismsto the Analysis of DNA Recovered From Sperm, ”Journal of Forensic Sciences 31;409-417, 1986.Givens, R. A., Botein, Hays & Sklar, New York,NY, personal communication, July 1989.Goldenberg, M., “Standards, Public Welfare De-fenses, and the Antitrust Laws,’ Business Lawyer42(3):629-674, 1987.Grunbaum, B. W., Moraga, CA, personal communi-cation, June 1989.Grunbaum, B. W., “Physiological State Evidence:Guidelines to Assure Quality Analysis,” The Cali-fornia Defender 1:20-26, 1984.Harrisburg Hospital v. Thornburgh, 616 F. Supp.699 (D.C. Pa. 1985).Hicks, J. W., Federal Bureau of Investigation, Wash-ington, DC, personal communication, February 1990.Higuchi, R., and Blake, E, T., “Applications of thePolymerase Chain Reaction in Forensic Science,”DNA Technology and Forensic Science, J. Ballan-tine, G. Sensabaugh, J. Witkowski (eds.) (ColdSpring Harbor, NY: Cold Spring Harbor LaboratoryPress, 1989).Higuchi, R., von Beroldingen, C. H., Sensabaugh,G. F., et al., “DNA Typing From Single Hairs,”Nature 332:543-546, 1988.Institute of Medicine, Council on Health CareTechnology, Committee to Improve the NationalInstitutes of Health Consensus Development Pro-gram, Consensus Development at the NIH: Improv-ing the Program (Washington, DC: National Acad-emy Press, 1990).International Society for Forensic Haemogenetics,“DNA Recommendation,” Vox Sang 57:276-277,1989.


46.

47.

48.

49.

50.

51.

52.

53.

54.

55.

56.

57.

58.

59.

60.

61.

Jeffreys, A. J., University of Leicester, Leicester,England, remarks at An International Symposiumon the Forensic Aspects of DNA Analysis: U.S.Department of Justice, Federal Bureau of Investiga-tion, Quantico, VA, June 1989.Jeffreys, A. J., Wilson, V., Neumann, R., et al.,“Amplification of Human Minisatellites by thePolymerase Chain Reaction: Towards DNA Finger-printing of Single Cells,” Nucleic Acids Research16:10953-10971, 1988.Jeffreys, A. J., Wilson, V., and Thein, S.L., “Hyper-variable Minisatellite Regions in Human DNA, ”Nature 314:67-73, 1985.Jeffreys, A. J., Wilson, V., and Thein, S. L., “Indi-vidual Specific ‘Fingerprints’ of Human DNA, ”Nature 316:76-79, 1985.Kanter, E., Baird, M., Shaler, R., et al., ‘Analysis ofRestriction Length Polymorphisms in Deoxyribo-nucleic Acid (DNA) Recovered From Dried Blood-stains,” Journal of Forensic Sciences 31:403408,1986.Kearney, J. J., Federal Bureau of Investigation, Quan-tico, VA, personal communication, August 1989.Kearney, J. J., Federal Bureau of Investigation, Quan-tico, VA, presentation before the U.S. Congress,Office of Technology Assessment, Advisory Panelon Forensic Uses of DNA Tests, April 1989.Kiely, T., “Making Medical Labs Measure Up,”Technology Review, pp. 14-15, July 1989.Kilshaw, D., “Quality Assurance. 1. Philosophyand Basic Principles, ’ Medical Laboratory Sci-ences 43:377-381, 1986.Kilshaw, D., “Quality Assurance. 2. Internal Qual-ity Control,” Medical Laboratory Sciences 44:73-83, 1987.Kilshaw, D., “Quality Assurance. 3. External Qual-ity Assessment” Medical Laboratory Sciences 44: 178-186, 1987.Kirby, L. T., DNA Fingerprinting: An Introduction(New York NY: Stockton Press, 1990).Kosecoff, J., Kanouse, D.E., Rogers, W.H., et al.,“Effects of the National Institutes of Health Con-sensus Development Program on Physician Prac-tice,’ Journal of the American Medical Association258:2708-2713, 1987.Kuo, M., Orange County Sheriff-Coroner’s CrimeLaboratory, Santa Ana, CA, personal communica-tion, July 1989.Kuo, M. (chair), “California Association of CrimeLaboratory Directors: DNA Committee Report tothe Board of Directors, Report #6,’ Orange CountySheriff-Coroner’s Crime Laboratory, Santa Ana,CA, 1988.Kwok S., and Higuchi, R., “Avoiding False Posi-tives With PCR,” Nature 339:237-238, 1989.

62.

63.

64.

65.

66.

67.

68.

69.

70.

71.

72.

73.

74.

75.

76.

77.

Lander, E. S., “DNA Fingerprinting On Trial,”Nature 339:501-505, 1989.Lander, E. S., “Reliability and Validity of DNATyping for Forensics,” contract paper prepared forthe Office of Technology Assessment, U.S. Con-gress, October 1989.Li, H., Gyllensten, U. B., Cui, X., et al., ‘ ‘Amplifica-tion and Analysis of DNA Sequences in SingleHuman Sperm and Diploid Cells,’ Nature 355:414-417, 1988.Lucas, D., Leete, C., and Field, K., “An AmericanProficiency Testing Program,” Forensic SciencesInternational 27:71, 1985.McNally, L., Shaler, R. C., Giusti, A., et al., “Evalu-ation of Deoxyribonucleic Acid (DNA) IsolatedFrom Human Bloodstains Exposed to UltravioletLight, Heat, Humidity, and Soil Contamination,”Journal of Forensic Sciences 34(5):1059-1069, 1989.McNally, L., Shaler, R. C., Giusti, A., et al., “TheEffects of Environment and Substrata on Deoxyri-bonucleic Acid (DNA): The Use of CaseworkSamples From New York City,” Journal of Foren-sic Sciences 34(5):1070-1077.Moenssens, A.A., Inbau, F.E., and Starrs, J.E.,Scientific Evidence in Criminal Cases, 3rd ed.(Mineola, NY: The Foundation Press, Inc., 1986).Mueller, L. D., University of California, Irvine,Irvine, CA, personal communication, August 1989.Neufeld, P. J., and Colman, N., “When ScienceTakes the Witness Stand,” Scientific American262:46-53, 1990.Perry, S., “The NIH Consensus Development Pro-gram: A Decade Later,” New England Journal ofMedicine 317:485-488, 1987.Peterson, J.L., Department of Criminal Justice, TheUniversity of Illinois at Chicago, Chicago, IL,personal communication, August 1989.Peterson, J. L., Field, K., Fabricant, E., et al., CrimeLaboratory Proficiency Testing Research Program(Washington, DC: U.S. Government Printing Of-fice, 1978).Peterson, J. L., Mihajlovic, S., and Bedrosian, J. L.,‘‘The Capabilities, Uses and Effects of the Nation’sCriminalistics Laboratories,” Journal of ForensicSciences 30(1):10-23, 1985.Rau, R. M., National Institute of Justice, Washing-ton, DC, personal communication, July 1989.Reeder, D. J., National Institute of Standards andTechnology, Gaithersburg, MD, personal commu-nication, April 1989.Reynolds, R., von Beroldingen, C., and Sensa-baugh, G. F., “Effects of DNA Degradation onAmplification by the Polymerase Chain Reaction,’abstract, An International Symposium on the Foren-sic Aspects of DNA Analysis: U.S. Department of


78.

79.

80.

81.

82.

83.

84.

85.

86.

87.

88.

89.

90.

Justice, Federal Bureau of Investigation, Quantico,VA, June 1989.Saiki, R.K., Gelfand, D.H., Stoffel, S., et al., “Primer-directed Enzymatic Amplification of DNA With aThermostable DNA Polymerase,’ Science 239:487-494, 1988.Schaeffer, M. (cd.), Federal Legislation and theClinical Laboratory (Boston, MA: G.K. Hall Medi-cal Publishers, 1981).Sensabaugh, G.F., “Consequences of NucleotideMisincorporation During the Polymerase ChainReaction, abstract, An International Symposiumon the Forensic Aspects of DNA Analysis: U.S.Department of Justice, Federal Bureau of Investiga-tion, Quantico, VA, June 1989.Sensabaugh, G. F., “Genetic Typing of BiologicalEvidence: Comments for the Cooper Amicus Brief,”California Association of Criminalists Newsletter,pp. 11-19, July 1987.Sensabaugh, G. F., University of California,Berkeley, CA, personal communication, August1989.Southern Association of Forensic Scientists, Foren-sic Serology Training Program, E. Quarles (cd.)(Decatur, GA: Bureau of Forensic Science, 1988).Southern, E., “Detection of Specific SequencesAmong DNA Fragments Separated by Gel Electro-phoresis,” Journal of Molecular Biology 98:503-517, 1975.Technical Working Group on DNA Analysis Meth-ods (TWGDAM), ‘‘Guidelines for a Quality Assur-ance Program for DNA Restriction FragmentLength Polymorphism Analysis,” Crime Labora-tory Digest 16(2):40-59, April-July 1989.Thompson, W. C., and Ford, S., “DNA TypingAcceptance and Weight of the New Genetic Identi-fication Tests,” Virginia Law Review 75:45-108,1989.Tribe, L., American Constitutional Law (Mineola,NY: Foundation Press, 1978).U.S. Congress, General Accounting Office, Em-ployee Drug Testing: Regulation of Drug TestingLaboratories, GAO/GGD-88-127FS, Washington,DC, 1988.U.S. Congress, House of Representatives, Commit-tee on Energy and Commerce, Clinical LaboratoryImprovement Amendments of 1988, Rpt. 100-899 toaccompany H.R. 5150 (Washington, DC: U.S. Gov-ernment Printing Office, 1988),U.S. Congress, Office of Technology Assessment,Development of Medical Technology: Opportuni-

91.

92.

93.

94.

95.

96.

97.

98.

99.

100.

101.

ties for Assessment, OTA-H-34 (Springfield, VA:National Technical Information Service, 1976).U.S. Congress, Office of Technology Assessment,Genetic Testing in the Workplace (Washington,DC: U.S. Government Printing Office, forthcom-ing, 1990).U.S. Congress, Office of Technology Assessment,Infertility: Medical and Social Choices, OTA-BA-358 (Washington, DC: U.S. Government PrintingOffice, 1988).U.S. Congress, Office of Technology Assessment,Mapping Our Genes-Genome Projects: How Big,How Fast?, OTA-BA-373 (Washington, DC: U.S.Government Printing Office, 1988).U.S. Congress, Office of Technology Assessment,New Developments in Biotechnology: Public Per-ceptions of Biotechnology, OTA-BP-BA-45 (Wash-ington, DC: U.S. Government Printing Office, 1987).U.S. Congress, Office of Technology Assessment,Technologies for Detecting Heritable Mutations inHuman Beings, OTA-H-298 (Washington, DC: U.S.Government Printing Office, 1986).U.S. Congress, Senate, Committee on Governme-ntal Affairs, Subcommittee on Oversight of Gover-nment Management, Health Care Financing Admini-stration’s Management of Medical Laboratories,hearings Mar. 23-24, 1988, S. Hrg. 100-765 (Wash-ington, DC: U.S. Government Printing Office, 1988).U.S. Department of Health, Education, and Welfare,Ethics Advisory Board, Report and Conclusions:HEW Support of Research Involving Human InVitro Fertilization and Embryo Transfer (Washing-ton, DC: U.S. Government Printing Office, 1979).von Beroldingen, C. H., Blake, E. T., Higuchi, R., etal., “Applications of PCR to the Analysis of Bio-logical Evidence,” PCR Technology: Principlesand Applications for DNA Amplification, H.A.Erlich (cd.) (New York NY: Stockton Press, 1989).Walsh, S., Blake, E., and Higuchi, R.,‘‘PCR Inhibi-tion and Bloodstains, ” abstract, An InternationalSymposium on the Forensic Aspects of DNA Anal-ysis: U.S. Department of Justice, Federal Bureau ofInvestigation, Quantico, VA, June 1989.Werrett, D. J., remarks at Banbury Conference on“DNA Technology and Forensic Science,” ColdSpring Harbor, NY, November 1988.Wing, K. R., The Law and the Public’s Health, 2ded. (Ann Arbor, MI: Health Administration Press,1985).

validity, reliability, and quality assurance

Documents