cia’s kent school: improving training for new · pdf filestephen marrin cia’s kent...

STEPHEN MARRIN

CIA’s Kent School: Improving Trainingfor New Analysts

The Central Intelligence Agency’s (CIA’s) Career Analyst Program (CAP)for new analysts seeks to increase their on-the-job effectiveness and abilityto produce more accurate analysis. The program is located within theSherman Kent School for Intelligence Analysis, which CIA’s seniormanagers created in 2000 to increase the expertise of officers within itsDirectorate of Intelligence (DI). The school provides analyst andmanagerial training for the DI, as well as housing the Kent Center whichacquires and disseminates information regarding analytic ‘best practices.’1

According to an Agency press release, the CAP is CIA’s ‘‘firstcomprehensive training program for professional intelligence analysts,’’and is a noticeable improvement on prior training efforts.2 The CAPprovides new analysts with the knowledge and skills that enable them tobe more effective in the production of finished intelligence, as well as intheir overall job performance. It also provides them with the means toproduce more accurate analysis by teaching them the causes of—andmeans to avoid—intelligence failure, and cognitive tools to assist them instructuring their analysis more along the lines of the scientific method.

Whether the CAP will lead to an improvement in the quality of the CIA’sanalytic output will depend on whether analysts have the opportunity toapply their newly acquired expertise when back on the job after training.Ultimately, the institutional assessments of analytic production processeswill determine whether analytic training programs allow the CIA to

Stephen Marrin, a former Central Intelligence Agency analyst and contractor,is in the doctoral program at the University of Virginia, specializing inintelligence studies. An earlier version of this article was presented at theannual meetings of the Intelligence Studies Section of the InternationalStudies Association, New Orleans, Louisiana, in March 2002.

International Journal of Intelligence and CounterIntelligence, 16: 609–637, 2003

Copyright # Taylor & Francis Inc.

ISSN: 0885-0607 print/1521-0561 online

DOI: 10.1080/08850600390198779

AND COUNTERINTELLIGENCE VOLUME 16, NUMBER 4 609

improve its production of accurate, timely, and tailored intelligence that fitsthe needs of the Agency’s national security policymaking customers.

IMPROVING THE CIA’S ANALYSES

Just as every reform is intended to be a solution to a particular problem, theCIA’s leaders created the Kent School as a way to address the Agency’sanalytic weaknesses. In May 1998, Director of Central Intelligence (DCI)George J. Tenet ‘‘gave a classified internal briefing on the CIA’s problemsand what he intends to do about them,’’ according to an article in USNews and World Report.3 In articulating his vision for the future of theCIA—a vision that became known as his ‘Strategic Direction’—Tenetemphasized the importance of changing past practices to improve theAgency’s contribution to national security policymaking. A year later, in18 October 1999, he told an audience at Georgetown University inWashington, D.C.:

[When] I launched a Strategic Direction plan for the CIA . . . I told ourpeople that we had to take charge of our destiny. That we would do allwithin our power to maintain our edge and our vibrancy. That we hadto streamline and realign ourselves and adopt business practices likethe best in the private sector. That we would think big and thinkdifferent. That we would work smarter and in new ways so that wewould have the agility and the resiliency to do what the President—this President or a future President—wants and the American peopleexpect.4

One week after this speech, the CIA failed to warn American policymakersof India’s intention to test nuclear weapons. This failure highlighted theAgency’s limitations and likely accelerated the implementation of reformsstemming from Tenet’s Strategic Direction plan.5 Post hoc analyses citedmany factors which may have contributed to the failure, but notable wasthe DI’s ‘‘lack of critical thinking and analytic rigor,’’ causing it to fail toadd together all indications of a possible nuclear test.6,7 According to TheWall Street Journal’s Carla Anne Robbins, Admiral David Jeremiah—whoheaded the official investigation into the failure—‘‘recommended hiringmore analysts, improving their training and increasing contact with outsideexperts to challenge conventional wisdom.’’8 She also reported that DCITenet said he would ‘‘make it my highest priority to implement [AdmiralJeremiah’s recommendations] as quickly as possible.’’9

Tenet subsequently commissioned a number of task forces to implementhis Strategic Direction. The ‘‘Analytic Depth and Expertise Task Force,’’made up of eight DI officers, was given free rein to investigate ways to‘‘develop true world class experts’’ consistent with the Strategic Direction’sgoals, according to Kent School program director Denis Stadther.10 The

610 STEPHEN MARRIN

INTERNATIONAL JOURNAL OF INTELLIGENCE

officers met several times a week through the latter half of 1998,brainstorming ideas to increase the expertise of junior, mid-level, andsenior analysts. The task force ruled out a training course for mid-levelofficers because they believed that the broadening experience provided byrotational assignments, such as those overseas or to policymakinginstitutions, would provide greatest value to the Agency. The task forcealso ruled out training for senior analysts. It noted that the DI’s careerpaths provided greater advancement possibilities for senior analysts willingto switch to management, and as a result the more ambitious analystsbecame managers rather than continuing to use their knowledge in ananalytic capacity. Therefore, the task force proposed the creation of amore desirable career track in which senior analysts could continue to usetheir expertise rather than shift into management. Finally, the task forceconcluded that junior officers would be best served with a trainingprogram intended to ‘‘build a common foundation upon which analyticexpertise could be built.’’

The task force submitted two recommendations to then-DDI John E.McLaughlin, who approved both, and subsequently attributed theirimplementation to DCI Tenet’s support and tenure that ‘‘lasted longerthan the ‘task force phase,’’’ according to Vernon Loeb of the WashingtonPost.11 In 2000, the career track allowing ‘‘analysts to rise to very seniorrank without branching out into management’’ took form as the SeniorAnalytic Service.12 Also that year, the improved new analyst programbegan operating through the newly created Sherman Kent school. Theschool was named for Kent, chairman of the CIA’s Board of NationalEstimates from 1952 to 1967, and considered ‘‘a leader in developing theprofession of intelligence analysis.’’13 At the dedication of the Kent Schoolin May 2000, Tenet praised Kent as a ‘‘brilliant teacher to generations ofanalysts,’’ and expressed his wish that ‘‘this school . . . [will] always produceanalysts of whom he would be proud.’’14

MECHANISM OF IMPROVEMENT

The Kent School was created to improve CIA’s analytic quality, in part byincreasing the expertise of its analysts. At the opening ceremonies, Tenetstated that the Kent School ‘‘will prepare generations of men and womenfor the . . .profession of intelligence analysis in the 21st Century . . . [byteaching] the best of what we as an Agency have learned about the craft ofanalysis.’’15 According to Martin Petersen, director of the CIA’s humanresources department, the Kent School’s creation ‘‘sends a very, verypowerful message about what you value, and the value here . . . is analyticexpertise.’’16 The Kent School improves the CIA’s analytic production by

611CIA’S KENT SCHOOL: IMPROVING TRAINING FOR NEW ANALYSTS

AND COUNTERINTELLIGENCE VOLUME 16, NUMBER 4

providing analysts with knowledge that they otherwise would acquirehaphazardly or not at all in on-the-job training.

Its mission of increasing analyst expertise corresponds with the efforts ofmany other professions, and even other intelligence organizations, thattrain employees as part and parcel of organizational improvementprograms. Training programs, in general, provide information specific tothe needs of the institution, and different institutions within thegovernmental foreign policy process use training programs to bolster theirunique informational needs. For example:

� The State Department’s Foreign Service Institute provides area and languagestudies helpful to outgoing State Department Foreign Service Officers andother government officials;

� The Department of Defense’s (DOD) command colleges—including the ArmyWar College, Naval War College, and National Defense University amongothers—provide rising military officers with a more advanced conceptual andinformational context within which to make command decisions;

� The Joint Military Intelligence Training Center (JMITC) specializes in providingshort-course training to the DOD’s military intelligence analysts, according toKen Olson, the chief of JMITC’s General Intelligence Training Branch. Itscompanion institution, the Joint Military Intelligence College (JMIC), providesbroader education consisting of degree and certificate programs in intelligenceat the graduate and undergraduate level.17

Training is intended to increase expertise, which is ‘‘the skill of an expert,’’and this skill is crucial to the production of accurate intelligence analysis.18

‘‘In the lexicon of US intelligence professionals,’’ observes intelligencescholar Loch K. Johnson, ‘‘analysis’ refers to the interpretation by expertsof unevaluated (‘raw’) information collected by the intell igencecommunity.’’19 The more expert an analyst is, the more likely that he orshe will produce a higher quality product. DCI Tenet described theimportance of an analyst’s expertise at the Kent School’s dedication:

In our [DI] it is not enough just to make the right call. That takes luck.You have to make the right call for the right reasons. That takesexpertise. It is expertise—built up through study and experience—thatcombines with relevance and rigor to produce something that is veryimportant: insight. . . . [O]ur analysts blend a scholar’s mastery of detailwith a reporter’s sense of urgency and clarity. At its best, the result isinsight. And it is insight that wins the confidence of our customers andmakes them want to read our publications and listen to our briefings.20

The DI’s exploitation of analytic expertise in its production of finishedintelligence entails the integration of a complex web of analytic specialtiesto produce multi-disciplinary analysis. Not hired by the CIA or even theDI, most analysts are hired by the individual DI offices, assigned to

612 STEPHEN MARRIN


‘‘groups’’ which cover specific geographic areas, and then assigned afunctional specialty—‘‘discipline’’ or ‘‘occupation’’ in DI terminology—such as political, military, economic, leadership, or scientific, technical, andweapons intelligence.21 For the most part, CIA analysts possess a verysmall area of direct responsibility, defined by a combination of theirregional area and discipline, work in country ‘‘teams’’ with analysts ofother disciplines, and interact with other regional or disciplinary specialistsas the need arises. Therefore, an analyst’s expertise can vary, depending onthe relative possession of regional knowledge, disciplinary theory, andintelligence methods in general:

� Regional expertise is essentially area studies: a combination of the geography,history, sociology, and political structures of a specific geographic region. TheDI’s regional offices, such as the Office of Near East, South Asia and Africa,are responsible for an analyst’s regional expertise and develop it by providingaccess to language training, regional familiarization through university courses,or in-house seminars.

� Procedural expertise is knowledge of ‘‘tradecraft’’; the sometimes unique methodsand processes required to produce intelligence analysis. The Kent School is theprimary repository for tradecraft information and its training programs are theprimary means of distribution to both analysts and managers.

� Disciplinary expertise is the theory and practice that underlies the individualanalytic occupations. For example, political, military, economic, andleadership analysis are all built on beds of theory derived from the academicdisciplines of political science, military science, economics, and politicalpsychology, respectively. Disciplinary expertise can be acquired throughprevious academic coursework, on-the-job training, and through occupational‘‘representatives’’ distributed throughout the offices as focal points fordisciplinary resources.

The CIA’s small analytic niches create specialists, but their specialties mustbe reintegrated in order to provide high-level policymakers with a biggerpicture that is more accurate and balanced than the limited perspective orknowledge of the niche analyst. This process of reintegration includes aprocedure known as ‘‘coordination,’’ in DI parlance, which allows analystsof all kinds to weigh in with their niche expertise on pieces of finishedintelligence before they are disseminated. According to CIA analyst FrankWatanabe: ‘‘We coordinate to ensure a corporate product and to bring thesubstantive expertise of others to bear.’’22 Accordingly, the bureaucraticnorm is that an analyst will make every effort to coordinate a draft withother analysts in related regional or disciplinary accounts prior tosubmitting the draft to management for editing and dissemination. As aresult, while the expertise of the primary drafter of the piece of intelligenceanalysis usually has primary influence on the accuracy of the final piece,the coordination process exerts a strong influence as well.



Inaccuracies in the analytic product can occur as a result of insufficientsubstantive and disciplinary expertise. Columbia University ProfessorRobert Jervis points out that ‘‘a grav(e) danger lies in not having sufficientexpertise about an area of a problem to detect and interpret importanttrends and developments. To make up for such deficiency, analysts tend toimpose on the information the concepts, models, and beliefs that they havederived elsewhere.’’23 In addition, in 1991 former DCI Stansfield Turnernoted that: ‘‘Another reason for many of the analytic shortcomings is thatour analytical agencies do not have an adequate grasp of the cultures ofmany countries with which we must deal.’’ He goes on to suggest thatanalysts could use a ‘‘a better opportunity to attend academic institutions,participate in professional conferences, travel and live abroad, acquirelanguage skills and thus become true experts in their areas.’’24 RobertJervis adds ‘‘adequate training’’ and ‘‘first-hand exposure to the country’’as additional methods to increase expertise.25

Expertise is not necessarily sufficient to produce accuracy or preventfailure, though. Jervis notes that: ‘‘experts will [not] necessarily get theright answers. Indeed, the parochialism of those who know all the factsabout a particular country that they consider to be unique, but lack theconceptual tools for making sense of what they see, is well known.’’26

In addition, ‘‘even if the organizational problems . . . and perceptualimpediments to accurate perception were remedied or removed, we couldnot expect an enormous increase in our ability to predict events’’ because‘‘the impediments to understanding our world are so great’’ that‘‘intelligence will often reach incorrect conclusions.’’27 Human cognitivelimitations require analysts to simplify reality through the analytic process,but reality simplified is no longer reality. As a result, ‘‘even experts can bewrong because their expertise is based on rules which are at best bluntapproximations of reality. In the end any analytic judgment will be anapproximation of the real world and therefore subject to some amount oferror,’’ and analytic inaccuracies—and sometimes intelligence failure—willbe inevitable.28,29

In sum, analyst expertise is an important component of analytic quality,but even if the Kent School’s analytic training were ideal it could neverincrease expertise to the point that perfection would be attainable. Inaddition, the CIA’s interdependent analytic structure implies that trainingcan increase aggregate capabilities only through the cumulative effects ofmultiple kinds of targeted training. Even so, that these factors limit theKent School’s potential to improve analyst quality does not precludeimprovement on the margins. If training can improve analytic quality itdoes so by increasing the knowledge and skills at the analyst’s disposal.

The CIA’s intelligence analysts require an understanding of their rolewithin the foreign policy process, tools that help them structure and

614 STEPHEN MARRIN


analyze complicated issues, a theoretical framework to approach an issuefrom a specific disciplinary perspective, such as political or economicanalysis, and the presentational skills necessary for busy policymakers toincorporate into their decisionmaking processes. Yet, prior to the KentSchool’s creation, new analysts were not receiving the information theyneeded to do their jobs effectively.

OVERCOMING PREVIOUS TRAINING DEFICIENCIES

The CIA’s central role in providing high-level policymakers with analysis ofnational security threats and opportunities might be expected to requirestringent training for new analysts prior to conducting analysis and writingreports for senior policymakers. In fact, CIA analysts have been hired andassigned a desk at Agency headquarters without any analytic trainingwhatsoever, with two or four weeks of analytic training coming six monthsor so later.30 For example, in 1996 the introduction to the CIA for newanalysts consisted of just one week of pro forma briefings regardingsecurity and other administrative issues, with nothing specific to analysis.Within weeks, and in some cases days, these newly hired analysts wereproducing finished intelligence—albeit limited and under the careful watchof more senior analysts—without any official training whatsoever insubstance or methods.

This minimal formal training appears to have historical roots in the CIA.According to former DDI and current Deputy Director of CentralIntelligence (DDCI) John McLaughlin: ‘‘Dick Lehman, the creator of whatwe now call the President’s Daily Brief, once remarked that the basicanalytic training he got back in 1949 came down to a single piece of advicefrom his boss: ‘Whatever you do, just remember one thing—the SovietUnion is up to no good!’ Simple, but that said it.’’31 In addition, MarkLowenthal, a former staff director of the House Permanent SelectCommittee on Intelligence, has said that CIA does not ‘‘do a lot oftraining. . . . They say, ‘Congratulations, you’re the Mali analyst, have anice day.’’’32 The training process usually relied upon the analyst’s priorformal education, combined with an initial period of sink-or-swimadaptation to the DI. The sink-or-swim analogy is used frequently insidethe Agency to describe its junior analyst acculturation. In May 2001, theKent School’s former dean, Frans Bax, likened previous DI training tobeing thrown into the deep end of a pool, and added that if the training ormentoring ‘‘missed,’’ the analyst ‘‘floundered.’’33

In 1996, the CIA improved its training for junior analysts with the creationof a month-long survey course for new analysts entitled ‘‘Fundamentals ofDI Tradecraft’’ (FDIT). Some of FDIT’s content was based on atradecraft course developed under the auspices of former DDI Douglas



J. MacEachin, and delivered to all analysts and managers.34 Nonetheless,although FDIT was intended only as a limited survey course, it received afair amount of criticism internally in its first few runnings due to the slowpace and inconsistent quality of instruction.35 It also demonstrated that amere month of training was too short to familiarize analysts with therange of information that they needed to know to do their jobs correctly.But these weaknesses became strengths, as the criticisms may haveprovided the Analytic Depth and Expertise task force with indicators thatthere was greater potential inherent in the new analyst training than wasbeing actualized through FDIT. In the end, the task force was able to usethe lessons derived from FDIT to inform the structure of the Kent Schooland the content of its new analyst training program.

One member of the Analytic Depth and Expertise task force had beenintimately involved in FDIT’s development as a DI representative to theCIA’s Office of Training and Education (OTE), and he contributed thelessons he learned from this to the task force.36 After deciding torecommend training for junior analysts, the task force began to preparethe outlines of a proposal. Prior DI training programs run through OTE,located in CIA’s Directorate of Administration, had difficulty obtainingqualified instructors from the DI because of the prevalent belief that non-analytic assignments in OTE were not career-enhancing. In addition,OTE’s separate structure led to difficulties providing training tailored tothe DI’s needs. To overcome these problems, the task force recommendedthat training be run through the DI so that it could be better integratedwith the DI’s requirements. Also suggested was a school structure thatwould ensure continuity and a knowledge base for future training efforts.A final suggestion was that the school be headed by a Senior IntelligenceService officer with a presence on the DI’s corporate board in order toprovide greater ‘‘bureaucratic weight and budgetary authority’’ for theschool to meet its goals.37 Then-DDI McLaughlin implemented all thesuggestions. As a result, the Kent School’s location within the DI andbureaucratic structure provide it with a solid base from which to deliver itsinformational product.

Although the CAP used lessons derived from FDIT, it was, according toDenis Stadther, ‘‘a clean-sheet exercise in developing new training that wasfounded in the belief that comprehensive training for new analysts mustgo beyond skills development to include history, mission, and values ofthe DI as well as orientation to specialized knowledge and skills likeoperations, intelligence collection, and counterintelligence issues.’’38

Comprehensive training could not be shoehorned into a one-month courseakin to FDIT, however, and the CAP’s length became a matter of somedispute. According to the school’s former dean, Frans Bax, DCI Tenet had‘‘chided the DI for not taking training as seriously as the [CIA’s

616 STEPHEN MARRIN


Directorate of Operations or DO]’’—which has a year-long program fortrainees—and ‘‘instructed the DI to create a program as long as theDO’s.’’39 As a result, the Analytic Depth and Expertise task force initiallyproposed a ten-month long training program. But this was opposed by theDI offices that were facing tight personnel resource constraints andreluctant to lose their new analysts for that long. To accommodate theirconcerns, the program was reduced from ten months to twenty-five weeks,and yet retained ‘‘all [its] training components’’ by providing analysts withfewer interim assignments to other components.40 The CAP’s curriculumundergoes constant revision derived from analyst and manager feedback ina search to make the program more effective. According to Denis Stadther,between 2000 and 2002, the CAP’s length was reduced to twenty-twoweeks, as the curriculum was refined due to student feedback and acomprehensive review of the curriculum by the CAP faculty.

CAP’S TRAINING CURRICULUM41

By February 2002, the CAP had become a twenty-two-week program which‘‘teaches newly hired analysts about the CIA’s and the DI’s history andvalues and develops the basic skills essential to an intelligence analyst’ssuccessful career in the directorate.’’42 The initial week entails anintroduction to intelligence topics, including the history, mission, andvalues of the CIA, and a unit on the history and literature of intelligencethat is taught by the history staff of the Agency’s Center for the Study ofIntelligence.43 The following few weeks introduce analysts to a variety ofDI skills, including analytic thinking, writing and self-editing, briefing,data analysis techniques, and a teamwork exercise akin to an ‘‘outwardbound’’ for analysts. Training in these kinds of basic skills is necessary,even for CIA analysts with advanced degrees, because intelligence analysisfocuses on the informational needs of policymakers who ask differentquestions than those encountered in academia. Also, briefing skills are notemphasized in academia but are vital for the effective distribution ofinformation within government. In addition, analysts are not accustomedto writing in the specialized writing style of the DI, which emphasizesbottom-line judgments at the beginning of the piece, while strategicallyusing evidence to bolster the main conclusions. This training is reinforcedthroughout the CAP, as analysts write an intelligence paper in which theyself-consciously apply lessons learned to a real-world analytic problem.44

The first five weeks of training culminate in a task force exercise thatprovides analysts with the opportunity to take the skills they have learnedand practiced in a classroom setting and apply them in a more realistic one.

After five weeks in the classroom the CAP students go on a four-weekinterim assignment that helps them understand how the DI relates to other



components of the CIA, the intelligence community, and the policymakingagencies they serve. The analysts—in consultation with their managers—identify positions within the bureaucracy that are related to theirassignment, and provide working knowledge of units that each analyst willeither work with or rely upon throughout his or her career.45 In particular,seeing how intelligence analysis is viewed and used in other components ofthe policymaking bureaucracy can help new analysts understand their roleand perform it better, according to Denis Stadther. These interimassignments can entail a stint at a military command—such as the PacificCommand headquartered at Pearl Harbor—which provides the analystwith insight into working with a military customer. These assignments canhave other benefits as well. In early 2001, DDCI McLaughlin spokeenthusiastically of the insight that a new analyst can acquire from visitingPearl Harbor and seeing first-hand the reason the CIA was created, andhow he or she fits into a fifty-year tradition of working to prevent similardestruction.46

After this four-week interim assignment, the analysts return to theclassroom for four more weeks of training in more advanced topics, suchas writing and editing longer papers and topical modules that addressissues such as denial and deception, and indicators and warning.47

According to Stadther, these are special kinds of analysis that requireapplication of more advanced and sophisticated analytic tradecrafttechniques. The analysts also broaden their knowledge of other positionswithin the intelligence community by visiting other intelligence agenciesand participating in an ‘‘ops familiarization course’’ which provides a briefexposure to the intense training that a DO case officer receives.48 Theanalysts are also provided with greater information on tradecraft topics—consisting of concepts, techniques, and methodologies—that form the coreof the intelligence analysts’ specialized skill set, which they can use torigorously and self-consciously analyze raw intelligence data. This skill setincludes adapting the study of intelligence issues to the needs of the user,presenting complicated issues in simple terms stripped of long-windedassumptions, qualifications, and background, and addressing issues relatedto controversial foreign policies and policy goals without taking a policyposition.49 Specifically, the CAP uses Sherman Kent’s ‘‘Principles forIntelligence Analysis’’ as guidelines for its students to follow as they learnthe craft of analysis.50 These guidelines— listed here in full in theAppendix—emphasize the importance of intellectual rigor, the consciouseffort to avoid analytical biases, a willingness to consider other judgments,and the systemic use of outside experts as a check on in-house blinders, aswell as the importance of candidly admitting analytic shortcomings andlearning from mistakes.

618 STEPHEN MARRIN


The CAP also teaches other tradecraft skills, such as alternative analysistechniques including, but not limited to, scenario building and competitive‘‘A=B Team’’ approaches that entail periodic checks on analysis byexternal experts.51,52 In addition, the CAP assigns Richards Heuer’swritings to provide analysts with tips and techniques for overcoming flawsand limitations in the human thinking process.53 The CAP also usesanalytic failure as a teaching tool for what not to do. Since much of theintelligence analysis tradecraft has been derived from intelligence failure,‘‘the lessons of . . . high-profile CIA screw-ups form the core of the KentSchool curriculum,’’ according to reporter Bob Drogin of the Los AngelesTimes. Drogin quotes the school’s dean as saying, ‘‘We spend a lot of timein this course studying mistakes.’’54

The analysts then go for a second four-week interim assignment, and comeback for a final four-week classroom session that deals with more advancedtopics. For example, a session on politicization and professional ethicsteaches analysts ‘‘how to walk the fine line that separates effectiveintelligence support for policymakers from prescribing policy,’’ accordingto Stadther. Another session, entitled ‘‘Writing for the President,’’ istaught by senior DI editors, provides insight and lessons learned in writingfor the CIA’s most senior and demanding policymakers. The CAP alsoprovides new analysts with a ‘‘roundtable on customer relations’’ in whichexperienced analysts provide their hard-earned lessons in dealing withCongress, the public, and the press, and a ‘‘secrets of success panel’’providing tips of what does and does not work from analysts with a fewyears of experience.

Once again, the classroom session ends with a task force exercise. This timeit entails a two-day terrorist crisis simulation that takes place outside theclassroom, and requires total immersion to handle the fifteen-hour days.The CAP instructors, using this as an opportunity to present analysts withsituations they might see later in their careers, spike the simulation withdilemmas to force the analysts to improvise and apply the skills theylearned throughout the course. As reporter Bob Drogin notes, it is‘‘deliberately designed to be intensive and stressful.’’55 Finally, at the endof the program, the CAP provides written evaluations of the students totheir home offices, documenting their skills, abilities, and performanceduring trainings.56 In sum, the CAP’s twenty-two weeks provide analystswith information unique to the realm of intelligence analysis, as well as‘‘greater opportunity for individual growth in understanding of theprofession, and hopefully a greater self-conscious rigor in the use ofanalytical tools.’’57

In addition to more rigorous content, the CAP utilizes interactive teachingmethods to maximize analyst learning and acquisition of expertise.



The training is intended to improve analyst performance. According to KentSchool Dean Dan Wagner, rather than risk student disengagement fromabstract lecture courses, the programs use varied and active teachingmethods which are ‘‘very important’’ to provide multiple pathways forstudents to apply their unique learning styles.58 In addition, the CAP alsoemploys the case study method to make the lessons more fungible to real-world situations.59 Former CIA instructor Thomas Shreeve has noted that‘‘cases fit best in those parts of the CIA curriculum in which the teachingobjectives included helping students learn how to analyze complex,ambiguous dilemmas and make decisions about how to resolve them.’’60

In sum, the CAP provides analysts with knowledge that they could notacquire elsewhere, and improves on prior efforts in this area. If theselessons are absorbed and subsequently applied, analytic quality mayimprove. But training effectiveness and the growth of analyst expertise willdepend on several variables, including student motivation, peer groupinfluence, and teaching methods. The CAP’s use of interactive teachingmethods is aimed at maximizing learning, and provides an assessment tothe analyst’s home office similar to and with the incentives of a reportcard. Some of that knowledge will fall by the wayside, however, asanalysts discard the lessons that are not relevant to their specific accounts.Other lessons, such as the promulgation of Sherman Kent’s principles tonew analysts, are more goals than standards, and may be modified byinevitable workplace pressures. For example, professional practices andpromotional concerns may mitigate against the universal application of‘‘candid admission of shortcomings.’’ Nonetheless, the CAP’s emphasis onpractices such as coordination, cooperation, and admission of one’s ownanalytic limitations provides the impetus to change the sometimeshidebound internal DI culture. As its former dean, Frans Bax, noted in2001, the Kent School is ‘‘trying to embed the concept of career-longlearning within the culture of the DI.’’61 By disseminating tradecraft andtechniques widely throughout the DI, individual analyst expertise willincrease. But concluding that this increased expertise may or will causeimprovement to the DI’s analytic quality has yet to be justified.

INCREASING ANALYST EFFECTIVENESS

The CAP appears to be an upgrading of the CIA’s past efforts to provideanalysts with job-relevant expertise, although measuring this improvementfrom outside the institution is impossible. Hypothetically, evaluatingthe CAP’s impact is possible through a simple pretest-posttest controlgroup research design.62 For example, to establish the efficacy of theschool in increasing new analyst expertise a test could be administeredboth before and after training to measure any increases in analyst

620 STEPHEN MARRIN


knowledge. A control group composed of new analysts who remain on thejob instead of receiving training could be tested as well, and any changesin analyst knowledge between the two groups would be attributable to thetraining alone.

In October 2001, the Kent School’s director implemented a version ofthis alternative design by hiring an outside contractor to interview CAPalumni and their managers to assess the effect of training on their jobperformance.63 CAP program director Denis Stadther provided anoverview of its conclusions. According to Stadther, all intervieweesconsidered the CAP to have been beneficial. In particular, line managersnoted that new analysts who went through the program appeared topossess greater confidence and maturity in applying their skills on thejob. In addition, CAP analysts created lasting ties with their classmates,and the use of these contacts increased their ability to work the systemin such job-related tasks as obtaining information from a counterpartagency, coordinating with experts elsewhere within the government, andfinding a fellow analyst within the CIA who covers a particular issue.Managers appreciated the analysts’ greater institutional expertise becauseit meant that they required less intense mentoring to perform well on thejob. Stadther also noted that, contrary to the belief of early CAPstudents that their promotion prospects might be adversely affectedby being pulled off the job for five months of training, anecdotalevidence suggests that their promotion rates have been as good as or betterthan those who did not attend the CAP. Stadther states that a goodperformance by analysts in the CAP appears to translate well into goodjob performance.

But he also emphasized that not all CAP student feedback was positive,and that this led to adjustments in the curriculum. For example, whenanalysts had difficulty relating to a theoretical unit intended to improveinterpersonal skills, the CAP’s program manager refocused it on thepractical application of those skills. Nonetheless, even though feedbackindicates that some aspects of the CAP’s curriculum require modification,in the aggregate the CAP improves a new analyst’s individual effectiveness,and this may improve the DI’s effectiveness on the margins.

Training also provides other benefits to analyst effectiveness that Stadtherdid not mention was part of the study. For example, CAP promotes a betterfit between analyst and assignment by encouraging analysts to pursuetransfers to units where their skills or interests may be more effectivelyapplied.64 The CIA’s placement policies are cumbersome due to the six-month or greater lag between the time a vacancy opens up and a qualifiedperson can be hired because of the background checks and other mattersnecessary to obtain security clearances. This lengthy process can lead toinitial assignments that are a misfit for an analyst’s knowledge and



abilities. In 2001, Frans Bax noted that the CAP ‘‘provides the opportunityfor officers to find a better fit for themselves,’’ and that ‘‘there is an infiniterange of career possibilities and if an analyst sees something better theyshould go for it.’’65 He added that, in the one year the CAP had been inoperation, a number of analysts had switched offices and even directorates,and that this was an intentional attempt to break out of the apprenticeshipmodel that had formerly formed the core human resource approach for theDI.66 Therefore, the CAP provides CIA with the opportunity to shiftpersonnel in a way that allows for interest and skill mixes to be appliedmore effectively towards institutional goals.

In sum, the CAP provides analysts with a greater knowledge of theinstitution and its practices that can make their job performance moreeffective, and even side benefits like providing the opportunity to refineplacement policies with transfers can improve institutional performance onthe margins. But neither addresses the core reason the Kent School wascreated: to improve the CIA’s analytic quality.

TOWARD IMPROVING ACCURACY

More fundamental to the mission of intelligence analysis than bureaucraticsavvy or presentational ability, however, is the accuracy of the finishedintelligence product. According to the CIA’s Website, its mission includes‘‘providing accurate, evidence-based, comprehensive, and timely foreignintelligence related to national security.’’67 Directly measuring the CAP’simpact on analytic quality is impossible from outside the institution, dueto the fact that intelligence analysis is inherently classified. In addition, sofar as I know, neither the CIA in general nor the school in particularmeasures the accuracy of the analytic product because accuracy inintelligence analysis is a very difficult concept to operationalize.

Since intelligence analysis can influence what American national securitypolicymakers decide to do, and what they do has the potential to promptor preclude actions of other international actors, a hit-or-miss yardstickwould not effectively capture the quality of the analysis.68 For example, ifthe CIA predicts that a terrorist bombing is imminent and policymakersimplement security procedures based on CIA warnings, and the terroristsare deterred due to the increased security measures, then the intelligenceprediction may well be considered inaccurate even though it helped preventthe bombing. This causal dynamic exists for all intelligence issues—including political, economic, and scientific—due to the nature of theintelligence mission. Therefore, post-hoc assessment of intelligenceaccuracy may not provide a true sense of the quality of the analysis.Instead, both the Kent School’s and the CIA’s evaluations focus on the

622 STEPHEN MARRIN


soundness of the analytic process because it is implicitly a modified version ofthe scientific method.

Analytic Processes Familiar

The CIA analyst uses a version of the scientific method to create intelligenceanalysis. The traditional term ‘‘intelligence cycle’’ describes how an analystintegrates information collected by numerous entities and disseminates thisinformation to policymakers. As former DCI William Colby noted, ‘‘Atthe center of the intelligence machine lies the analyst, and he is the fellowto whom all the information goes so that he can review it and think aboutit and determine what it means.’’69 While the ‘‘intelligence cycle’’ presentsthis process in sequential terms, more accurately the analyst is engaged innever-ending conversations with collectors and policymakers over thestatus of international events and their implications for United Statespolicy. In converting raw intelligence data into finished analysis, analystsinterpret the international environment through an information processingmethodology approximating the scientific method.70 As intelligence authorWashington Platt noted in 1957: ‘‘The so-called ‘scientific method’ meansdifferent things to different people, but the basic features are much thesame. These features are: collection of data, formation of hypotheses,testing the hypotheses, and so arriving at conclusions based on theforegoing which can be used as reliable sources of prediction.’’71

An analyst builds an operating hypothesis upon constructs foundin a particular country’s history, culture, regional dynamics, governmentalworkings, or whatever aspect may be relevant to determine how thatcountry’s leaders might respond to foreign challenges. The Kent Schoolbolsters this knowledge with both procedural and disciplinary expertise toprovide a methodological and theoretical base for the production ofintelligence analysis. For example, academia develops theories to explainoutcomes in the realms of economics, politics, psychology, andinternational relations, and these theories form the basis for any goodintelligence analysis. The analyst’s reigning conceptual framework isgrounded in disciplinary theory modified by knowledge of regional orcountry-specific substance. Analysts simultaneously—and for the mostpart instinctively—use inductive reasoning to find patterns amidst theflood of data, and use deductive reasoning to provide meaning and contextfor the patterns they find. Incoming bits of intelligence data are filteredthrough this framework, and those that fit are imbued with meaning andcontext, while those that do not fit are set aside as potential modifiers ofthe concepts. As a result, intelligence analysis, when done accurately, is amodel of social scientific inductive and deductive reasoning in action.



Yet, despite the similarities to social science research, intelligence analysisis more complex. Two methodologists note that intelligence analysis hasdifficulties— including externally imposed time constraints whichnecessitate analysis prior to acquisition of sufficient information, aninability to control the variables under study, an unknown data qualitydue to imperfections in the collection process and the possibility ofdeception, an emphasis on prediction, and a focus on utility—that,combined, make intelligence analysis more difficult than social scientificresearch in an academic setting.72

Also, like many specialties in the study of international relations,intelligence analysts cannot test their hypotheses or replicate outcomes inthe real world, and instead must allow the progression of events to falsifythem. In addition, international relations writ large has a minimalempirical base relative to other fields such as medicine or physics, meaningthat the etiology of certain kinds of activities, such as war or revolution,cannot be aggregated into anything other than a very general anduncertain theory. Unlike academics, though, the intelligence analyst doesnot create theory, but must interpret facts that are themselves subject todistortion and deception, at times under tight deadlines.73 This means thathigh levels of uncertainty are implicit in most analytic judgments.

As a result of the high levels of uncertainty, intelligence analysis rarely hascertainty or proof and, for the most part, conclusions are tentative. In whathas become a cliche, ‘‘evidence collected by intelligence agencies is oftenambiguous and can lead to differing conclusions.’’74 In addition, Israelischolars Yitzhak Katz and Ygal Vardi cite an experiment demonstratingthat the same data can lead to differing conclusions, depending on theconceptual structures applied.75 The result of such ambiguity is anenvironment of conflict and debate over the interpretation and implicationsof intelligence data. Loch K. Johnson says that within the institutionalcontext ‘‘analysis is usually a messy affair, with incomplete informationand often heated debates over what the available facts may really meanabout the intentions of secretive adversaries in distant capitals. . . .Uncertainty, ambiguity, debate, and partial answers to tangled questionswill remain an existential condition of the analytical process.’’76

Given this complex information environment, not knowing or negligencein applying social science techniques can lead to analytic inaccuracies. AsRobert Jervis observed in 1986: ‘‘Good intelligence demands threeinterrelated conditions. Analysts should present alternatives and competingexplanations for a given even[t], develop the evidence for each of thealternatives, and present their arguments as fully as is necessary to dojustice to the subject matter. This is not to imply, of course, that if theseconditions are met the resulting analyses will always be excellent but onlythat their omission will substantially reduce the probability that the

624 STEPHEN MARRIN


product will be of high quality.’’77 At its most extreme outcome, analyticinaccuracies due to failure to apply tradecraft can lead to intelligencefailures. As intelligence scholar Robert Folker notes, ‘‘the root cause ofmany critical intelligence failures has been analytical failure.’’78 Katz andVardi also argue that, although Israeli intelligence possessed ‘‘datapointing to the probability of war,’’ it was surprised by the 1973 YomKippur War because it ‘‘was incorrectly interpreted.’’79 In addition,Georgetown Professor Walter Laqueur notes that sources of analyticfailure include ‘‘bias . . . [which is] an unwillingness . . . to accept evidencecontrary to their preconceived notions, . . . ignorance, lack of politicalsophistication and judgment, [l]ack of training and experience, lack ofimagination, and the assumption that other people behave more or less aswe do,’’ which is known in intelligence parlance as ‘mirror-imaging.’80 Insum, failure to apply appropriate social science standards of rigor andprocedure can lead to analytic inaccuracy. If so, then accuracy can beimproved on the margins by providing analysts with greater knowledge offact, theory, or procedure, and the application of methodologies tosimplify and structure their analysis.

Kent School: Improving CIA’s Analytic Procedure

The Kent School’s CAP provides analysts with improved methodologicalknowledge and analytical tools that should provide a more rigorous basisfor their analysis. This greater rigor, approximating social science methodsshould increase the chance that the analytic product will be more accuratethan it otherwise would have been without training. Just as increases in thequality of research methodology allow a researcher to more effectivelyoperationalize and measure the constructs of interests, better training inmethodology would hypothetically provide an intelligence analyst with abetter foundation from which to produce more accurate analysis.Therefore, if the theory holds, then the Kent School will provide analystswith the capability to produce more accurate analysis, so long as itsprograms provide information that enables a more effective application ofthe scientific method.

Many possible methodological tools can be used to organize and simplifythe process of intelligence analysis. As Washington Platt noted in 1957: ‘‘Inintelligence production as yet we find little study of methods as such. Yet asystematic study of methods in any given field by an open-minded expertin that field nearly always leads to worthwhile improvement.’’81 Forexample, one JMIC publication—a distillation of a student’s Mastersthesis—demonstrates that ‘‘analysts who apply a structured method(specifically) hypothesis testing . . . to an intelligence problem, outperformthose who rely on . . . the intuitive approach.’’82 Just as social science is



structured upon a foundation of hypothesis falsification and revision whichrequires that hypotheses be compared for accuracy and relevance,intelligence analysts self-consciously compare hypotheses in a processtermed ‘‘competitive analysis.’’ This technique can be applied in particularcircumstances—e.g., when an issue of particular importance is imbuedwith partisan implications—to get at underlying assumptions that mightbe biasing the analysis in one direction or the other.83 Some have evenadvocated the structural incorporation of this method through aninstitutionalized ‘‘Devil’s Advocate’’ position or ‘‘Team B’’ exercises.According to Richards Heuer, other tools that help analysts structure themost appropriate approach to a particular intelligence problem include‘‘outlines, tables, diagrams, trees, and matrices, with many sub-species ofeach. For example, trees include decision trees and fault trees. Diagramsincludes causal diagrams, influence diagrams, flow charts, and cognitivemaps. . . . [In addition, a matrix can be used] to array evidence for andagainst competing hypotheses to explain what is happening now orestimate what may happen in the future.’’84 Finally, R.V. Jones—the‘‘father of modern scientific and technical intelligence’’—suggests thatanalysts apply Occam’s Razor—or ‘‘the simplest hypothesis that isconsistent with the information’’—as a way to simplify the analysis ofcontradictory or insufficient information.85

The analytic tradecraft standards and techniques taught in the CAP weredeveloped in the 1990s. According to former CIA officer Jack Davis, formerDDI Douglas MacEachin articulated ‘‘corporate tradecraft standards foranalysts . . . aimed . . . at ensuring that sufficient attention would be paid tocognitive challenges in assessing complex issues.’’86 Davis added:

MacEachin advocated an approach to structured argumentation called‘‘linchpin analysis,’’ to which he contributed muscular terms designedto overcome many CIA professionals ’ distaste for academicnomenclature. The standard academic term ‘‘key variables’’ becamedrivers . ‘ ‘Hypotheses’’ concerning drivers became l inchpins—assumptions underlying the argument—and these had to be explicitlyspelled out. . . . MacEachin thus worked to put in place systematic andtransparent standards for determining whether analysts had met theirresponsibilities for critical thinking.87

MacEachin then formalized the promulgation of methodologies to all DIanalysts. In 1996 and 1997, ‘‘nearly all DI managers and analysts attended’’ anew course based on the standards entitled ‘‘Tradecraft 2000.’’88 In addition,in 1997 Jack Davis authored ‘‘a series of notes on analytical tradecraft’’ thatencapsulated the lessons from the new course.89 According to the CIAWebsite—where portions of these notes can be found—the notes‘‘elaborate on some of the skills and methods used by DI intelligence

626 STEPHEN MARRIN


analysts’’ and have ‘‘become a standard reference within CIA forpractitioners and teachers of intelligence analysis.’’90 These methodologieshave become, in essence, analytic doctrine, and through them CIA analystshave become familiarized with social science methodologies, if with amodified vocabulary. In 2001, the CIA Website noted that: ‘‘Invariably,analysts work on the basis of incomplete and conflicting information. DIanalysts are taught to clearly articulate what is known (the facts), how it isknown (the sources), what drives the judgements (linchpin assumptions),the impact if these drivers change (alternative outcomes), and whatremains unknown.’’91 The analytic standards were subsequentlyincorporated into the existing training courses for analysts, and maintaintheir place in the Kent School’s curriculum today.

In sum, the Kent School’s CAP provides analysts with the expertisenecessary to produce more accurate analysis. This conclusion, however, isinsufficient to address whether the school’s CAP actually improves thequality of the CIA’s analytic output.

BUREAUCRATIC PROCESSES CAN STIFLE POTENTIAL

Hypothetically, the CAP could improve an analyst’s analytic accuracy, butits potential could be stifled if the DI’s business practices preventapplication of the hard-earned expertise. Intelligence analysis takes placewithin a specific organizational context, and an organization’s processescan promote or impede the application of lessons learned in training, andaffect the ultimate accuracy of the intelligence product.

For example, the length of finished intelligence product emphasized bythe DI can impact expertise acquisition. The DI produces many types offinished intelligence—some reportorial, some analytical, some estimative—largely as a result of its attempt to meet policymakers’ varying needs forinformation.92 The reportorial products—known as current intelligence—tend to be shorter but more targeted to policymakers’ needs, while themore estimative and analytic products can be longer and potentially lessrelevant for policymakers due to the time of production and length ofproduct. Over time, the DI tends to emphasize the benefits of one productlength over another. In the 1980s, the DI emphasized the production oflonger pieces for the knowledge that they create, but after argumentssurfaced that these longer papers were of decreased uti l i ty forpolicymakers, the DI promptly reversed course in the mid-1990s andlimited ‘‘most DI analytical papers . . . to 3 to 7 pages, including graphicsand illustrations.’’93 Presumably, this was done to increase the relevance ofDI products for policymakers.

But, while the production of a single piece of current intelligence haslimited effect on expertise because it draws on the knowledge and tools



that an analyst has developed through training and prior analysis, expertiseacquisition and application problems become endemic if the institutionemphasizes current intelligence over longer products. If the analyst doesnot have time to think, learn, or integrate new information with old tocreate new understandings, knowledge of facts and events may increase,but the ability to accurately interpret these events decreases, as the analystdoes not have the opportunity to reinforce knowledge through the morein-depth research and drafting processes. The reliance on brevity alsodeprives analysts of the opportunity to apply ‘‘best practices,’’ includingthe structured tradecraft methodologies and benefits arising fromdisciplinary theory. By 1997, DI management was making efforts toaddress the decreased expertise levels resulting from excessive currentintelligence production.94 In the end, the Kent School’s effectiveness inimproving analytic accuracy may depend in part on whether the CIAprovides its analysts with assignments that allow them to utilize theexpertise acquired in training, while still providing relevant current analysis.

The length of time that the DI encourages an analyst to spend on anaccount can also impact an analyst’s expertise, and by extension, accuracy.In 2001, former CIA Inspector General Frederick P. Hitz argued thatCIA’s analysts have ‘‘tended to be 18-month wonders who hopscotch fromsubject to subject without developing any broad, long-term expertise in anyone area,’’ and as a result ‘‘are ill-equipped to grapple with theinformation that arrives on their desks.’’95 An analyst cannot applysophisticated methodological tools to intelligence data until he or shebecomes knowledgeable about the specific regional area or discipline of theaccount. Frequent changes will prevent an analyst from achieving thepotential inherent in the analytic tools that the Kent School provides newanalysts. Therefore, for the Kent School to improve analytic accuracythrough the application of increased expertise, ‘‘analysts should beexpected to make a career of their functional or geographic specialty sothat the daily flows of information are added to an already solid base ofknowledge,’’ as Hitz suggests.

In sum, the DI must adapt its practices to provide analysts with thegreatest opportunity to apply the expertise they have acquired both on-the-job and in training if the potential inherent in the Kent School’sprograms to improve the DI’s analytic accuracy is to be fully realized. Yet,despite the importance of institutional context in determining the CIA’sanalytic quality, few have studied the interplay between analyst andinstitution. As Robert Jervis has concluded, ‘‘perhaps the intelligencecommunity has paid too [little attention] . . . to how the community’sinternal structure and norms might be altered to enable intelligence to beworth listening to.’’96 Further studies of analytic structures and processesmust be done to determine which reforms would have the greatest

628 STEPHEN MARRIN


improvement on analytic quality, and perhaps their implementation will takeCIA a step closer to creating ‘‘intelligence worth listening to.’’

IMPROVING TO MEET FUTURE CHALLENGES

In 1988, Loch Johnson said ‘‘probably the most important problem faced byany intelligence agency [is] how to improve the quality of its product.’’97 TheKent School is part of the CIA’s effort to do this. The school has apparentlyachieved its goal of increasing analytic expertise by improving the Agency’straining for new analysts. This training improves the analysts’—andconsequently the DI’s—effectiveness, and should provide each analyst witha greater ability to produce more accurate intelligence analysis. But, even ifincreases in analyst expertise due to training can be documented, theanalyst may not have the time, inclination, or opportunity to applyrecently acquired expertise in the production of finished intelligenceanalysis. Analytic training may be a necessary component for improvementin analytic accuracy writ large, but is not sufficient in and of itself.Therefore, complementary modifications may have to be implemented tothe DI’s business practices if the desired outcome—increased analyticaccuracy—is to occur. As such, the Kent School is a step in the rightdirection, but further assessments are required to determine which changesin business practices or organizational structure might magnify possibleimprovements.

More broadly, other issues have implications for American intelligence.The tragic events of September 2001 have highlighted the limitations ofinstitutionalized intelligence, including the problems inherent in intelligenceanalysis, and present policymakers with the opportunity to redefine theintelligence community of the next decades. This time, according to anintelligence expert advising President George W. Bush on ways to improvethe workings of the intelligence community, ‘‘We are trying to treat theintelligence community more like a corporation that should have goals anda way to measure success or failure against those goals.’’98

Increased analytic quality, achieved through the Kent School’s CareerAnalyst Program helps CIA reach that goal. Yet, in the broader context ofnational security policymaking, improving the accuracy of the CIA’sanalytic output on the margins may not necessarily lead to improvedpolicymaking. Intelligence analysis is only one of many informationstreams that the policymaker listens to, and may not be the mostimportant. In addition, a policymaker’s decisions will be affected by policyagendas, tradeoffs, and other cognitive limitations that create ‘‘bounded’’rationality. Therefore, if the researcher’s primary concern is more effectivenational security policymaking, then it may be more useful in this limited-resource environment to assess how intelligence production can be



modified to better fit policymaker needs. In the end, the utility of intelligenceanalysis may be just as important a criterion of analytic quality as isaccuracy.99

Nevertheless, understanding the options available to improvepolicymaking is crucial, and improving the CIA’s analytic accuracy is oneof those options. As Robert Jervis has said, ‘‘We will never be able to doas well as we would like, but this does not mean that we cannot do betterthan we are doing now.’’100

The Kent School’s Career Analyst Program is a step in the right direction,but more steps may be necessary in order to produce a measurableimprovement in the CIA’s analytic quality.

APPENDIX

Sherman Kent’s Principles for Intelligence Analysis. The CareerAnalyst Program (CAP) Teaches These Principles:1. Intellectual Rigor

Judgments are supported by facts or credible reporting.All sources are reviewed and evaluated for consistency, credibility.Uncertainties or gaps in information are made explicit.

2. Conscious Effort to Avoid Analytical Biases

State working assumptions and conclusions drawn from them explicitly.Subject assumptions and conclusions to structured challenge: what developments

would indicate they would be wrong.If uncertainties or the stakes of being wrong are high, identify alternative

outcomes and what it would take for each to occur.

3. Willingness to Consider Other Judgments

Recognize the limits to your own expertise and avoid treating your account asyours alone.

Seek out expertise that will complement your own as a product is being prepared.Strong differences of view should be made explicit.

4. Collective Responsibility for Judgment

Seek out and allow time for formal coordination of your product.Represent and defend all Agency and DI views.Make it clear when you express individual views; do so only when asked.

5. Precision of Language

Provide your most unique or new insight or fact quickly.Use active voice and short sentences; avoid excessive detail; minimize the use of

technical terms. Follow DI writing guidelines.Shorter is always better.

6. Systematic Use of Outside Experts as a Check on In-House Blinders

630 STEPHEN MARRIN


Seek out new external studies and experts relevant to your account and disciplineon a continuing basis.

Keep up with news media treatment of your account and consider whether theirperspective offers unique insight.

On key issues, indicate where outsiders agree or disagree with your judgments.

7. Candid Admission of Shortcomings and Learning from Mistakes

Recognize that intelligence analysis will sometimes be wrong because it mustfocus on the tough questions or uncertainties.

Review periodically past judgments or interpretations; what made them right orwrong; how could they have been better.

Alert the policymaker if you determine that a previous line of analysis waswrong. Explain why and what it means.

8. Attentiveness To and Focus On Policymaker Concerns

Deliver intelligence that is focused on and timed to the policymaker’s currentagenda.

Make clear the implications of your analysis for U.S. policy.Provide ‘‘actionable’’ intelligence that can help the policymaker handle a threat,

make a decision, or achieve an objective.

9. Never Pursue a Policy Agenda

Personal policy preferences must not shape the information presented or theconclusions of intelligence analysis.

Politely but clearly deflect policymaker requests for recommendations on policy.Intelligence helps the policymaker by reducing the range of uncertainty and

risk, and by identifying opportunities for action. It does not make thechoice for him.

REFERENCES1CIA Website, ‘‘Tenet Dedicates New School for Intelligence Analysis,’’ CIAPress Release, 4 May 2000. http:==www.cia.gov=cia=public_affairs=press_release=archives=2000=pr050400.html

2CIA Website, ‘‘Tenet Lauds Appointment of McLaughlin as Acting DDCI,’’Press Release, 29 June 2000. http:==www.cia.gov=cia=public_affairs=press_release=archives=2000=pr06292000.html

3Bruce B. Auster, ‘‘What’s Really Gone Wrong with the CIA,’’ U.S. News andWorld Report, 1 June 1998, Vol. 124, No. 21, p. 27.

4George J. Tenet, ‘‘The CIA and the Security Challenges of the New Century,’’International Journal of Intelligence and CounterIintelligence, Vol. 13, No. 2,Summer 2000, pp. 140–141.

5Various newspaper articles including: Bob Drogin, ‘‘School for New Brand ofSpooks,’’ The Los Angeles Times, 21 July 2000, p. A-1.; Tim Weiner, ‘‘U.S.Intelligence Under Fire in Wake of India’s Nuclear Test,’’ The New York



Times, 13 May 1998; Walter Pincus, ‘‘CIA Chief Cited Loss of Agency’sCapabilities; Remarks Preceded Indian Bomb Tests,’’ The Washington Post,25 May 1998, p. A4.

6For a good presentation of the specific indicators, see Tim Weiner, ‘‘U.S.Intelligence Under Fire in Wake of India’s Nuclear Test,’’ op. cit.

7James R. Asker, ‘‘Same Ol’, Same Ol’ ’’ Aviation Week and Space Technology, 8June 1998.

8Carla Anne Robbins, ‘‘Failure to Predict India’s Tests Is Tied to SystemwideIntelligence Breakdown,’’ The Wall Street Journal, 3 June 1998, p. A8.

9Ibid.

10Denis Stadther was a member of this task force and provided all the informationon the Kent School’s creation during an interview on 14 May 2001, except whereotherwise indicated.

11Vernon Loeb, ‘‘CIA Goes Deep Into Analysis; Agency Opens School, ElevatesAnalysts,’’ The Washington Post, 4 May 2000, p. A23.

12CIA Website, ‘‘Tenet Lauds Appointment of McLaughlin as Acting DDCI.’’Also see: CIA Website, ‘‘John E. McLaughlin: Deputy Director of CentralIntelligence.’’ http:==www.cia.gov=cia=information=mclaughlin.html

13CIA Website, ‘‘Tenet Dedicates New School for Intelligence Analysis,’’ CIAPress Release, 4 May 2000.

14CIA Website, ‘‘Remarks of the Director of Central Intelligence George J. Tenetat the Dedication of the Sherman Kent School,’’ 4 May 2000. http:==www.cia.gov=cia=public_affairs=speeches=archives=2000=dci_speech_05052000.html

15CIA Website, ‘‘Remarks of the Director of Central Intelligence George J. Tenetat the Dedication of the Sherman Kent School, 4 May 2000.’’

16Vernon Loeb, ‘‘CIA Goes Deep Into Analysis; Agency Opens School, ElevatesAnalysts.’’ For more on Martin Petersen, see Vernon Loeb, ‘‘Tenet, KrongardAlter CIA Power Structure,’’ The Washington Post, 1 May 2001, p. A21.

17Ken Olson, Personal Interview, 16 May 2001; Defense Intelligence Agency(DIA) Website, ‘‘About the JMIC.’’ http:==www.dia.mil=Jmic=about.html

18Merriam-Webster Website: http:==www.m-w.com

19Loch K. Johnson, ‘‘Analysis For a New Age,’’ Intelligence and National Security,Vol. 11, No. 4, October 1996, p. 657.

20CIA Website, ‘‘Remarks of the Director of Central Intelligence George J. Tenetat the Dedication of the Sherman Kent School, 4 May 2000.’’

21CIA Website. Directorate of Intelligence: Organizational Componentshttp:==www.odci.gov=cia=di=mission=components.html (see: Council ofIntelligence Occupations). See also CIA Website. Intelligence Disciplines,http:==www.odci.gov=cia=di=work=disciplines.html

22Frank Watanabe, ‘‘Fifteen Axioms for Intelligence Analysts,’’ Studies inIntelligence, unclassified edition, 1997. http:==www.odci.gov=csi=studies=97unclass=axioms.html

632 STEPHEN MARRIN


23Robert Jervis, ‘‘What’s Wrong with the Intelligence Process?’’ InternationalJournal of Intelligence and CounterIntelligence, Vol. 1, No. 1, Spring 1986, pp.31–32.

24Stansfield Turner, ‘‘Intelligence for A New World Order,’’ Foreign Affairs, Fall1991, p. 164.

25Robert Jervis, ‘‘What’s Wrong with the Intelligence Process?’’, p. 33.

26Ibid, pp. 31–32.

27Ibid, pp. 28, 30.

28Stephen Marrin, ‘‘Complexity Is in the Eye of the Beholder,’’ DI DiscussionDatabase, 6 May 1997.

29The inevitability of intelligence failure appears to be a consensus position in theintelligence failure literature and stems from Richard Betts’s 1978 article.Richard K. Betts, ‘‘Analysis, War and Decision: Why Intelligence Failures AreInevitable,’’ World Politics, Vol. XXX1, No. 1, October 1978, pp. 61–89.

30Author’s personal experience, circa 1996.

31CIA Website, ‘‘Remarks of the Deputy Director of Central Intelligence John E.McLaughlin at the Conference on CIA’s Analysis of the Soviet Union, 1947–1991, Princeton University. 9 March 2001.’’ http:==www.cia.gov=cia=public_affairs=speeches=ddci_speech_03092001.html

32Tim Weiner, ‘‘Naivete at the CIA: Every Nation’s Just Another U.S.’’ The NewYork Times, 7 June 1998.

33Frans Bax, Presentation at an Association of Former Intelligence Officers(AFIO) Luncheon, Fort Myer, VA, 22 May 2001.

34Jack Davis, ‘‘Improving Intelligence Analysis at CIA: Dick Heuer’sContribution to Intelligence Analysis,’’ Psychology of Intelligence Analysis,Center for the Study of Intelligence, CIA, 1999, p. xix.

35Author’s experience in one of FDIT’s earlier runnings in the fall of 1996.

36Denis Stadther, Personal interview, 14 May 2001.

37That the Kent School dean resides on the DI’s corporate board is from CIA’sinternal ‘‘newspaper,’’ What’s News at CIA, Issue number 703. Unclassifiedarticle: ‘‘DDI Establishes Sherman Kent School for Intelligence Analysis.Frans Bax Named Dean.’’ Unknown date.

38Denis Stadther, personal correspondence, 6 September 2001.

39Frans Bax, presentation at AFIO Luncheon.

40Denis Stadther, personal correspondence, 6 September 2001.

41Most of the information on CAP was provided by its program director DenisStadther during an interview on 14 May 2001 and a second interview on 26February 2002. All direct information on CAP stems from these twointerviews, except where otherwise indicated.

42Kent School publication, ‘‘Career Analyst Program: Preparing for a Career inthe Directorate of Intelligence,’’ June 2001.




44Bob Drogin, ‘‘School for New Brand of Spooks’’; Denis Stadther, personalinterview, 14 May 2001.

45Denis Stadther, personal interview, 14 May 2001. Re-emphasized on 26February 2002.

46John McLaughlin, personal conversation, 9 March 2001.


48Ibid.

49For greater information on intelligence tradecraft, see: Douglas J. MacEachin,et al., ‘‘The Tradecraft of Analysis: Challenge and Change in the CIA,’’Consortium for the Study of Intelligence, Washington DC, 1994. Also see theDI Analytic Toolkit which contains ‘‘excerpt(s) from Notes on AnalyticTradecraft, published between 1995 and 1997, which elaborate on some of theskills and methods used by DI intelligence analysts. These notes become astandard reference within CIA for practitioners and teachers of intelligenceanalysis.’’ It can be found at: http:==www.odci.gov=cia=di=toolkit=index.html

50Sherman Kent’s ‘‘Principles for Intelligence Analysis’’ acquired frominformational materials provided by Kent School officers, 14 May 2001. SeeAppendix for the full text of the document.


52The ‘A=B Team’ approach stems from 1976 when former DCI George H. W.Bush commissioned a team of outside experts to review the same informationthat had led to a controversial National Intelligence Estimate on Sovietmilitary spending. This outside ‘‘B Team’’ based its analysis on morepessimistic assumptions of Soviet intentions, interpreted intelligence of Sovietmilitary spending accordingly, and came to a judgment more skeptical thanthe IC’s that was consistent with reigning conservative views. See Robert C.Reich, ‘‘Re-examining the Team A–Team B Exercise,’’ International Journal ofIntelligence and CounterIntelligence, Vol. 3, No. 3, Fall 1989, pp. 387–388. Seealso Kevin P. Stack, ‘‘A Negative View of Competitive Analysis, ’ ’International Journal of Intelligence and CounterIntelligence, Vol. 10, No. 4,Winter 1997–1998, pp. 456–464.

53Richards J. Heuer, Jr., Psychology of Intelligence Analysis (Washington, DC:Center for the Study of Intelligence, CIA, 1999).

54Bob Drogin, ‘‘School for New Brand of Spooks.’’

55Ibid.


57Stephen Marrin, ‘‘The CIA’s Kent School: A Step in the Right Direction,’’ TheIntelligencer: Journal of U.S. Intelligence Studies. Vol. 11, No. 2, Winter 2000,pp. 55–57.

58Dan Wagner, personal interview, 23 August 2001.

59Frans Bax, presentation at AFIO Luncheon. Denis Stadther, personal interview,14 May 2001.

634 STEPHEN MARRIN


60Thomas W. Shreeve and James J. Dowd, Jr. ‘‘Building a Learning Organization:Teaching with Cases at CIA,’’ International Journal of Intelligence and Counter-Intelligence, Vol. 10, No. 1, Spring 1997, p. 104.


62Donald T. Campbell and Julian C. Stanley, Experimental and Quasi-Experimental Designs for Research (Boston: Houghton Mifflin, 1963), pp.13–22.

63Denis Stadther, personal interview, 26 February 2002.

64Frans Bax, personal interview, 14 May 2001.


66Ibid.

67CIA Website. ‘‘CIA Vision, Mission, and Values.’’ http:==www.cia.gov=cia=information=mission.html

68For more on the limitations of using accuracy to measure intelligence, see thestrategic surprise and intelligence estimation literature. Of particular note, seeSteve Chan, ‘‘The Intelligence of Stupidity: Understanding Failures inStrategic Warning,’’ The American Political Science Review, Vol. 73, No. 1,March 1979, pp. 171–180.

69William Colby, ‘‘Retooling the Intelligence Industry,’’ Foreign Service Journal,January 1992, p. 21.

70For more information on the processes of intelligence analysis, see Robert M.Clark, Intelligence Analysis: Estimation and Prediction (Baltimore, MD:American Literary Press Inc., 1996); Research: Design and Methods,(Washington, DC: Joint Military Intelligence College, September 2000); DavidSchum, Evidence and Inference for the Intelligence Analyst (Lanham, MD:University Press of America, 1987).

71Washington Platt, Strategic Intelligence Production: Basic Principles (New York:Frederick A. Praeger, 1957), p. 75.

72Jerome K. Clauser, and Sandra M. Weir, Intelligence Research Methodology(State College, PA: HRB Singer, Inc., 1976), pp. 37–46.

73Robert Jervis, ‘‘What’s Wrong with the Intelligence Process?,’’ p. 29.

74Michael Dobbs, ‘‘U.S., Russia At Odds on Iranian Deal,’’ The Washington Post,15 June 2001, p. A01.

75Yitzhak Katz and Ygal Vardi, ‘‘Strategies for Data Gathering and Evaluation inthe Intelligence Community,’’ International Journal of Intelligence and Counter-Intelligence, Vol. 5, No. 3, Fall 1991, appendix, pp. 325–327.

76Loch K. Johnson, ‘‘Analysis For a New Age,’’ p. 661.


78Robert D. Folker, ‘‘Intelligence in Theater Joint Intelligence Centers: AnExperiment in Applying Structured Methods,’’ Joint Military IntelligenceCollege Occasional Paper Number Seven, Washington, DC, January 2000, p. 3.

79Yitzhak Katz and Ygal Vardi, ‘‘Strategies for Data Gathering and Evaluation inthe Intelligence Community,’’ p. 313.



80Walter Laqueur, ‘‘The Future of Intelligence,’’ Society, Vol. 35, No. 2, January–February 1998, p. 301.

81Washington Platt, Strategic Intelligence Production: Basic Principles (New York:Frederick A. Praeger, 1957), p. 151.

82Robert Folker, ‘‘Intelligence in Theater Joint Intelligence Centers,’’ p. 2.

83Richards Heuer devotes an entire chapter to this approach entitled ‘‘Analysis ofCompeting Hypotheses,’’ Psychology of Intelligence Analysis, pp. 95–109.

84Ibid, p. 89.

85R.V. Jones, Reflections on Intelligence (London: Heineman, 1989), pp. 87–88. Thecharacterization of R.V. Jones is attributed to former DCI James Woolsey at:http:==www.odci.gov=cia=public_affairs=press_release=archives=1997=pr122997.html

86Jack Davis, ‘‘Improving Intelligence Analysis at CIA,’’ p. xvii.

87Douglas J. MacEachin, ‘‘The Tradecraft of Analysis: Challenge and Changein the CIA,’’ p. 1; Jack Davis. ‘‘Improving Intelligence Analysis at CIA,’’ pp.xvii–xix.

88Jack Davis, ‘‘Improving Intelligence Analysis at CIA,’’ p. xix.

89The reference to Davis authoring the tradecraft notes is not on the CIA’s Website.Instead, the citation was in the foreword of the hardcopy version, which can be foundat: http:==intellit.muskingum.edu=intellsite=analysis_folder=di_catn_Folder=foreword.html; Jack Davis, ‘‘Improving Intelligence Analysis at CIA,’’ p. xix.

90CIA Website, ‘‘DI Analytic Toolkit.’’ http:==www.odci.gov=cia=di=toolkit=index.html

91CIA Website, ‘‘Intelligence Analysis in the DI: Frequently Asked Questions.’’http:==www.odci.gov=cia=di=work=analyst.html

92CIA Website, ‘‘Analytical Products of the DI.’’ http:==www.odci.gov=cia=di=work=major.html

93For the emphasis in the 1980s on longer papers, see Loch K. Johnson, ‘‘Makingthe Intelligence ‘Cycle’ Work,’’ International Journal of Intelligence and Counter-Intelligence, Vol. 1, No. 4, Winter 1986, pp. 6–7. For the benefits of longerpapers, see Arthur S. Hulnick, ‘‘Managing Intelligence Analysis: Strategies forPlaying the End Game,’’ International Journal of Intelligence and Counter-Intelligence, Vol. 2, No. 3, Fall 1988, pp. 322–323; and Loch K. Johnson,‘‘Analysis For a New Age,’’ p. 665. For the backlash against longer papers,see Jay T. Young, ‘‘US Intelligence Assessment in a Changing World: TheNeed for Reform,’’ Intelligence and National Security, Vol. 8, No. 2, April1993, pp. 129, 134. For the DI’s change in the 1990s, see ‘‘ ‘Say It Ain’t So,Jim’: Impending Reorganization of CIA Looks Like Suppression, Politicizingof Intelligence,’’ Publications of the Center for Security Policy, No. 94-D 74,15 July 1994. http:==www.security-policy.org=papers=1994=94-D74.html. Forthe backlash against shorter papers, see John Gentry, ‘‘A Framework forReform of the U.S. Intelligence Community.’’

94John E. McLaughlin, New Challenges and Priorities for Analysis, DefenseIntelligence Journal, Vol. 6, No. 2, 1997, pp. 16–17. Also on the CIA Websiteat http:==www.cia.gov=cia=di=speeches=428149298.html

636 STEPHEN MARRIN


95Frederick P. Hitz, ‘‘Not Just a Lack of Intelligence, a Lack of Skills,’’ TheWashington Post, 21 October 2001, p. B3.


97Loch K. Johnson, A Season of Inquiry (Chicago: Dorsey Press, 1988), p. 197.

98Walter Pincus, ‘‘Intelligence Shakeup Would Boost CIA,’’ The Washington Post,8 November 2001, p. A1.

99In fact, in 2001 the Kent School’s former dean noted his preference for a utilitymeasure over an accuracy one. Frans Bax, personal interview, 14 May 2001.




cia’s kent school: improving training for new · pdf filestephen marrin cia’s kent...

Documents