Asian Journal of Psychiatry 4 (2011) 304–308

Practicing evidence-based psychiatry. 3. Interpreting treatment guidelines

Ashley Clark, Babu Rankupalli, Rajiv Tandon *

University of Florida, Gainesville, FL, USA

A R T I C L E I N F O

Keywords:

Treatment guideline

Practice guideline

EBM

Evidence-based

Antipsychotic

Schizophrenia

Treatment

Pharmacotherapy

A B S T R A C T

Practicing evidence-based medicine (EBM) requires the ability to evaluate relevant evidence for the

purpose of making an evidence-based treatment decision. Broadly, there are three available sources of

information available to the practicing clinician: individual studies, literature reviews, and practice

guidelines. In the first two articles in the series, we described the threats-to-validity (T2V) approach in

evaluating evidence and specifically discussed the different threats to validity in applying the findings of

a single study or a systematic review to an individual patient. In this article, we describe the elements of a

treatment practice guideline and evaluate threats to validity at each of these steps. We illustrate the

method by evaluating a practice guideline relevant to the treatment question presented in a clinical

vignette. We briefly review practice guidelines for the pharmacological treatment of schizophrenia,

discuss recommendations from an exemplar guideline about antipsychotic choice in patients with

schizophrenia and consider application of its conclusions to the question of which antipsychotic to select

for the particular patient with schizophrenia utilizing the T2V approach.

� 2011 Published by Elsevier B.V.

Contents lists available at SciVerse ScienceDirect

Asian Journal of Psychiatry

jo u rn al h om epag e: ww w.els evier .c o m/lo cat e/a jp

1. Introduction

Evidence-based treatment refers to the practice of makingclinical treatment decisions based on the best available scientificinformation and is therefore predicated on the clinician’s ability toadequately appraise relevant evidence with the objective ofmaking the best possible treatment choice for an individualpatient. Broadly, there are three available sources of informationavailable to the practicing clinician: individual studies, reviews ofthe literature, and practice guidelines. In the first article in thisseries (Rankupalli and Tandon, 2010), we discussed the ‘‘threats tovalidity’’ approach (Cook and Campbell, 1979) towards evaluatingand applying findings of a single relevant study. In the secondarticle (Haj-Ibrahim and Tandon, 2011), we summarized types ofintegrative literature and reviewed threats to the validity ofconclusions derived from systematic reviews and meta-analyses.In the current article, we discuss the proper evaluation of clinicalpractice guidelines and their optimal application towards choosingthe best-possible treatment for a given patient in a specifiedclinical situation.

The limitations of relying on a single study as the sole source ofevidence on which to base clinical decisions have been discussedin prior articles in this series. Literature reviews and practiceguidelines are two broad approaches towards synthesizinginformation from multiple studies that can be then be utilized

* Corresponding author.

E-mail address: tandon@ufl.edu (R. Tandon).

1876-2018/$ – see front matter � 2011 Published by Elsevier B.V.

doi:10.1016/j.ajp.2011.11.002

in making evidence-based clinical decisions. Whereas a system-atic literature overview of treatment provides a summary aboutthe effects of treatments A–Z on patients with condition Y, aclinical practice guideline provides explicit guidance about thehierarchy of selection among treatments A–Z in patients withcondition Y. Decision analysis and cost-effectiveness analysis aretwo statistical approaches that are often utilized in developingclinical practice guidelines and these will be discussed in the nextpaper in the series along with a discussion of critical thinking. Inthis article, we focus on treatment practice guidelines andconsider the challenges in applying their specific recommenda-tions to a particular patient. We first outline the series of steps inapplying findings of a systematic review to making a treatmentdecision for a patient. We then examine the threats to validity(‘T2V’) at each of these steps that must be considered in the properapplication of the review’s findings to a specified clinical situation.We next provide a clinical vignette and summarize relevanttreatment guidelines that can inform the treatment choice whichneeds to be made.

2. Clinical practice guidelines

The Institute of Medicine defines clinical practice guidelines as‘‘systematically developed statements to assist practitioners andpatients make decisions about appropriate health care for specificclinical circumstances’’ (Field and Lohr, 1992). In contrast tosystematic reviews that summarize evidence on a particular topic,practice guidelines provide precise recommendations about theorder of choices among a menu of different treatment options. The

A. Clark et al. / Asian Journal of Psychiatry 4 (2011) 304–308 305

prescriptive nature of treatment guidelines is therefore extremelyattractive in an era of exploding information and diminishingclinician time to process all relevant evidence and properly apply itto a specific clinical situation in an individual patient. Althoughclinical practice guidelines vary widely in terms of their scope andspecificity, they all attempt to translate a vast amount of medicalknowledge into a clinician-friendly, readily usable hierarchy ofappropriate medical choices for a particular clinical condition. Thissimplicity and explicit nature of clinical treatment guidelinesrepresent both a major virtue and potential shortcoming ofpractice guidelines. Furthermore, practice guidelines vary widelywith regards to their intended goals and perspectives as also withrespect to the methodological rigor with which they are compiled.It is therefore essential that the clinician systematically evaluatethe validity and applicability of a treatment guideline prior toutilizing it in the care of an individual patient.

A treatment guideline provides its recommendations in thefollowing manner: ‘‘in the treatment of persons with condition Y,utilize the following treatments A, B, C, D, . . . in the following sequenceand in the following manner’’. This is analogous to the framing of anEBM treatment question (Rankupalli and Tandon, 2010): ‘‘which ofthe following treatments A, B, C, D, . . . provide the greatest likelihoodof the best outcome in my patient with condition Y’’. Thisintervention-comparison-outcome-population (‘ICOP’) reframing ofthe treatment question allows one to search for treatment guidelinesthat might provide relevant information. Having selected a particulartreatment guideline, one begins the following five-step process ofevaluating the guideline’s recommendations and their applicability toanswering the above clinical question:

1. Is the clinical problem addressed in the guideline explicit andproperly framed and is it relevant (i.e., matching ‘ICOP’)?

2. Does the guideline incorporate all valid information byincluding an explicit and proper literature search and retrievalstrategy?

3. Does the treatment guideline utilize a rigorous method toanalyze the data and thereby integrate available information?

4. Does the treatment guideline provide a concise and clearrecommendation, and is it appropriately presented?

5. Does the treatment guideline’s recommendations apply to mypatient?

The Threats to Validity (T2V) approach is a useful tool in theexamination of these issues. As previously defined (Rankupalli andTandon, 2010), validity is defined as the extent to which a statedconclusion approximates the truth (‘SCATT-ER’). In the context of anEBM-treatment decision based on recommendations of a practiceguideline, it refers to the confidence with which the stated findingsof the practice guideline can be applied to a population of patients‘‘like the one I am treating’’. Corresponding to the ICOP system offraming the clinician’s treatment question, treatment guidelinesgenerally summarize their results in a similar format: ‘‘Werecommend that in the treatment of persons with condition Y,utilize the following treatments A, B, C, D, . . . in the followingsequence and in the following manner.’’ Starting from theperspective that the stated guideline’s recommendations accuratelysummarize the available evidence, one could be partially orcompletely wrong about this conclusion because of potential threatsto validity at each of the five steps and these issues are brieflysummarized:

2.1. Problem formulation

As with systematic reviews (discussed in Haj-Ibrahim andTandon, 2011), treatment practice guidelines must address an

identifiable clinical population and range of covered treatmentoptions. Furthermore, the intended scope of and audience for thetreatment guideline should be specified; i.e., is it for thespecialist or the general practitioner; is it for the communityor tertiary care setting? The guideline must clearly describe thedesirable clinical outcomes on the basis of which differentavailable treatments were compared and ranked in the pre-scribed hierarchy of recommendations made in the guideline.Finally, the guidelines’ goals and perspectives must be explicitsince guidelines are prescriptive and its biases may be difficult todiscern. While all guidelines are developed with the objective ofproviding guidance in areas of clinical uncertainty and with thebroad objective of improving the effectiveness and efficiency oftreatment, their primary driver can differ: (i) reduce practicevariation; (ii) reduce costs of care; (iii) enhance efficacy ofprescribed treatments; (iv) reduce adverse effects of prescribedtreatments. Some guidelines evolve principally from the per-spective of the treating clinician, others are developed primarilyfrom a patient perspective, some are constructed from thepayor’s point of view, and yet others derive from the hospitalviewpoint. The clinician must therefore evaluate the fit betweenthe conceptualization of the clinical problem and its operatio-nalization in the treatment guideline and also assess anyfinancial conflicts or other biases (e.g., Kraemer and Gostin,2005) that might influence the recommendations of the practiceguideline. The following questions are usefully asked withreference to the treatment guideline:

(i) Is the target population to which the practice guideline is to beapplied clearly defined?

(ii) Is the range of treatments evaluated for the purpose of theguideline specified?

(iii) Are the desired outcomes on the basis of which differenttreatments are compared explicitly stated?

(iv) What are the goals and perspectives of the guideline?(v) Are the sources of funding, other support, and affiliations of

the authors of the guideline identified?(vi) How might the answers to these questions influence one’s

confidence in these guidelines?

2.2. Collection of all valid information

Treatment guidelines incorporate information from both thepublished medical literature and expert opinion. The validity ofany treatment guideline is therefore partly determined by theextent to which the entirety of the relevant literature is included. Abroad, ‘‘unbiased’’, and exhaustive search of the literature forrelevant studies is central to this step. With regard to expertopinion, it is important that input from all key stakeholders such aspatients, treating physicians, and other professionals be incorpo-rated. The reliability and rigor of this step can be assessed by askingthe following questions:

(i) Were clear inclusion and exclusion criteria for ‘‘studies to beincluded’’ specified a priori?

(ii) Was the literature search strategy comprehensive andexplicitly described?

(iii) Was there a rigorous assessment of the validity of individualstudies and systematic reviews that were incorporated? In theprocess of assessing the quality of evidence, an explicitevidence rating system (e.g., Harbour and Miller, 2001;National Institute of Health and Clinical Excellence, 2007)should optimally be utilized.

(iv) Did the guideline incorporate comprehensive stakeholder andmulti-disciplinary input?

A. Clark et al. / Asian Journal of Psychiatry 4 (2011) 304–308306

2.3. Rigorous method of integrating data

This step considers the process of integrating differentsources of information and combining evidence and expertopinion. The group of individuals that authors treatmentguidelines can utilize one of the following methods to developthe guidelines: (i) informal consensus and formal consensus; (ii)evidence-based guidelines explicitly linked to strength ofevidence and effectiveness of recommended treatment options.To assess potential threats to the validity of the meta-analyticmethod utilized to integrate data, the following question shouldbe asked to assess whether necessary underlying assumptionsare satisfied:

(i) Was the process by which the recommendations wereformulated explicit?

(ii) How were expert and other stakeholder opinion combinedwith the evidence?

2.4. Proper method of summarizing and presenting data

The fourth step in the development of a practice guideline is thetranslation of the evidence into explicit treatment recommenda-tions that will constitute the final guideline. Evidence-basedguidelines generally grade their recommendations according tothe strength of the evidence that supports the guideline. Forexample a 5-grade system (US Preventive Task Force, 1996) may beemployed as follows:

(A) There is good evidence to support the recommendation.(B) There is fair evidence to support the recommendation.(C) There is insufficient evidence for or against the recommenda-

tion.(D) There is fair evidence against the recommendation.(E) There is good evidence against the recommendation.

A more explicit evidence-based guideline will additionallytabulate the specific benefits, harms, and costs of the varioustreatment recommendations in addition to noting patient pre-ferences and probabilities of various outcomes. A formal utility orcost-effectiveness analysis may be presented. Therefore, theclinician should ask the following questions in evaluating thisstep of the guideline development:

(i) Is the guideline based on consensus (formal or informal)?(ii) If based on evidence, are various recommendations graded

according to the strength of supporting evidence?(iii) Are the specific benefits, harms, probabilities, costs, and

patient preferences tabulated?

2.5. Application of findings of systematic review to individual patient

As one next seeks to apply the recommendations of atreatment guideline to an individual patient, it is critical toevaluate whether the spectrum of subjects in the guideline issimilar to the specific patient to whom conclusions of theguideline are to be applied. Are the demographic characteristicsand range of disease severity of the subjects in the studiesincluded in the guideline adequately described and match thatof the patient to whom the overview conclusions are to beapplied? Does the range of interventions covered in theguideline cover the specific treatments of interest beingconsidered for one’s patient? Are outcome variables measuredin the studies relevant?

To evaluate this final step in the individualized application of atreatment practice guideline, the following questions should beasked:

(i) Are the subjects included in the various studies utilized in theguideline development adequately described and similar to‘‘my patient’’?

(ii) Are the treatments evaluated in the treatment guidelinesimilar to the ones I am considering for ‘‘my patient’’?

(iii) Are the outcomes (efficacy, safety, tolerability, costs, etc.)considered in guideline development relevant to ‘‘my pa-tient’’?

(iv) Will the guidelines’ recommendations benefit my individualpatient?

3. Applying a clinical practice guideline to an individualtreatment decision

3.1. CASE VIGNETTE (described in detail in Rankupalli and Tandon,

2010)

Mr. A. is a 30-year-old single male with a 3-year history ofparanoid schizophrenia. He presents to you with a 6-monthpsychotic exacerbation . . .. . .. . .. . . What antipsychotic treatmentwould you begin?

3.2. Operationalizing the question

One specific issue that needs to be addressed is selection of anantipsychotic agent. Framing the clinical question in the ICOPformat, the intervention being considered may be a particularantipsychotic agent (let us say, olanzapine), the comparatorswould include other first- and second-generation antipsychoticagents, the outcome might be freedom from psychosis or improvedsocial function, the target population would be young adults withschizophrenia, and the question would be articulated as ‘‘How doesolanzapine compare to other antipsychotic agents in reducingpsychosis and improving social function in adults with schizo-phrenia’’.

3.3. Finding a relevant guideline

Utilizing the specific ICOP terms (olanzapine, antipsychotic,outcome, schizophrenia, adults, treatment guideline) to search theliterature, one finds a number of relevant treatment guidelines forthe pharmacological treatment of schizophrenia. Treatmentguidelines can be evaluated by a specific instrument (AGREECollaboration, 2003a,b) and approximately 25 schizophreniatreatment practice guidelines have been compared utilizing thisinstrument (Moore, 2011; Gaebel et al., in press). The recommen-dations of four of these guidelines are summarized in Table 1. Ascan be gleaned from the table, there are substantial similarities anddifferences between the recommendations made by the differentguidelines. A different form of guidance is provided by the WorldPsychiatry Association Section on Pharmacopsychiatry and issummarized in Table 2.

With regard to the specific question about selection of aparticular antipsychotic in a first-episode patient with schizo-phrenia posed in the clinical vignette, there are differentrecommendations made by the guidelines. Whereas the WPAand NICE guidelines advocate selecting any first-generation or

second antipsychotic other than clozapine (with guidance as tohow to choose among them given individual patient factors), theAPA and TMAP guidelines specifically recommend selecting asecond-generation antipsychotic (and not a first-generation agent)

Table 1Comparison of four guidelines for the pharmacological treatment of schizophrenia.

APA PORT TMAP NICE

2004 2009, Buchanan et al. (2010) 2006, Moore et al. (2007) 2009

1st line SGA other than clozapine SGA or FGA other than clozapine SGA Any SGA or FGA except clozapine

2nd line Different SGA, FGA

LAI (for non-adherence)

Clozapine

Different SGA, FGA

LAI (for non-adherence)

Different SGA Any different SGA or FGA

except clozapine

3rd line Different SGA, FGA

LAI

Clozapine or ECT

Clozapine Clozapine Clozapine

4th line Different SGA, FGA, LAI or ECT – Clozapine + (FGA, SGA or ECT) Clozapine Augmentation

5th line – – FGA or SGA not tried in 1st or 2nd –

6th line – – Combination therapy (SGA + FGA, 2SGAs,

FGA/SGA + ECT, FGA/SGA + mood stabilizer)

–

Table 2Guidelines to optimize antipsychotic treatment of Schizophrenia. Guidance from

the World Psychiatry Association Section on Pharmacopsychiatry, Tandon et al.

(2008).

1. Considerations in selecting the best antipsychotic for a particular patient

Equivalent efficacy across agents

Individual variability in response

No good predictor of individual response to different agents yet

Different agents have different side effects

Different patients have different vulnerabilities and preferences

Switching is risky so it is important to try to select the first agent right

Best outcomes achieved by matching patient’s side-effect vulnerabilities to

the agent’s pharmacologic profile

2. Proper antipsychotic trial sequence

Begin with systematic 6–10-week trial of one antipsychotic with optimal

dosing

If inadequate response (for reasons of inefficacy, non-adherence, or poor

tolerability), follow with systematic trial of monotherapy with one or more

other antipsychotics at adequate dose and duration, choosing appropriate

agent based on reason for non-response

If inadequate response, follow with a trial of clozapine or a long-acting

antipsychotic

Follow with a trial of clozapine, if not tried before

Only then consider other strategies (e.g., antipsychotic polypharmacy)a

3. Good practice guidelines for ongoing antipsychotic treatment

Measurement-based individualized care

Ongoing careful monitoring essential

Repeated assessment of efficacy using reliably defined treatment targets

(facilitated by use of standard rating scales)

Careful assessment of adverse effects

Care consistent with health monitoring protocols (e.g., of the American

Diabetes Association)

Standard protocols customized to individual vulnerabilities/needs and

specific agent

Ongoing collaboration with patient in decision-making

a Given limitations of antipsychotics for treating the various symptom domains

of schizophrenia, clinicians often use combinations of antipsychotics and

adjunctive treatment with other agents, but evidence of the effectiveness of these

approaches for schizophrenia is generally weak at best.

A. Clark et al. / Asian Journal of Psychiatry 4 (2011) 304–308 307

other than clozapine. Finally, the PORT guidelines advocateselecting any antipsychotic agent other than clozapine orolanzapine. It should be noted that the guidelines which selectivelyadvocate use of a second-generation antipsychotic were publishedearlier than the ones that advocate selection of either a first-generation or second-generation antipsychotic agent (2004 and2006 versus 2008, 2009, and 2010). Secondly, one of the three morerecent guidelines recommend against the initial use of olanzapinein first-episode patients – ostensibly because of the greatermetabolic adverse effects associated with its use. Third, some ofthe guidelines provide more specific recommendations based onthe predominant symptomatology in the particular patient(Tandon et al., 2009, 2010).

4. Conclusion: what is the clinician to do?

Clinical practice guidelines are designed to provide the clinicianwith specific guidance about choosing between different treat-ment options in specific clinical situations. What should theclinician do when different treatment guidelines come up withdifferent recommendations?

As the treating physician realized with good randomizedclinical trials (Rankupalli and Tandon, 2010), and with systematicreviews (Haj-Ibrahim and Tandon, 2011), the clinician shouldalso recognize that there is no perfect treatment practiceguideline that applies to all patients. As with other kinds ofevidence, treatment practice guidelines have their particularbiases and points of emphasis that must be considered as theirrecommendations are applied to individual patients. Whiletreatment guidelines are extremely valuable in the practice ofEBM and the provision of the best-possible treatment with thegreatest likelihood of a good outcome for individual patients,they should only be seen as a starting point for the process ofindividualized treatment selection. The biases and limitations ofthe treatment guideline must be considered along with thepreferences, specific risks, and desired outcomes of the individ-ual patient. Critical thinking and the methods of decisionanalysis, which are central to the provision of evidence-basedand individualized treatment, will be discussed in the next articlein this series.

References

AGREE Collaboration, 2003a. Appraisal of Guidelines for Research and Evaluation(AGREE) Instrument. www.agreecollaboration.com.

AGREE Collaboration, 2003b. Development and validation of a international apprai-sal instrument for assessing the quality of clinical practice guidelines: theAGREE project. Qual. Saf. Health Care 12, 18–23.

American Psychiatric Association, 2004. Practice guidelines for the treatment ofschizophrenia, 2nd ed. Am. J. Psychiatry 161 (Suppl.), 1–56.

Buchanan, R.W., Kreyenbuhl, J., Kelly, D.L., et al., 2010. The 2009 schizophreniaPORT psychopharmacological treatment recommendations and summarystatements. Schizophr. Bull. 36, 71–93.

Field, M.J., Lohr, K.N., 1992. Guidelines for Clinical Practice: From Development toUse. Institute of Medicine/National Academy Press, Washington, DC.

Haj-Ibrahim, J., Tandon, R., 2011. Practicing evidence-based psychiatry. 2. Inter-preting integrative literature: systematic reviews and meta-analyses. Asian J.Psychiatry 4, 80–85.

Harbour, R., Miller, J., 2001. A new system for grading recommendations in evidencebased guidelines. Br. Med. J. 323, 334–336.

Moore, T.A., 2011. Schizophrenia treatment guidelines in the United States. Clin.Schizophr. Related Psychoses 4, 40–49.

Moore, T.A., Buchanan, R.W., Buckley, P.F., et al., 2007. The Texas MedicationAlgorithm Project antipsychotic algorithm for schizophrenia, 2006 update. J.Clin. Psychiatry 68, 1751–1762.

National Institute for Health and Clinical Excellence, 2007. How NICE ClinicalGuidelines are Developed: An Overview for Stakeholders, the Public, and theNHS. National Health Service, London.

National Institute for Health and Clinical Excellence, 2009. SchizophreniaCore Interventions in the Treatment and Management of Schizophrenia in

A. Clark et al. / Asian Journal of Psychiatry 4 (2011) 304–308308

Adults in Primary and Secondary Care (Update). National Health Service,London.

Rankupalli, B., Tandon, R., 2010. Practicing evidence-based psychiatry. 1. Apply-ing a study’s findings: the threats to validity approach. Asian J. Psychiatry 3,35–40.

Tandon, R., Belmaker, R.H., Gattaz, W.F., et al., 2008. World Psychiatric Associa-tion Pharmacopsychiatry Section statement on comparative effectivenessof antipsychotics in the treatment of schizophrenia. Schizophr. Res. 100,20–38.

Tandon, R., Nasrallah, H.A., Keshavan, M.S., 2009. Schizophrenia, ‘‘Just the facts’’. 4.Clinical features and conceptualization. Schizophr. Res. 110, 1–23.

Tandon, R., Nasrallah, H.A., Keshavan, M.S., 2010. Schizophrenia, ‘‘Just the facts’’. 5.Treatment and prevention. Schizophr. Res. 122, 1–23.

US Preventive Task Force, 1996. Guide to Clinical Preventive Services, 2nd ed.William and Wilkins, Philadelphia.

Further reading

Cook, T.D., Campbell, D.T., 2002. Quasi-Experimentation: Design and Analysis Issuesfor Field Settings. Rand-McNally, Chicago.

Guyatt, G.H., Rennie, D., 2001. Users Guides to the Medical Literature. AMA Press,Chicago.

Kane, J.M., Leucht, S., Carpenter, D., Docherty, J.P., 2003. Expert consensus guidelineseries. Optimizing pharmacological treatment of psychotic disorders. Introduc-tion, methods, commentary, and summary. J. Clin. Psychiatry 64 (Suppl. 12), 5–19.

Kraemer, J.D., Gostin, L.O., 2009. Science, politics, and values: the politicization ofprofessional practice guidelines. J. Am. Med. 301, 665–667.

US Department of Health and Human Services, Agency for Healthcare Policy andResearch, 1992. Categories of Evidence Relevant to Specific Causal Relationshipsand Treatments. AHCPR Publications, Rockville, MD.