LETTERS

NOTES & COMMENTS

A case of comparing effectiveness sinecomparative effectiveness

To the Editor: In reporting their recent randomizedclinical trial, Sivanesan et al1 framed their hypothesisin terms of comparing the effectiveness of psoralenplus ultraviolet A light phototherapy (PUVA) withother psoriasis therapies. Unfortunately, their trialcould not have accomplished that task, raising seri-ous concerns about the rigor of the trial.

A reduction of 75% or greater in the Psoriasis Areaand Severity Index (PASI 75) has gained prominenceas an effectiveness endpoint for psoriasis.2 PUVAwasdeveloped in the preePASI 75 era.3 Sivanesan et alwrote: ‘‘The lack of [PASI 75] data for PUVA makes itdifficult to compare, even if only very roughly, theefficacy of PUVA relative to other treatment options.’’But the study did not actually study effective psori-asis therapies aside from PUVA. Instead, the studyrandomly assigned participants to treatment withPUVA or ultraviolet A light (UVA) therapy alone,statistically evaluated proportions of patients in eachgroup that achieved PASI 75, and then informallycompared results from their study to those of recentpsoriasis trials.

The study’s results are not surprising. For decades,dermatologists have known that PUVA—but not UVAalone—is a very effective therapy for psoriasis.4 ThatPUVA greatly outperformed UVA provides no newinformation. As for the informal comparison ofeffectiveness, the authors write that PUVA ‘‘may becomparable with infliximab and cyclosporine,’’ evenas they correctly state that ‘‘relative efficacy of treat-ments can only be estimated, because the studieswere not equivalently designed and no studies havedirectly compared PUVA with other systemic treat-ments.’’1 But the authors have actually provided noformal estimates of relative efficacy. They have sim-ply eyeballed the findings. Nevertheless, they write,‘‘the ability to make even a rough comparison’’between therapies is ‘‘invaluable’’ to dermatologists.1

We find that argument unconvincing. Takingthe idea of comparative effectiveness seriously de-mands designing and conducting studies that yieldrigorous comparisons.5 Informal comparisons, suchas that offered by Sivanesan et al, are not scientif-ically valid ways of assessing comparative effective-ness, because trial designs and participants—and,therefore, results—can differ substantially in waysthat eyeballing their results neither captures noraccounts for.5

716 OCTOBER 2010

A meaningful comparison of the effectiveness ofPUVA relative to other therapies for psoriasis canonly be done in a trial that includes those othertherapies as comparison arms, as other trials havedone.6,7 European regulatory guidelines already‘‘strongly recommend’’ an active comparison arm,to enable ‘‘proper assessment of benefit/risk ratio.’’8

The alternative is no trial at all. Scientifically unsoundresearch—such as a trial that by design cannotanswer the question it purports to address—is inva-lid.9,10 Moreover, such research squanders resourcesthat ought to be used to undertake rigorous science.Dermatologists, clinical investigators, institutionalreview boards, and journals should take that man-date seriously for the sake of our specialty and, moreimportantly, our patients.

Kenneth A. Katz, MD, MSc, MSCE,a and SamualGarner, MBEb

Division of Dermatology,a Department of Medicine,University of California, San Diego, San Diego,Californiab

Funding sources: None.

Conflicts of interest: Dr Katz has received anhonorarium from Stiefel.

Correspondence to: Kenneth A. Katz, MD, MSc,MSCE, HIV, STD, and Hepatitis Branch, SanDiego County, 3851 Rosecrans St, Ste 207, SanDiego, CA 92110

E-mail: Kenneth.Katz@gmail.com

REFERENCES

1. Sivanesan SP, Gattu S, Hong J, Chavez-Frazier A, Bandow GD,

Malick F, et al. Randomized, double-blind, placebo-controlled

evaluation of the efficacy of oral psoralen plus ultraviolet A for

the treatment of plaque-type psoriasis using the Psoriasis Area

Severity Index score (improvement of 75% or greater) at 12

weeks. J Am Acad Dermatol 2009;61:793-8.

2. Feldman SR, Krueger GG. Psoriasis assessment tools in clinical

trials. Ann Rheum Dis 2005;64(suppl 2):ii65-ii68.

3. Parrish JA, Fitzpatrick TB, Tanenbaum L, Pathak MA. Photo-

chemotherapy of psoriasis with oral methoxsalen and long-

wave ultraviolet light. N Engl J Med 1974;291:1207-11.

4. Wolff K. Should PUVA be abandoned? N Engl J Med 1997;

336:1041-5.

5. Agency for Healthcare Research and Quality. Methods refer-

ence guide for effectiveness and comparative effectiveness

reviews, version 1.0. Available at: http://effectivehealthcare.

ahrq.gov/repFiles/2007_10DraftMethodsGuide.pdf. Accessed

February 8, 2010.

J AM ACAD DERMATOL

J AM ACAD DERMATOL

VOLUME 63, NUMBER 4

Letters 717

6. Heydendael VM, Spuls PI, Opmeer BC, de Borgie CA, Reitsma

JB, Goldschmidt WF, et al. Methotrexate versus cyclosporine in

moderate-to-severe chronic plaque psoriasis. N Engl J Med

2003;349:658-65.

7. Griffiths CE, Strober BE, van de Kerkhof P, Ho V, Fidelus-Gort R,

Yeilding N, et al. Comparison of ustekinumab and etanercept for

moderate-to-severe psoriasis. N Engl J Med 2010;362:118-28.

8. European Medicines Agency, Committee for Medicinal Pro-

ducts for Human Use (CHMP). Guideline on clinical investigation

of medicinal products indicated for the treatment of psoriasis.

Available at: http://www.ema.europa.eu/pdfs/human/ewp/

245402en.pdf. Accessed February 8, 2010.

9. International Conference on Harmonisation of Technical

Requirements for Registration of Pharmaceuticals for Human

Use. ICH harmonised tripartite guideline: guideline for good

clinical practice E6(R1). Available at: http://www.ich.org/LOB/

media/MEDIA482.pdf. Accessed February 8, 2010.

10. Council for International Organizations of Medical Sciences

(CIOMS). International ethical guidelines for biomedical research

involving human subjects. Geneva, Switzerland. Available at:

http://www.cioms.ch/frame_guidelines_nov_2002.htm. Accessed

February 8, 2010.

doi:10.1016/j.jaad.2010.04.049

Reply

To the Editor: In raising concerns about our recentpublication,1 Katz and Garner2 described two mainobjections. First, theyclaimed that our research ‘‘placesparticipants at undue risk and squanders resources’’under the assumption that the goal of our trial was toprove psoralen plus ultraviolet A light phototherapy(PUVA) more efficacious than ultraviolet A light (UVA)therapy alone—a concept they called ‘‘no new infor-mation.’’ Second, they objected to our making anyinference regarding the efficacy of PUVA in referenceto other available psoriasis treatment options.

With regard to the first objection, we think Katz andGarner misunderstood our intent: our purpose wasnot to compare the efficacy of PUVA versus UVA. UVAwas usedonly because there are no other scientificallyvalid ways of making the trial randomized andplacebo-controlled. More specifically, the placebogroup would know immediately that they were inthe control arm if they were not receiving any form ofphototherapy. By giving participants an oral placebocapsule identical to 8-methoxypsoralen (OxsoralenUltra) and having both groups receive phototherapy,all participants and investigators were blinded totreatment arms. It was critical for this study to beplacebo-controlled because noneplacebo-controlledtrials may result in unintentional PASI 75 inflation.

With regard to the second objection, we clearlystated in the publication that ‘‘in the absence of headto head trials, scores between treatment modalitiescannot be reliably compared.’’1 However, this doesnot mean that we cannot make any inferences indiscussing clinical trial results. Because the hottest

current topic in psoriasis therapy is the biologics,there is no way to conduct the type of ‘‘head to head’’comparative trial Katz and Garner require without amassive funding source. Realistically, only ‘‘bigpharma’’ biologic companies are able to fund suchstudies. Therefore, if the scientifically extreme posi-tion Katz and Garner propose is taken seriously, onlybig pharma research papers would be able to makeany comment about the relative efficacy of variouspsoriasis treatment options. All treatment optionswithout a ‘‘deep pocket’’ sponsor, such as prebiologictherapies, would be completely shut out from publi-cation. In effect, this leads to a situation where onlymega-finances can ensure publishability. We feel thatthis would be an unethical way of advancing medicalknowledge and clinical decision making regardingappropriate patient care.

Swarna Priya Sivanesan, MD, MPH,a Shilpa Gattu,BS,b Arianne Chavez-Frazier, MD,c and JohnKoo, MDd

Department of Dermatology,a Pittsburgh Univer-sity, Pittsburgh, Pennsylvania; University ofCalifornia, Irvine,b Irvine, California; DaytonSkin Surgery Center/Wright State University,c

Dayton, Ohio; and the Department of Dermato-logy,d University of California, San Francisco,San Francisco, California

Funding sources: None.

Conflicts of interest: Dr Koo has been a clinicalresearcher and/or consultant and/or speaker forValeant, Allergan, Warner-Chilcott, Amgen, Bio-gen, Bristol-Myers Squibb, Centacor, Connetics,Elan, Astellas, Galderma, Genentech, GlaxoS-mithKlein, Novartis, Roche, and Serono. MsGattu was a consultant for Valeant Pharmaceu-ticals. Drs Sivanesan and Chavez-Frazier haveno conflicts of interest to declare.

Correspondence to: Swarna Priya Sivanesan, MD,MPH, Department of Dermatology, University ofPittsburgh, 3601 5th Ave, Pittsburgh, PA 15238

E-mail: ssivane@yahoo.com

REFERENCES

1. Sivanesan SP, Gattu S, Hong J, Chavez-Frazier A, Bandow GD,

Malick F, et al. Randomized, double-blind, placebo-controlled

evaluation of the efficacy of oral psoralen plus ultraviolet A for

the treatment of plaque-type psoriasis using the Psoriasis Area

Severity Index score (improvement of 75% or greater) at 12

weeks. J Am Acad Dermatol 2009;61:793-8.

2. Katz KA, Garner S. A case of comparing effectiveness sine

comparative effectiveness. J Am Acad Dermatol 2010;63:716-7.

doi:10.1016/j.jaad.2010.04.050