derivation and validation of a clinical prediction rule

Derivation and validation of a clinical prediction rulefor uncomplicated ureteral stone—the STONE score:retrospective and prospective observational cohortstudies

OPEN ACCESS

Christopher L Moore associate professor 1, Scott Bomann emergency physician 1, Brock Danielsemergency physician2, Seth Luty research assistant3, Annette Molinaro associate professor1, DineshSingh assistant professor 4, Cary P Gross professor 5 6

1Department of Emergency Medicine, Yale University School of Medicine, New Haven, CT, USA; 2Wellington Hospital, Wellington, New Zealand;3Yale New Haven Hospital, New Haven, CT, USA; 4Departments of Neurosurgery and Epidemiology and Biostatistics, University of California SanFrancisco Medical School, San Francisco, CA, USA; 5Department of Urology, Yale University School of Medicine, New Haven, CT, USA; 6Departmentof Internal Medicine, Yale University School of Medicine, New Haven, CT, USA

AbstractObjective To derive and validate an objective clinical prediction rule forthe presence of uncomplicated ureteral stones in patients eligible forcomputed tomography (CT). We hypothesized that patients with a highprobability of ureteral stones would have a low probability of acutelyimportant alternative findings.

Design Retrospective observational derivation cohort; prospectiveobservational validation cohort.

Setting Urban tertiary care emergency department and suburbanfreestanding community emergency department.

Participants Adults undergoing non-contrast CT for suspecteduncomplicated kidney stone. The derivation cohort comprised a randomselection of patients undergoing CT between April 2005 and November2010 (1040 patients); the validation cohort included consecutiveprospectively enrolled patients from May 2011 to January 2013 (491patients).

Main outcome measures In the derivation phase a priori factorspotentially related to symptomatic ureteral stone were derived from themedical record blinded to the dictated CT report, which was separatelycategorized by diagnosis. Multivariate logistic regression was used todetermine the top five factors associated with ureteral stone and thesewere assigned integer points to create a scoring system that wasstratified into low, moderate, and high probability of ureteral stone. Inthe prospective phase this score was observationally derived blinded to

CT results and compared with the prevalence of ureteral stone andimportant alternative causes of symptoms.

Results The derivation sample included 1040 records, with five factorsfound to be most predictive of ureteral stone: male sex, short durationof pain, non-black race, presence of nausea or vomiting, andmicroscopichematuria, yielding a score of 0-13 (the STONE score). Prospectivevalidation was performed on 491 participants. In the derivation andvalidation cohorts ureteral stone was present in, respectively, 8.3% and9.2% of the low probability (score 0-5) group, 51.6% and 51.3% of themoderate probability (score 6-9) group, and 89.6% and 88.6% of thehigh probability (score 10-13) group. In the high score group, acutelyimportant alternative findings were present in 0.3% of the derivationcohort and 1.6% of the validation cohort.

Conclusions The STONE score reliably predicts the presence ofuncomplicated ureteral stone and lower likelihood of acutely importantalternative findings. Incorporation in future investigations may help tolimit exposure to radiation and over-utilization of imaging.

Trial registrationwww.clinicaltrials.gov NCT01352676.

IntroductionKidney stones are estimated to occur at some point in nearly 1in 11 people in the United States, with flank or kidney painresulting in over two million annual visits to the emergencydepartment.1 2 Computed tomography (CT) has been described

Correspondence to: C L Moore [email protected]

Extra material supplied by the author (see http://www.bmj.com/content/348/bmj.g2191?tab=related#datasupp)

Patient factors assessed in deriving factors for score (greyed out factors were gathered for observational purposes but not considered for use inscore)Diagram for inclusion of patients in derivation (retrospective) phaseEnrollment diagram for validation (prospective) phase

No commercial reuse: See rights and reprints http://www.bmj.com/permissions Subscribe: http://www.bmj.com/subscribe

BMJ 2014;348:g2191 doi: 10.1136/bmj.g2191 (Published 26 March 2014) of 12

Research

RESEARCH

on 19 October 2021 by guest. P

rotected by copyright.http://w

ww

.bmj.com

/B

MJ: first published as 10.1136/bm

j.g2191 on 26 March 2014. D

ownloaded from

http://creativecommons.org/licenses/by-nc/3.0/

http://www.clinicaltrials.gov/

http://www.bmj.com/content/348/bmj.g2191?tab=related#datasupp

http://www.bmj.com/permissions

http://www.bmj.com/subscribe

http://crossmark.crossref.org/dialog/?doi=10.1136/bmj.g2191&domain=pdf&date_stamp=2014-03-26

http://www.bmj.com/

as the “best imaging study to confirm the diagnosis of a urinarystone” and is now the first line imaging study for suspectedkidney stone in the United States.3-5 Though accurate, CT iscostly, involves the use of ionizing radiation, and does not seemto have impacted patient centered outcomes, such as rates ofdiagnosis or hospital admission, in those with suspected kidneystones.3-7

Many patients with flank pain will not benefit from a CT scan,as most kidney stones will pass spontaneously. Moreover, it isunlikely that a CT scan in the setting of flank pain will detectacutely important alternative findings in patients without signsof infection.8 Hence an objective clinical prediction rule forrenal colic that could reliably identify patients highly likely tohave a ureteral stone (and thus unlikely to have an importantalternative diagnosis) may allow patients to be safely managedwithout imaging, or imaged with other approaches such asultrasonography or reduced dose CT.We derived and validated a clinical prediction score for ureteralstones that cause symptoms, identifying patients with either avery high or a very low probability of having an uncomplicatedureteral stone. We hypothesized that patients who are highlylikely to have a kidney stone are unlikely to harbor an importantalternative diagnosis, and may be appropriate for imagingchoices other than standard dose CT.

MethodsStudy design and settingWe performed a retrospective derivation and prospectivevalidation of a clinical scoring system for ureteral stones thatcause symptoms in two separate emergency departments withthe same medical record systems.9-11 The Yale New HavenHospital emergency department is an urban, tertiary careteaching hospital and trauma center that sees over 80 000 adultsannually. The ShorelineMedical Center emergency departmentis a freestanding eight bed suburban facility without residents,which sees approximately 20 000 adults and children annually.At the time of this study both sites utilized a templated,handwritten, scanned emergency department patient care record(Lynx Medical Systems, Bellevue, WA), with laboratory anddictated radiology reports on Sunrise ClinicalManage (Eclipsys,Atlanta, GA). The human investigation committee of the Yaleinstitutional review board approved the derivation (retrospective)phase with a waiver of informed consent, and the validation(prospective) phase involved written informed consent from allpatients.

Derivation phaseWe electronically retrieved the dictated reports of all patientsreceiving a CT “flank pain protocol” (the name given at bothsites to a non-contrast enhanced CT protocol for suspectedkidney stone) at either of the two emergency department sitesbetween April 2005 and November 2010. Patients were eligibleif the CT was performed in the emergency department and theywere 18 years of age or older at the time of imaging. From anoriginal set of over 5000 computed tomograms, we selectedapproximately one third of records (estimated to yield about1000 records that met the inclusion criteria) for full recordreview using a random number spreadsheet function (MicrosoftExcel, Redmond,WA). Exclusion criteria were lack of any flankor back pain, history of trauma, evidence of infection (subjectiveor objective fever or presence of leukocytes on urine dipstickanalysis), known active malignancy, known renal disease(including creatinine >1.5 mg/dL or 133 μmol/L), or previousurologic procedure (including lithotripsy or ureteral stent).8

Power calculation—derivation and validation setsOur selection of about 1000 records was based on pilot data andearlier studies indicating that about 50% of patients undergoingCT would have a ureteral stone, and about 20% of these wouldundergo intervention for ureteral stone (or 10% of overallpopulation, about 100 patients). As a general rule, when usinglogistic regression, each independent element of a clinicalprediction requires approximately 10 events.12 This would haveallowed us to incorporate a maximum of 10 elements in a ruleto predict the need for intervention as well as being sufficientlypowered to derive a rule for the more common outcome (anyureteral stone).For the validation set we set minimally acceptable values forthe classification probabilities of false and true positive fractions,of 0.05 and 0.95, respectively. All conclusions were to be basedon a 90% (α=0.1) rectangular confidence region, using onesided exact confidence limits. As such we would attain 85%power with a minimum of 80 ureteral stones and a minimumof 256 non-stones.

Data abstractionBased on clinical experience and review of the literature, fivephysician co-investigators from three specialties (emergencymedicine, internal medicine, and urology) identified an a priorilist of factors thought to potentially be predictive of ureteralstone (see supplementary appendix 1).We conducted a literaturereview using key word searches in PubMed and relevantcitations through Web of Science (Thomson Reuters). Thesefactors were then abstracted from medical records blinded toCT reports.10 The Lynx medical record used by emergencyclinicians during the study period is a templated, handwrittenchart that specifically prompts clinicians for the presence orabsence of factors related to the chief complaint selected(typically flank or back pain), and was well suited to determiningthe presence or absence of factors. We abstracted the presenceor absence of factors into a standardized form on an electronicdatabase (Filemaker Pro 12, FileMaker, Santa Clara, CA).We blindly abstracted and categorized the results of the dictatedCT reports as previously described.8 The reports were reviewedprimarily to determine whether a kidney stone was causingsymptoms or whether the computed tomogram showed anothercause of symptoms. We considered a kidney stone to be thecause of symptoms if it was located from the renal pelvis to theureterovesical junction (parenchymal stones were not consideredto cause symptoms) or if signs of passed ureteral stone werespecifically mentioned in the CT report. We also documentedacutely important alternative causes of symptoms (such asappendicitis, diverticulitis, and others).8Other factors associatedwith kidney stone were also noted, including stone size, location,presence and degree of hydronephrosis or hydroureter, presenceof perinephric or ureteral stranding, and asymptomatic stonesas well as incidental findings (defined as unrelated to patientsymptoms). We abstracted the CT results into a standard formon a separate FileMaker database.

Inter-rater reliabilityTo determine inter-rater reliability of elements abstracted fromthe medical record, we blindly re-reviewed a subset of 50randomly selected records. A priori, any element with a κ ofbelow 0.6 was not eligible for inclusion in the prediction rule.We performed inter-rater reliability of categorization of CT scanresults from a random selection of records.8



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

Constructing the scoring systemAll variables included were considered through univariatelogistic regression analysis, with estimation of prevalence andodds ratios with corresponding 95% confidence intervals. Weperformed multivariate logistic regression, employing forwardselection and 10-fold cross validation for model selectionincluding estimation of two measures of prediction accuracy:the misclassification rate and the area under the receiveroperating characteristic curve (AUC). Misclassification is ameasure of prediction error, and ranges from 0 to 1, with lowerscores indicating fewer errors in prediction. AUC ranges from0.5 to 1, with higher scores indicating better prediction. Thebest model was the one that had a low cross validatedmisclassification rate and a high AUC. Subsequently, weincluded all observations to provide the most accurate estimatesof the coefficients for the selected model and to derive acorresponding integer scoring system following the methodsused in the Framingham study.11 The simplicity of this scoringsystem allows a patient’s risk to be calculated without the needfor a calculator. Initially, we organized variables in the finalmultivariate model into meaningful categories, each with aspecific reference value. We then assigned a referent risk foreach factor with the base risk assigned 0 points in the scoringsystem, such that a higher point total conveys more risk. Next,we calculated the difference in terms of regression units betweeneach category and the corresponding base category. We set theconstant, B, as the number of regression units that correspondsto 1 point. We then computed the points for each risk factor’srisk categories as the difference in regression units betweeneach category and its base category divided by B. Subsequentlywe calculated the risk associated with each point total throughthe multiple logistic regression equation. We used a weightedκ test is used to verify the agreement between risk estimatesbased on the point system and those based on the multivariatelogistic regression model. In addition to estimating AUC forsummarizing the model’s discrimination, we used the Hosmerand Lemeshow test to test for goodness of fit and calibration.While the odds ratios (coefficients) from the multivariateregression analysis can be used to estimate the probability ofan event (in this case ureteral stone), we sought to construct amore straightforward scoring system for clinical use withoutthe use of complicated calculations. We assigned integer pointsto the presence of risk factors for ureteral stone using thecoefficients from a multivariate analysis based on allobservations, as described in the methods used to estimate therisk of cardiovascular disease in the Framingham study.11 Wecomputed points for each factor as the difference in regressionunits between each category and its base category, which wasgiven a value of zero.To assess the difference in accuracy between the integer pointsystem and the logistic regression model we calculated themisclassification rate, AUC, and weighted κ based ondifferences in classification for each model. In addition toestimating AUC for summarizing the model’s discrimination,we used the Hosmer and Lemeshow test to determine thegoodness of fit and calibration.After the point system was constructed from the derivationphase but before analysis of prospective data, the research teamselected three categories for risk (low, moderate, and high) basedon estimated clinical utility for the probability of ureteral stoneby point total in each category.

Prospective validationFrom May of 2011 to February of 2013, consecutive patientspresenting during defined periods to the emergency departmentsites in whom the clinician intended to obtain a CT scan forkidney stone were approached for enrollment. Both cliniciansand enrolling staff were not aware of the specific elements ofthe rule derived in the retrospective phase. Defined enrollmentshifts included overnights, weekends, and holidays, and anautomatic paging system was set up to notify the researchassociate of all CTs ordered for renal colic. Review of thehospital imaging system was conducted daily to monitor anypatients missed during enrollment or when enrollment was nottaking place.Before analysis of the validation data, the scoring system wasdeveloped from the derivation set as described previously,yielding a 0-13 point scale. Also before analysis of theprospective data we stratified this scale based on estimatedclinical utility into low (about 10%), moderate (about 50%),and high (about 90%) probability of ureteral stone. Estimatedclinical utility of cut points on the scale were arrived at throughconsensus of all investigators, including physicians fromemergency medicine, internal medicine, and urology.Stratification into three groups enabled the derivation andvalidation sets to be compared for clinical utility fordiscrimination of risk as well as allowing estimates of theprevalence of more rare important alternative findings in eachgroup.The research associated recorded all relevant factors (listed insupplementary appendix 1) from the derivation phase for theenrolled patients before the results of the CT were known.Research associates were not aware of the elements of theSTONE score when prospective data were collected. Theyassigned point values of 0-13 and category of risk to each patientin the validation cohort blinded to the CT result, and the CTresult was categorized blinded to the clinical factors (exceptlaterality of pain) and point total. We used bootstrapping toestimate Hosmer-Lemeshow test and discrimination (AUC)with AUC point estimates and 95% confidence intervals.

ResultsDerivation sampleOf 5383 “flank pain protocol” CT scans (that is, the name fora non-enhanced CT scan using a renal colic protocol at ourinstitution) performed in the emergency departments on patients18 years of age or older during the retrospective period, 1853(34.4%) were randomly selected for full record review. Of these,1040 were complete records with no exclusion criteria (figure⇓,also see supplementary figure 1). Table 1⇓ lists thecharacteristics of the derivation and validation cohorts.Approximately half (49.5%; 515 of 1040) of the patients had aureteral stone that was causing symptoms on their computedtomogram, whereas 2.9% (30 of 1040) had acutely importantalternative causes of symptoms. Inter-rater reliability forcategorization of the CT result yielded a κ of 0.75-0.80,indicating excellent agreement. Table 2⇓ shows the factors thatwere significant for the presence or absence of ureteral stoneon univariate analysis.

STONE scoreMultivariate analysis yielded five factors that were mostsignificantly associated with the presence of a ureteral stone:male sex, acute onset of pain, non-black race, presence of nauseaor vomiting, and microscopic hematuria (table 3⇓). Previous



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

visits to an emergency department were also significantlyassociated with a lower probability of ureteral stone but, tomaximize generalizability between centers, were not includedin the model. These five factors were incorporated into theSTONE score with associated integer point values (table 3),yielding a total score ranging from 0-13.11 The multivariatelogistic regression model had a misclassification rate of 0.23(95% confidence interval 0.22 to 0.23) and an AUC of 0.86(95% confidence interval 0.79 to 0.93), whereas the STONEscore had a misclassification rate of 0.23 (0.22 to 0.23) and anAUC of 0.82 (0.74 to 0.90). Agreement between the riskestimates based on the STONE score and those based on themultivariate logistic regression model demonstrated a weightedκ of 0.87 (95% confidence interval 0.86 to 0.87), indicatingminimal loss of accuracy by assigning integer points to thefactors.

Prospective validationFrom 25 May 2012 to 24 January 2013, 491 patients withoutexclusion criteria were enrolled (see supplementary figure 2).The characteristics of patients approached did not differsignificantly from those that were not approached (table 1). Forthe validation cohort, the STONE score grouped into three levelsof risk had an AUC of 0.792 (95% confidence interval 0.756 to0.828) and the Hosmer-Lemeshow χ2=1.95 was not significant(P=0.38), indicating good discrimination and calibration.

Comparison of derivation and validation setsIn the derivation and validation sets, respectively, 19.8% and15.5% of patients were classified as having a low probabilityof kidney stone, 49.6% and 46.8% as moderate, and 30.6% and37.7% as high. The prevalence of ureteral stone by group in thederivation and validation sets was, respectively, 8.3% and 9.2%in the low probability group, 51.6% and 51.3% in the moderategroup, and 89.6% and 88.6% in the high group (figure). Overall,acutely important alternative causes of symptoms were foundon CT scan in 2.9% and 3.7% of the derivation and validationcohorts, with acutely important alternative causes in 0.3% and1.6% of the high probability group, respectively. Table 4⇓ showsthe causes and frequency of acutely important alternativefindings in the overall derivation and validation sets.

DiscussionThis study showed that a clinical scoring system accuratelypredicts the likelihood of ureteral stone, which is inverselyassociated with likelihood of an acutely important alternatecause of symptoms. To our knowledge this is the first clinicalscoring system to be derived and validated for prediction ofuncomplicated ureteral stone in patients attending emergencydepartments in whom CT imaging is deemed indicated. Aprevious study from the intravenous pyelography era derivedfactors from 203 patients and validated the findings in 73patients, finding four elements to be predictive of ureteral stone:flank pain, hematuria, acute onset of pain, and positive findingson a plain radiograph.13 Our data show that the quantitativeeffects of the five factors incorporated into the STONE scorecan accurately predict ureteral stone and allow stratification ofpatients in the emergency department with suspected kidneystone into one of three groups: low probability (≤10% chanceof stone), moderate probability (about 50% chance of stone),and high probability (about 90% chance of stone).Additionally, we found that the likelihood of an acutelyimportant alternative finding is inversely proportional to theprobability of a ureteral stone being present, as predicted by the

STONE score. While the overall presence of acutely importantalternative findings was 2.9% in the derivation set and 3.8% inthe validation set, the prevalence of clinically importantalternative diagnoses in the high probability group was less thanhalf of this: 0.3% and 1.6% in the derivation and validationcohorts, respectively.

Clinical and policy implicationsIn deriving and validating this clinical prediction rule (ratherthan a decision rule), we are not necessarily stating that patientswith a high stone score should not undergo CT imaging—thoughthis may not be an unreasonable approach in certain situations.In any clinical situation the risk of a test (in this case fromexposure to radiation) and the resources required to do the testwill need to be balanced against the tolerance for uncertaintyand risk of misdiagnosis on the part of both the clinician andthe patient. In some patients—perhaps particularly younger oneswho are more susceptible to radiation and less likely to havecertain diagnoses such as diverticulitis, aortic disease, ormalignancy—this score may be used to provide objective datato help balance the cost and risk of performing a CT. The otherpossibility is that this clinical prediction rule could be used todetermine which patients may be most appropriate forsubstantially reduced dose CT, which has been shown to reliablyidentify ureteral stones, particularly large ones that may requireintervention.14

CT use in the United States, and public healthimplicationsSince the landmark paper by Smith and colleagues in 1996, CThas become the first line test for kidney stone in the UnitedStates.3-15 However, despite a 10-fold increase in the utilizationof CT scanning for diagnosis of kidney stone from 1996-2007,the proportion of patients with a diagnosis of kidney stone,findings of significant alternative diagnoses, or hospitaladmission has not changed.3 7 This suggests that the increase inCT use for diagnosis of this condition may not be substantiallyimproving patient centered outcomes.16 Outside of the UnitedStates, CT is not necessarily the first line test for suspectedkidney stone.17-20 In 2011 the European Urology Associationreleased comprehensive guidelines on urolithiasis in which itstated that “ultrasonography should be used as the primaryprocedure.”21 In 2007, the yearly rate of CT scanning in theUnited States was nearly 228 per 1000 population—more thandouble the rate in Canada and nearly four times the rate in theUnited Kingdom.22 23 These data are not specific to imaging inkidney stones and do not include patient outcomes, but thepresence of wide regional variation (particularly in a conditionthat is not life threatening) suggests an opportunity for moreappropriate utilization.24 25

While the health risk attributable to a single CT scan is small,in a country of 310 million people (approximate US population)it is important to note a lifetime incidence of nephrolithiasis ofapproximately 10%.1 If half of these people undergo a CT scanto detect nephrolithiasis (likely a conservative estimate as kidneystones are often recurrent and many patients undergo multipleCT scans26), we could expect 15 million CT scans to beperformed on current US residents. In addition to the cost ofthis imaging, it could be estimated that exposure to ionizingradiation from CT would cause between 10 000 and 30 000additional malignancies (using risk estimates of between 1 in500 and 1 in 1500 for renal colic CT scans).27

In this setting CT was performed nearly as often in women asin men in both phases of the study (48.1% of CT scans in women



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

in the derivation phase; 44.4% in the validation phase).However, the diagnostic yield (percentage of patients withureteral stones on CT) for men was much higher: 68.8% in thederivation phase and 66.7% in the validation phase comparedwith women (28.7% and 41.7%, respectively). The lowerdiagnostic yield in women coupled with a higher risk fromradiation of the pelvis with CT suggests that women (especiallyyounger women) may be a group that could benefit from morejudicious use of CT radiation.

Use of the score to select appropriate patientsfor reduced dose CT or ultrasonographyIn terms of potential clinical utility, if a CT scan is beingconsidered for suspected kidney stone and a patient has a highSTONE score (which occurred in about a third of patients:30.6% in the derivation cohort and 37.7% in the validationcohort), then the patient is very likely to have a kidney stoneand very unlikely to have an important non-kidney stone causeof symptoms. Thus, if the STONE score is high a CT might beavoided entirely or a reduced dose CT could be performed (toensure that there is not a large stone that may requireintervention). It is important to note that it is still possible tomiss an important alternative diagnosis in the high probabilitygroup if CT is not performed (of the roughly 10% of patientsin the high group, about 10% of these, or 1-2% of the overallgroup, had an important alternative finding), However, theSTONE score offers objective data to both the clinician and thepatient that could help guide shared decision making about CTscanning, which is not without risk in terms of radiation andincidental findings that may lead to further testing orintervention. Our hope is that this score can be incorporatedinto imaging decisions for suspected renal colic to decreaseexposure to radiation and over-utilization of imaging (that is,imaging without improvement in patient care).28 Furtherinvestigation, potentially including a randomized trial, may helpto elucidate this.Most kidney stones (smaller stones, about 80% in this study asis generally the case) will pass spontaneously with treatment ofthe symptoms. Patients with a very high probability of ureteralstone thus may not require any imaging and could be managedwith pain control and drugs to enhance stone expulsion, withdefinitive diagnosis using a urine strainer. Clinicians may,however, still want to perform a CT to exclude potentiallyserious alternative causes of symptoms29 and to determine thesize and location of any stone (with implications for prognosisand intervention).30 In this case, patients with a high STONEscore may be ideally suited for substantially reduced dose CTscanning. Though data on low dose protocols have beenpublished outside of the United States31-33 and the AmericanCollege of Radiology states reduced dose techniques are“preferred,”4 data from the Dose Imaging Registry (part of theAmerican College of Radiology National Radiology of DataRegistry: www. nrdr.acr.org) indicates that themean institutionaldose for CT for renal colic is still greater than 10 mSv, andreduced dose techniques are rarely used in US hospitals (inpress).34

Reduced dose CT has been shown to be accurate for kidneystones, particularly larger ones that may require intervention,but has not been widely used in the United States, likely becauseof concerns about accuracy in an unselected population.14 34

Reluctance to implement reduced dose CT protocols for renalcolic may result from fear of missing other disease. Aninvestigator looking at reduced dose CT for renal colic notedthat to put these reduced dose protocols into practice they“would want to target it at patients who have a high pretest

probability of calculi and obstructive uropathy, since the abilityto detect other pathology is hindered.”35 In addition to predictingkidney stone, our data show that the group that is most likelyto have kidney stones is also unlikely (<2%) to have animportant alternative cause of symptoms. A probability ofdisease under 2% has been identified as a testing threshold (pointat which the negatives of a test outweigh the positives) for CTuse in detecting other important diseases, such as pulmonaryembolism.36 Identifying patients in this group could safely directsome patients with suspected kidney stone to low dose or ultralow dose CT.Ultrasound is another option that may be used for imaging insuspected renal colic, and ultrasonography is often a first linetest outside of the United States.19 21 It has the advantage ofavoiding radiation entirely and is sometimes definitivelydiagnostic: identifying the presence, size, and location of akidney stone that is causing symptoms. Often, however,ultrasonography may show indirect evidence of obstruction(hydronephrosis) without visualizing the actual ureteral stone,which may be obscured by bowel. We did find the presence ofhydronephrosis on CT to be highly predictive of ureteral stone,and future work will incorporate the presence of hydronephrosison ultrasonography into the STONE score.At our institution, the STONE score has been incorporated intothe computerized physician order entry system (Epic, VeronaWI).When a clinician orders a CT for kidney stone the questionsasked and a STONE score with risk category accompanies theradiology order. This has been welcomed by the radiologistswho were often unsure of the perceived likelihood of kidneystone on the part of the ordering physician. We have found thatthe STONE score is easily entered and calculated using ourelectronic health record.We are also currently using the STONEscore in a prospective study to select patients who areappropriate for either expectant management (no CT) or an ultralow dose CT, with a radiation dose that is about 90% lower thanconventional CT (effective dose of around 1 mSv, about thatof a plain abdominal radiograph). On a population basis,assuming the no threshold linear model suggested by theBiologic Effects of Ionizing Radiation report (currently BEIRVII), an equivalent reduction in cancer risk could be expected.37The current average effective dose of CT in the United Statesis 11.2 mSv, with only 2% of CT scans done using low doses.38

Strengths and limitations of this studyAn important limitation of this study is that gestalt clinicianpretest probability for kidney stone (that is, the overall clinicianestimate for likelihood of kidney stone after initial clinicianevaluation) has not been thoroughly investigated, and it ispossible that it would perform similarly to an objective clinicalprediction rule. A study by Abramson and colleagues showedthat the pretest probability of emergency department physiciansobtaining CT for suspected kidney stone clustered in the 41-60%and 71-90% ranges.39However, the use of a relatively objectivescoring system has the advantage that it is not dependent onclinician experience. In pulmonary embolism, for example,while gestalt pretest probability has been shown to be reasonablyaccurate, authors comparing gestalt pretest probability toobjective scoring systems conclude that they “advocate the useof a clinical prediction rule because it has been shown to beaccurate and can be used by less-experienced clinicians.”40 Thisstudy is also limited by being derived and validated in the sameclinical setting; it is not known how well it would perform inother settings.



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from

http://www/



http://www.bmj.com/

ConclusionWe have derived and validated a clinical prediction score forthe presence of ureteral stones that cause symptoms.Multicentervalidation and evaluation of incorporating the STONE scoreinto imaging algorithms is warranted.

We thank Christal Esposito, project manager, and research associatesPasquale Cicarella and Richelle Jessey and other research associatesfor their help in collecting and organizing data. The authors have followedthe STROBE checklist in collecting and reporting their data. Elementsof the checklist are incorporated into the manuscript. Further details ofany particular checklist item are available on request from thecorresponding author ([email protected]).Contributors: CLM conceived the work, monitored data collection forthe whole trial, oversaw data cleaning and statistical analysis, anddrafted and revised the manuscript. He is guarantor. SB helped withproject conception and participated in data analysis and manuscriptrevision. BD assisted with data analysis and participated in manuscriptrevisions. SL was lead research assistant and data manager, heparticipated in data collection, designed data collection tools, oversawcleaning and management of data, and assisted with presentation ofresults in themanuscript. AM provided input into methodology, performedstatistical analyses, and participated in manuscript revisions. DSparticipated in study design, analysis of data, and manuscript revisions.CPG participated in study design, analysis of data, and manuscriptrevisions. The authors confirm that data were collected, results analyzed,and the manuscript prepared without influence from funding agencies.Funding: This study was funded through a grant from the Agency forHealthcare Research and Quality (5R01HS018322-03): Identifyingunnecessary irradiation of patients with suspected renal colic. Thisfunding provided resources to assist with the collection, management,analysis and interpretation of the data, and preparation and review ofthe manuscript. The AHRQ provided funding and oversight of funding,but was not directly involved in collection or cleaning of data, analysisof results, or drafting of the manuscript.Competing interests: All authors have completed the ICMJE uniformdisclosure form at www.icmje.org/coi_disclosure.pdf and declare: Allauthors with the exception of BD and SB were funded in part by aresearch grant from the Agency for Healthcare Research and Quality(5R01HS018322-03); CPG received support from the Robert WoodJohnson clinical scholars program; CLM has received compensationas a consultant from Philips Healthcare and Sonosite (a subsidiary ofFujiFilm); CPG is a scientific advisory board member for Fair Health;CPG receives funding from Medtronic as a collaborator on the YaleUniversity Open Data Access project; there are no other relationshipsor activities that could appear to have influenced the submitted work.Ethical approval: This study was approved by the Yale HumanInvestigation Committee (institutional review board).Data sharing: The raw data from this study is not currently availableonline; however, the authors are willing to share a deidentified datasetof primary data with interested parties. Available on request from thecorresponding author ([email protected]).Transparency: CLM, the lead author and guarantor, affirms that themanuscript is an honest, accurate, and transparent account of the studybeing reported; that no important aspects of the study have been omitted;and that any discrepancies from the study as planned have beenexplained.

1 Scales CD, Smith AC, Hanley JM, Saigal CS. Prevalence of kidney stones in the UnitedStates. Eur Urol 2012;62:160-5. doi:10.1016/j.eururo.2012.03.052.

2 Pearle MS, Calhoun EA, Curhan GC. Urologic diseases in America project: urolithiasis.J Urol 2005;173:848-57. doi:10.1097/01.ju.0000152082.14384.d7.

3 Hyams ES, Korley FK, Pham JC, Matlaga BR. Trends in imaging use during the emergencydepartment evaluation of flank pain. J Urol 2011;186:2270-4. doi:10.1016/j.juro.2011.07.079.

4 Imaging EP on U. American College of Radiology appropriateness criteria: acute onsetflank pain—suspicion of stone disease. 2011. www.acr.org/~/media/ACR/Documents/AppCriteria/Diagnostic/AcuteOnsetFlankPainSuspicionStoneDisease.pdf.

5 Teichman JMH. Acute renal colic from ureteral calculus. N Engl J Med 2004;350:684-93.6 Gottlieb RH, Thao CL, Erturk EN, Sotack JL, Voci SL, Holloway RG, et al. CT in detecting

urinary tract calculi: influence on patient imaging and clinical outcomes. Radiology2002;225:441-9.

7 Westphalen AC, Hsia RY, Maselli JH, Wang R, Gonzales R. Radiological imaging ofpatients with suspected urinary tract stones: national trends, diagnoses, and predictors.Acad Emerg Med 2011;18:699-707. doi:10.1111/j.1553-2712.2011.01103.x.

8 Moore CL, Daniels B, Singh D, Luty S, Molinaro A. Prevalence and clinical importanceof alternative causes of symptoms using a renal colic computed tomography protocol inpatients with flank or back pain and absence of pyuria. Acad Emerg Med 2013;20:470-8.doi:10.1111/acem.12127.

9 Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rulesin emergency medicine. Ann Emerg Med 1999;33:437-47. www.ncbi.nlm.nih.gov/pubmed/10092723.

10 Gilbert EH, Lowenstein SR, Koziol-McLain J, Barta DC, Steiner J. Chart reviews inemergency medicine research: where are the methods? Ann Emerg Med 1996;27:305-8.www.ncbi.nlm.nih.gov/pubmed/8599488.

11 Sullivan LM, Massaro JM, D’Agostino RB. Presentation of multivariate data for clinicaluse: the Framingham Study risk score functions. Stat Med 2004;23:1631-60. doi:10.1002/sim.1742.

12 Wasson JH, Sox HC, Neff RK, Goldman L. Clinical prediction rules—applications andmethodological standards. N Engl J Med 1985;313:793-9.

13 Elton TJ, Roth CS, Berquist TH, Silverstein MD. A clinical prediction rule for the diagnosisof ureteral calculi in emergency departments. J Gen Intern Med 1993;8:57-62. www.ncbi.nlm.nih.gov/pubmed/8441076.

14 Niemann T, Kollmann T, Bongartz G. Diagnostic performance of low-dose CT for thedetection of urolithiasis: a meta-analysis. Am J Roentgenol 2008;191:396-401. doi:10.2214/AJR.07.3414.

15 Smith RC, Verga M, McCarthy S, Rosenfield AT. Diagnosis of acute flank pain: value ofunenhanced helical CT. Am J Roentgenol 1996;166:97-101.

16 Swensen SJ. Patient-centered Imaging. Am JMed 2012;125:115-7. doi:10.1016/j.amjmed.2011.06.002.

17 Patlas M, Farkas A, Fisher D, Zaghal I, Hadas-Halpern I. Ultrasound vs CT for the detectionof ureteric stones in patients with renal colic. Br J Radiol 2001;74:901-4. www.ncbi.nlm.nih.gov/pubmed/11675305.

18 Park SJ, Yi BH, Lee HK, Kim YH, Kim HC. Evaluation of patients with suspected ureteralcalculi using sonography as an initial diagnostic tool. J Ultrasound Med 2008;27:1441-50.

19 Moore CL, Scoutt L. Sonography first for acute flank pain? J Ultrasound Med2012;31:1703-11. www.ncbi.nlm.nih.gov/pubmed/23091240.

20 Ripollés T, Agramunt M, Errando J, Martínez MJ, Coronel B, Morales M. Suspectedureteral colic: plain film and sonography vs unenhanced helical CT. A prospective studyin 66 patients. Eur Radiol 2004;14:129-36. doi:10.1007/s00330-003-1924-6.

21 Tiselius HG, Ackermann D, Alken P, Buck C, Conort P, Gallucci M. Guidelines onurolithiasis. Eur Urol 2011;40:362-71. www.ncbi.nlm.nih.gov/pubmed/19585096.

22 Rampell C. Using, and overusing, medical technologies. New York Times . 16 Dec 2009.23 Berdahl CT, Vermeulen MJ, Larson DB, Schull MJ. Emergency department computed

tomography utilization in the United States and Canada. Ann Emerg Med2013;62:486-94.e3. doi:10.1016/j.annemergmed.2013.02.018.

24 Song Y, Skinner J, Bynum J, Sutherland J, Wennberg JE, Fisher ES. Regional variationsin diagnostic practices. N Engl J Med . 2010;363:45-53. doi:10.1056/NEJMsa0910881.

25 Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas F, Pinder EL. The implicationsof regional variations in medicare spending. Part 2 : health outcomes and satisfactionwith care. Ann Intern Med 2013;138:288-98.

26 Katz SI, Saluja S, Brink J, Forman HP. Radiation dose associated with unenhanced CTfor suspected renal colic: impact of repetitive studies. Am J Roentgenol 2006;186:1120-4.doi:10.2214/AJR.04.1838.

27 Smith-Bindman R, Lipson J, Marcus R, Kim KP, Mahesh M, Gould R, et al. Radiationdose associated with common computed tomography examinations and the associatedlifetime attributable risk of cancer. Arch Intern Med 2009;169:2078-86. doi:10.1001/archinternmed.2009.427.

28 Brenner DJ. Minimising medically unwarranted computed tomography scans. Ann ICRP2012;41:161-9. doi:10.1016/j.icrp.2012.06.004.

29 Hoppe H, Studer R, Kessler TM, Vock P, Studer UE, Thoeny HC. Alternate or additionalfindings to stone disease on unenhanced computerized tomography for acute flank paincan impact management. J Urol 2006;175:1725-30; discussion 1730. doi:10.1016/S0022-5347(05)00987-0.

30 Papa L, Stiell IG, Wells G, Ball I, Battram E, Mahoney JE. Predicting intervention in renalcolic patients after emergency department evaluation. CJEM 2005;7:78-86. www.ncbi.nlm.nih.gov/pubmed/17355656.

31 Poletti P-A, Platon A, Rutschmann OT, Schmidlin FR, Iselin CE, Becker CD. Low-doseversus standard-dose CT protocol in patients with clinically suspected renal colic. Am JRoentgenol 2007;188:927-33. doi:10.2214/AJR.06.0793.

32 Kluner C, Hein PA, Gralla O, Hein E, Hamm B, Romano V, et al. Does ultra-low-dose CTwith a radiation dose equivalent to that of KUB suffice to detect renal and ureteral calculi?J Comput Assist Tomogr 2006;30:44-50. www.ncbi.nlm.nih.gov/pubmed/16365571.

33 Kim BS, Hwang IK, Choi YW, Namkung S, Kim HC, Hwang WC, et al. Low-dose andstandard-dose unenhanced helical computed tomography for the assessment of acuterenal colic: prospective comparative study. Acta radiol 2005;46:756-63. doi:10.1080/02841850500216004.

34 Lukasiewicz A, Weinreb J, Coombs LP, Bhargavan M, Ghita M, Moore CL. Radiationdose index of CTs for kidney stone performed in the United States. Acad Emerg Med2013;20:S54.

35 Frei R. Ultra-low CT dose accurately detects renal stones. Ren Urol News 2012. 2013.www.renalandurologynews.com/ultra-low-ct-dose-accurately-detects-renal-stones/article/248315/.

36 Kline J. Further illumination of the test threshold approach in the care of emergencydepartment patients with symptoms of pulmonary embolism. Ann Emerg Med2010;55:327-30. doi:10.1016/j.annemergmed.2010.01.005.

37 National Academy of Sciences. Health risks from exposure to low levels of ionizingradiation: BEIR VII—phase 2 (executive summary). 2006. www.nap.edu/openbook.php?record_id=11340&page=1.



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from

http://www.icmje.org/coi_disclosure.pdf

http://dx.doi.org/10.1016/j.eururo.2012.03.052

http://dx.doi.org/10.1097/01.ju.0000152082.14384.d7

http://dx.doi.org/10.1016/j.juro.2011.07.079

http://dx.doi.org/10.1016/j.juro.2011.07.079

http://www.acr.org/~/media/ACR/Documents/AppCriteria/Diagnostic/AcuteOnsetFlankPainSuspicionStoneDisease.pdf

http://www.acr.org/~/media/ACR/Documents/AppCriteria/Diagnostic/AcuteOnsetFlankPainSuspicionStoneDisease.pdf

http://dx.doi.org/10.1111/j.1553-2712.2011.01103.x

http://dx.doi.org/10.1111/acem.12127

http://www.ncbi.nlm.nih.gov/pubmed/10092723



http://dx.doi.org/10.1002/sim.1742

http://dx.doi.org/10.1002/sim.1742



http://dx.doi.org/10.2214/AJR.07.3414


http://dx.doi.org/10.1016/j.amjmed.2011.06.002

http://dx.doi.org/10.1016/j.amjmed.2011.06.002




http://dx.doi.org/10.1007/s00330-003-1924-6


http://dx.doi.org/10.1016/j.annemergmed.2013.02.018

http://dx.doi.org/10.1056/NEJMsa0910881


http://dx.doi.org/10.1001/archinternmed.2009.427

http://dx.doi.org/10.1001/archinternmed.2009.427

http://dx.doi.org/10.1016/j.icrp.2012.06.004

http://dx.doi.org/10.1016/S0022-5347(05)00987-0

http://dx.doi.org/10.1016/S0022-5347(05)00987-0





http://dx.doi.org/10.1080/02841850500216004

http://dx.doi.org/10.1080/02841850500216004

http://www.renalandurologynews.com/ultra-low-ct-dose-accurately-detects-renal-stones/article/248315/

http://www.renalandurologynews.com/ultra-low-ct-dose-accurately-detects-renal-stones/article/248315/

http://dx.doi.org/10.1016/j.annemergmed.2010.01.005

http://www.nap.edu/openbook.php?record_id=11340&page=1

http://www.nap.edu/openbook.php?record_id=11340&page=1



http://www.bmj.com/

What is already known on this topic

Kidney stones are common, and imaging with computed tomography (CT) is now the first line diagnostic testHowever, CT has not been shown to improve patient centered outcomesAn objective, validated clinical prediction rule for uncomplicated ureteral stone has not been demonstrated and could help decreaseexposure to radiation or over-utilization of imaging

What this study adds

A clinical prediction rule was derived and validated that can identify patients with a high probability of uncomplicated ureteral stone andabsence of other important cause of symptomsResults from this study may be used to select patients who could benefit from management without CT, or from reduced dose CT

38 Lukasciewicz A, Bhargavan-Chatfield M, Coombs L, Ghita M,Weinreb J, GunabushanamG, et al. Radiation dose index of renal colic protocol CT studies in the United States: areport from the American College of Radiology National Radiology Data Registry.Radiology2014. http://pubs.rsna.org/doi/full/10.1148/radiol.14131601.

39 Abramson S,Walders N, Applegate KE, Gilkeson RC, RobbinMR. Impact in the emergencydepartment of unenhanced CT on diagnostic confidence and therapeutic efficacy inpatients with suspected renal colic. Am J Roentgenol 2000;175:1689-95.

40 Chunilal SD, Eikelboom JW, Attia J, Miniati M, Panju AA, Simel DL, et al. Does this patienthave pulmonary embolism? JAMA 2003;290:2849-58.

Accepted: 7 March 2014

Cite this as: BMJ 2014;348:g2191This is an Open Access article distributed in accordance with the Creative CommonsAttribution Non Commercial (CC BY-NC 3.0) license, which permits others to distribute,remix, adapt, build upon this work non-commercially, and license their derivative workson different terms, provided the original work is properly cited and the use isnon-commercial. See: http://creativecommons.org/licenses/by-nc/3.0/.



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from

http://pubs.rsna.org/doi/full/10.1148/radiol.14131601

http://creativecommons.org/licenses/by-nc/3.0/



http://www.bmj.com/

Tables

Table 1| Demographics of derivation and validation cohorts. Values are numbers (percentages) unless stated otherwise

Validation cohort (n=491)Derivation cohort (n=1040)Characteristics

45.8 (14.7)44.8 (14.9)Mean (SD) age (years)

218 (44.4)501 (48.1)Female sex

Race:

411 (83.7)883 (84.9)White

57 (11.6)110 (10.6)Black

23 (4.7)47 (4.5)Other

Location of enrolment:

357 (72.7)722 (69.4)Yale-New Haven Hospital ED

134 (27.3)318 (30.6)Shoreline Medical Center ED

Cause of symptoms on CT:

274 (55.8)515 (49.5)Symptomatic ureteral stone

18 (3.7)30 (2.9)Acutely important alternative cause

Disposition:

52 (10.6)71 (6.8)Admit

439 (89.4)969 (93.2)Discharge

ED=emergency department; CT=computed tomography.



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

Table 2| Significant predictors for presence or absence of ureteral stone (univariate analysis of derivation set), with odds ratios and 95%confidence intervals

Odds ratio (95% CI)No (%) of those with

factor with ureteral stoneNo (%) of total with

factorBaseline (odds ratio 1.0)Factors

Personal characteristics:

3.4 (2.6 to 4.4)371 (68.8)539 (51.8)Female sexMale sex

6.1 (3.7 to 9.9)547 (58.8)930 (89.4)Black raceNon-black race

1.4 (1.0 to 2.0)96 (61.5)156 (15.0)Arrival by other modeArrival by ambulance

History of present illness:

3.8 (2.2 to 6.9)551 (56.6)973 (93.6)No flank painAny flank pain

0.5 (0.4 to 0.6)134 (42.5)315 (30.3)No back painAny back pain

1.5 (1.1 to 2.0)480 (56.3)853 (82.0)Symptoms non-lateralizedSymptoms lateralized

3.5 (2.7 to 4.6)420 (66.7)630 (61.0)Pain onset gradual or unknownPain onset abrupt or sudden

1.5 (1.1 to 1.9)223 (60.8)367 (35.3)Pain course not constantPain course constant

0.5 (0.4 to 0.7)91 (41)222 (21.3)No pain with movementPain with movement

5.8 (4.1 to 8.2)292 (77.9)375 (36.1)Pain course 1 day to 1 weekPain duration <6 hours

1.8 (1.3 to 2.6)137 (52.9)259 (24.9)Pain course 1 day to 1 weekPain duration 6 h-1 day

0.4 (0.2 to 0.7)23 (20.4)113 (10.9)Pain course 1 day to 1 weekPain >1 week

2.1 (1.6 to 2.8)445 (59.8)744 (71.5)Pain not severe or <7 out of 10Pain severe or 7-10 out of 10

2.3 (1.8 to 3.0)229 (68.2)336 (32.3)No radiation of pain to groinRadiation of pain to groin

1.9 (1.4 to 2.6)176 (56.6)311 (29.9)No nausea or vomitingNausea alone

4.1 (3.0 to 5.7)219 (73.5)298 (28.7)No nausea or vomitingNausea with vomiting

0.5 (0.3 to 0.9)21 (39.6)53 (5.1)Absence of diarrheaPresence of diarrhea

1.9 (1.5 to 2.5)129 (61.1)211 (20.3)Dysuria not presentPresence of dysuria

2.0 (1.5 to 2.8)139 (67.8)205 (19.7)Subjective hematuria not presentSubjective hematuria

Medical, family, and social history:

0.5 (0.4 to 0.6)143 (42.7)335 (32.2)No allergy presentPresence of any allergy

3.7 (2.9 to 4.8)404 (68.2)592 (56.9)Prior visits to emergency departmentdocumented

No prior visits to emergency departmentdocumented

3.4 (1.9 to 6.6)50 (79.4)63 (6.1)No family history or not mentionedFamily history of kidney stones

0.5 (0.4 to 0.7)78 (40)195 (18.8)No history of smokingAny history of smoking

1.4 (1.0 to 1.7)194 (59.5)326 (31.3)No history of kidney stonesHistory of kidney stones

0.6 (0.5 to 0.8)141 (46.7)302 (29)No surgical historyAny surgical history

0.7 (0.5 to 0.8)227 (48.9)464 (44.7)No drugs documentedTaking any drugs

Physical examination:

1.2 (1.1 to 1.2)NANAMean 134 (SD 35) mm HgRaised systolic blood pressure, each 10mm Hg

1.3 (1.2 to 1.4)NANAMean 85 (SD 13) mm HgRaised diastolic blood pressure, each 10mm Hg

0.8 (0.8 to 0.9)NANAMean 83 (SD 15) beats/minRaised pulse, per 10 beats/min

1.5 (1.1 to 2.1)107 (62.6)171 (16.4)No right lower quadrant tendernessRight lower quadrant tenderness

1.4 (1.1 to 1.8)198 (60.0)330 (31.7)No right or left lower quadranttenderness

Right or left lower quadrant tenderness

0.6 (0.4 to 0.9)38 (41.8)91 (8.8)No upper tendernessUpper abdominal tenderness

0.4 (0.2 to 0.6)36 (33.0)106 (10.2)Lumbar or back tenderness

Laboratory values:

4.7 (3.5 to 6.2)473 (66.0)717 (68.9)No erythrocytes in urineAny erythrocytes in urine

0.013 (0.012 to 0.014)NANA88.4 (SD 35.4) μmol/L (1.0 SD 0.4mg/dL)

Creatinine, each 8.84 μmol/L (0.1 mg/dL)increase

NA=not applicable.



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

Table 3| STONE score, factors, and categories

PointsOdds ratio (95% CI)STONE score by factors and categories

Sex

Sex:

01Female

24.31 (3.13 to 5.98)Male

Timing

Duration of pain to presentation:

01>24 hours

11.85 (1.27 to 2.70)6-24 hours

36.34 (4.26 to 9.33)<6 hours

Origin

Race:

01Black

36.77 (3.79 to 12.64)Non-black

Nausea

Nausea and vomiting:

01None

11.98 (1.38 to 2.86)Nausea alone

25.26 (3.53 to 7.93)Vomiting alone

Erythrocytes

Hematuria (on urine dipstick):

01Absent

35.61 (3.96 to 8.04)Present

0-13Total



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

Table 4| Types and frequency of acutely important alternative causes of symptoms in derivation and validation sets, listed by decreasingfrequency in derivation set

Validation set (n=491)Derivation set (n=1040)Acutely important alternative cause of symptoms

46Diverticulitis

45Appendicitis

14Malignancy or concerning mass

14Ovarian or adnexal cause

13Pyelonephritis

02Ruptured angiomyolipoma

21Cholecystitis

11Pnemonia

01Retroperitoneal fibrosis

01Perforated viscous

11Bowel obstruction

11Colitis

10Aortic aneurysm

10Pancreatitis

18 (3.8%)30 (2.9%)Total No (%)



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/

Figure

Prevalence of ureteral stone by STONE score category in derivation and validation cohorts. Percentages at top of barsindicate prevalence of ureteral stone in group. Values under bars indicate number within derivation and validation sets thatfell within risk stratums



RESEARCH



ww

.bmj.com

/B


j.g2191 on 26 March 2014. D

ownloaded from



http://www.bmj.com/