improved safety event reporting in outpatient, … · is time-consuming (about 15 minutes for entry...

66 JCOM February 2015 Vol. 22, No. 2 www.jcomjournal.com

safety event reporting

ABSTRACT• Objective: To evaluate the effect of an educational

intervention with regular audit and feedback on re-porting of patient safety events in a nonacademic,communitypracticesettingwithanestablishedreport-ingsystem.

• Methods:Aquasi-experimentalwithcomparatorde-sign was used to compare a 6-practice collabora-tivegroupwitha27-practicecomparatorgroupwithregardtosafetyeventreportingrates.Baselinedatawere collected for a 12-month period followed byrecruitmentof6practices(3familymedicine,2pedi-atric,and1generalsurgery).Aneducationalinterven-tionwascarriedoutwitheach,andthiswasfollowedby monthly audit and regular written and in-personfeedback. Practice-level comparisons were madewithspecialty-andsize-matchedpractices for the6practicesinthecollaborativegroup.

• Results: In the 12-month period following the inter-ventioninMarch2013,the6practicesreported175patientsafetyeventscomparedwithonly19eventsintheprevious12-monthperiod.Eachpracticeatleastdoubled reporting rates, and5of the6 significantlyincreased rates. In contrast, rates for comparatorpracticeswereunchanged, with 84events reportedfor the pre-intervention period and 81 for the post-interventionperiod.Eventclassificationandtypesofevents reported were different in the collaborativepracticescomparedwiththecomparatorsforthepost-interventionperiod.Forthecollaborativegroup,nearmiss events predominated as did diagnostic testingandcommunicationeventtypes.

• Conclusion: An initial educational session stressinganonymous,voluntarysafetyeventreportingtogetherwithmonthlyauditandfeedbackandafocusonde-velopinganonpunitiveenvironmentcansignificantlyenhancereportingofsafetyevents.

Multiple challenges in the outpatient setting make establishing a culture of safety and improving care delivery more difficult than for inpatient

settings. In the outpatient setting, care is often inaccessible, not well coordinated between providers and between facili-ties and providers, and delivered in many locations. It may also involve multiple sites and providers for a single patient, may require multiple visits in a single location, and can be provided by phone, email, mail, video, or in person [1]. Errors and adverse events may take long periods of time to become apparent and are more often errors of omission compared with those in the inpatient setting [2].

Incident reporting systems are considered important in improving patient safety [3], and their limitations and value have recently been reviewed [4]. However, limited research has been conducted on medical errors in ambula-tory care, and even less is available on optimal monitoring and reporting strategies [5–12]. Reporting in our system is time-consuming (about 15 minutes for entry of a single report), is not tailored for outpatient practices, may be considered potentially punitive (staff may believe that reporting may place themselves at risk for performance downgrade or other actions), and marked under-reporting of safety events was suspected. Most but not all of the sug-gested characteristics considered important for hospital-based reporting systems are fulfilled in our ambulatory reporting system [13].

Several academic groups have reported much im-proved reporting and a much better understanding of the types of errors occurring in their respective outpatient settings [14–16]. The most compelling model includes a voluntary, nonpunitive, anonymous reporting approach and a multidisciplinary practice-specific team to analyze

Improved Safety Event Reporting in Outpatient, Nonacademic Practices with an Anonymous, Nonpunitive ApproachHerbert W. Clegg II, MD, Terri Cardwell, RPh, PharmD, MHA, Aaron M. West, BS, and Felicia Ferrell, BA

From Novant Health and Novant Health Medical Group, Winston-Salem, NC.

www.jcomjournal.com Vol. 22, No. 2 February 2015 JCOM 67

reported errors and to enact change through a continu-ous quality improvement process [14,15].

We implemented a project to significantly improve reporting of safety events in an outpatient, nonacademic 6-practice collaborative by using education, monthly audit, and regular feedback.

METHODSSettingNovant Health Medical Group is a consortium of over 380 clinic sites, nearly 1300 physicians, and over 500 advanced practice clinicians. Clinic locations are found in Virginia, North Carolina, and South Carolina. Medical group members partner with physicians and staff in 15 hospitals in these geographic locations. Novant Health utilizes Epic (Epic Systems, Verona, WI) as an electronic health record. Safety event reporting is accomplished electronically in a single software program (VIncident, Verge Solutions, Mt. Pleasant, SC), used for all patients in our integrated care system (inpatient and outpatient facilities).

InterventionWe designed a quasi-experimental study enrolling a 6-practice collaborative of 3 family medicine practices, 2 pediatric practices, and 1 general surgery practice. These practices was selected because each had a proven physi-cian leader and an experienced practice manager willing to participate in this initiative. We developed a compen-dium of patient safety events (see Appendix) that had been reported over time in our safety event reporting program. Historically, reports were made electronically in the program by a single reporter in clinics, and these reports were initially verbally communicated by a staff member or provider to the reporter.

Two of the authors (HWC and TC) met in March 2013 with the lead physician, practice manager, and patient safety coach at each clinic for approximately 1 hour. We discussed current reporting practice, delivered education for the safety event compendium, and detailed an anony-mous, voluntary, and nonpunitive approach (stressing the use of the term “safety event” and not “error”) to report-ing using a single page, 8-question paper report about the event. The report was not to be signed by the person completing the event data with placement in a drop box for later collection and electronic reporting as per usual practice in the clinic. We agreed that clinic leaders would stress to staff and providers that the initiative was nonpu-nitive and anonymous and that the goal was to report all

known safety events, as an improvement project. Patient safety coaches were selected for each of the 6

practices by the manager. Patient safety coaches are vol-unteer clinical or nonclinical staff members whose role is to observe, model, and reinforce pre-established patient safety behaviors and use of error prevention tools among peers and providers. Training requirements include an initial 2-hour training session in which they learn funda-mentals of patient safety science, high reliability principles, coaching techniques for team accountability, and concepts for continuous quality improvement. Additionally, they attend monthly meetings where patient safety concepts are discussed in greater detail and best practices are shared. Following this training, each clinic’s staff was educated on the project, a process improvement team (lead physician, manager, and patient safety coach) was constituted, and the project was begun in April 2013. In quarter 3 of 2013, each practice team selected a quality improvement project based upon reported safety events in their clinic. We asked our medical group risk managers to continue event dis-cussion with practice managers as usual, as each event is discussed briefly after a report is made.

We audited reports monthly and provided feedback to the practice team with a written report at the end of each 3-month period starting in June 2013 and ending in June 2014 (5 reports). Individual on-site visits to meet and discuss progress were completed in September 2013 and March 2014, in addition to the initial visit in March 2013.

EvaluationWe compared reported monthly safety events for each of the 6 practices and for the 6-practice collaborative in the aggregate for the 12-month pre-intervention period (April 2012 through March 2013) and post-intervention period (April 2013 through March 2014). Each prac-tice was compared with 3 specialty- and size (number of providers)-matched practices, none of whom received education or feedback on reporting or had patient safety coaches in the practice. In addition, for each of the 3 fam-ily medicine practices in the collaborative, we matched 1:3 other family medicine practices for specialty, size, and presence of a designated and trained patient safety coach. For the duration of the project, only 50 of 380 practices in the medical group had a trained patient safety coach.

The rate of safety events reported (ie, number of safety events reported/number of patient encounters) was com-

RepoRts fRom the field



pared for the 2 time periods using Poisson regression or zero-inflated Poisson regression. SAS enterprise guide 5.1 was used for all analyses. A P value of < 0.05 was consid-ered statistically significant. The protocol was reviewed by the institutional review board of Novant Health Presbyterian Medical Center and a waiver for informed written consent was granted.

RESULTS

For the year preceding the recruitment and education of the collaborative practices (pre-intervention year), report-ing rates for the 6 collaborative practices (1.2 or 19 events reported/154,148 patient encounters) and for the aggre-gate of 27 comparators (1.5 or 84/568,417) were very similar (P = 0.47). For the post-intervention year, the col-laborative practices’ rate increased to 11.5 (175/152,610, P < 0.001), while the rate for the comparator practices remained stable at 1.5 (81/554,608). Rates remained unchanged as well for all other Novant Health Medical Group practices (Figure 1).

Each of the 6 collaborative clinics experienced at least a doubling of reporting rates, and 5 of the 6 clin-ics significantly increased reporting rates (Figure 2). Practices 1 through 5 had pre-intervention rates of 0 to 2.4 and post-intervention rates of 6.0 to 8.2. Practice 6

increased from 5.9 to 164.6, an increase largely due to reporting of communication issues for this practice. In practice 1, a general surgery practice, reporting increased from 0 events (in 5093 encounters) to 4 events (in 5071 encounters, a rate of 7.9) for the 2 time periods. How-ever, this increase did not reach the level of significance (P = 0.09). The 3 general surgery clinic comparators to-gether reported 0 safety events for the post-intervention year among 13,793 encounters in clinic.

To control for the presence of patient safety coaches in practices, the 3 family medicine clinics (clinics 4 through 6, Figure 2) were each matched 1:3 for size (number of providers) and specialty (other family medicine clinics), also with a patient safety coach. While the rates were significantly increased for the 3 collaborative family medicine clinics (P < 0.001), only 1 of the comparators clinic’s rate changed significantly (0.2 or 1/44,580 to 1.3 or 6/45,157), and this change was marginally significant (P = 0.048). This practice was the only one of the 27 comparator clinics to demonstrate any increased rate.

We also compared the classification and types of safety events reported for each of 3 groups for the year April 2013 through March 2014 (Figures 3 and 4; see Appendix for definitions of safety event types and classification). In the collaborative group, 88% of events

Figure 1. Safetyeventreportingratesper10,000encountersforallNovantHealthMedicalGrouppractices,January2012–June2014.

Educationalintervention

Janu

ary

2012

Febr

uary

201

2M

arch

201

2Ap

ril2

012

May

201

2Ju

ne2

012

July

201

2Au

gust

201

2

Sept

embe

r201

2

Oct

ober

201

2

Nov

embe

r201

2

Dec

embe

r201

2Ja

nuar

y20

13

Febr

uary

201

3M

arch

201

3Ap

ril2

013

May

201

3Ju

ne2

013

July

201

3Au

gust

201

3

Sept

embe

r201

3O

ctob

er2

013

Nov

embe

r201

3

Dec

embe

r201

3Ja

nuar

y20

14

Febr

uary

201

4M

arch

201

4Ap

ril2

014

May

201

4Ju

ne2

014

Collaborativepractices(6) Comparatorpractices(27) Allotherpractices(315)

Eve

nt

Rat

e p

er 1

0,00

0 E

nco

un

ters

35

30

25

20

15

10

0

5


reported were near miss events and precursor safety events. Only 12% were non-safety events, as compared with 65% for comparator practices and 63% for all oth-ers. Two serious safety events were reported in the “all other practices” group. Too few events occurred in the pre-intervention collaborative group to make meaningful pre- and post-intervention comparisons.

A different pattern of event type was seen for the collaborative group. Falls accounted for only 6% and diagnostic testing (16%) and communications issues (34%) totaled 50%. In contrast, for comparator practices, falls accounted for 51% of reported safety events and only 3% of events were reported as diagnostic testing (2%) and communication issues (1%). This pattern accounts for the


Figure 2. Safety event reporting rates for 6 collaborative practices for pre-intervention and post-intervention time periods,April2012–March2013andApril2013–March2014.Numbersabovebarsrefertorateper10,000encounters.Barsrepresentactualnumberofevents.

Figure 3. ClassificationofsafetyeventsforallNovantHealthMedicalGrouppractices,April2013–March2014.

Nu

mb

er o

f E

ven

ts

80

70

60

50

40

30

10

20

0

Practice1 Practice2 Practice3 Practice4 Practice5 Practice6

P=0.09 P<0.001 P<0.01 P<0.001 P<0.001 P<0.001

2012–2013

2013–2014

7.9

8.1

7.48.2

164.6

6.0

5.90.91.02.40.50

70%

60%

50%

40%

30%

10%

20%

0%

12%

65% 63%

37%

30% 29%

51%

5%8%

0% 0% 0.4%

Non-SafetyEvent PrecursorSafetyEvent NearMissEvent SeriousSafetyEvent

Collaborativepractices(6)

Comparatorpractices(27)

Allotherpractices(315)



marked differences in event classification, as falls are usu-ally classified as non-safety events.

DISCUSSION

In our nonacademic community practices, patient safety reporting rates improved following an initial educational session stressing anonymous, voluntary safety event re-porting together with monthly audit and feedback. Our findings corroborate those of others in academic ambu-latory settings, who found that an emphasis on patient safety reporting, particularly if an anonymous approach is taken in a nonpunitive atmosphere, can significantly increase the reporting of patient safety events [14–16]. We demonstrated marked under-reporting and stability of patient safety event reporting throughout our ambula-tory practice group and a 10-fold increase in reporting among the 6-practice collaborative.

An unexpected finding was that with the exception of 1 practice, we found no increased reporting in compara-tor practices that had a patient safety coach. Additionally, we noted that general surgery practices report (or experi-ence) very few ambulatory safety events, as a total of 4 events were reported for all 4 general surgery practices in 18 months.

We chose a quasi-experimental with a comparison group and pre-test/post-test design since randomization of practices was not feasible [17]. We used a 2-year pe-riod to control for any seasonal trends and to allow time after the intervention to see if meaningful improvement in reporting over time would continue. We attempted to address the potential for nonequivalence in the com-

parison group by matching for specialty and size of practice.

There are several limitations to this study. Bias in the selection of collaborative practices may have occurred since each had a proven leader, and this may have led to more rapid adoption and utilization of this reporting approach. Also, our findings may not be generalizable to other inte-grated health systems given the unique approaches to pa-tient safety culture development and the disparate nature of reporting systems. In addition, with our study design we could not be certain that anonymous reporting was a key factor in the increase in reporting rates, but de-briefing interviews indicated that both anonymous reporting and declaring a nonpunitive, supportive approach in each prac-tice was important to enhanced reporting.

We expect increased reporting to decline over time without consistent feedback, as has been demonstrated in other studies [18], and we will continue to monitor rates over time.

As our current reporting system requires consider-able reporter time for data input and discussion with risk managers, is not specifically configured for ambulatory reporting, is considered by staff and providers poten-tially punitive, and marked under-reporting is clear, we have proposed moving to a new system that is more user-friendly, ambulatory-focused, and has a provision for anonymous reporting.

Presented in part at the Institute for Healthcare Improvement 15th Annual International Summit on Improving Patient Care in the Office Practice and the Community, Washington DC, March 2014.

Figure 4. TypeofsafetyeventreportedforallNovantHealthMedicalGrouppractices,April2013–March2014.

60%

50%

40%

30%

10%

20%

0%

Collaborativepractices(6)

Comparatorpractices(27)

Allotherpractices(315)

Fall MedicationEvent

DiagnosticTesting

Communication Other(injurynotafall)

Security/HIPAA

Miscellaneous ProcedureRelated

51%

6%

33%

25% 25%

35%

16%

2%

7%

34%

1%4%

11%12%

19%

7% 6%

<1% <1%0.6% 2% 1% 0% 0%


Acknowledgements: We gratefully acknowledge the work of collaborative practice team members, including Chris-topher Isenhour MD, Janet White, Shelby Carlyle, Mark Tillotson MD, Maria Migliaccio, Melanie Trapp, Jennifer Ochs, Gary DeRosa MD, Margarete Hinkle, Scott Wag-ner, Kelly Schetselaar, Timothy Eichenbrenner MD, Sandy Hite, Jamie Shelton, Raymond Swetenburg MD, James Lye MD, Kelly Morrison, Jan Rapisardo, Jane Moss, Rhett Brown MD, Dorothy Hedrick, Camille Farmer, and Wil-liam Anderson, MS, for assistance with analysis.

Corresponding author: Herbert Clegg, MD, 108 Providence Road, Charlotte NC, 28207, [email protected].

Financial disclosures: None.

REFERENCES1. Tang N, Meyer GS. Ambulatory patient safety: The time is

now. Arch Intern Med 2010;170:1487–9.2. Ghandi TK, Lee TH. Patient safety beyond the hospital. N

Engl J Med 2010;363:1001–3.3. Institute of Medicine. To err is human: Building a safer

health system. Washington DC: National Academies Press; 1999.

4. Pham JC, Girard T, Pronovost PJ. What to do with healthcare incident reporting systems. J Public Health Res 2013;2:e27.

5. Elder NC, Dovey SM. Classification of medical errors and preventable adverse events in primary care: A synthesis of the literature. J Fam Pract 2002;51:927–32.

6. Mohr JJ, Lannon CM, Thoma KA, et al. Learning from er-rors in ambulatory pediatrics. In: Henriksen K, Battles JB, Marks ES, et al, editors. Advances in patient safety: from research to implementation. Washington, DC: Agency for Healthcare Research and Quality; 2005: 355–68. Avail-able at www.ahrq.gov//downloads/pub/advances/vol1/Mohr.pdf.

7. Phillips RL, Dovey SM, Graham D, et al. Learning from different lenses: reports of medical errors in primary care by clinicians, staff, and patients. J Patient Saf 2006;2:140–6.

8. Singh H, Thomas EJ, Khan MM, Peterson LA. Identifying diagnostic errors in primary care using an electronic screen-ing algorithm. Arch Intern Med 2007;167:302–8.

9. Rappaport DI, Collins B, Koster A, et al. Implementing medication reconciliation in outpatient pediatrics. Pediat-rics 2011;128:e1600-e1607.

10. Bishop TF, Ryan AK, Casalino LP. Paid malpractice claims for adverse events in inpatient and outpatient settings. JAMA 2011;305:2427–31.

11. Wynia MK, Classen DC. Improving ambulatory patient safety. Learning from the last decade, moving ahead in the next. JAMA 2011;306:2504–5.

12. Neuspiel DR, Stubbs EH. Patient safety in ambulatory care. Pediatr Clin North Am 2012;59:1341–54.

13. Leape LL. Patient safety. Reporting of adverse events. N Engl J Med 2002;347:1633–8.

14. Neuspiel DR, Stubbs EH, Liggin L. Improving reporting of outpatient medical errors. Pediatrics 2011;128:e1608–e1613.

15. Neuspiel DR, Gizman M, Harewood C. Improving error reporting in ambulatory pediatrics with team approach. In: Henriksen K, Battles JB, Keyes MA, et al, editors. Advances in patient safety: new directions and alternative approaches. Vol 1. Agency for Healthcare Research and Quality; 2008. Available at www.ncbi.nlm.nih.gov/books/NBK43643/.

16. Plews-Ogan ML, Nadkarni MM, Forren S, et al. Patient safety in the ambulatory setting: a clinician-based approach. J Gen Intern Med 2004;19:719–25.

17. Harris AD, McGregor JC, Perencevich EN, et al. The use and interpretation of quasi-experimental studies in medical informatics. J Am Med Inform Assoc 2006;13:16–23.

18. Gerber JS, Prasad PA, Fiks AG, et al. Durability of ben-efits of an outpatient antimicrobial stewardship interven-tion after discontinuation of audit and feedback. JAMA 2014;312:2569–70.

Appendix. Safety Event Types and Classification

Medication/Vaccine Event• Administration• Adverse event• Dispensing• Medication reconciliation• Monitoring• Order entry/transcribing/verification

• Prescribing/ordering• Samples• Mislabeling• Wrong drug or vaccine given• Vaccine missed or administered at wrong time• Drug ordered (or given) for allergic patient




Falls

Communication Isssues/Documentation/ID Band• Reception process (eg, wrong patient registered)• Handoff inappropriate/incomplete• Incomplete documentation• No interpreter when needed• Error in entering patient information in PM or EMR• Patient not given requested appointment or referral

Diagnostic Issues• Lab

1. Mislabeling, not labeled, order not done, wrong test or study ordered

2. Delay in test performance3. Wrong patient4. Wrong collection tube5. Delay/failure in review of test results6. Delay/failure in reporting test results to patient

• Imaging1. Wrong test/study ordered2. Wrong patient3. Delay in performance of test/study4. Delay/failure in review of test results5. Delay/failure in reporting test results to patient

• Misdiagnosis• Delay in treatment

Security/Personal Property/HIPAA• Release of information without consent• Improper informed consent• Other HIPAA issue• Self-induced injury

Sharps/Body Fluid/Exposures• Sharps wound• Contact• Exposure/spill chemotherapeutic agent

Acute Care Related• Misdiagnosis• Delay in office care• Delay in treatment

• Delay in test reporting• Referral not kept• Left without being seen• Traumatic injury• Other

Behavior(Patient/Person involved)• Drug-seeking behavior documented• Patient refused treatment• Non-compliance• Self-induced injury• Violent-threatening behavior

Other Event Type• Injury other than a fall (equipment, IV,

phlebotomy, injection, etc)• Retained foreign body• Unexpected code/death• Unplanned removal/damage to tissue/organ• Wrong procedure/patient/site• Pause not performed if procedure• Medical device related

Safety Event Classification*1. Serious safety event (SSE): a deviation from generally

accepted performance standards that reached a patient and results in moderate to severe harm or death. A root cause analysis is required following these events.

2. Precursor safety event (PSE): a deviation that reaches the patient and results in minimal to no detectable harm. A root cause analysis or apparent cause analysis is required following these events.

3. Near miss safety event (NME): a deviation that does not reach the patient due to a defensive barrier or a lucky catch. There is no formal analysis required fol-lowing one of these events.

4. Non-safety event (NSE)

*Based upon a classification by Healthcare Per-formance Improvement, Virginia Beach, VA and adopted by Novant Health for the system’s safety program: First, Do No Harm.

Copyright 2015 by Turner White Communications Inc., Wayne, PA. All rights reserved.

improved safety event reporting in outpatient, … · is time-consuming (about 15 minutes for entry...

Documents