Title of article: Introducing a Standard Operating Procedure in the Retinoblastoma Pathway of Care

Authors: 1*Raumil Patel, 1Kaitlyn Hougham, 2 1Stephanie Kletke, 1,3Arshia Javidan, 4Jason Hu, 5Wei Sim, 1,2Sameh Solimaon and 1,2,5Brenda L. Gallie

Word count (excluding title page, abstract, references, figures and tables):

Abstract

Purpose: Based on the Canadian Retinoblastoma Guidelines for Care, The Hospital for Sick Children (SickKids) Retinoblastoma (retinoblastoma ) program developed 27 Standard Operating Procedures (SOPs). This study aims to determine evaluate the impact of a SOP for examinations under anesthesia (EUA) on retinoblastoma staff’s adherence to essential EUA steps, case times, and frequency of disruptions. This study also determines assesses retinoblastoma staff’s perceptions and attitudes towards SOPs usagee.Study Design: Prospective quality improvement study.

Methods: This study was approved by the SickKids Quality Management office. Prior to SOP implementation, a survey was administered to evaluate retinoblastoma team member perceptions of SOPs and a multidisciplinary group refined and updated the SOP. Two unbiased observers measured adherence to essential EUA steps, case-times, and frequency of disruptions before and after SOP implementation.

* Correspondence: Brenda L. Gallie |Brenda@gallie.ca| 416-294-9729

Postal: 555 University Ave, Toronto M5G 1X81Department of Ophthalmology and Vision Sciences, The Hospital for Sick Children, Toronto, Canada; 2Department of Ophthalmology and Vision Sciences, University of Toronto, Toronto, Canada; 3Faculty of Medicine, University of Toronto, Toronto, Canada; 4Faculty of Medicine, University of Ottawa, Ottawa, Canada;5Faculty of Medicine, Queen’s University, Kingston, Canada

Results: Sixty-four percent of the retinoblastoma team completed the pre-implementation survey. Forty-five percent did not routinely use SOPs, however most believed SOPs would improve patient care (72.0%) and surgical team function (63.0%). A total of 45 EUAs were performed, 22 occurring prior to SOP implementation and 23 following SOP implementation. Staff’s adherence to essential EUA steps increased (60.0% to 80.0%, P<0.0001) following SOP implementation. At least one disruption occurred in 47% of EUAs. The most frequent disruptions were related to equipment and miscommunication related, accounting for 57.0% and 43.0% of all disruptions, respectively. The frequency of disruptions decreased following SOP implementation (63.6% to 30.4%, P=0.026). After SOP implementation, the time required to complete an EUA (0:21:24 ± 0:11:32 to 0:19:22 ± 0:09:16, P=0.492) and total case time (0:52:38 ± 0:21:14 to 0:50:19 ± 0:13:12, P=0.661) decreased, but this did not reach statistical significance. at a statistically insignificant level.

Conclusions: This is the first study to prospectively evaluate SOPs developed for surgical treatment in a pediatric setting. The rretinoblastoma team’s favourablefavorable perceptions to SOP use likely facilitated SOP integration into the operating roomOR. The EUA SOP wasis associated with increased staff adherence to essential EUA steps and decreased frequency of disruptions. This may facilitate improvements in efficiency, communication, and quality of care while minimizing errors, distractions, and waste. These results are encouraging, and additional research of SOP use in surgical care is required to consolidate our findings.

Keywords: Standard operating procedures, Checklist, Quality improvement, Hospital, Retinoblastoma, Examination under Anesthesia, Communication, Surgery, Standardization, Patient safety

INTRODUCTIONRetinoblastoma is the most prevalent intraocular cancer of childhood, affecting the eyes unilaterally or bilaterally1-3. Ample awareness of retinoblastoma within the Canadian population and accessibility to genetic services allows for prompt diagnosis and provision of care4. Treatment options include focal therapies, such as laser therapy and cryotherapy, chemotherapy, and enucleation5. The Hospital for Sick Children (SickKids) has a multidisciplinary retinoblastoma team consisting of pediatric ophthalmologists, pediatric oncologists, genetic counsellors, ophthalmic imaging specialists, and social workers. The team approach to retinoblastoma care facilitates improvements in family support, decision-making, and patient outcomes6. However, management of retinoblastoma is intricate and a multidisciplinary approach risks inefficiencies, waste, and errors3. In order to maintain optimal quality of care in the multidisciplinary approachsetting, standardization of care is recommended3 7 8. Hospital policies on standardization are broad and difficult to tailor to individual team needs. Standardization through standard operating procedures (SOPs) may overcome these limitations.SOPs convert routine procedures into a series of steps to ensure consistent procedures are performanceed consistently at the desired level of level of quality3 7-9. They are written collaboratively by individuals with expertise in the field so they maycan be customized to fit the team’s specific needs, desires, and organizational culture7. Checklists offer another method foractualize introducing the SOP in everyday usestandardization. Implementation of the World Health Organization’s surgical safety checklist or a hospital-specific checklist is associated with improvements in communication, patient-safety awareness, and reduction in surgical complications10-19. Introduction of the surgical safety checklist in a tertiary obstetric centre increased familiarity and communication within the team, and compliance with operative checks13. Compared to checklists, SOPs are more comprehensive. In addition to as besides outlining the steps forto successful procedure completion, they mention include the materials and personnel required for the procedure, and the procedure’s purpose, risks, and benefits7. SOPs effectively combine evidence-based medicine with the realities of clinical practice into one document7 8. In medicine, SOPs are currently used in clinical trials, laboratory procedures, and provision of patient care to patients, but studies evaluating SOP use in surgical care are limited8 9 20 21. Moreover, when introducing SOPs or checklists, staff involvement in the design and implementation is necessaryhelps to ensure staff buy-in and maximize compliance. Since 2012, the retinoblastoma team at SickKids has developed 27 SOPs pertaining to different facets of retinoblastoma care, such as surgery, genetic counselling, and administrative duties. However, these SOPs have not yet been implemented. We therefore investigated the impact of implementing an SOP specifically for the retinoblastoma Examination Under Anesthesia (EUA)

procedure, wherein the patient’s eyes are examined while the patient is under general anesthesia, on the retinoblastoma pathway of care. The primary outcome measuress we measured were theincluded adherence to EUA steps listed in the SOP, frequency of disruptions, and case times. We introduced the EUA SOP in the form of a checklist for ease-of-use in the operating room (OR). We specifically introduced the EUA SOP, as EUAs are the most common retinoblastoma surgical procedure performed by our team, being used to diagnosis retinoblastoma and monitor response to treatment. We additionally measured staff’s perceptions and attitudes towards SOPs prior to before SOP implementation. To our knowledge, this study is the first to investigate the impact of SOPs in a pediatric and ophthalmic surgical environment.

METHODSStudy design This study was a prospective quality improvement study approved by the SickKids Quality Management office. The study was performed in the SickKids retinoblastoma operating room (OR) where patients underwentreceived EUAs. Prior toBefore SOP implementation, in order to update the dated EUA SOP to current practices and habits of the retinoblastoma team, a team comprised ofing individuals who would frequently use the EUA SOP was createdupdated the SOP to reflect current best practices. This team consisted of two pediatric ophthalmologists and an OR nurse. The team met three times over the course of a one month period to update the SOP. Given that the SOP was relatively extensive and as a result , not amenable to convenient perioperative use, the SOP was transformed into a one page checklist. This study was conducted in three phases:[i.] Observations to determine relevant variables: Data on a wide range of EUA

variables surrounding EUAs was collected. From the collected data, the primary outcomes of sStaff’s adherence to EUA steps, case times and the frequency of intraoperative disruptions were identified as primary outcome measuresdecided. A questionnaire was also distributed to members of the team to assess team perceptions and attitudes regarding checklist use.

[ii.] SOP update and implementation: involved theThe reiterative refinement of the EUA SOP was reiteratively refined and the subsequent development of an associated checklist was developed for perioperative convenience of use perioperatively. This checklist was piloted in the OR and was reiteratively modified with suggestions from the retinoblastoma team until saturation of feedback was achieved.

[iii.] Post-SOP iImplementation data collection: This involved official checklist implementation in the OR, and measuring the same metrics described in phase one.

The data collected for each variable is available in supplementary file 1.In the SOP update and implementation phase, we introduced and reiteratively refined the checklist until staff wereas satisfied with the final product. Any changes made to the checklist were reflected in the SOP. .

The refinement of the EUA SOP was a collaborative team-led initiativeeffort involving various members of the team, and it was also a team-led initiative. Involving the individuals who will use the checklist in the design process has been shown to increase compliance22.Post- SOP implementation data collection only began once all feedback had been incorporated into the checklist and SOP. Prior to the OR day, an OR nurse would print an individual EUA checklist was printed for each patient. The team had shared responsibility of the checklist completion, but it was primarily the responsibilitythat of the ophthalmology fellow and staff ophthalmologists to complete. The team alternated between a read-do approach, in which the where they would read the checklist was read before completing the steps, and a do-read approach, where they would in which the procedures were performed and perform the procedures and refer to the checklistthen the checklist was reviewed to detect missed steps. After the case, the completed checklist would bewas inserted into a folder specific for each patient.Patient SamplePatients of all ages for retinoblastoma treatmentundergoing examination under anesthesia for retinoblastoma were included in the study. Patients that were seenexamined by the retinoblastoma team that wereand not diagnosed with retinoblastoma were excluded from the study. Additionally, retinoblastoma patients requiring intra-arterial chemotherapy were also excluded, as . This is because intra-arterial chemotherapy isthis treatment is delivered by the image-guided therapy team in a separate OR and involves a different team andand procedures differ from a routine EUA a different overall procedure than the standard retinoblastoma case. Outcomes The primary outcomes measures included d were a ddeviations from the EUA SOP, case times, and the frequency of disruptions. A deviation from the EUA SOP was defined asis missing an essential step outlined in the SOP. The SOP contains both essential and optional steps (Ssupplementary Ffigure 1). Essential steps should be performed duringin each EUA, however due to the unique clinical history of the patients, not all steps , including essential steps, are applicable to each patient. We measured three types of disruptions: equipment, miscommunication, or procedure-related. We also measured three cCase-times included: total case time, total EUA

time, and port access time. We measured three types of disruptions including : equipment, miscommunication, andor procedure-related. The definitions used for the disruptions and case times are listed in Ttable 1.Data CollectionBesides our primary outcomes, we additionally measured variables thought to be influenced by SOP implementation or affect SOP implementation, but not directly affect the performance of an EUA. These variables includeAdditional outcome measures related to SOP implementation included : sample characteristics, case start time accuracy, case duration accuracy, number of imaging techniques performed, number of post-EUA procedures performed, number of individuals in the OR, presence of staff ophthalmologist and , and presence of clinical fellow. These variables are defined in Table 1. The clinicalClinical characteristics were obtained from review of electronic patient chartsmedical records and the eCancerCare retinoblastoma (eCC) database. These variables and their definitions are listed in table 1.

All patients in the study underwent an EUA before any additional procedures were performed. The data wereData was collected by two independent observers unrelated to the retinoblastoma team. One observer collected all the pre-SOP implementation data (RP), and the other observer collected all post-SOP implementation data (AJ). In order to minimize subjectivity between the two observers in the data collected, each observer underwent a two-week data collection training period using standardized instructions on data collection, and to familiarize themselves with the SOP and checklist. Instructions on collecting data in a standardized manner were also developed. Additionally, both observers collected datadata collected by both observers on four4 sample4 EUAs together which was compared to ensure data was collected consistentlyconsistency. The observers were instructed to avoid communication and interactions with the team. Data collection began at the start of the day with the OR huddle and typically lastedcontinued until the final patient had left the OR. Observers would remain in the room until the patient had left the room. Case- specific data collection started when the patient entered the room and finished when the patient left the room. All data collected was verified by the observers and a member of the retinoblastoma team. The de-identified data was stored in a Microsoft Excel sheet.

Figure 1: CONSORT diagram for the data collection process. Due to the unique disease history for the patient, nNot all variables were applicable to each patient.

Data AnalysisFisher’s exact test, Pearson’s chi-square test, and the independent sample’s T-test were used to evaluate differences between the pre-SOP implementation and post-SOP implementation samplesdata. Spearman’s Rho, Mann-Whitney U test and the independent samples T-test were used to identify significant associations between measured variables and the primary outcomes. Metrics found to significantly correlate with the primary outcomes were controlled when analyzing SOP impact on primary outcomes. To determine the effect of SOP implementation on the frequency of disruptions, a logistic regression model was used. ANCOVA and the independent samples T-test were used to evaluate changes in case times due to SOP implementation. The Mann-Whitney U test was used to evaluate changes in mean adherence to essential EUA steps after SOP implementation. IBM SPSS 25.0 statistical package was used for all data analysis. A P value of less than 0.05 was considered statistically significant. Data collected from cases where the prime reason for anesthetic was not examination was not used.

RESULTS

Patient cohortsA total of 45 EUAs were performed, involving with 33 unique patients (Ttable 1). With respect to clinical characteristics, there was no statistically significant difference between the groups (Ttable 1).

Table 1: Comparison of the before pre- and afterpost implementation samples on certain clinical variables.

Pre-SOP implementation

Post-SOP implementation

P-value

Number of EUAs performed

22 23

Unique Patients 16/22 17/23

Staging EUAs 2/22 0/23 0.23

Bilateral cases 18/22 21/23 0.41

Days since last EUA 37 42 0.33

Treatment given in last 3 EUAs

20/22 19/23 0.71

Treatment given in current EUA

22/22 16/23 0.18

SOP Adherence The compliance to 13 out of 14 essential EUA steps increased after SOP implementation. The greatest increase (14.3% to 82.6%, X2(1, n = 44) = 20.50, P<0.0001) was in providing the anesthesiologist a 10-minute notice to case end. Steps performed consistently before SOP implementation had statistically significant increases as well after SOP implementation: updating the horizontal sheet (81.8% to 100%, X2(1, n=45) =4.59, P=0.032), and updating the patient’s eCC (77.3% to 100%, X2(1, n=45) =5.88, P=0.015). To calculate the total adherence to essential EUA SOP steps, the sum of essential steps successfully completed were divided by the sum of all essential steps for each patient. A Mann-Whitney U test revealed a statistically significant difference in the adherence to essential EUA steps before SOP-implementation (Md = 75.0%, n = 22) and after SOP-implementation (Md = 92.0%, n = 23), U=56.00, z=-4.52, P< 0.0001, r=-0.67. Table 2: Frequency of completion of essential EUA steps before and after SOP-implementation, per case.

Variable Pre-SOP Implementation Frequency

Post-SOP Implementation Frequency

P-value

EUA equipment 19/22 23/23 0.07

Port access equipment

6/6 11/11 -

Ophthalmologist in huddle

19/19 23/23 -

Review eCC before consent

3/18 10/23 0.067

Review eCC before EUA

5/17 16/22 0.007

Review RetCam before EUA

14/18 22/22 0.02

Head towel 21/22 23/23 0.301

Time out 19/22 20/23 0.953

Notice to anesthesiologist

3/21 19/23 0.0001

Update horizontal sheet

18/22 23/23 0.032

Update eCC 17/22 23/23 0.015

Dictate OR note 21/22 22/23 0.97

Update fundus diagram

21/22 23/23 0.301

Frequency of DisruptionsOf the 45 EUAs, 21 (46.7%) had at least one disruption. The majority of these cases (95.0%) experienced a single disruption. No cases had more than one disruption in the three disruption categories: equationequipment, miscommunication or procedure related. The most frequent disruptions were equipment-related (57.0% of all disruptions), followed by miscommunication-related disruptions (43.0%). Procedural disruptions did not occur in any of the 45 EUAs. The cumulative frequency of disruptions significantly decreased after SOP implementation (63.6% to 30.43%, X2(1, n=45) =4.98 P=0.026) yet, individually the frequency of equipment related disruptions and miscommunication related discussions did not significantly change after SOP implementation.There was a negative correlation between the number of individuals in the OR and the frequency of disruptions, r=-0.34, n=45, P=0.021. Direct logistic

regression was performed to evaluate the influence of SOP implementation and number of individuals in the OR on the likelihood of a disruption occurring in the case. The model contained two independent variables (number of individuals in the OR, and SOP implementation). The model with all predictors was statistically significant, X2(2, n =45) = 11.03, P=0.004. The model explained 21.7% (Cox & Snell R square) and 29.0% (Nagelkerke R squared) of the variance in frequency of disruptions, and correctly classified 60.0% of cases. As seen in Ttable 3, both independent variables made a unique statistically significant contribution to the model. The strongest predictor of a disruption occurring was the number of individuals in the OR, with an odds ratio of 0.61. Controlling for the number of people in the OR, the implementation of an SOP was also a predictor of a disruption occurring, with an odds ratio of 0.20. Table 3: Frequency of intraoperative disruptions occurrence before and after SOP implementation per case.

Variable Pre-SOP Implementation

Post-SOP Implementation

Statistical Analysis

Frequency Frequency Statistical Test

P-value

Disruption (one or more)

14/22 7/23 Direct Linear Regression

0.004

Equipment Disruption

8/22 4/23 Pearson’s Chi-Square

0.15

Miscommunication Disruption

6/22 4/23 Pearson’s Chi-square

0.64

Procedural Disruption

0/22 0/23 - -

Case times

Table 4: Results from direct logistic regression assessing the effect of number of individuals in the OR and the implementation of the SOP on frequency of disruptions.

Variable B S.E. Wald

Df

P Odds Ratio

95.0% C.I. for Odds Ratio

Lower Upper

Number of individuals in the OR

-0.50 0.23 4.76 1 0.029 0.61 0.39 0.95

Group (pre- or post-SOP implementation)

-1.63 0.71 5.68 1 0.022 0.20 0.05 0.79

Case time data collected from patients that underwent enucleations oror case time data from non-t routine EUAs was not usedwere excluded which decreased the sample size for some case times.

There was a strong positive correlation between the number of imaging procedures performed in the EUA and the time required to perform an EUA, r=0.52, n=38, P=0.001. Total case time positively correlateds with the number of imaging procedures performed in the EUA (r=0.43, n=40, P=0.006), and the number of post-EUA procedures performed (r=0.40, n=40, P=0.01). The time required to perform an EUA decreased from 0:21:24 ± 0:11:32 to 0:19:22 ± 0:09:16 after SOP implementation. An ANCOVA [between-subject factors: total EUA time; covariate: number of imaging procedures] revealed no main effects of SOP implementation, F(1, 35)=0.31, P=0.58, ηp

2=0.009. The total case time also decreased from 0:56:32 ± 0:35:17 to 0:50:19 ± 0:13:12 after SOP implementation. An ANCOVA [between-subject factors: total case time; covariate: number of imaging procedures and number of post-EUA procedures] revealed no main effects of SOP implementation, F(1, 37 )=1.53, P=0.22, ηp

2=0.04.

The time required to access the patient’s port (0:02:41 ± 0:0035 to 0:02:12 ± 0:00:54, t(14)=1.15, P=0.27, two-tailed) decreased after SOP implementation.

After SOP implementation, there was an increase in case duration accuracy (50.0% to 87.0%, X2(1, n = 45) = 4.59, P=0.032). Additionally, case start accuracy wasis positively associated with the presence of the lead staff ophthalmologist, r =-0.37, n = 45, P = 0.011. A direct logistic regression was performed to ascertain the effect of the presence of the stafflead ophthalmologist and SOP implementation on case start accuracy. The logistic regression model was statistically significant X2(2, n =45) = 11.72, P=0.003. The model correctly classified 75.6% of the cases and explained 32.3% (Nagelkerke R 2) of the variance in case start time accuracy. When accounting for the presence of the lead staff ophthalmologist, implementation of the SOP is significantly associated with an increase in case start time accuracy (P = 0.023).

Survey

Surveys to assess staff’s attitudes and perceptions towards SOP use before SOP implementation were distributed to 11 retinoblastoma staff team members (, and results are highlighted in Ttable 4). 45% of staff did not routinely use SOPs, and believed they performed well without them. The majority of staff were unaware of where the SOPs wereare located (64.0%). RegardlessHowever, most believed SOP use wouldill have a beneficial impact on patient care (72.0%), clinical practice/administrative roles (72.0%), and team functioning (64.0%).

Table 4 Results of Questionnaire Designed to Measure Rretinoblastoma Staff's Opinions and Attitudes towards on SOPs

Pre-intervention (%)

Question Yes/Positive

No/Negative Somewhat/Neutral

Reasons I routinely use retinoblastoma SOPs Include:

I do not routinely use applicable retinoblastoma SOPs 45 55

Improves patient care 45 55

Improves team functioning 54 46

Improves personal confidence in clinical practice 36 64

Comply with administrative orders 27 73

Other 18 82

Reasons I do not routinely use retinoblastoma SOPs include:

I routinely use applicable retinoblastoma SOPs 0 100

I did not know there were retinoblastoma SOPs 27 73

I do not have time to use retinoblastoma SOPs 9 91

I am unsure where to find current retinoblastoma SOPs 36 64

I am performing well without retinoblastoma SOPs 45 55

Other 45 55

Your opinion of the retinoblastoma team’s receptiveness to SOP implementation 45 18

Impact of the retinoblastoma SOPs on patient care 72 0 18

Impact of the retinoblastoma SOPS on team functioning 64 0 36

Impact of the retinoblastoma SOPs on your clinical practice and/or administrative roles

72 0 18

In your opinion, how do patients perceive the use of the retinoblastoma SOPs? 36 0 64

DISCUSSIONIn this study, we investigated the impact of SOP implementation in the retinoblastoma pathway of care and determined staff’s perception and attitudes towards SOP use. R We had retinoblastoma staff lead the development and implementation of the EUAa SOP for EUAs. SOPs facilitate enable standardization ofin medical care, minimizing which can facilitate improvements in efficiencies and minimize waste, inefficiencies and miscommunication7-9 23. However, there is a lack of research evaluating SOP use in the pediatric population, and ophthalmic surgical treatments. Our results indicate SOP implementation is associated with increased adherence to essential EUA steps, and negatively associated with the frequency of disruptions per case. Additionally, our survey results indicate that retinoblastoma staff arewere receptive towards of SOP use, and believed patient care can be improved through SOP use.

SOP implementation is associated with improvements inimproved adherence to EUA steps. A potential explanation could be that a physical checklist in the OR serves as a reminder for staff to address or contemplate all EUA steps. The greatest increase in adherenceimprovement was in providing a 10-minute notice to the anesthesiologist. The retinoblastoma team identified this as an important step as it promotes improved communication with the anesthesiologist and may, but can also minimize the time between case end and patient regaining consciousness. Steps performed most consistently occurred at the end of the case. This could be due to downtime as the case ends, facilitating easier recall of the steps required to be completed. Steps most frequently missed occurred during the EUA. Therefore, introducing a checklist with the steps written facilitated improved compliance to these activities. Although overall adherence to EUA steps improved significantly from 75.00% before SOP implementation andto 92.00% SOP post-implementation, we did not reach 100.00% adherence. A possible explanation could be that steps with the greatest increase in adherence provided more tangible benefits to the retinoblastoma staff. For instance, the benefits of shorter case times from providing the anesthesiologist a 10- minute notice are easily noticeable compared to ensuring an appropriate head towel is always prepared. The increased compliance to procedure steps upon introduction of a checklist is a common improvement seen after checklist implementation9 10 23. This is particularly important in a setting involving rotating team members, including improvements in adherence is important to our team where it’s common to have fellows, residents, and OR nurses rotate. With SOP introductionAn SOP enables seamless integration of , new team members of the team are provided with instructions on how the team performs to facilitate seamless integration.

Disruptions are a common source of inefficiencies in the OR24-26. Minimization of disruptions is essential to increasing work quality. A primary benefit of standardization is reducing incorrect performances of a procedure as the checklist lists the essential steps requiredidentifies essential steps. In our study, we detected zero procedural disruptions,

indicating that the retinoblastoma team has a strong grasp on the procedures performed in the OR. However, equipment and miscommunication disruptions were common. The frequency of equipment disruptions can may be attributed to the many imaging equipment modalities utilized and the corresponding imaging specialists required. For instance, use of RetCam and OCT imaging wereis common during an EUA , and is typicallyoften performed by imaging specialists. Additionally, because of the nature of EUAs whereFurthermore, it may be difficult to predict which imaging techniques may be required, especially in the context of new or lack of findings may warrant additional imaging techniques, it’s difficult to predict which imaging technique will be requiredwarranting investigation. Therefore, in certain scenarios RetCam imaging might not suffice and optical coherence tomography or a B-scan ultrasonography is required, but the machinery is not made available beforehand. Introduction of the EUA checklist is associated with a statistically significant decrease in the frequency of disruptions. This could be because tThe checklist may remindreminds staff to prepare sufficient quantities of EUA equipment, and to reviewse the patient’s previous RetCam images and medical recordseCC before the patient enters. This revision can help identify if further imaging techniques beyond manual observation or RetCam will be needed.

The decrease in miscommunication disruptions and improvements in communication seen in our study is a common theme observed in other studies as well11 13 15 18 19. This may have resulted from the inclusion of a checklist item One possible explanation is that we have an item in the checklist to ensures a time-out is completedensuring completion of a surgical time-out. Although the adherence to the time-out did not significantly changed before and after SOP implementation, after training and use withof the SOP, there was an increase retinoblastoma staff were more focused and attentive as evidenced by the increase in adherence to other steps. Therefore, it is possible thePerhaps attention and focus during retinoblastoma staff had greater focus in these time-outs improved after SOP implementation. This is beneficial as an intraoperative pause has been identified to improve communication within teams by halting extraneous conversations and providing all members an opportunity to discuss issues or problems27. Another factor is that our checklist was not designed to be completed by only one person. The entire team in the OR shared responsibility for completing the checklist, which promoted better communication within the team. Miscommunication has been identified as a problem in many surgical cases, extending case times, promoting errors and reducing teamwork26 28-30, but the introduction of a checklist with shared responsibility can reduce the frequency of these disruptions to improve efficiency and work quality.

Introduction of the SOP had no significant effects on case time. This is likely due to individual variation between the cases. The checklist reminds the staff on certain procedural steps, but it remains broad and does not specify how to perform each individual aspect of the EUA, such as RetCam imaging. SOPs on imaging techniques and retinoblastoma treatments have already also been developed by the retinoblastoma team, and future studies would need tomay examine if whether the introductions of

these SOPs facilitate decreaseddecreases EUA and case times. The checklist provides a reminder to ensure equipment for port-access is available, but it does not direct the staff through the procedures. Our results indicate no differences in the availability of port-access equipment before and after SOP implementation, therefore the checklist is unlikely to affect port-access time. However, the checklist facilitated an improvement in providing the anesthesiologist a 10-minute notice to case end, increases in case start time and case duration accuracy, and decreases in disruptions which haves been identified to reduce the time required for procedures31. Our limited sample size and the variability of procedures and imaging techniques used in retinoblastoma cases may have prevented us from observing similar results. Despite the statistically insignificant results, they demonstrate that checklist introduction does not prolong any aspects of surgery. Fear of extending case times has been identified as a barrier to staff adopting checklist use32 33.

Checklist introduction facilitates a significant decrease in the patient preparation time. Patient preparation times are often extended due to insufficient dilation of the pupils, but the checklist introduces an item to check forincludes sufficient eye pupillary dilation before the patient enters the room. If the eyes are insufficiently dilateddilation is insufficient, additional dilating drops are given, decreasing the patient preparation time. Additionally, when a parent or guardian accompanies the patient to the OR, the OR can become crowded and uncomfortable for the child, extending the time required for the patient to be anesthetized. Our checklist recommends in these scenarios that staff who are not immediately helping the patient to momentarily leave the OR and prevent crowdedness.

Refusal to adopt checklist use by staff has been identified as a barrier to implementing checklists33. We did not encounter this problem, as indicated by the survey results. Most staff were receptive towards SOP useuse and believed SOP use can lead to improvements in the pathway of care. We also provided a training period where staff had the chance tofor staff to increase familiarity with theutilize the checklist and request preferential changes be made to the checklistany modifications. This period identified difficulties staff had to using the checklist such as being unsure of when to pause during the surgery and refer to the checklist which were addressed. Additionally, a team leader was assigned to ensure all members in the OR understood the importance of the checklist and how it is to be used. Cumulatively, theseThese efforts ensured high rates of checklist compliance to the checklist.

This study had notable strengths. To our knowledge, this is the first study to evaluate the impact of SOP implementation in a pediatric and ophthalmic surgical environmentSOP implementation in a pediatric ophthalmology setting. The results of this study identify improvements in surgical care associated with SOP implementation. Furthermore, thisThis study involved measurement of a variety of frameworksassessed several aspects of SOP implementation, including such as adherence, disruptions, and case-times which allowed for a more complete understanding of the effects of SOP implementation on surgical care. Moreover, as the intervention was staff-led and

appropriate steps were taken to increase compliance, we did not experience difficulty in integrating the checklist into the OR.

The exploratory nature of the study is a limitation of the study. Due to the numerous variables collected and the multiple comparisons made, the results are at risk for a type I error. Other limitations of the study include data collection through observation which prevents us from determining causal relationships between SOP implementation and our outcomes. Additionally, the data is also subject to observer bias. Staff acknowledgement of the presence of the observers may have affected their performance34. Similarly, the survey results collected did not come from all members of the retinoblastoma team, and are subject to responder bias. The staff that did respond to the survey may have a more favorable appreciation for SOP use than staff that did not respond. Moreover, the generalizability of this study is limited as the EUA SOP and accompanying checklist are specifically tailored to the retinoblastoma team at SickKids. However, we have prepared a SOP that provides instructions on how to implement a SOP that can be generalized to other institutions. Another limitation is the use of two separate observers to collect pre-and post-SOP implementation data. However, we attempted to minimize discrepancies in the data collection method between the two observers through a training period, development of a standardized approach to collect data, and independent review of the data. Lastly, we were limited by our sample size, which decreased the power of our study and could have prevented us from identifying statistically significant results with respect to case-times.

In conclusion, our study demonstrates that the standardization of procedures through SOP implementation can improve facets of surgical care such as lowering the frequency of disruptions, related case times and promoting adherence to essential procedural steps. These improvements help maintain the desired quality of care while decreasing inefficiencies and waste. Future studies that establish causal relationships between SOP implementation and beneficial outcomes are required. In the future, SOPs for other aspects of retinoblastoma care, such as administration of a new patient will be created and implemented. We also plan to share the SOPs with other retinoblastoma treatment centers.

SUPPLEMENTARY MATERIAL

EUA supplementary file 1 – this is an excel file that has all of the raw data collected. Available on docshare.

Supplementary Figure 1: EUA Checklist Introduced into retinoblastoma OR

Supplementary Table 1: Definitions of variables

Variable Definition Note

Sample Characteristics

Staging EUA Patient’s first EUA to stage retinoblastoma

Bilateral If patient has bilateral retinoblastoma

Time since last EUA Days elapsed between current EUA and patient’s last EUA

Not applicable if patient does not have prior EUA

Treatment in last 3 EUAs

Not including current EUA, did the patient receive treatment in last three EUAs?

Not applicable if patient does not have prior EUA

Treatment in current EUA

Was the patient given retinoblastoma specific treatment in the current EUA?

Essential EUA Steps

EUA equipment Performed correctly if all equipment required for the EUA is present at case start

Port access equipment

Performed correctly if all equipment required for the port access is present at case start

Ophthalmologist in huddle

Performed correctly if at least one ophthalmologist partakes in morning huddle

Review eCC before consent

Performed correctly if at least one ophthalmologist checks patient’s eCC before consent

Not applicable if patient does not have an eCC profile created, or eCC profile is empty

Review eCC before EUA

Performed correctly if at least one ophthalmologist checks patient’s eCC before EUA

Not applicable if patient does not have an eCC profile created, or eCC profile is empty

Not required if ophthalmologist checks patient’s eCC before consent

Review RetCam before EUA

Performed correctly if at least one ophthalmologist reviews patient’s

Not applicable if patient does not have prior

previous RetCam images RetCam images

Head towel Performed correctly if the appropriate head towel is prepared for each patient

Not applicable if ophthalmologist do not require head towel

Time out Performed correctly if OR nurse calls for a time out

Notice to anesthesiologist

Performed correctly if any member of the provides notice to anesthesiologist of case ending soon

Update horizontal sheet

Performed correctly if updated at any point during surgery

Not applicable if horizontal sheet not prepared for patient

Update eCC Performed correctly if updated at any point during surgery

Not applicable if eCC profile not created for patient

Dictate OR note Performed correctly if dictated at any point during surgery

Update fundus diagram

Performed correctly if updated at any point during surgery

Not applicable if fundus diagram not prepared for patient

Total adherence Average of the overall adherence to EUA steps for each case

Disruptions

Equipment Equipment is not available, malfunctioning, the staff required to operate the equipment is missing, or staff unaware of how to operate equipment

Procedure Staff unaware of procedure or perform procedure incorrectly

Miscommunication Commands going unheard, or incorrect interpretations of commands

Case Time

Total case time Time elapsed between patient entering OR and patient leaving OR

Total EUA time Time elapsed between start of EUA and end of EUA

Port access time Time taken for ophthalmologist to access port

Not applicable if patient does not require port access

Other Prominent Variables

Case start accuracy A case start time is accurate if the case started as scheduled with a 5-minute leeway

Case duration accuracy

A case duration is accurate if the case did not extend for more than allotted time

Number of individuals in the OR

Number of individuals in the OR at the start of EUA

Number of imaging techniques performed

Number of imaging techniques performed as part of the EUA

Number of post-EUA procedures performed

Number of procedures performed as part of treatment after EUA

Not applicable if only EUA performed as part of surgery

Presence of lead ophthalmologist

If the lead ophthalmologist is present during EUA start

Presence of staff ophthalmologist

If the staff ophthalmologist is present during EUA start

REFERENCES

1. Dimaras H, Corson TW, Cobrinik D, et al. Retinoblastoma. Nature reviews Disease primers 2015;1:15021. doi: 10.1038/nrdp.2015.21 [published Online First: 2015/01/01]

2. Dimaras H, Kimani K, Dimba EAO, et al. Retinoblastoma. The Lancet 2012;379(9824):1436-46. doi: 10.1016/s0140-6736(11)61137-9

3. Valenzuela A, Chan HSL, Heon E, et al. Pediatric Retina: Springer, Berlin, Heidelberg 2011.

4. Dimaras H, Dimba EA, Gallie BL. Challenging the global retinoblastoma survival disparity through a collaborative research effort. Br J Ophthalmol 2010;94(11):1415-6.

doi: 10.1136/bjo.2009.174136 [published Online First: 2010/08/04]

5. Canadian Retinoblastoma S. National Retinoblastoma Strategy Canadian Guidelines for Care: Strategie therapeutique du retinoblastome guide clinique canadien. Can J

Ophthalmol 2009;44 Suppl 2:S1-88. doi: 10.3129/i09-194

6. Elzomor H, Taha H, Nour R, et al. A multidisciplinary approach to improving the care and outcomes of patients with retinoblastoma at a pediatric cancer hospital in Egypt.

Ophthalmic Genet 2017;38(4):345-51. doi: 10.1080/13816810.2016.1227995 [published Online First: 2017/01/14]

7. Guidance for Preparing Standard Operating Procedures (SOPs). In: Agency USEP, ed.: U.S. Environmental Protection Agency, 2007.

8. Rao TS, Radhakrishnan R, Andrade C. Standard operating procedures for clinical practice. Indian J Psychiatry 2011;53(1):1-3. doi: 10.4103/0019-5545.75542

9. Chen C, Kan T, Li S, et al. Use and implementation of standard operating procedures and checklists in prehospital emergency medicine: a literature review. Am J Emerg Med

2016;34(12):2432-39. doi: 10.1016/j.ajem.2016.09.057 [published Online First: 2016/10/16]

10. Byrnes MC, Schuerer DJ, Schallom ME, et al. Implementation of a mandatory checklist of protocols and objectives improves compliance with a wide range of

evidence-based intensive care unit practices. Crit Care Med 2009;37(10):2775-81. doi: 10.1097/CCM.0b013e3181a96379 [published Online First: 2009/07/08]

11. Cabral RA, Eggenberger T, Keller K, et al. Use of a Surgical Safety Checklist to Improve Team Communication. AORN J 2016;104(3):206-16. doi:

10.1016/j.aorn.2016.06.019

12. El Boghdady M, Tang B, Alijani A. The effect of a performance-based intra-procedural checklist on a simulated emergency laparoscopic task in novice surgeons. Surgical endoscopy 2017;31(5):2242-46. doi: 10.1007/s00464-016-5223-0 [published

Online First: 2016/09/03]

13. Kearns RJ, Uppal V, Bonner J, et al. The introduction of a surgical safety checklist in a tertiary referral obstetric centre. BMJ Qual Saf 2011;20(9):818-22. doi:

10.1136/bmjqs.2010.050179 [published Online First: 2011/06/23]

14. Morgan L, New S, Robertson E, et al. Effectiveness of facilitated introduction of a standard operating procedure into routine processes in the operating theatre: a

controlled interrupted time series. BMJ Qual Saf 2015;24(2):120-7. doi: 10.1136/bmjqs-2014-003158

15. Pugel AE, Simianu VV, Flum DR, et al. Use of the surgical safety checklist to improve communication and reduce complications. J Infect Public Health 2015;8(3):219-

25. doi: 10.1016/j.jiph.2015.01.001

16. Salzwedel C, Bartz HJ, Kuhnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: a randomized controlled trial. International journal for quality in health care : journal of the International Society for Quality in Health Care

2013;25(2):176-81. doi: 10.1093/intqhc/mzt009 [published Online First: 2013/01/31]

17. Sherren B, Tricklebank S, Glover G. Development of a standard operating procedure and checklist for rapid sequence induction in the critically ill. Scand J Trauma

Resusc Emerg Med 2014;41(22)

18. Takala RS, Pauniaho SL, Kotkansalo A, et al. A pilot study of the implementation of WHO surgical checklist in Finland: improvements in activities and communication. Acta

Anaesthesiol Scand 2011;55(10):1206-14. doi: 10.1111/j.1399-6576.2011.02525.x

19. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual Saf 2014;23(4):299-318. doi: 10.1136/bmjqs-2012-

001797

20. Baretinoblastoma é B, Verdonck K, Mukendi D, et al. The Art of Writing and Implementing Standard Operating Procedures (SOPs) for Laboratories in Low-

Resource Settings: Review of Guidelines and Best Practices. PLOS Neglected Tropical Diseases 2016;10(11):e0005053. doi: 10.1371/journal.pntd.0005053

21. Best R, Harris H, Walsh L, et al. Pediatric Drowning A Standard Operating Procedure to Aid the Prehospital Management of Pediatric Cardiac Arrest Resulting

from Submersion. Pediatr Emerg Care 2017;0(0)

22. Conley DM, Singer SJ, Edmondson L, et al. Effective surgical safety checklist implementation. J Am Coll Surg 2011;212(5):873-9. doi:

10.1016/j.jamcollsurg.2011.01.052

23. Nachtigall I, Deja M, Tafelski S, et al. Adherence to Standard Operating Procedures is Crucial for Intensive Care Unit Survivial of Elderly Patients. J Int Med Res

2008;36:438-59.

24. Parker SE, Laviana AA, Wadhera RK, et al. Development and evaluation of an observational tool for assessing surgical flow disruptions and their impact on surgical

performance. World J Surg 2010;34(2):353-61. doi: 10.1007/s00268-009-0312-z [published Online First: 2009/12/17]

25. Sevdalis N, Forrest D, Undre S, et al. Annoyances, disruptions, and interruptions in surgery: the Disruptions in Surgery Index (DiSI). World J Surg 2008;32(8):1643-50. doi:

10.1007/s00268-008-9624-7 [published Online First: 2008/05/21]

26. Wiegmann DA, ElBardissi AW, Dearani JA, et al. Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery

2007;142(5):658-65. doi: 10.1016/j.surg.2007.07.034 [published Online First: 2007/11/06]

27. Erestam S, Angenete E, Derwinger K. The Surgical Teams' Perception of the Effects of a Routine Intraoperative Pause. World J Surg 2016;40(12):2875-80. doi:

10.1007/s00268-016-3632-9

28. Gillespie BM, Chaboyer W, Fairweather N. Interruptions and miscommunications in surgery: an observational study. Aorn j 2012;95(5):576-90. doi:

10.1016/j.aorn.2012.02.012 [published Online First: 2012/05/01]

29. Sevdalis N, Healey AN, Vincent CA. Distracting communications in the operating theatre. Journal of evaluation in clinical practice 2007;13(3):390-4. doi: 10.1111/j.1365-

2753.2006.00712.x [published Online First: 2007/05/24]

30. Yoong W, Khin A, Ramlal N, et al. Interruptions and distractions in the gynaecological operating theatre: irritating or dangerous? Ergonomics 2015;58(8):1314-

9. doi: 10.1080/00140139.2015.1005171 [published Online First: 2015/02/13]

31. Ahmed A, Ahmad M, Stewart CM, et al. Effect of distractions on operative performance and ability to multitask--a case for deliberate practice. The Laryngoscope

2015;125(4):837-41. doi: 10.1002/lary.24856 [published Online First: 2014/07/31]

32. Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the implementation of surgical safety checklists: a systematic review of the qualitative

evidence. BMJ Qual Saf 2015;24(12):776-86. doi: 10.1136/bmjqs-2015-004021

33. Fourcade A, Blache JL, Grenier C, et al. Barriers to staff adoption of a surgical safety checklist. BMJ Qual Saf 2012;21(3):191-7. doi: 10.1136/bmjqs-2011-000094

34. McCarney R, Warner J, Iliffe S, et al. The Hawthorne Effect: a randomised, controlled trial. BMC Med Res Methodol 2007;7:30. doi: 10.1186/1471-2288-7-30