Title of article: Introducing a Standard Operating Procedure in the Retinoblastoma Pathway of Care

Authors: 1*Raumil Patel, 1Kaitlyn Hougham, 2Stephanie Kletke, 1,3Arshia Javidan, 4Jason Hu, 5Wei Sim, 1,2,6 Sameh Soliman and 1,2,5Brenda L. Gallie

Word count (excluding title page, abstract, references, figures and tables):s

AbstractPurpose: Based on the Canadian Retinoblastoma Guidelines for Care, The Hospital for Sick Children (SickKids) Retinoblastoma program developed 27 Standard Operating Procedures (SOPs). This study aims to evaluate the impact of one SOP for examinations under anesthesia (EUA) on retinoblastoma team adherence to key steps, case times, and frequency of disruptions. This study also assesses the staff’s perceptions and attitudes to using SOPs.

Study Design: Prospective quality improvement study.

Methods: This study was approved by the SickKids Quality Management office. A survey was administered to evaluate retinoblastoma team member perceptions of SOPs. A multidisciplinary group refined and updated the EUA SOP. Two unbiased observers measured adherence to key EUA steps, case-times, and frequency of disruptions before and after SOP implementation.

Results: 64.0%Sixty-four percent of the retinoblastoma team (11/17) completed the pre-implementation survey. Forty-five percent45.0% (5/11) did not routinely use SOPs, however most 72.0% (8/11) believed SOPs would improve patient care 63% (7/11) believed they would help (72.0%) and surgical team function (63.0%).function. A total of 45 EUAs were evaluated, 22 prior to and 23 following SOP implementation. Staff’s adherence to key EUA steps increased (60.0% to 80.0%, P<0.0001) following SOP implementation. At least one disruption occurred in 47.0% of EUAs. The most frequent disruptions were related to equipment and miscommunication, 57.0% and 43.0% of all disruptions, respectively. The frequency of disruptions decreased following SOP

* Correspondence: Brenda L. Gallie |Brenda@gallie.ca| 416-294-9729

Postal: 555 University Ave, Toronto M5G 1X81Department of Ophthalmology and Vision Sciences, The Hospital for Sick Children, Toronto, Canada; 2Department of Ophthalmology and Vision Sciences, University of Toronto, Toronto, Canada; 3Faculty of Medicine, University of Toronto, Toronto, Canada; 4Faculty of Medicine, University of Ottawa, Ottawa, Canada;5Faculty of Medicine, Queen’s University, Kingston, Canada6 Department of Ophthalmology, University of Alexandria, Alexandria, Egypt

implementation (63.6% to 30.4%, P=0.026). After SOP implementation, the time required to complete an EUA was observed to (21:24 ±11 to 19:22 ± 09:16, P=0.49) and total case time (52:38 ± 21:14 to 50:19 ± 13:12, P=0.66) decrease, but did not reach statistical significance.

Conclusions: This is the first report of evaluation of an SOP developed for a pediatric ophthalmic operative assessment surgery. The team’s favorable perceptions to SOP use likely facilitated SOP integration into the operating room. The EUA SOP was associated with increased staff adherence to key EUA steps and decreased frequency of disruptions. This may facilitate further improvements in efficiency, communication, and quality of care while minimizing errors, distractions, and waste.

Keywords: Standard operating procedures, Checklist, Quality improvement, Hospital, Retinoblastoma, Examination under Anesthesia, Communication, Surgery, Standardization, Patient safety

INTRODUCTIONRetinoblastoma is the most prevalent intraocular cancer of childhood, affecting both or one eye1-3. Ample awareness of retinoblastoma within the Canadian population and accessibility to genetic services allows for prompt diagnosis and provision of care4. Treatment options include focal therapies, such as laser therapy and cryotherapy, chemotherapy, and enucleation5. The Hospital for Sick Children (SickKids) has a multidisciplinary retinoblastoma team consisting of pediatric ophthalmologists, pediatric oncologists, genetic counsellors, ophthalmic imaging specialists, and social workers. This team-oriented approach to retinoblastoma care facilitates improvements in family support, decision-making, and patient outcomes6. However, management of retinoblastoma is intricate and a multidisciplinary approach risks inefficiencies, waste, and errors3. In order to maintain optimal quality of care in the multidisciplinary setting, standardization of care is recommended3 7 8. Hospital policies on standardization are broad and difficult to tailor to individual team needs. Standard operating procedures (SOPs) may overcome these limitations.

SOPs convert routine procedures into a series of steps to ensure consistent performance at the desired level of quality3 7-9. They are written collaboratively by individuals with expertise in the field so they may be customized to fit the team’s specific needs and organizational culture7. SOPs may be used to teach staff how to perform a new procedure or ensure already trained personnel adhere to established procedural steps. Checklists actualize the SOP in everyday use. Implementation of the World Health Organization’s surgical safety checklist or a hospital-specific checklist is associated with improvements in communication, patient-safety awareness, and reduction in surgical complications10-19. . Introduction of the surgical safety checklist in a tertiary obstetric centre increased familiarity and communication within the team, and compliance with operative checks13. Compared to checklists, SOPs are more comprehensive. In addition to outlining the steps for successful procedure completion, they include the materials and personnel required, and the procedure’s purpose, risks, and benefits7. SOPs effectively combine evidence-based medicine with the realities of clinical practice into one document7 8. In medicine, SOPs are currently used in clinical trials, laboratory procedures, and provision of patient care, but studies evaluating SOP use in surgical care are limited8 9 20 21. Moreover, when introducing SOPs or checklists, staff involvement in the design and implementation helps to ensure staff buy-in and maximize compliance.

Since 2012, the retinoblastoma team at SickKids has developed 27 SOPs pertaining to different facets of retinoblastoma care, such as surgery, genetic counselling, and administrative duties such as the registration of a new RB patient. However, these SOPs have not yet been implemented. We therefore investigated the impact of implementing an SOP specifically for the retinoblastoma Examination Under Anesthesia (EUA) procedure, wherein the patient’s eyes are examined under general anesthesia. The primary outcome measures included adherence to EUA steps listed in the SOP, frequency of disruptions, and case times. We introduced the EUA SOP in the form of a

checklist for ease-of-use in the operating room (OR). We specifically introduced the EUA SOP, as EUAs are the most common retinoblastoma surgical procedure performed by our team, used to diagnosis retinoblastoma and monitor response to treatment. We additionally measured staff’s perceptions and attitudes towards SOPs prior to implementation. To our knowledge, this study is the first to investigate the impact of SOPs in a pediatric ophthalmic surgical environment.

METHODS

Study design This study was a prospective quality improvement study approved by the SickKids Quality Management office. The study was performed in the SickKids retinoblastoma operating room (OR) where patients underwent EUAs. We specifically introduced the EUA SOP, as EUAs are the most common retinoblastoma surgical procedure performed by our team, used to diagnosis retinoblastoma and monitor response to treatment.

Prior to SOP implementation, a team of two pediatric ophthalmologists (authors SS, BG) and an OR nurse who frequently perform the EUA, met three times over a one month period and updated the EUA SOP to reflect current best practices. . The SOP was transformed into a one page checklist for ease-of-use in the OR.

The checklist SOP (supplementary figure 1) contains both essential and optional steps (Supplementary Figure 1). The checklist is divided into four sections encompassing steps to be done at the morning huddle, before patient enters the room, during the EUA, and after the EUA. Essential steps (e.g. checking patient’s previous RetCam Images) should be performed during each EUA, however not all steps (e.g. checking the eCC for a patient that does not have one yet) are applicable to each patient.

Patient Sample

Children with retinoblastoma of all ages undergoing examination under anesthesia were included. Patients examined by the retinoblastoma team and not diagnosed with retinoblastoma were excluded from the study. Children undergoing a planned surgical treatment as intra-arterial chemotherapy, intravitreal chemotherapy or enucleation were excluded, as these procedures differ from a routine EUA.

This study was conducted in three phases:

i. Observations to determine relevant variables: Data on a wide range of EUA variables was collected. Staff’s adherence to EUA steps, case times and the frequency of intraoperative disruptions were identified as primary outcome measures. A questionnaire was also distributed to members of the team to assess perceptions and attitudes regarding checklist use.

[ii.] SOP update and implementation: The EUA SOP was reiteratively refined and an associated checklist was developed for perioperative convenience. This checklist

was piloted in the OR and was reiteratively modified with suggestions from the retinoblastoma team to team until saturation of feedback was achievedcreategenerate the official checklist used in post-SOP implementation data collection..

ii.[iii.] Post-SOP implementation data collection: This involved official checklist implementation in the OR and measuring the same metrics described in phase one.

The data collected for each variable is available in supplementary file 1.

In the SOP update and implementation phase, we introduced and reiteratively refined the checklist until staff were satisfied with the final product. Any changes made to the checklist were reflected in the SOP. The refinement of the EUA SOP was a collaborative team-led initiative. Involving the individuals who will use the checklist in the design process has been shown to increase compliance22.

Post-SOP implementation data collection began once all feedback had been incorporated into the checklist and SOP. Prior to the OR day, an individual EUA checklist was printed for each patient. The team had shared responsibility of checklist completion, but it was primarily that of the ophthalmology fellow and staff ophthalmologists. The team alternated between a read-do approach, in which the checklist was read before completing the steps, and a do-read approach, in which the procedures were performed and then the checklist was reviewed to detect missed steps. After the case, the completed checklist was inserted into a folder specific for each patient.

Data Collection

Two independent observers unrelated to the retinoblastoma team (Authors RP and AJ) collected pre-SOP implementation data (RP) and post-implementation data (AJ). In order to minimize subjectivity between observers, each observer underwent a two-week data collection training period before phase 1 and phase 3 using standardized instructions. Additionally, in phase 3 data collected by both observers on four sample EUAs was compared to ensure consistency. The observers were instructed to avoid communication and interactions with the team. Data collection began at the start of the day with the OR huddle and continued until the final patient had left the OR. Case-specific data collection started when the patient entered the room and finished when the patient left the room. All data collected was verified by a member of the retinoblastoma team. The de-identified data was stored in a Microsoft Excel sheet.

Figure 1: CONSORT diagram for the data collection process. Not all variables were applicable to each patient. Do you mean “POST- implementation”????/

Outcomes

The primary outcome measures included deviations from the EUA SOP, case times, and the frequency of disruptions. A deviation from the EUA SOP was defined as missing an essential step outlined in the SOP. Case-times included total case time, total EUA time, and port access time. We measured three types of disruptions including equipment, miscommunication, and procedure-related. The definitions used for the disruptions and case times are listed in supplementary table 1.

Additional outcome measures related to SOP implementation included sample characteristics, case start time accuracy, case duration accuracy, number of imaging techniques performed, number of post-EUA procedures performed, number of individuals in the OR, presence of staff ophthalmologist and clinical fellow. These variables are defined in Table 1. Clinical characteristics were obtained from review of electronic medical records and the eCancerCare retinoblastoma (eCC) database.

Data Analysis

Fisher’s exact test, Pearson’s chi-square test, and the independent sample’s T-test were used to evaluate differences between pre- and post-SOP implementation data.

Spearman’s Rho, Mann-Whitney U test and the independent samples T-test were used to identify significant associations between measured variables and the primary outcomes. Metrics found to significantly correlate with the primary outcomes were controlled when analyzing SOP impact on primary outcomes. To determine the effect of SOP implementation on the frequency of disruptions, a logistic regression model was used. ANCOVA and the independent samples T-test were used to evaluate changes in case times due to SOP implementation. The Mann-Whitney U test was used to evaluate changes in mean adherence to essential EUA steps after SOP implementation. IBM SPSS 25.0 statistical package was used for all data analysis. A P value of less than 0.05 was considered statistically significant.

RESULTS

The data collected for each variable is available in supplementary file 1.

Patient cohorts

A total of 45 EUAs were performed, involving 33 patients (Table 1). With respect to clinical characteristics, there was no statistically significant difference between groups (Table 1).

Table 1: Comparison of the pre- and post-implementation samples on certain clinical variables.

Pre-SOP implementation Post-SOP implementation P-value

Number of EUAs performed 22 23

Unique Patients 16/22 17/23

Staging EUAs 2/22 0/23 0.23

Bilateral cases 18/22 21/23 0.41

Days since last EUA 37 42 0.33

Treatment given in last 3 EUAs

20/22 19/23 0.71

Treatment given in current EUA

22/22 16/23 0.18

SOP Adherence

The compliance to 13 out of 14 essential EUA steps increased after SOP implementation. The greatest increase (14.3% to 82.6%, X2(1, n = 44) = 20.50,

P<0.0001) was in providing the anesthesiologist a 10-minute notice to case end. Steps performed consistently before SOP implementation had statistically significant increases after SOP implementation: updating the paper work-sheet tracking treatments (81.8% to 100%, X2(1, n=45) =4.59, P=0.032), and updating the patient’s eCC (77.3% to 100%, X2(1, n=45) =5.88, P=0.015). To calculate the total adherence to key EUA SOP steps, the sum of key steps successfully completed were divided by the sum of all key steps for each patient. A Mann-Whitney U test revealed a statistically significant difference in the adherence to key EUA steps before (Md = 75.0%, n = 22), and after (Md = 92.0%, n = 23), U=56.00, z=-4.52, P< 0.0001, r=-0.67 SOP-implementation.

Table 2: Frequency of completion of essential EUA steps before and after SOP-implementation, per case.

Variable Pre-SOP Implementation Frequency

Post-SOP Implementation Frequency

Improvement/Neutral/Worsen

P-value

EUA equipment

19/22 23/23 Improve 0.07

Port access equipment

6/6 11/11 Neutral -

Ophthalmologist in huddle

19/19 23/23 Neutral -

Review eCC before consent

3/18 10/23 Improve *0.067

Review eCC before EUA

5/17 16/22 Improve *0.007

Review RetCam before EUA

14/18 22/22 Improve *0.02

Head towel 21/22 23/23 Improve 0.301

Time out 19/22 20/23 Neutral 0.953

Notice to anesthesiologist

3/21 19/23 Improve *0.0001

Update horizontal sheet

18/22 23/23 Improve *0.032

Update eCC 17/22 23/23 Improve *0.015

Dictate OR note

21/22 22/23 Improve 0.97

Update fundus diagram

21/22 23/23 Improve 0.301

* Indicates statistically significant values (P<0.05)

Frequency of Disruptions

Of the 45 EUAs, 21 (46.7%) had at least one disruption. The majority of these cases (95.0%) experienced a single disruption. No cases had more than one disruption in the three disruption categories: equipment, miscommunication or procedure related. The most frequent disruptions (Table 3) were equipment-related (57.0% of all disruptions), followed by miscommunication-related disruptions (43.0%). Procedural disruptions (staff unaware of procedure or perform procedure incorrectly) did not occur in any of the 45 EUAs. The cumulative frequency of disruptions significantly decreased after SOP implementation (63.60% to 30.43%, X2(1, n=45) =4.98, P=0.026) yet, individually the frequency of equipment related disruptions (36.4% to 17.4% X2 (1, n=45) =2.07, P=0.15) and miscommunication (27.3% to 17.4% X2 (1, n=45) =0.64, P=0.42) related discussions did not significantly change after implementation.

There was a negative correlation between the number of individuals in the OR and the frequency of disruptions, r=-0.34, n=45, P=0.021. Direct logistic regression was performed to evaluate the influence of SOP implementation and number of individuals in the OR on the likelihood of a disruption occurring. The model contained two independent variables (number of individuals in the OR and SOP implementation). The model with all predictors was statistically significant, X2(2, n =45) = 11.03, P=0.004. The model explained 21.7% (Cox & Snell R square) and 29.0% (Nagelkerke R squared) of the variance in frequency of disruptions, and correctly classified 60.0% of cases. As seen in Table 3, both independent variables made a unique statistically significant contribution to the model. The strongest predictor of a disruption occurring was the number of individuals in the OR, with an odds ratio of 0.61. This model indicates when controlling for the number of people in the OR, the implementation of an SOP was also a predictor of a disruption occurring, with an odds ratio of 0.20.

Table 3: Frequency of intraoperative disruptions before and after SOP implementation per case.

Variable Pre-SOP Implementation

Post-SOP Implementation

Statistical Analysis

Frequency Frequency Statistical Test

P-value

Disruption (one or more)

14/22 7/23 Direct Logistic Regression

*0.004

Equipment Disruption

8/22 4/23 Pearson’s Chi-Square

0.15

Miscommunication Disruption

6/22 4/23 Pearson’s Chi-square

0.64

Procedural Disruption

0/22 0/23 - -

* Indicates statistically significant values (P<0.05)

Table 4: Results from direct logistic regression assessing the effect of the number of individuals present in the OR at the start of the EUA and the implementation of the SOP on frequency of disruptions.

Variable B S.E. Wald Degree of Freedoms

P-Value

Odds Ratio

95.0% C.I. for Odds Ratio

Lower Upper

Number of individuals in the OR

-0.50 0.23 4.76 1 *0.029 0.61 0.39 0.95

Group (pre- or post-SOP implementation)

-1.63 0.71 5.68 1 *0.022 0.20 0.05 0.79

* Indicates statistically significant values (P<0.05)

Case times

Case time data collected from patients that underwent enucleations or non-routine EUAs were excluded.

There was a strong positive correlation between the number of imaging procedures performed in the EUA and the time required to perform an EUA, r=0.52, n=38, P=0.001. Total case time positively correlated with the number of imaging procedures performed in the EUA (r=0.43, n=40, P=0.006), and the number of post-EUA procedures performed (r=0.40, n=40, P=0.01). The time required to perform an EUA decreased from 21:24 ± 11:32 to 19:22 ± 09:16 after SOP implementation. An ANCOVA [between-subject factors: total EUA time; covariate: number of imaging procedures] revealed no main effects of SOP implementation, F(1, 35)=0.31, P=0.58, ηp

2=0.009. The total case time also decreased from 56:32 ± 0:35:17 to 50:19 ± 13:12 after SOP implementation. An ANCOVA [between-subject factors: total case time; covariate: number of imaging procedures and number of post-EUA procedures] revealed no main effects of SOP implementation, F(1, 37 )=1.53, P=0.22, ηp

2=0.04.

The time required to access the patient’s port (0:02:41 ± 0:0035 to 02:12 ± 00:54, t(14)=1.15, P=0.27, two-tailed) decreased after SOP implementation.

After SOP implementation, there was an increase in case duration accuracy (50.0% to 87.0%, X2(1, n = 45) = 4.59, P=0.032). Additionally, case start accuracy was positively associated with the presence of the staff ophthalmologist, r =-0.37, n = 45, P = 0.011. A

direct logistic regression was performed to ascertain the effect of the presence of the staff ophthalmologist and SOP implementation on case start accuracy. The logistic regression model was statistically significant X2(2, n =45) = 11.72, P=0.003. The model correctly classified 75.6% of the cases and explained 32.3% (Nagelkerke R2) of the variance in case start time accuracy. When accounting for the presence of the staff ophthalmologist, implementation of the SOP is significantly associated with an increase in case start time accuracy (P = 0.023).

Survey

Surveys to assess attitudes and perceptions towards SOP use before implementation were distributed to 11 retinoblastoma team members (Table 4). 45.0% of staff did not routinely use SOPs, and believed they performed well without them. The majority of staff were unaware of where the SOPs were located (64.0%). However, most believed SOP use would have a beneficial impact on patient care (72.0%), clinical practice/administrative roles (72.0%), and team functioning (64.0%).

Table 5:4 Results of Questionnaire Designed to Measure Retinoblastoma Staff's Opinions and Attitudes towards SOPs

Pre-intervention (%)

Question Yes/Positive

No/Negative Somewhat/Neutral

Reasons I routinely use retinoblastoma SOPs Include:

I do not routinely use applicable retinoblastoma SOPs 45 55

Improves patient care 45 55

Improves team functioning 54 46

Improves personal confidence in clinical practice 36 64

Comply with administrative orders 27 73

Other 18 82

Reasons I do not routinely use retinoblastoma SOPs include:

I routinely use applicable retinoblastoma SOPs 0 100

I did not know there were retinoblastoma SOPs 27 73

I do not have time to use retinoblastoma SOPs 9 91

I am unsure where to find current retinoblastoma SOPs 36 64

I am performing well without retinoblastoma SOPs 45 55

Other 45 55

Your opinion of the retinoblastoma team’s receptiveness to SOP implementation 45 18

Impact of the retinoblastoma SOPs on patient care 72 0 18

Impact of the retinoblastoma SOPS on team functioning 64 0 36

Impact of the retinoblastoma SOPs on your clinical practice and/or administrative roles

72 0 18

In your opinion, how do patients perceive the use of the retinoblastoma SOPs? 36 0 64

DISCUSSIONIn this study, we investigated the impact of SOP implementation in the retinoblastoma pathway of care and determined staff’s perception and attitudes towards SOP use. Retinoblastoma staff led the development and implementation of the EUA SOP. Our results indicate SOP implementation is associated with increased adherence to essential EUA steps, and negatively associated with the frequency of disruptions per case. Additionally, our survey results indicate that retinoblastoma staff are receptive of SOP use.

SOP implementation is associated with improved adherence to EUA steps.One explanation for the improvement in adherence to essential EUA steps following SOP implementation is A potential explanation could be that a physical checklist in the OR serves as a reminder for staff to address or contemplate all EUA steps. The greatest improvement was in providing a 10-minute notice to the anesthesiologist. The retinoblastoma team identified this as an important step as it promotes improved communication with the anesthesiologist and may minimize the time between case end and patient regaining consciousness. Steps performed most consistently occurred at the end of the case. This could be due to downtime as the case ends, facilitating easier recall of the steps required to be completed. Steps most frequently missed occurred during the EUA.between EUA start and EUA end. Therefore, introducing a checklist with the steps written facilitated improved compliance to these activities. Although overall adherence to EUA steps improved significantly from 75.00% to 92.00% post-implementation, we did not reach 100.00% adherence. A possible explanation could be that steps with the greatest increase in adherence provided more tangible benefits to the retinoblastoma staff. For instance, the benefits of shorter case times from providing the anesthesiologist a 10-minute notice are easily noticeable compared to ensuring an appropriate head towel is always prepared. Standardization is associated with increased compliance to procedure steps9 10 23. This is particularly important in a setting involving rotating team members, including fellows, residents, and OR nurses. An SOP enables seamless integration of new team members.

Disruptions are a common source of inefficiencies in the OR24-26. A primary benefit of standardization is reducing incorrect performances of a procedure as the checklist identifies essential steps. In our study, we detected zero procedural disruptions, which suggests that the performance of the surgical procedures is not a source of inefficiency. However, equipment and miscommunication disruptions were common. The frequency of equipment disruptions may be attributed to the many imaging modalities utilized and imaging specialists required. For instance, use of RetCam and OCT imaging were common and often performed by imaging specialists. Furthermore, it may be difficult to predict which imaging techniques may be required, especially in the context of new findings warranting investigation. Introduction of the EUA checklist is associated with a statistically significant decrease in the frequency of disruptions. The checklist may remind staff to prepare sufficient quantities of EUA equipment, and to review previous RetCam images and medical records before the patient enters.

We noticed a decrease in the number of miscommunication disruptions following SOP implementation, this result is commonly observed in studies involving the introduction of standardization.11 13 15 18 19. This may have resulted from the inclusion of a checklist item ensuring completion of a surgical time-out. Although adherence to the time-out did not significantly change before and after SOP implementation, after training and use of the SOP, there was an increase in adherence to other steps. Perhaps attention and focus during these time-outs improved after SOP implementation. This is beneficial as an intraoperative pause has been identified to improve communication within teams by halting extraneous conversations and providing all members an opportunity to discuss issues or problems27. Another factor is that our checklist was not designed to be completed by only one person. The entire team in the OR shared responsibility for completing the checklist, which promoted better communication within the team. Miscommunication has been identified as a problem in many surgical cases, extending case times, promoting errors and reducing teamwork26 28-30, but the introduction of a checklist with shared responsibility can reduce the frequency of these disruptions.

Introduction of the SOP had no significant effect on case time. This is likely due to individual variation between the cases. The checklist reminds the staff on certain procedural steps, but does not specify how to perform each individual aspect of the EUA, such as RetCam imaging. SOPs on imaging techniques and retinoblastoma treatments have also been developed by the retinoblastoma team, and future studies may examine whether the introduction of these SOPs decreases EUA and case times. The checklist provides a reminder to ensure equipment for port-access is available, but does not direct staff through the procedures. Our results indicate no difference in the availability of port-access equipment before and after SOP implementation, therefore the checklist is unlikely to affect port-access time. However, the checklist facilitated an improvement in providing the anesthesiologist a 10-minute notice to case end, increases in case start time and case duration accuracy, and decreases in disruptions which have been identified to reduce the time required for procedures31. Our limited sample size and the variability of procedures and imaging techniques used in retinoblastoma cases may have prevented us from observing similar results.

Despite the statistically insignificant results, they demonstrate that checklist introduction does not prolong any aspects of surgery. Fear of extending case times has been identified as a barrier to staff adopting checklist use32 33.

Checklist introduction facilitates a significant decrease in the patient preparation time. Patient preparation times are often extended due to insufficient dilation of the pupils, but the checklist includes sufficient pupillary dilation before the patient enters the room. If dilation is insufficient, additional drops are given, decreasing the patient preparation time. Additionally, when a parent or guardian accompanies the patient to the OR, the OR can become crowded and uncomfortable for the child, extending the time required for the patient to be anesthetized. Our checklist recommends in these scenarios that staff who are not immediately helping the patient to momentarily leave the OR and prevent crowdedness.

Refusal to adopt checklist use by staff has been identified as a barrier to implementing checklists33. The favorable reception staff had towards SOP use as indicated by our results might have helped us mitigate this problem. We also provided a training period for staff to utilize the checklist and request any modifications. Additionally, a team leader was assigned to ensure all members understood the importance of the checklist. These efforts ensured high rates of checklist compliance.

To our knowledge, this is the first study to evaluate SOP implementation in a pediatric ophthalmology setting. This study assessed several aspects of SOP implementation, including adherence, disruptions, and case-times. Moreover, as the intervention was staff-led and appropriate steps were taken to increase compliance, we did not experience difficulty in integrating the checklist into the OR.

The exploratory nature of the study is a limitation. Due to the numerous variables collected and the multiple comparisons made, the results are at risk for a type I error. Other limitations of the study include data collection through observation which prevents us from determining causal relationships between SOP implementation and our outcomes. Additionally, the data is also subject to observer bias. Staff acknowledgement of the presence of the observers may have affected their performance34. Similarly, the survey results collected did not come from all members of the retinoblastoma team, and are subject to responder bias. The staff that did respond to the survey may have a more favorable appreciation for SOP use than staff that did not respond. Moreover, the generalizability of this study is limited as the EUA SOP and accompanying checklist are specifically tailored to the retinoblastoma team at SickKids. However, we have prepared a SOP that provides instructions on how to implement a SOP that can be generalized to other institutions. Another limitation is the use of two separate observers to collect pre-and post-SOP implementation data. However, we attempted to minimize discrepancies in the data collection method between the two observers through a training period, development of a standardized approach to collect data, and independent review of the data. Lastly, we were limited by our sample size, which decreased the power of our study and could have prevented us from identifying statistically significant results with respect to case-times.

In conclusion, our study demonstrates that the standardization of procedures through SOP implementation can improve facets of surgical care such as lowering the frequency of disruptions, related case times and promoting adherence to essential procedural steps. These improvements help maintain the desired quality of care while decreasing inefficiencies and waste. Future studies that establish causal relationships between SOP implementation and beneficial outcomes are required. Currently, we are developing a meta-SOP for the management and implementation of other SOPs. This will be used in the future to implement SOPsIn the future, SOPs for other aspects of retinoblastoma care, such as administration of a new patient will be created and implemented. TheseAdditionally, these SOPs will be translated into e-learning modules. These modules will have video explanations for procedural steps and include quizzes to test retinoblastoma staff’s comprehension of procedures. We believe this will make

SOPs more accessible and promote their use. Additionally, this will allow us to track valuable metrics on how staff engage with SOPs. WeIn the future, we also plan to share the SOPs with other retinoblastoma treatment centers.

SUPPLEMENTARY MATERIAL

EUA supplementary file 1 – this is an excel file that has all of the raw data collected. Available on docshare.

Supplementary Figure 1: EUA Checklist Introduced into retinoblastoma OR

Supplementary Table 1: Definitions of variables

Variable Definition Note

Sample Characteristics

Staging EUA Patient’s first EUA to stage retinoblastoma

Bilateral If patient has bilateral retinoblastoma

Time since last EUA Days elapsed between current EUA and patient’s last EUA

Not applicable if patient does not have prior EUA

Treatment in last 3 EUAs Not including current EUA, did the patient receive treatment in last three EUAs?

Not applicable if patient does not have prior EUA

Treatment in current EUA Was the patient given retinoblastoma specific treatment in the current EUA?

Essential EUA Steps

EUA equipment Performed correctly if all equipment required for the EUA is present at case start

Port access equipment Performed correctly if all equipment required for the port access is present at case start

Ophthalmologist in huddle Performed correctly if at least one ophthalmologist partakes in morning huddle

Review eCC before consent

Performed correctly if at least one ophthalmologist checks patient’s eCC before consent

Not applicable if patient does not have an eCC profile created, or eCC profile is empty

Review eCC before EUA Performed correctly if at least one ophthalmologist checks patient’s eCC before EUA

Not applicable if patient does not have an eCC profile created, or eCC profile is empty

Not required if ophthalmologist checks patient’s eCC before consent

Review RetCam before EUA

Performed correctly if at least one ophthalmologist reviews patient’s previous RetCam images

Not applicable if patient does not have prior RetCam images

Head towel Performed correctly if the appropriate head towel is prepared for each patient

Not applicable if ophthalmologist do not require head towel

Time out Performed correctly if OR nurse calls for a time out

Notice to anesthesiologist Performed correctly if any member of the provides notice to anesthesiologist of case ending soon

Update horizontal sheet Performed correctly if updated at any point during surgery

Not applicable if horizontal sheet not prepared for patient

Update eCC Performed correctly if updated at any point during surgery

Not applicable if eCC profile not created for patient

Dictate OR note Performed correctly if dictated at any point during surgery

Update fundus diagram Performed correctly if updated at any point during surgery

Not applicable if fundus diagram not prepared for patient

Total adherence Average of the overall adherence to EUA steps for each case

Disruptions

Equipment Equipment is not available, malfunctioning, the staff required to operate the equipment is missing, or staff unaware of how to operate equipment

Procedure Staff unaware of procedure or perform procedure incorrectly

Miscommunication Commands going unheard, or incorrect interpretations of commands

Case Time

Total case time Time elapsed between patient entering OR and patient leaving OR

Total EUA time Time elapsed between start of EUA and end of EUA

Port access time Time taken for ophthalmologist to access port

Not applicable if patient does not require port access

Other Prominent Variables

Case start accuracy A case start time is accurate if the case started as scheduled with a 5-minute leeway

Case duration accuracy A case duration is accurate if the case did not extend for more than allotted time

Number of individuals in the OR

Number of individuals in the OR at the start of EUA

Number of imaging techniques performed

Number of imaging techniques performed as part of the EUA

Number of post-EUA procedures performed

Number of procedures performed as part of treatment after EUA

Not applicable if only EUA performed as part of surgery

Presence of lead ophthalmologist

If the lead ophthalmologist is present during EUA start

Presence of staff ophthalmologist

If the staff ophthalmologist is present during EUA start

REFERENCES

1. Dimaras H, Corson TW, Cobrinik D, et al. Retinoblastoma. Nature reviews Disease primers 2015;1:15021. doi: 10.1038/nrdp.2015.21 [published Online First: 2015/01/01]

2. Dimaras H, Kimani K, Dimba EAO, et al. Retinoblastoma. The Lancet 2012;379(9824):1436-46. doi: 10.1016/s0140-6736(11)61137-9

3. Valenzuela A, Chan HSL, Heon E, et al. Pediatric Retina: Springer, Berlin, Heidelberg 2011.

4. Dimaras H, Dimba EA, Gallie BL. Challenging the global retinoblastoma survival disparity through a collaborative research effort. Br J Ophthalmol 2010;94(11):1415-6. doi: 10.1136/bjo.2009.174136 [published Online First: 2010/08/04]

5. Canadian Retinoblastoma S. National Retinoblastoma Strategy Canadian Guidelines for Care: Strategie therapeutique du retinoblastome guide clinique canadien. Can J Ophthalmol 2009;44 Suppl 2:S1-88. doi: 10.3129/i09-194

6. Elzomor H, Taha H, Nour R, et al. A multidisciplinary approach to improving the care and outcomes of patients with retinoblastoma at a pediatric cancer hospital in Egypt. Ophthalmic Genet 2017;38(4):345-51. doi: 10.1080/13816810.2016.1227995 [published Online First: 2017/01/14]

7. Guidance for Preparing Standard Operating Procedures (SOPs). In: Agency USEP, ed.: U.S. Environmental Protection Agency, 2007.

8. Rao TS, Radhakrishnan R, Andrade C. Standard operating procedures for clinical practice. Indian J Psychiatry 2011;53(1):1-3. doi: 10.4103/0019-5545.75542

9. Chen C, Kan T, Li S, et al. Use and implementation of standard operating procedures and checklists in prehospital emergency medicine: a literature review. Am J Emerg Med 2016;34(12):2432-39. doi: 10.1016/j.ajem.2016.09.057 [published Online First: 2016/10/16]

10. Byrnes MC, Schuerer DJ, Schallom ME, et al. Implementation of a mandatory checklist of protocols and objectives improves compliance with a wide range of evidence-based intensive care unit practices. Crit Care Med 2009;37(10):2775-81. doi: 10.1097/CCM.0b013e3181a96379 [published Online First: 2009/07/08]

11. Cabral RA, Eggenberger T, Keller K, et al. Use of a Surgical Safety Checklist to Improve Team Communication. AORN J 2016;104(3):206-16. doi: 10.1016/j.aorn.2016.06.019

12. El Boghdady M, Tang B, Alijani A. The effect of a performance-based intra-procedural checklist on a simulated emergency laparoscopic task in novice surgeons. Surgical endoscopy 2017;31(5):2242-46. doi: 10.1007/s00464-016-5223-0 [published Online First: 2016/09/03]

13. Kearns RJ, Uppal V, Bonner J, et al. The introduction of a surgical safety checklist in a tertiary referral obstetric centre. BMJ Qual Saf 2011;20(9):818-22. doi: 10.1136/bmjqs.2010.050179 [published Online First: 2011/06/23]

14. Morgan L, New S, Robertson E, et al. Effectiveness of facilitated introduction of a standard operating procedure into routine processes in the operating theatre: a controlled interrupted time series. BMJ Qual Saf 2015;24(2):120-7. doi: 10.1136/bmjqs-2014-003158

15. Pugel AE, Simianu VV, Flum DR, et al. Use of the surgical safety checklist to improve communication and reduce complications. J Infect Public Health 2015;8(3):219-25. doi: 10.1016/j.jiph.2015.01.001

16. Salzwedel C, Bartz HJ, Kuhnelt I, et al. The effect of a checklist on the quality of post-anaesthesia patient handover: a randomized controlled trial. International journal for quality in health care : journal of the International Society for Quality in Health Care 2013;25(2):176-81. doi: 10.1093/intqhc/mzt009 [published Online First: 2013/01/31]

17. Sherren B, Tricklebank S, Glover G. Development of a standard operating procedure and checklist for rapid sequence induction in the critically ill. Scand J Trauma Resusc Emerg Med 2014;41(22)

18. Takala RS, Pauniaho SL, Kotkansalo A, et al. A pilot study of the implementation of WHO surgical checklist in Finland: improvements in activities and communication. Acta Anaesthesiol Scand 2011;55(10):1206-14. doi: 10.1111/j.1399-6576.2011.02525.x

19. Treadwell JR, Lucas S, Tsou AY. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual Saf 2014;23(4):299-318. doi: 10.1136/bmjqs-2012-001797

20. Barbé B, Verdonck K, Mukendi D, et al. The Art of Writing and Implementing Standard Operating Procedures (SOPs) for Laboratories in Low-Resource Settings: Review of Guidelines and Best Practices. PLOS Neglected Tropical Diseases 2016;10(11):e0005053. doi: 10.1371/journal.pntd.0005053

21. Best R, Harris H, Walsh L, et al. Pediatric Drowning A Standard Operating Procedure to Aid the Prehospital Management of Pediatric Cardiac Arrest Resulting from Submersion. Pediatr Emerg Care 2017;0(0)

22. Conley DM, Singer SJ, Edmondson L, et al. Effective surgical safety checklist implementation. J Am Coll Surg 2011;212(5):873-9. doi: 10.1016/j.jamcollsurg.2011.01.052

23. Nachtigall I, Deja M, Tafelski S, et al. Adherence to Standard Operating Procedures is Crucial for Intensive Care Unit Survivial of Elderly Patients. J Int Med Res 2008;36:438-59.

24. Parker SE, Laviana AA, Wadhera RK, et al. Development and evaluation of an observational tool for assessing surgical flow disruptions and their impact on surgical performance. World J Surg 2010;34(2):353-61. doi: 10.1007/s00268-009-0312-z [published Online First: 2009/12/17]

25. Sevdalis N, Forrest D, Undre S, et al. Annoyances, disruptions, and interruptions in surgery: the Disruptions in Surgery Index (DiSI). World J

Surg 2008;32(8):1643-50. doi: 10.1007/s00268-008-9624-7 [published Online First: 2008/05/21]

26. Wiegmann DA, ElBardissi AW, Dearani JA, et al. Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery 2007;142(5):658-65. doi: 10.1016/j.surg.2007.07.034 [published Online First: 2007/11/06]

27. Erestam S, Angenete E, Derwinger K. The Surgical Teams' Perception of the Effects of a Routine Intraoperative Pause. World J Surg 2016;40(12):2875-80. doi: 10.1007/s00268-016-3632-9

28. Gillespie BM, Chaboyer W, Fairweather N. Interruptions and miscommunications in surgery: an observational study. Aorn j 2012;95(5):576-90. doi: 10.1016/j.aorn.2012.02.012 [published Online First: 2012/05/01]

29. Sevdalis N, Healey AN, Vincent CA. Distracting communications in the operating theatre. Journal of evaluation in clinical practice 2007;13(3):390-4. doi: 10.1111/j.1365-2753.2006.00712.x [published Online First: 2007/05/24]

30. Yoong W, Khin A, Ramlal N, et al. Interruptions and distractions in the gynaecological operating theatre: irritating or dangerous? Ergonomics 2015;58(8):1314-9. doi: 10.1080/00140139.2015.1005171 [published Online First: 2015/02/13]

31. Ahmed A, Ahmad M, Stewart CM, et al. Effect of distractions on operative performance and ability to multitask--a case for deliberate practice. The Laryngoscope 2015;125(4):837-41. doi: 10.1002/lary.24856 [published Online First: 2014/07/31]

32. Bergs J, Lambrechts F, Simons P, et al. Barriers and facilitators related to the implementation of surgical safety checklists: a systematic review of the qualitative evidence. BMJ Qual Saf 2015;24(12):776-86. doi: 10.1136/bmjqs-2015-004021

33. Fourcade A, Blache JL, Grenier C, et al. Barriers to staff adoption of a surgical safety checklist. BMJ Qual Saf 2012;21(3):191-7. doi: 10.1136/bmjqs-2011-000094

34. McCarney R, Warner J, Iliffe S, et al. The Hawthorne Effect: a randomised, controlled trial. BMC Med Res Methodol 2007;7:30. doi: 10.1186/1471-2288-7-30