Cumulative Sum (CUSUM) charts for medical student peripheral venous cannulation;
development of a difficulty-adjusted CUSUM
Dr Harry MurgatroydSpR Anaesthesia
Leeds Teaching Hospitals Trust
Sumaiyah KolaMedical Student
Leeds University Medical School
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Runcie CJ. Assessing the performance of a consultant anaesthetist by control chart methodology. Anaesthesia. 2009; 64(3): 293-296
CUSUM Chart
Graphical presentation over timeUsed to determine
competency
Monitors performance
Learning Curves
Developed initially to look at industrial processes
Audit of quality clinical practice
Theory
QuickTime™ and aTIFF (Uncompressed) decompressor
are needed to see this picture.
Set:definition of success / failureacceptable failure rateunacceptable failure rateerror
Collect:binary data
Algorithm:Score falls with successScore increases with failureGraphBoundary Lines
Failure seen as a rise in the graph
Success is seen as a fall in the graph
Each point represents a Each point represents a single cannulation attemptsingle cannulation attempt
Sequential cannulation attempts
Score derived using the CUSUM formula
Problems
•Patient variability
•Standard CUSUM
•Constant failure and success rates
•Risk adjusted CUSUM
•Complicated
•Not intuitive
•Failure rates•Set by user
•Can affect results considerably
• Aims– Proficiency at
intravenous cannulation
– Plot individual CUSUM charts
– Develop a ‘difficulty-adjusted’ CUSUM technique
• Time Scale: – 5 weeks
Medical student project
How to insert an intravenous cannula 101
Methodology
• Setting– Teaching hospital
– Elective surgical lists
• Procedures– Verbal consent
– Peripheral venous cannulation • Standard technique• Size of cannula appropriate to surgical
procedure
– Data collection• Success or failure• Appearance of vein• Size of cannula• Patient awake or anaesthetised
Conventional CUSUM
• Definition of ‘success and failure’
• Acceptable and unacceptable failure rates– Consultant consensus– Literature– 0.2 and 0.4 respectively
• Calculation– Published formulae– Error rates = 0.1– MS Excel
Williams et al. BMJ 1992;304:1359-61.
de Oliveira. Anesth Analg 2002;95:411-6.
Failure rates
Upper and lower boundries
0= failure,1= success
Data is plotted sequentially
Example if “IF” formula in Excel
Running total, CUSUM
Conventional CUSUM
Difficulty Adjusted CUSUM
• Difficulty score– Appearance of vein– Cannula size– Awake or anaesthetised
• Different failure rates– Two stages
• Vein adjusted• All three variables
– Intervention line• Average of all prior lines
Difficulty Adjusted CUSUMVein adjusted
Vein appearance
Acceptable failure rate
Unacceptable failure rate
Visible, palpable
0.15 0.3
Just visible 0.3 0.6
Table shows the scoring of each of the recorded variables. These are then added up to give the total score for the cannulation attempt
Shows the standard CUSUM formula, whilst incorporating different failure rates and scores dependent on the difficulty of the variables recorded.
Spreadsheet showing the final added up scores of the different variables. Hence including the vein, consciousness and cannula size. Using “IF” formulas the correct value of S is selected from the table above and the CUSUM then plotted in the same way as before.
Three variable methodology
Difficulty Adjusted CUSUM
• Successful difficult cannulation– Large fall in score
• Failed difficult cannulation– Small rise in score
• Successful easy cannulation– Small fall in score
• Failed easy cannulation– Large rise in score
Conventional CUSUM vs Vein DA-CUSUMAn example CUSUM and difficulty-adjusted CUSUM chart for student A
-5
-4
-3
-2
-1
0
1
2
3
0 10 20 30 40 50 60
Patient number
CU
SU
M s
core
Conventional CUSUM
DA-CUSUM
DA-CUSUM intervention line
Conventional CUSUM intervention line
Conventional CUSUM vs Three Variable DA-CUSUM
-5
-4
-3
-2
-1
0
1
2
3
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55
Standard
Including Difficulty
Three variable DA-CUSUM for two students
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
2.5
1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 51 53 55
Series
CU
SU
M Sumi
Dave
Monday Morning
POSITIVES of CUSUM
•Objective
•Simple Calculations
•Shows improvement in learners
•Early detection of poor performance
•Allows comparisons between students
LIMITATIONS of CUSUM
•Only technical skills
•Must have binary outcome
•Relies on logbooks and honesty of user
•Time consuming
•Open to manipulation
•Does not show improvements that do not change binary outcomeBolson S, Colon M. Int J Health Care Qual Assur
2000;12:433-438.
Kestin IG. BJA 1995;75:805-809.
Difficulty adjusted CUSUM
• POSITIVES
• Potentially corrects for patient variability
• Easier and more intuitive than other methods of adjustment
• LIMITATIONS
• Failure rates set by the user
• The more variables ‘corrected’ for, the more layers of estimation and inaccuracy
• Loss of statistical element of conventional CUSUM
Summary
• Easy technique– Handheld devices– Electronic logbooks
• Objective• Can be adjusted for patient
variability• Allows
– Charting of ‘learning curve’ – Comparison between practitioners– Identification of poor performance
?