Using Stata Graphs to Visually Monitor the
Progress of Multi-centre Randomized Clinical
Trials
Alexandra Whate & Glenn Jones
General Background• Clinical trials are increasingly multi-centre
and global• Routine meetings about trial status become
logistically difficult and expensive to hold• Investigators have different levels of training
and motivation in carrying out clinical trials• Monthly performance monitoring reports have
traditionally been text and table based -adaptations need to be made
Our Context• In 6 trials Investigators from over 30 countries
collect data and submit them to the DMC• Monthly Performance Monitoring Reports are
created by the DMC and submitted to Investigators and Technical Officers (TO)• Used to evaluate investigator performance • Used to monitor study progress
– Accrual, follow-up, survival, adverse events– Provide investigators with the “big picture” to
encourage continued accrual, adherence to protocol and patient follow-up
– Detection of emerging issues or bias
Principles to Optimize Monthly Reports:
• Visual – to overcome language barriers• Relatively Simple – to match the level of
training of Investigators• Discriminating – To easily identify those that
are adhering to protocol and those who are not
• Efficient – Quick to read and understand• Strategic – Motivating Investigators; minimize
bias; ensure patient safety
Applications of Stata
1. Monitoring Accrual2. Monitoring Survival 3. Monitoring Clinical Trajectories4. Monitoring Follow-up
1. AccrualAccrual should be relatively rapidAll centres should participate in
adding patients to the study on a regular basis
Using Stata To Monitor Accrual• Line plots show total accrual over the time of
the study (standard)• Bar graphs show monthly accrual• Bar graphs can also be made ‘by centre’ to
show accrual for each centre involved• These can demonstrate
– Trends in accrual – Interruptions due to issues with staffing and
equipment
0
20
40
60
80
100
120
140cu
mula
tive a
ccr
ua
l
13 3 3
56
7
11
56
2
54
56 6
3
9
6
2
12
6 68
02468
101214
nu
mbe
r of p
atien
ts a
ccru
ed
Oct
200
7
No
v 20
07
De
c 20
07
Jan 2
008
Feb
200
8
Marc
h 2
00
8
Ap
ril 20
08
May 2
00
8
June
200
8
July
20
08
Au
g 2
008
Se
pt 2
00
8
Oct
200
8
No
v 20
08
De
c 20
08
Jan 2
009
Feb
200
9
Marc
h 2
00
9
Ap
ril 20
09
May 2
00
9
June
200
9
July
20
09
Au
g 2
009
Se
pt 2
00
9
Total Accrual
Graph bar var, over(month var, label(angle(ver))) blabel(total) ytitle ylabel(0(2)14) ylabel(,angle(hor)) saving(graphstata) twoway line var var, clwidth(medthick) ylabel(,angle(hor)) xlabel(,nolabel) xtitle("") saving(graphlinestata)
graph combine graphstata.gph graphlinestata.gph, col(1) title(“Total Accrual”)
34
6
12
4
12
3
1
3
1
32
1111 12
12 2
1 11
3222
4
112
12
11
3
11
5
32
1
21
33 32
3
1
32
11
34
211 11
21
0
2
4
6
8
10
0
2
4
6
8
10
Oct
2007
Nov
2007
Dec
2007
Jan 2
00
8F
eb 2
00
8M
arc
h 2
008
April 2
008
Ma
y 2008
June 2
008
July
2008
Aug 2
008
Sept 2
008
Oct
2008
Nov
2008
Dec
2008
Jan 2
00
9F
eb 2
00
9M
arc
h 2
009
April 2
009
Ma
y 2009
June 2
009
July
2009
Aug 2
009
Sept 2
009
Oct
2007
Nov
2007
Dec
2007
Jan 2
00
8F
eb 2
00
8M
arc
h 2
008
April 2
008
Ma
y 2008
June 2
008
July
2008
Aug 2
008
Sept 2
008
Oct
2008
Nov
2008
Dec
2008
Jan 2
00
9F
eb 2
00
9M
arc
h 2
009
April 2
009
Ma
y 2009
June 2
009
July
2009
Aug 2
009
Sept 2
009
Oct
2007
Nov
2007
Dec
2007
Jan 2
00
8F
eb 2
00
8M
arc
h 2
008
April 2
008
Ma
y 2008
June 2
008
July
2008
Aug 2
008
Sept 2
008
Oct
2008
Nov
2008
Dec
2008
Jan 2
00
9F
eb 2
00
9M
arc
h 2
009
April 2
009
Ma
y 2009
June 2
009
July
2009
Aug 2
009
Sept 2
009
Oct
2007
Nov
2007
Dec
2007
Jan 2
00
8F
eb 2
00
8M
arc
h 2
008
April 2
008
Ma
y 2008
June 2
008
July
2008
Aug 2
008
Sept 2
008
Oct
2008
Nov
2008
Dec
2008
Jan 2
00
9F
eb 2
00
9M
arc
h 2
009
April 2
009
Ma
y 2009
June 2
009
July
2009
Aug 2
009
Sept 2
009
Oct
2007
Nov
2007
Dec
2007
Jan 2
00
8F
eb 2
00
8M
arc
h 2
008
April 2
008
Ma
y 2008
June 2
008
July
2008
Aug 2
008
Sept 2
008
Oct
2008
Nov
2008
Dec
2008
Jan 2
00
9F
eb 2
00
9M
arc
h 2
009
April 2
009
Ma
y 2009
June 2
009
July
2009
Aug 2
009
Sept 2
009
Oct
2007
Nov
2007
Dec
2007
Jan 2
00
8F
eb 2
00
8M
arc
h 2
008
April 2
008
Ma
y 2008
June 2
008
July
2008
Aug 2
008
Sept 2
008
Oct
2008
Nov
2008
Dec
2008
Jan 2
00
9F
eb 2
00
9M
arc
h 2
009
April 2
009
Ma
y 2009
June 2
009
July
2009
Aug 2
009
Sept 2
009
Country 1 Country 2 Country 3
Country 4 Country 5 Country 6
nu
mbe
r o
f pa
tien
ts a
ccru
ed
Graphs by centre
Accrual by Centre
graph bar var, by(centre) over(month var, label(labsize(vsmall) angle(ver))) ytitle("number of patients accrued") ylabel(0(2)10) ylabel(,angle(hor)) blabel(total, gap(.25)) title(“Accrual by Centre”)
Monitoring Randomization• We must ensure that randomization
processes are working– Require by the end of accrual that there are
equal number of patients on each treatment arm of the study
– Requires that patients are being placed at an equal rate on the study arms – real time randomization
Two-Arm Trial8
64
20
24
68
nu
mbe
r o
f pa
tien
ts r
an
do
miz
ed
01o
ct2
00
7
01n
ov2
00
7
01d
ec2
00
7
01ja
n2
00
8
01fe
b2
00
8
01m
ar2
00
8
01a
pr2
00
8
01m
ay2
00
8
01ju
n2
00
8
01ju
l20
08
01a
ug
20
08
01se
p20
08
01o
ct2
00
8
01n
ov2
00
8
01d
ec2
00
8
01ja
n2
00
9
01fe
b2
00
9
01m
ar2
00
9
01a
pr2
00
9
01m
ay2
00
9
01ju
n2
00
9
01ju
l20
09
01a
ug
20
09
01se
p20
09
01o
ct2
00
9
Date of Randomization
Arm 1 Arm 2
bihist var, by(var)
Multi-Arm Trial
0
.2
.4
.6
.8
1C
um
ulat
ive
Pro
babi
lity
Jul-
05S
ep-0
5N
ov-
05Ja
n-0
6M
ar-0
6M
ay-0
6Ju
l-06
Sep
-06
No
v-06
Jan
-07
Mar
-07
May
-07
Jul-
07S
ep-0
7N
ov-
07Ja
n-0
8M
ar-0
8M
ay-0
8Ju
l-08
Sep
-08
No
v-08
Jan
-09
Mar
-09
May
-09
Jul-
09S
ep-0
9N
ov-
09
DOrand
Arm 1 Arm 2 Arm 3 Arm 4
cdfplot var, by(txarm) legend(col(4)) xlabel(,angle(ver))
2. Monitoring Survival
Ethical requirement to monitor patterns of mortality over the span of the study
Monitoring Survival• Kaplan-Meier plots allow us to estimate event rates
over time– We create a “survival” variable with date of event or
last follow-up minus the date of randomization gen SURV=date of last follow-up – date of randomization stset SURV, failure(death==1)
• Stata allows for graphical demonstration of survival estimates for different baseline groups (different disease, stage of disease etc) especially important for sample size
0.000.100.200.300.400.500.600.700.800.901.00
41 36 34 28 20 16 13 9 8 7 7 7 5 4 3 3 3 2 2 1 1 1 1 1 1 1 1 1 1 0M1:69 66 62 57 52 46 41 34 31 26 25 23 21 20 18 15 14 12 11 10 8 8 7 5 3 2 0 0 0 0M0:
Risk
0 20
40
60
80
10
012
014
016
018
020
022
024
026
028
030
032
034
036
038
040
042
044
046
048
050
052
054
056
058
0
days from randomization
Disease 1 Disease 2
Kaplan-Meier survival estimates
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
0 50
10
0
15
0
20
0
25
0
30
0
35
0
40
0
45
0
50
0
55
0
60
0
days from randomization
110 88 62 41 32 25 18 13 9 6 3 1 0 Risk
95% CI Survivor function
Kaplan-Meier survival estimate sts graph, cim censored(single) risktable(,title(Risk)) ylabel(,angle(hor)) ylabel(#10) xlabel(#20) xtitle("days from randomization")xlabel(,angle(ver))
sts graph, cim censored(single) risktable(,title(Risk)) ylabel(,angle(hor)) ylabel(#10) xlabel(#20) xtitle("days from randomization")xlabel(,angle(ver))
1
3. Clinical Trajectory
Demonstrates the extent to which protocol-required activities are properly
ordered and on timeDemonstrates the homogeneity or patterns
in clinical trajectories and identifies clear outliers (protocol violation, patient choice)
Clinical Trajectory• In Stata, graphic representations of clinical
trajectories can be created by plotting dates of interest:
1. Date of diagnosis2. Dates of imaging/scans/biopsies3. Date of surgery4. Date of randomization5. Dates of treatment6. Dates of follow ups
• Plot deceased and living patients separately
Jul-07
Oct-07
Jan-08
Apr-08
Jul-08
Oct-08
Jan-09
Apr-09
Jul-09
Oct-09
Oct
-07
No
v-07
De
c-07
Jan
-08
Fe
b-0
8
Mar-
08
Apr-
08
May-
08
Jun
-08
Jul-
08
Aug
-08
Sep
-08
Oct
-08
No
v-08
De
c-08
Jan
-09
Fe
b-0
9
Mar-
09
Apr-
09
May-
09
Jun
-09
date of randomization
Follow-up CT scan Biopsy 1st Treatment2nd Treatment 3rd Treatment Randomization Death
Clinical Trajectories of Deceased Patients
twoway scatter var var var var var var var var if deathflag==1, xti("date of randomization")yti("") xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4)) ti("Clinical Trajectories of Deceased Patients") || scatter var var if status==1,
msymbol(x) mcolor(gold) legend(lab(8 "Death")) xlabel(#20) ylabel(#10)
Jul-07Oct-07Jan-08Apr-08Jul-08Oct-08Jan-09Apr-09Jul-09Oct-09
Jul-07Oct-07Jan-08Apr-08Jul-08Oct-08Jan-09Apr-09Jul-09Oct-09
Oct
-07
No
v-0
7D
ec-
07
Jan-
08
Feb
-08
Mar-
08A
pr-0
8M
ay-
08
Jun-
08
Jul-0
8A
ug-0
8S
ep-0
8O
ct-0
8N
ov-
08
De
c-0
8Ja
n-0
9F
eb-0
9M
ar-
09A
pr-0
9M
ay-
09
Jun-
09
Oct
-07
No
v-0
7D
ec-
07
Jan-
08
Feb
-08
Mar-
08A
pr-0
8M
ay-
08
Jun-
08
Jul-0
8A
ug-0
8S
ep-0
8O
ct-0
8N
ov-
08
De
c-0
8Ja
n-0
9F
eb-0
9M
ar-
09A
pr-0
9M
ay-
09
Jun-
09
Oct
-07
No
v-0
7D
ec-
07
Jan-
08
Feb
-08
Mar-
08A
pr-0
8M
ay-
08
Jun-
08
Jul-0
8A
ug-0
8S
ep-0
8O
ct-0
8N
ov-
08
De
c-0
8Ja
n-0
9F
eb-0
9M
ar-
09A
pr-0
9M
ay-
09
Jun-
09
Country 1 Country 2 Country 3
Country 4 Country 5 Country 6
follow-up randomization CT scan biopsy
treatment 1 treatment 2 treatment 3 death
date of randomization
Graphs by centre
twoway scatter var var var var var var var if death==1 , xti("date of randomization")yti("") xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4)) || scatter var var, msymbol(x) mcolor(gold) legend(lab(8 “death")) xlabel(#20) ylabel(#10) by(centre)
Scatter Plots for Living Patients
Dec-07Feb-08Apr-08Jun-08Aug-08Oct-08Dec-08Feb-09Apr-09Jun-09Aug-09Oct-09
Jan
-08
Fe
b-0
8
Mar-
08
Apr-
08
May-
08
Jun
-08
Jul-
08
Aug
-08
Sep
-08
Oct
-08
No
v-08
De
c-08
Jan
-09
Fe
b-0
9M
ar-
09
Apr-
09
May-
09
Jun
-09
Jul-
09
Aug
-09
Sep
-09
Oct
-09
No
v-09
Date of Randomization
Follow Up Treatment1 Treatment 2 Treatment 3CT scan Biopsy Randomization
Clinical Trajectories
twoway scatter var var var var var var var, msize(small) xti("Date of Randomization") xlabel(,angle(ver)) ylabel(,angle(hor)) legend(col(4) ti("Clinical Trajectories")yline(18048)xlabe(#25)ylabel(#10)
4. Tracking Follow-UpIdentify patients that are at risk of
being lost to follow-upFollow-up is required by protocol in
specific intervals
Ensuring Regular Follow-Up• Each month we are interested in monitoring the
date of last follow-up for each patient to determine whether follow-up submission is up to date – We can use a strip plot separated by country to show
the date of last follow-up for each patient– Dot plots can be used to isolate patients based on
follow-up date (mlabel)– Goal is to inform Investigators about the specific cases
that are missing follow-up
• Allows synchronization of records
Strip Plot: Follow-Up
stripplot var, over(var) xlabel(,angle(ver)) xline(18110) separate(centre) legend(col(3))
Oct
-07
No
v-0
7
De
c-0
7
Jan
-08
Fe
b-0
8
Ma
r-0
8
Ap
r-0
8
Ma
y-0
8
Jun
-08
Jul-
08
Au
g-0
8
Se
p-0
8
Oct
-08
No
v-0
8
De
c-0
8
Jan
-09
Fe
b-0
9
Ma
r-0
9
Ap
r-0
9
Ma
y-0
9
Jun
-09
Jul-
09
Au
g-0
9
Se
p-0
9
Oct
-09
DOlfu
Country 1 Country 3 Country 5 Country 7 Country 9
Country 2 Country 4 Country 6 Country 8
Dot Plots
IN025
CR009
CR012
IN024 CR010
IN015
IN012
IN018
IN029
INO16
IN027
CH021
IN030 CH001 IC002
IN031 IN032 IN033
IN034
IN035
IN039
IN036
SA015 IN037
IC003
IN038
TL006 CH029
CH026
TL005 CH027 CH025
CH008
CH031 CR013 CR003 CR007
CH019 CH017 CH020 SA025 CH028
CH010 CH012 CH030
SA027 CH006
IN040 CR015 SA023 CH032 CR011 CH033 SA031 SA026 SA033 SA028 SA032 SA034 SA029
CH034
SA030
May-09
Jun-09
Jul-09
Aug-09
Sep-09
date
of l
ast
follo
w-u
p s
ubm
issi
on
dotplot var, ylabel(,angle(hor)) ylabel(#5) mlabel(patid) mlabsize(tiny) xscale(off) yline(18110)
Strategic Monitoring of Follow-up
• Scatter plots and strip plots give a visual clue to the proportion of patients that are out of date for follow-up
• A more sophisticated look at combined follow-up for the whole trial allows us to determine the proportion exactly– Plotting cumulative date of last follow-up with
CDF plots estimates permanent loss to follow-up
Ideal Cumulative Follow-up
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
%
Apr
-09
May
-09
Jun
-09
Jul-
09
Aug
-09
Sep
-09
date of last follow-up
Cumulative Follow-up
CDF Plot - Cumulative Follow-up
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
%
No
v-07
De
c-07
Jan
-08
Fe
b-0
8M
ar-
08
Apr-
08
May-
08
Jun
-08
Jul-
08
Aug
-08
Sep
-08
Oct
-08
No
v-08
De
c-08
Jan
-09
Fe
b-0
9M
ar-
09
Apr-
09
May-
09
Jun
-09
Jul-
09
Aug
-09
Sep
-09
Oct
-09
No
v-09
date of last follow-up
Cumulative Follow-up
Line to denote the time since
which follow-up forms should
have been received
cdfplot var, ylabel(#10) xlabe(#5) xlabel(,angle(ver)) xline(18130) xtitle(date of last follow-up) ytitle (%) legend(col(3))
Determining Contribution• CDF plots can be made by centre to show how each
centre is contributing to the global CDF• Then scatter plots can also be modified to list
specific patients that are behind on follow-up– We generate cumulative date variables cumul var, gen(newvar) and scatter these dates to create a “Scatter CDF plot” – By plotting only those patients that are overdue for
follow-up we can “zoom in” on the CDF plot to identify the specific patients that are contributing to sections 1 or 2 of the overall CDF plot.
Cumulative Follow-Up by Centre
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
%
No
v-07
De
c-07
Jan
-08
Fe
b-0
8M
ar-0
8
Apr
-08
May
-08
Jun
-08
Jul-
08
Aug
-08
Sep
-08
Oct
-08
No
v-08
De
c-08
Jan
-09
Fe
b-0
9M
ar-0
9
Apr
-09
May
-09
Jun
-09
Jul-
09
Aug
-09
Sep
-09
Oct
-09
No
v-09
date of last follow-up
Country 1 Country 3 Country 5 Country 7 Country 9
Country 2 Country 4 Countrry 6 Country 8
Cumulative Follow-up
cdfplot var, by(centre) ylabel(#10) xlabe(#5) xlabel(,angle(ver)) xline(18110) xtitle(date of last follow-up) ytitle (%) legend(col(3))
Scatter CDF
CH021
CH001
CR009CR012
CR010
IN025
IN024
IN015
IN012
IN018IN029
INO16
IN027
IN030
IN031IN032IN033
IN034
IN035IN039
IN036
IN037
IN038
SA015
TL006
.4
.45
.5
.55
.6
.65
.7
cdf d
ate
of la
st fo
llow
-up
Apr
-09
May
-09
Jun
-09
Jul-
09
Aug
-09
date of last follow-up
Country 1 Country 2 Country 3 Country 4 Country 5 Country 6
twoway (scatter cdf var if centre==1, mlabel(patid)) (scatter cdf var if centre==2, mlabel(patid)) (scatter cdf var if centre==3, mlabel(patid)) (scatter cdf var if centre==4, mlabel(patid)) (scatter cdf var if centre==5, mlabel(patid)) (scatter cdf var if centre==6, xlabel(,angle(ver)) legend(col(6)) xlabel(#5) ylabel(,angle(hor)) mlabel(patid))
Follow-Up Over Time
• We can merge serial .dta files (just key variables) to enable simultaneous plotting of each months CDF for dates of last follow-up in patients that are/were alive
• This shows the structure of follow-up across the months of the study
CDF Plot-Merged Over Time
0
.1
.2
.3
.4
.5
.6
.7
.8
.9
1
Cu
mula
tive
Pro
ba
bili
ty
May-
08
Jun
-08
Jul-
08
Aug
-08
Sep
-08
Oct
-08
No
v-08
De
c-08
Jan
-09
Fe
b-0
9
Mar-
09
Apr-
09
May-
09
Jun
-09
Jul-
09
Aug
-09
Sep
-09
Oct
-09
DOlfu
Jan 2009 Feb 2009 March 2009 April 2009
May 2009 June 2009 July 2009 Aug 2009
Sept 2009
Summary• Stata’s graphing capabilities can be used to monitor
accrual, survival and follow-up status and other aspects of protocol adherence
• The time interval between reports can be short– reports can be produced efficiently using do files
• Minimum Investigator training is required to interpret reports about the whole study and their own performance
• The trial can be visually monitored by TO’s and the DMC
Summary
• Rapid identification of problems by centre and patient can lead to more timely responses
• Different patterns of problems can be detected for which different strategies for resolution can be pursued – this can give direction to Investigators as to how to solve problems
• Using Stata to visually monitor trails helps to improve patient safety and trial quality
Questions?