prof neville yeomans director of research, austin lifesciences
DESCRIPTION
Session 6: Basic Statistics Part 1 (and how not to be frightened by the word). Prof Neville Yeomans Director of Research, Austin LifeSciences. So now we’ve got some results? How can we make sense out of them?. What I will cover (over 2 sessions, August 28 and September 25). - PowerPoint PPT PresentationTRANSCRIPT
Session 6: Basic Statistics Part 1(and how not to be frightened by the word)
Prof Neville YeomansDirector of Research, Austin LifeSciences
So now we’ve got some results? How can we
make sense out of them?
What I will cover(over 2 sessions, August 28 and September 25)
• Sampling populations• Describing the data in the samples• How accurately do those data reflect the ‘real’
population the samples were taken from?• We’ve compared two groups. Are they really from
different populations, or are they samples from the same population and the measured differences are just due to chance?
What I will cover (contd.)• Tests to answer the question ‘Are the differences
likely to be just due to chance?’– Data consisting of values (e.g. hemoglobin
concentration)(‘continuous variables’)– Data consisting of whole numbers – frequencies,
proportions (percentages)– Tests for just two groups; tests for multiple groups
• Tests that examine relationships between two or more variables (correlation, regression analysis, life-table)
What I will cover (contd.)
• How many subjects should I study to find the answer to my question? (Power calculations)
• Statistical packages and other resources
We’ve got some numbers. How are we going to describe them to others?
Suppose we’ve measured heights of a number of females (‘a sample’) picked off the street in Heidelberg
Subject #
Height (cm)
1 179
2 162
3 150
4 155
5 168
6 175
7 159
8 152
Subject #
Height (cm)
9 156
10 157
11 161
12 164
13 165
14 159
15 161
16 163
Subject #
Height (cm)
17 167
18 173
19 159
20 170
21 168
22 162
23 171
24 164
Heights of (sample of) Heidelberg women
Person number
0 2 4 6 8 10 12 14 16 18 20 22 24 26
Hei
ght (
cm)
145
150
155
160
165
170
175
180
185
How could we more concisely describe these data – using just one or two numbers that would give us useful information about the sample as a whole?
1. A measure of ‘central tendency’2. A measure of how widely the values are spread
Frequency of women with each height in sample
Height
145 150 155 160 165 170 175 180 185
Freq
uenc
y
0.5
1.0
1.5
2.0
2.5
3.0
3.5The median (middle value) = 162.5
The range (150-179)a poor measure for describing the whole population because it depends on sample size – range is likely to be wider with larger samples
Interquartile range (25th percentile to 75th
percentile of values: 159-168)
what we should always use with the median – it’s largely independent of sample size
Frequency distribution of height of Heidelberg women
Height
130 140 150 160 170 180 190 200
Freq
uenc
y
0
1
2
3
4
5
6 the Mean (average)(Ʃx/N) = 163.3 cm
the Standard Deviation*± 7.2 cm
*In Excel, enter formula ‘=STDEV(range of cells)’
- doesn’t vary much with sample size (except very small samples)
- approx. 67% of values will lie within ± 1 SD either side of mean**
- approx 95% of values will lie within ± 2 SD either side of mean**
(amalgamated into 3cm ranges: e.g.141-143, 144-146 etc.)
** Provided the population is ‘normally distributed’
The ‘Normal distribution’Mean = Median in a ‘perfect’normal distribution
Standard deviations away from mean
We measured the mean height of our sample of 25 women ... (it was 163.3 cm)
• But what is the average height of the whole population – of ALL Heidelberg women?
• We didn’t have time or resources to track them all down – that’s why we just took what we hoped was a representative sample.
• What I’m asking is: how good an estimate of the true population mean is our sample mean?
• This is where the Standard Error of the Mean* (or just Standard Error, SE) comes in.
*It’s sometimes called the Standard ESTIMATE of the Error of the mean
The Standard Error (contd.)• The mean height of our sample of 25 women was 163.3 cm• We calculated the Standard Deviation (SD) of the sample to
be 7.3 cm (that value, on either side of the mean, that should contain about 2/3 of those measured)
• Standard Error of the mean = SD/√N , i.e. 7.3/ √25 = 7.3/5 = 1.46
• So now we can express our results for the height of our sample as 163.3 ± 1.5 (Mean ± SEM) ...... But what does this really tell us?
• The actual true mean height of the whole population of women has a 67% likelihood of lying within 1.5 cm (i.e. 1 SEM) either side of the mean we found in our sample; and a 95% likelihood of lying within 3 cm (i.e. 2 x SEM) either side of that sample mean. (It’s actually 1.96xSEM for a reasonably large sample - e.g. roughly 30 – and wider for small samples, but let’s keep it simple).
The concept of the standard error of the mean (SEM) – e.g. serum sodium values
130 134 138 142 146 150 154 mmol/L
True population mean142 mmol/L (SD=4.0)
Sample mean =142.8 mmol/L
(SD of sample = 3.4)
1 x SEM = (i.e. 3.4/√10) = 1.1 mmol/L
2 x SEM (~’95% confidence interval’) = 2.2 mmol/LRandom sample of 10
normal individuals
That means: ‘There is a 95% chance (19 chances out of 20) thatthe actual population mean, estimated from our random samplelies between 140.6 and 145.0 mmol/L)’
Why does Standard Error depend on population SD and sample size?
SE = SD/√N
A: Narrow populationspread (i.e. small SD)
B: Wide populationspread (i.e. large SD)
Increasing N decreases SE of mean.i.e. increases accuracy of our estimate of the population meanbased on results of our sample
Testing significance of differencesSamples of Heidelberg men and women
Height (cm)
130 140 150 160 170 180 190 200
Freq
uenc
y
0
1
2
3
4
5
6
7
Women:Mean = 163.3SE = 1.46
Men:Mean = 176.5SE = 1.43
On a quick, rough, check we can see that:(a) the 95% confidence interval for our estimate of the height of women is160.3-166.4 cm (approximately mean ± 2SE).(b) our estimate of the mean height of the men sampled is quite a lot outsidethe 95% confidence interval (range)for the women, so it looks improbable thatthey are from the same population
95% confidence intervals
Testing significance of differences ....How likely is it that the two random samples
came from the same population?Student’s t-test
Standard deviation either side of mean
Composite frequency distribution, created by pooling data from both samples
Mean: 170.7SD: 10.0 cm
0 321-1-2-3
Women
Men
How likely is it that these two samples (the pink and the blue) were taken from the SAME population? [this is called the NULL HYPOTHESIS]
Tested for statistical significance of difference: p<0.001i.e. there is less than 1 chance in 1000 that these two samplescame from the same population
In fact, though, running a Student’st-test on the two samples of height in the Heidelberg men and women slide, gives this error message:
Assumptions to be met before testing significance of differences with PARAMETRIC TESTS* – i.e. tests that use the mathematics of the normal curve distribution• The combined data should approximate a normal
curve distribution– in this instance the male data were skewed (not
evenly distributed around the mean) and spread a bit too far out into the tails of the frequency-distribution curve
• The variances (=SD2) of the groups should not differ significantly from each other
*Student’s t-test, Paired t-test, Analysis of Variance
An example of data where groups have different variance (spread), and one group is skewed
Means
Airway obstruction - controls vs smokers
Controls Smokers
FEV1
/FVC
30
40
50
60
70
80
90
100
Means
Medians
The equal variance test failed (P<0.05), and the normality test almostfailed (P=0.08) – so we should not use a parametric test such as t-test
Lower limit of ‘normal’ range
So what do we do if we can’t use a parametric test to check for significances of differences?
• Use a non-parametric test• These tests, instead of using the actual numerical
values of the data, put the data from each group into ascending order and assign a rank number for their place in the combined groups.
• The maths of the test is then done on these ranks• Examples: Rank sum test, Wilcoxon Rank Test, Mann
Whitney rank test, etc.– (the P value for our slide of heights of Heidelberg men and
women was calculated using Wilcoxon test)
Group 1 data
Rank when groups
combinedGroup 2
data
Rank when groups
combined12 1 16 513 2.5 19 6.513 2.5 26 815 4 40 1019 6.5 78 1127 9 101 12
Sum of ranks: 25.5 46.5
Mann-Whitney test: P = 0.026
Example of how a rank-sum (non-parametric) test is constructed manually*
*In reality, these days you’ll just feed the raw data into a program to do it for you
Tests to examine significance of differences between 3 or more groups
• Parametric tests (tests based on the mathematics of the ‘normal curve’)– Analysis of Variance (1-way, 2-way, factorial, etc.)
• Non-parametric tests (rank-sum tests)– Kruskal-Wallis test
[strictly this should read ... ‘tests to decide how likely are data from 3 or more samples to come from the same population’]
Parallel group trial of placebo versus Normopress treatmentfor hypertension
Normopress dose
0 mg 0.5 mg 2.0 mg
Mea
n 24
hou
r blo
od p
ress
ure
0
20
40
60
80
100
120
140Mean +/- SE
Parallel group trial of placebo versus Normopress treatmentfor hypertension
Normopress dose
0 mg 0.5 mg 2.0 mg
Mea
n 24
hou
r blo
od p
ress
ure
0
20
40
60
80
100
120
140Mean +/- SE
One variable (dose), comparedacross 3 groups .... So this gets tested with one-way ANOVA
This tells us that it is very unlikely the three groups belong to the same population..But which differ from which?
Parallel group trial of placebo versus Normopress treatmentfor hypertension
Normopress dose
0 mg 0.5 mg 2.0 mg
Mea
n 24
hou
r blo
od p
ress
ure
0
20
40
60
80
100
120
140Mean +/- SE
One Way Analysis of Variance Thursday, June 28, 2012, 3:39:25 PM
Normality Test: Passed (P = 0.786)Equal Variance Test: Passed (P = 0.694)
Group Name N Missing Mean Std Dev SEMControl 12 0 126.500 7.243 2.091Normopress 0.5 mg 12 0 125.083 5.551 1.602Normopress 2.0 mg 12 0 114.000 5.625 1.624
Source of Variation DF SS MS F P Between Groups 2 1124.389 562.194 14.679 <0.001Residual 33 1263.917 38.301Total 35 2388.306
The differences in the mean values among the treatment groups are greater than would be expected by chance; there is a statistically significant difference (P = <0.001).
All Pairwise Multiple Comparison Procedures (Holm-Sidak method):Overall significance level = 0.05Comparisons for factor: Comparison Diff of Means t Unadjusted P Critical Significant?
LevelControl vs. Normopress 2.0 mg 12.500 4.947 <0.001 0.017 YesNormopress 0.5 mg vs. Normopress 2.0 mg 11.083 4.387 <0.001 0.025 YesControl vs. Normopress 0.5 mg 1.417 0.561 0.579 0.050 No
Before and after data – paired tests
• Create a paired analysis of length of hair after going to hairdresser.
• Hypothesis: cutting hair makes it shorter
Two independent groupsDifference between means tested for significance with
Student’s t test
Group
1 2
Hai
r len
gth
(cm
)
0
5
10
15
20
9.7±1.9*8.2±1.7
*mean ± SE
P=0.58
Our actual data
Group
1 2
Hai
r len
gth
(cm
)
0
5
10
15
20
P=0.025
Difference between means tested for significancewith paired Student’s t test
Before After
The variation within each individual is much less than between individuals
The paired t-test examines the mean and standard error of the changesIn each individual, and tests how likely are the changes due to chance
So far we have been dealing with‘CONTINUOUS VARIABLES’– numbers such as heights, laboratory values, velocities, temperatures etc. that could have any value (e.g. many decimal points) if we could measure accurately enough.
– whole numbers, most often as proportions or percentages.
Now we’ll look at ...‘DISCONTINUOUS VARIABLES’
Rates and proportions
• In 1969, a home for retired pirates has 93 inmates, 42 of whom have only one leg.
• In 2004, a subsequent survey finds there are now 62 inmates, 6 of whom have only one leg.
Has there been a ‘real’ change (i.e. a change unlikely to be due to chance) in the proportion of one-legged pirates in the home between the two surveys?
Year Pirates with 1 leg (%)
Pirates with 2 legs (%)
Total pirates
1969 42 (45.2) 51 (54.8) 93
2004 6 (9.7) 56 (10.3) 62
Totals 48 107 155
Chi-square= 20.282 with 1 degrees of freedom. (P = <0.001)
i.e. The likelihood that the differencein proportions of 1-legged inmates between 1969 and 2004 is due to chance ... is less than 1:1000
Expected
2919
One trap with chi-square tests and small numbers ....
Treatment No. dead No. surviving
Placebo 9 5
Penicillin 1 8
Treatment
No. dead No. surviving
Totals
Placebo 9 5 14
Penicillin 1 8 9
Totals 10 13 23
Fisher’s exact test:
P = 10! x 13! x 14! x 9! = 0.029 9! x 5! x 1! x 8! x 23!
Penicillin treatment for pneumonia
Correlation• Fairly straightforward concept of how likely are two
variables to be related to each other• Examples:
– Do children’s heights vary with their age, and if so is the relation direct (i.e. get bigger as get older) or converse (get smaller as get older)?
– Does respiratory rate increase as pulse rate increases during exertion?
• The correlation coefficient, R, tells us how closely the two variables ‘travel together’
• P value is calculated to tell us how likely the relationship is to be ‘only’ by chance
Examples of regression (correlation) data
Hours of sunlight0 2 4 6 8 10 12
MW
gen
erat
ed
4
6
8
10
12
14
16
18
X Data
0 2 4 6 8 10 12
Y D
ata
4
6
8
10
12
14
16
18
20
22
R = 0.965P<0.0001
R = - 0.37P = 0.30
Some other common statistical analyses
• Life-table analyses – Observing and comparing events developing over
time; allows us to compensate for dropouts at varying times during the study
• Multiple linear and multivariate regression analyses– Looking for relationships between multiple
variables
Life table analyses
Scagliotti et al. J Clin Oncol 2012; 30: 2829
Advanced lung cancer. Trial compared motesanib + 2 conventionalchemo drugs ... with placebo plus the two other drugs
Multiple regression analysis• Examines the possible effect of more than one
variable on the thing we are measuring (the ‘dependent variable’)
Perret JL et al. The Interplay between the Effects of LifetimeAsthma, Smoking, and Atopy on Fixed Airflow Obstruction inMiddle Age. Am J Respir Crit Med 2013; 187: 42-8....from Institute of Breathing and Sleep (Austin),University of Melbourne, Monash University, Alfred Hospital, And others
Perret et al. 2013
Sample Size Calculations
How many patients, subjects, mice etc. do we need to study to reliably* find the answer to our research question?
*We can never be certain to do this, but should aim to be considerably more likely than not to find out the truth about the question
Sample size calculations (1)First we need to grapple with two types of ‘error’ in interpreting differences between means and/or medians of groups:• Type 1 (or α) error: ... that we think the difference is
‘real’ (data are from 2 or more different populations) when it is not– This is what we’ve dealt with so far, and the P-
values assess how likely the differences are due to chance
• Type 2 (or β) error: ... that our experiment, and the stats test we’ll apply to the results, will FAIL to show a significant difference when there REALLY IS ONE
Sample size calculations (2)
• If we end up with a Type 2 () error, it will be because our sample size(s) was too small to persuade us that the actual difference between means was unlikely due to chance (i.e. P<0.05)
• The smaller the real difference between population means, the larger the sample size needs to be to detect it as being statistically significant
Sample size calculations (3)How do we go about it?
Most of the good statistical packages have a function for calculating sample sizes
1. Decide what statistical test will be appropriate to apply to primary endpoint when study completes
2. Estimate the likely size of difference between groups, if the hypothesis is correct
3. Decide how confident you want to be that the difference(s) you observe is unlikely due to chance
4. Decide how much you want to risk missing a true difference (i.e. what power you want the study to have)
Note: We really should have done a sample size calculation before we started ourexperiments, but for this course we needed to deal with the basics of stats tests first
Sample size calculationsA worked example (i)
• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months
1. Decide what statistical test: chi square, to compare differences in frequencies in 2 groups
Sample size calculationsA worked example (i)
• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months
• We expect a 10% incidence of ulcers in the controls• We hypothesize that a 50% reduction (i.e. 5%) in
those treated with X would be clinically worthwhile
2. We’ve now decided the size of the difference between groups we are interested to look for
Sample size calculations (4)A worked example (i)
• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months
• We expect a 10% incidence of ulcers in the controls• We hypothesize that a 50% reduction (i.e. 5%) in
those treated with X would be clinically worthwhile• We decide to be happy with a likelihood of only 1:20
that difference observed is due to chance3. That is to say, we want to set P0.05 as the level of α (alpha) risk (the risk of concluding the difference is real when it’s actually due to chance)
Sample size calculations (4)A worked example (i)
• We want to see whether drug X will reduce the incidence of peptic ulcer in patients taking aspirin for 6 months
• We expect a 10% incidence of ulcers in the controls• We hypothesize that a 50% reduction (i.e. 5%) in
those treated with X would be clinically worthwhile• We decide to be happy with a likelihood of only 1:20
that difference observed is due to chance• We would like to have at least an 80% chance of
finding that 50% reduction (20% of missing it, i.e. of β risk)
4. That is, set Power of the study ≥80% (1-β) to detect such a difference (if it exists)
Sample size calculationsA worked example (i)
Summary of sample size calculation setting:• Estimated ulcer incidence in controls = 10%• Estimated incidence in group receiving drug X
= 5%• For P(α) 0.05, and Power (1-β) ≥80%• Data tested by chi square ......................................................................Calculated required sample size = 449 in each
group
Sample size calculationsA worked example (ii)
• We hypothesize that removing the spleen in rats will result in an increase in haemoglobin (Hb) from the normal mean of 14.0 g/L to 15.0 g/L
• We already know that the SD (Standard deviation) of Hb values in normal rats is 1.2 g/L (if we don’t know we’ll have to guess!)
• Testing will be with Student’s t test• We’ll set α (likelihood observed difference due to
chance) at 0.05• We want a power (1-β) of at least 80% to minimize risk
of missing such a difference if its real*
*More correctly, we should say if the samples really are from different populations
Sample size calculationsA worked example (ii)
Summary of sample size calculation setting:• Control mean = 14.0 g/L; Operated mean =
15.0 g/L• Estimated SD in both groups = 1.2 g/L• For P(α) 0.05, and Power (1-β) ≥80%• Data tested by Student t......................................................................Calculated required sample size = 24 in each
group
Summary of the most common statistical tests in biomedicine (1. Parametric tests)
Test Purpose Comments
Student t-test Compare 2 groups of ‘continuous’ data*
Only use if data are ‘normally distributed’ and variances of groups similar
Paired Student t-test Compare before-after data on the same individuals
The differences (between before-after) need to be ‘normally distributed’More powerful than ‘unpaired’ t-test because less variability within individuals than between them
1- way analysis of variance
Compare 3 or more groups of continuous data
Same requirements as for Studentt-test
2-way analysis of variance
Compare 3 or more groups , stratified for at least 2 variables
As above
*For measured values, not numbers of events (frequencies)
Summary of the most common statistical tests in biomedicine (2. non-parametric)
Test Purpose Comments
Rank-sum or Mann-Whitney test
Compare 2 groups of ‘continuous’ data, using their ranks rather than actual values
Use if t-test invalid because data not ‘normally distributed’ and/or variances of groups significantly different
Signed rank test Rank test to use instead of paired t-test
Use instead of paired t-test if the differences (between before and after) are not ‘normally distributed’
Non-parametric analysis of variance
Compare 3 or more groups of continuous data
As above (it’s the generalised form of Mann-Whitney test when there are >2 groups)
Some tools for statistical analyses• Excel spreadsheets – e.g. If column A contains
data in the 8 cells, A3 through A10– Mean : =average(a3:a10)– SD: =stdev(a3:a10)– SEM: =(stdev(a3:a10))/sqrt(8)
• Common statistical packages for significance testing– Sigmaplot– SPPS (licence can be downloaded from Unimelb)– STATA
Other resources
• Armitage P, Berry G, Matthews JNS. Statistical methods in medical research. 4th edn. Oxford: Blackwell Science, 2002.
• Dawson B, Trapp R. Basic and clinical biostatistics. 4th edn. New York: McGraw Hill 2004. (Electronic book in Unimelb electronic collection)
• Rumsey DJ. Statistics for Dummies. 2nd edn. Oxford: Wiley & sons 2011.