statistical tests for replicated experiments normal probability plots are a less formal diagnostic...
TRANSCRIPT
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Normal probability plots are a Normal probability plots are a lessless formal diagnostic tool for detecting formal diagnostic tool for detecting effectseffects
F-tests and t-tests provide a F-tests and t-tests provide a statistical teststatistical test of factor effects of factor effects
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Statistical tests Statistical tests areare possible for possible for unreplicated designs (unreplicated unreplicated designs (unreplicated pilot studies are essential tools in pilot studies are essential tools in sample size calculations)sample size calculations)
We will first focus on statistical tests We will first focus on statistical tests for replicated designsfor replicated designs
Statistical tests for Statistical tests for replicated experiments--replicated experiments--
ExampleExample Response--Pulse rate of subjectResponse--Pulse rate of subject FactorsFactors
– Treatment (Energy Drink, Placebo) Treatment (Energy Drink, Placebo) – Setting (Moderate, Difficult)Setting (Moderate, Difficult)– Machine (Stair climber, Recumbent bike)Machine (Stair climber, Recumbent bike)
Statistical tests for Statistical tests for replicated experiments--replicated experiments--
ExampleExampleT S M Responses Mean Variance
-1 -1 -1 109, 112, 116, 112 112.25 8.25
1 -1 -1 120, 123, 125, 120 122 6
-1 1 -1 130, 118, 118, 118 121 36
1 1 -1 125, 123, 130, 124 125.5 9.67
-1 -1 1 108, 110, 113, 111 110.5 4.33
1 -1 1 121, 123, 123, 118 121.25 5.58
-1 1 1 113, 117, 122, 119 117.75 14.21
1 1 1 127, 126, 125, 125 125.75 0.917
Statistical tests for Statistical tests for replicated experiments--replicated experiments--
ExampleExampleT S M TS TM SM TSM
8.25 6.00 -1.375 -2.00 1.125 -.125 .625
Statistical tests for Statistical tests for replicated experiments--replicated experiments--
ExampleExample
Standardized Effect
Perc
ent
76543210-1-2
99
95
90
80
70
605040
30
20
10
5
1
Factor NameA AB BC C
Effect TypeNot SignificantSignificant
B
A
Normal Probability Plot of the Standardized Effects(response is Pulse, Alpha = .05)
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Effect sizes depend on the Effect sizes depend on the measurement scalemeasurement scale
Statistical tests are based on Statistical tests are based on standardizedstandardized effects effects
To compute standardized effects, To compute standardized effects, start with an estimate of start with an estimate of experimental errorexperimental error
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Experimental error can be Experimental error can be summarized by the square root of summarized by the square root of the variance of the background the variance of the background noise (the standard deviation)noise (the standard deviation)
The experimental error measures The experimental error measures variation in a single observationvariation in a single observation
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
The variance is best estimated by The variance is best estimated by the Mean Square for Pure Error the Mean Square for Pure Error (MSPE)(MSPE)
8
) variancescell of Sum(MSPE
Statistical tests for Statistical tests for replicated experiments--replicated experiments--
ExampleExample
The standard deviation for each run The standard deviation for each run is ~3 beats per minuteis ~3 beats per minute
26.3
62108
9221146258
MSPE
....
MSPE
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
While the standard deviation for a While the standard deviation for a single response is the square root of single response is the square root of MSPE, the standard deviation of an MSPE, the standard deviation of an effect (its standard error) is:effect (its standard error) is:
n
MSPEk 22
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
We divide an effect in a k-factor We divide an effect in a k-factor experiment with n replications (e.g., experiment with n replications (e.g., A) by its standard error to compute a A) by its standard error to compute a t-test statistic :t-test statistic :
nMSPE
EffectT
k 22
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Test statistics for other effects are Test statistics for other effects are computed similarlycomputed similarly
U-do-it: Calculate the T-statistics of U-do-it: Calculate the T-statistics of all effects for the Exercise dataall effects for the Exercise data
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
When an effect is negligible, T has a When an effect is negligible, T has a t-distributiont-distribution
The shape of the t-distribution curve The shape of the t-distribution curve depends on the number of replicates depends on the number of replicates (“degrees of freedom”=2(“degrees of freedom”=2kk(n-1))(n-1))
The t-distribution curves have The t-distribution curves have slightly more spread than the bell-slightly more spread than the bell-shaped (“normal”) curveshaped (“normal”) curve
Statistical tests for Statistical tests for replicated experiments--t replicated experiments--t curve for 3-factor designcurve for 3-factor design
43210-1-2-3-4-5
0.4
0.3
0.2
0.1
0.0
4 reps2 reps
2 replications
4 replications
Statistics tests for replicated Statistics tests for replicated experimentsexperiments
If |T| is larger than the 99.5If |T| is larger than the 99.5thth or 97.5 or 97.5thth percentile of the t distribution, an percentile of the t distribution, an effect is significanteffect is significant
These percentiles are commonly These percentiles are commonly found in textbooks (but found in textbooks (but please please use a use a computer package instead)computer package instead)
Statistical tests for Statistical tests for replicated experiments--t replicated experiments--t critical value for 3-factor critical value for 3-factor
design (n=4)design (n=4)
43210-1-2-3-4
0.4
0.3
0.2
0.1
0.0
.025
t dist. with 3 reps (24 df)
2.064
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Sometimes, twice the area to the Sometimes, twice the area to the right of |T| is reported as a right of |T| is reported as a p-valuep-value. . Small p-values suggest that a Small p-values suggest that a standardized effect is distinguishable standardized effect is distinguishable from background noise from background noise
You definitely need a computer to You definitely need a computer to compute p-values--in the following compute p-values--in the following example, the p-value for the M effect example, the p-value for the M effect is 2*.122=.244is 2*.122=.244
Statistical test for replicated Statistical test for replicated experiments--Exampleexperiments--Example
43210-1-2-3-4-5
0.4
0.3
0.2
0.1
0.0
1.19
0
.122
t distribution with 24 degrees of freedom
Statistical tests for Statistical tests for replicated experiments--replicated experiments--
ExampleExample U-do-it: Compute p-values for the U-do-it: Compute p-values for the remaining effects. Which effects are remaining effects. Which effects are significant? Are these the same significant? Are these the same effects that the probability plot effects that the probability plot detected?detected?
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
F tests for individual effects are F tests for individual effects are equivalent to t-testsequivalent to t-tests
F tests allow several comparisons to F tests allow several comparisons to be tested simultaneouslybe tested simultaneously
The t-test can be used to help in The t-test can be used to help in computing the number of replications computing the number of replications needed in a factorial experimentneeded in a factorial experiment
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Hypothesis testing can be extended Hypothesis testing can be extended to combine estimates of error from to combine estimates of error from both both pure errorpure error and and negligible negligible effectseffects– Negligible effects can be selected Negligible effects can be selected a a
prioripriori or from effects plots or from effects plots– Degrees of freedom for t-tests and F-Degrees of freedom for t-tests and F-
tests should be adjusted accordinglytests should be adjusted accordingly
Statistical tests for Statistical tests for replicated experimentsreplicated experiments
Error estimated from negiglible terms Error estimated from negiglible terms (Lack of Fit) is similar to MSPE(Lack of Fit) is similar to MSPE
Source DF SS MS Source DF SS MS
Residual Error 28 300.38 10.73Residual Error 28 300.38 10.73
Lack of Fit 4 45.38 11.34Lack of Fit 4 45.38 11.34
Pure Error 24 255.00 10.63Pure Error 24 255.00 10.63