topic 23: diagnostics and remedies. outline diagnostics –residual checks anova remedial measures
TRANSCRIPT
Topic 23: Diagnostics and Remedies
Outline
• Diagnostics
– residual checks
• ANOVA remedial measures
Diagnostics Overview
• We will take the diagnostics and remedial measures that we learned for regression and adapt them to the ANOVA setting
• Many things are essentially the same
• Some things require modification
Residuals
• Predicted values are cell means, =
• Residuals are the differences between the observed values and the cell means Yij-
ijYi.Y
i.Y
Basic plots
• Plot the data vs the factor levels (the values of the explanatory variables)
• Plot the residuals vs the factor levels
• Construct a normal quantile plot and/or histogram of the residuals
KNNL Example
• KNNL p 777
• Compare 4 brands of rust inhibitor (X has r=4 levels)
• Response variable is a measure of the effectiveness of the inhibitor
• There are 10 units per brand (n=10)
Plots
• Data versus the factor
• Residuals versus the factor
• Normal quantile plot of the residuals
Plots vs the factor
symbol1 v=circle i=none;proc gplot data=a2; plot (eff resid)*abrand;run;
Data vs the factor
Means look different …common spread in Y’s
Residuals vs the factorOdd dist of points
QQ-plot
Due to odd (lack of and large)spread
Can try nonparametric analysis – last slides
General Summary
• Look for
–Outliers
–Variance that depends on level
–Non-normal errors
• Plot residuals vs time and other variables if available
Homogeneity tests
• Homogeneity of variance (homoscedasticity)
• H0: σ12 = σ2
2 = … = σr2
• H1: not all σi2 are equal
• Several significance tests are available
Homogeneity tests
• Text discusses Hartley, modified Levene
• SAS has several including Bartlett’s (essentially the likelihood ratio test) and several versions of Levene
Homogeneity tests
• There is a problem with assumptions–ANOVA is robust with respect to
moderate deviations from Normality–ANOVA results can be sensitive to
the homogeneity of variance assumption
• Some homogeneity tests are sensitive to the Normality assumption
Levene’s Test
• Do ANOVA on the squared residuals from the original ANOVA
• Modified Levene’s test uses absolute values of the residuals
• Modified Levene’s test is recommended• Another quick and dirty rule of thumb
2 2max( ) / min( ) 2i is s
KNNL Example
• KNNL p 785• Compare the strengths of 5 types
of solder flux (X has r=5 levels)• Response variable is the pull
strength, force in pounds required to break the joint
• There are 8 solder joints per flux (n=8)
Scatterplot
Levene’s Test
proc glm data=a1; class type; model strength=type; means type/ hovtest=levene(type=abs); run;
ANOVA Table
Source DFSum of
SquaresMean
Square F Value Pr > FModel 4 353.612085 88.4030213 41.93 <.0001Error 35 73.7988250 2.1085379Corrected Total
39 427.410910
Common variance estimated to be 2.11
Output
Levene's TestANOVA of Absolute Deviations Source DF F Value Pr > Ftype 4 3.07 0.0288Error 35
We reject the null hypothesis and assume nonconstant variance
Means and SDs
Level strengthtype N Mean Std Dev1 8 15.42 1.232 8 18.52 1.253 8 15.00 2.484 8 9.74 0.815 8 12.34 0.76
Remedies
• Delete outliers
– Is their removal important?• Use weights (weighted regression)
• Transformations
• Nonparametric procedures
What to do here?
• Not really any obvious outliers
• Do not see pattern of increasing or decreasing variance or skewed dists
• Will consider
–Weighted ANOVA
–Mixed model ANOVA
Weighted least squares
• We used this with regression–Obtain model for how the sd
depends on the explanatory variable (plotted absolute value of residual vs x)–Then used weights inversely
proportional to the estimated variance
Weighted Least Squares
• Here we can compute the variance for each level
• Use these as weights in PROC GLM
• We will illustrate with the soldering example from KNNL
Obtain the variances and weights
proc means data=a1; var strength; by type; output out=a2 var=s2;data a2; set a2; wt=1/s2;
NOTE. Data set a2 has 5 cases
Proc Means Output
Level oftype N
strength
Mean Std Dev1 8 15.4200000 1.23713956
2 8 18.5275000 1.25297076
3 8 15.0037500 2.48664397
4 8 9.7412500 0.81660337
5 8 12.3400000 0.76941536
Merge and then use the weights in PROC GLM
data a3; merge a1 a2; by type; proc glm data=a3; class type; model strength=type; weight wt; lsmeans type / cl; run;
Output
Source DFSum of
Squares Mean Square F Value Pr > FModel 4 324.213099 81.0532747 81.05 <.0001
Error 35 35.0000000 1.0000000
Corrected Total 39 359.213099
Data have been standardized to have a variance of 1
LSMEANS Output
typestrength
LSMEANStandard
Error Pr > |t|95% Confidence
Limits1 15.4200000 0.4373949 <.0001 14.53204
116.30795
92 18.5275000 0.4429921 <.0001 17.62817
819.42682
23 15.0037500 0.8791614 <.0001 13.21895
716.78854
34 9.7412500 0.2887129 <.0001 9.155132 10.32736
85 12.3400000 0.2720294 <.0001 11.78775
112.89224
9
Because of weights, standard errors simply based on sample variances of each level
Mixed Model ANOVA
• Relax the assumption of constant variance rather than including a “known” weight
• This involves moving to a mixed model procedure
• Topic will not be on exam but wanted you to be aware of these model capabilities
SAS Codeproc glimmix data=a1;
class type;
model strength=type / ddfm=kr;
random residual / group=type;
run;This allows the variance to differ in each level and a degrees of freedom adjustment is used to account for this
GLIMMIX OUTPUTFit Statistics
-2 Res Log Likelihood 122.11AIC (smaller is better) 132.11AICC (smaller is better) 134.18BIC (smaller is better) 139.88CAIC (smaller is better) 144.88HQIC (smaller is better) 134.79Generalized Chi-Square 35.00Gener. Chi-Square / DF 1.00
Covariance Parameter Estimates
Cov Parm Group EstimateStandard
ErrorResidual (VC) type 1 1.5305 0.8181Residual (VC) type 2 1.5699 0.8392Residual (VC) type 3 6.1834 3.3052Residual (VC) type 4 0.6668 0.3564Residual (VC) type 5 0.5920 0.3164
Type III Tests of Fixed Effects
EffectNum
DFDen DF F Value Pr > F
type 4 14.81 71.78 <.0001
Really 3 groups of variances
SAS Codeproc glimmix data=a1;
class type;
model strength=type / ddfm=kr;
random residual / group=type1;
run;Type1 was created to identify Type 1 and 2, Type 3, and Type
4 and 5 as 3 groups
GLIMMIX OUTPUTFit Statistics
-2 Res Log Likelihood 122.13AIC (smaller is better) 128.13AICC (smaller is better) 128.91BIC (smaller is better) 132.80CAIC (smaller is better) 135.80HQIC (smaller is better) 129.74Generalized Chi-Square 35.00Gener. Chi-Square / DF 1.00
Covariance Parameter Estimates
Cov Parm Group EstimateStandard
ErrorResidual (VC) Grp 1 1.5502 0.5859Residual (VC) Grp 2 6.1834 3.3052Residual (VC) Grp 3 0.6294 0.2379
Type III Tests of Fixed Effects
EffectNum
DFDen DF F Value Pr > F
type 4 19.8 77.68 <.0001
Better BIC but same general type conclusion
Transformation Guides
• When σi2 is proportional to μi, use
• When σi is proportional to μi, use log(y)
• When σi is proportional to μi2, use 1/y
• For proportions, use arcsin( )–arsin(sqrt(y)) in a SAS data step
• Box-Cox transformation
Y
Y
Example
• Consider study on KNNL pg 790
• Y: time between computer failures
• X: three locations data a3;
infile 'u:\.www\datasets512\CH18TA05.txt'; input time location interval; symbol1 v=circle; proc gplot; plot time*location; run;
Scatterplot
Outlier or skewed distribution? Can consider transformation first
Box-Cox Transformation
• Can consider regression
and 1-b1 is the power to raise Y by
• Can try various “convenient” powers
• Can use SAS directly to calculate the power
log( ) vs log( )i is y
E(logsig) = 0.90 + .79 logmu
Power should be 1-.79 ≈ 0.20
Using SAS
proc transreg data=a3;
model boxcox(time / lambda=-2 to 2
by .2) = class(location);
run;
OutputBox-Cox Transformation Information for time
Lambda R-Square Log Like-1.0 0.24 -73.040-0.8 0.27 -67.330-0.6 0.31 -62.316-0.5 0.33 -60.144-0.4 0.35 -58.239-0.2 0.38 -55.346 *0.0 + 0.39 -53.830 *0.2 0.38 -53.769 <0.4 0.36 -55.118 *0.5 0.34 -56.2730.6 0.32 -57.7120.8 0.29 -61.3141.0 0.25 -65.675
< - Best Lambda* - 95% Confidence Interval+ - Convenient Lambda
Transforming data in SAS
data a3;
set a3;
transtime = time**0.20;
symbol1 v=circle i=none;
proc gplot;
plot transtime*location;
run;
Much more constant spread in data!
Nonparametric approach
• Based on ranks
• See KNNL section 18.7, p 795
• See the SAS procedure NPAR1WAY
Rust Inhibitor Analysis
Source DFSum of
Squares Mean Square F Value Pr > FModel 3 15953.4660 5317.82200 866.12 <.0001Error 36 221.03400 6.13983Corrected Total 39 16174.5000
Highly significant F test. Even if there is a violation of Normality, the evidence is
overwhelming
Nonparametric AnalysisWilcoxon Scores (Rank Sums) for Variable eff
Classified by Variable abrand
abrand NSum ofScores
ExpectedUnder H0
Std DevUnder H0
MeanScore
1 10 128.0 205.0 32.014119 12.802 10 355.0 205.0 32.014119 35.503 10 255.0 205.0 32.014119 25.504 10 82.0 205.0 32.014119 8.20
Average scores were used for ties.
Kruskal-Wallis TestChi-Square 33.7041
DF 3
Pr > Chi-Square <.0001
Last slide
• We’ve finished most of Chapters 17 and 18.
• We used program topic23.sas to generate the output.