ISE 352: Design of Experiments

• Dr. Laura Moody• Course Objectives:

– Upon successful completion of this course, you should be able to do the following:• Plan an experiment.• Identify the appropriate experimental method.• Develop a detailed experimental design.• Conduct and analyze the results of an experiment.• Draw appropriate conclusions and specific

recommendations.

ISE 352 - Ch. 1&2 1

Experimentation

• What is an experiment?

• Give an example of an experiment you have conducted …

ISE 352 - Ch. 1&2 2

Why experiment?

• Process improvement– improve productivity & safety– reduce variability / improve quality– reduce development time– reduce overall costs

• Engineering design– evaluation / comparison of design configurations– evaluation of alternatives– select design parameters for robust design /

performance– explore product ideasISE 352 - Ch. 1&2 3

Basic Principles

• randomization

• replication

• blocking

ISE 352 - Ch. 1&2 4

Guidelines … with an example

• Example: can you improve the range of a remote car key?1. Recognition and statement of the problem:

2. Selection of the response variable:

3. Choice of factors, levels, and range:

ISE 352 - Ch. 1&2 5

Guidelines … with an example (cont.)

4. Choice of experimental design:

5. Performing the experiment:

6. Statistical analysis of the data:

7. Conclusions & recommendations:

ISE 352 - Ch. 1&2 6

Some basic statistical concepts

• Describing sample data– Random samples– Sample mean, variance, standard deviation– Populations versus samples– Population mean, variance, standard deviation– Estimating parameters

• Simple comparative experiments– The hypothesis testing framework– The two-sample t-test– Checking assumptions, validity

7ISE 352 - Ch. 1&2

Homework

• Problem 1.4, pg. 22• Read chapters 1 & 2 and review the remaining

slides in this slideshow.• Be prepared to solve the following in class on

Friday:– 2.4, 2.15, 2.19, 2.25– Helpful hints:

• An Excel data set is available through the textbook website (www.wiley.com/college/montgomery)

ISE 352 - Ch. 1&2 8

An Example:

• Portland cement formulation – See data, page 24

• ch 2 examples.xls

ISE 352 - Ch. 1&2 9

Graphical view of the data

• Dot diagram• fig. 2.1, pp. 24

• What do you see?

ISE 352 - Ch. 1&2 10

If you have a large sample, a histogram may be useful

ISE 352 - Ch. 1&2 11

• Box plots– Fig. 2.3, pp. 26

• What do you see here?

Another graphical comparison

ISE 352 - Ch. 1&2 12

The hypothesis testing framework

• Statistical hypothesis testing is a useful framework for many experimental situations

• Origins of the methodology date from the early 1900s

• For the Portland cement example, we will use the two-sample t-test

ISE 352 - Ch. 1&2 13

The hypothesis testing framework

• Sampling from a normal distribution• Statistical hypotheses:

ISE 352 - Ch. 1&2 14

0 1 2

1 1 2

:

:

H

H

Estimation of parameters

ISE 352 - Ch. 1&2 15

1

2 2 2

1

1 estimates the population mean

1( ) estimates the variance

1

n

ii

n

ii

y yn

S y yn

Portland cement example …

• Summary statistics (see pg. 36)– Can get these from Minitab, as well …

ISE 352 - Ch. 1&2 16

1

21

1

1

16.76

0.100

0.316

10

y

S

S

n

2

22

2

2

17.04

0.061

0.248

10

y

S

S

n

Modified Mortar Unmodified Mortar

The two-sample t-test:

• Use the sample mean to draw inferences about the population means …

• Comparison is …

• This suggests a statistic,

ISE 352 - Ch. 1&2 17

_____________________21 yy

ny

22 where

,means the in difference the of deviation standard

meansinsdifference

2

22

1

21

210

nn

yyZ

The two-sample t-test:

ISE 352 - Ch. 1&2 18

2 2 2 21 2 1 2

1 2

2 21 2

1 2

2 2 21 2

2 22 1 1 2 2

1 2

Use and to estimate and

The previous ratio becomes

However, we have the case where

Pool the individual sample variances:

( 1) ( 1)

2p

S S

y y

S Sn n

n S n SS

n n

The two-sample t-test:

• Values of t0 that are near zero are consistent with the null hypothesis

• Values of t0 that are very different from zero are consistent with the alternative hypothesis

• t0 is a “distance” measure-how far apart the averages are expressed in standard deviation units

• Note: this is an interpretation of t0 as a signal-to-noise ratioISE 352 - Ch. 1&2 19

1 20

1 2

The test statistic is

1 1

p

y yt

Sn n

For the Portland cement example

ISE 352 - Ch. 1&2 20

2 22 1 1 2 2

1 2

1 20

1 2

( 1) ( 1) 9(0.100) 9(0.061)0.081

2 10 10 2

0.284

16.76 17.04 2.20

1 1 1 10.284

10 10

The two sample means are a little over two standard deviations apart

Is t

p

p

p

n S n SS

n n

S

y yt

Sn n

his a "large" difference?

The two-sample (pooled) t-test

• So far, we haven’t really done any “statistics”• We need an objective basis for deciding how large the test

statistic t0 really is– In 1908, W. S. Gosset derived the reference distribution for t0 …

called the t distribution– Tables of the t distribution – see Appendix II, pg. 614– In Excel, use the function

» ________________________________________

• OR …

– In Minitab, …

ISE 352 - Ch. 1&2 21

The two-sample (pooled) t-test

• The P-value is the area (probability) in the tails of the t-distribution beyond -2.20 + the probability beyond +2.20 (it’s a two-sided test)

• The P-value is a measure of how unusual the value of the test statistic is given that the null hypothesis is true

• The P-value the risk of wrongly rejecting the null hypothesis of equal means (it measures rareness of the event)

• The P-value in our problem is P = 0.042• (Note the values for tcrit used in the traditional approach)

ISE 352 - Ch. 1&2 22

p = 0.042

Computer two-sample t-test results

ISE 352 - Ch. 1&2 23

t-Test: Two-Sample Assuming Equal Variances

  Modified UnmodifiedMean 16.764 17.042Variance 0.100138 0.061462222Observations 10 10Pooled Variance 0.0808Hypothesized Mean Difference 0df 18t Stat -2.18688P(T<=t) one-tail 0.021098t Critical one-tail 1.734064P(T<=t) two-tail 0.042197t Critical two-tail 2.100922  

Checking assumptions – the normal probability plot

ISE 352 - Ch. 1&2 24

Normality testing in Minitab

ISE 352 - Ch. 1&2 25

Importance of the t-Test

• Provides an objective framework for simple comparative experiments

• Could be used to test all relevant hypotheses in a two-level factorial design, because all of these hypotheses involve the mean response at one “side” of the cube versus the mean response at the opposite “side” of the cube

ISE 352 - Ch. 1&2 26

Confidence intervals (See pg. 44)

• Hypothesis testing gives an objective statement concerning the difference in means, but it doesn’t specify “how different” they are

• General form of a confidence interval

• The 100(1- α)% confidence interval on the difference in two means:

ISE 352 - Ch. 1&2 27

where ( ) 1 L U P L U

1 2

1 2

1 2 / 2, 2 1 2 1 2

1 2 / 2, 2 1 2

(1/ ) (1/ )

(1/ ) (1/ )

n n p

n n p

y y t S n n

y y t S n n

ISE 352 - Ch. 1&2 28

The Portland cement example(see pg. 45)

Other chapter topics

• Hypothesis testing when the variances are known

• One sample inference• Hypothesis tests on variances• Paired experiments

ISE 352 - Ch. 1&2 29

Homework (due Monday, Aug. 27)

• Finish and format the results of the problems you solved in class on Friday.

• Read Chapter 3 and review the PowerPoint slides.

• Be prepared to solve the following in class:– 3.3– 3.8, part a only (use Minitab or Excel)

• Bring your solutions to class on Monday.

ISE 352 - Ch. 1&2 30