chapter eleven. designing, conducting, analyzing, and interpreting experiments with multiple...

Chapter Eleven.Designing, Conducting, Analyzing, and Interpreting Experiments with Multiple Independent Variables

Experimental Design: Doubling the Basic Building Block A factorial design gives us the power we need to devise

an investigation of several factors (IVs) in a single experiment.

Experimental Design: Doubling the Basic Building Block Factors

Experimental Design: Doubling the Basic Building Block Factors

Synonymous with IVs

Experimental Design: Doubling the Basic Building Block Factors

Synonymous with IVs Independent variables (IVs)

Experimental Design: Doubling the Basic Building Block Factors

Synonymous with IVs Independent variables (IVs)

Stimuli or aspects of the environment that are directly manipulated by the experimenter to determine their influences on behavior.

Experimental Design: Doubling the Basic Building Block Factorial designs are the lifeblood of experimental psychology

because they allow us to look at combinations of IVs at the same time, a situation that is quite similar to the real world.

Experimental Design: Doubling the Basic Building Block Factorial designs are the lifeblood of experimental

psychology because they allow us to look at combinations of IVs at the same time, a situation that is quite similar to the real world.

A factorial design is more like the real world because there are probably few, if any, situations in which your behavior is affected by only a single factor at a time.

How Many IV’s? The factorial design gets its name because we refer to

each IV as a factor.

How Many IV’s? The factorial design gets its name because we refer to

each IV as a factor. Multiple IV’s yield a factorial design.

How Many IV’s? The factorial design gets its name because we refer to

each IV as a factor. Multiple IV’s yield a factorial design.

Theoretically, there is no limit to the number of IV’s that can be used in an experiment.

How Many IV’s? The factorial design gets its name because we refer to

each IV as a factor. Multiple IV’s yield a factorial design.

Theoretically, there is no limit to the number of IV’s that can be used in an experiment.

Practically speaking, however, it is unlikely that you would want to design an experiment with more than two or three IV’s.

How many Groups or Levels? Once you have two or more IV’s, you will use a factorial

design.

How many Groups or Levels? Once you have two or more IV’s, you will use a factorial

design. The number of levels of each factor is unimportant at

this point.

How many Groups or Levels? the simplest possible factorial design is known as a 2 X

2 design. This 2 X 2 shorthand notation tells us that we are dealing

with a design that has two factors (IV’s) because there are two digits given and that each of the two factors has two levels because each digit shown is a two.

How many Groups or Levels? The number of numbers tells us how many

IV’s there are.

How many Groups or Levels? The number of numbers tells us how many IV’s

there are. The value of each number tells us how many levels

each IV has.

How many Groups or Levels? Various factors are often designated by letters, so the

first factor is labeled Factor A, the second as Factor B, and so on.

How many Groups or Levels? Various factors are often designated by letters, so the

first factor is labeled Factor A, the second as Factor B, and so on.

The levels within a factor are often designated by the letter that corresponds to the factor and a number to differentiate the different levels.

How many Groups or Levels? Various factors are often designated by letters, so the

first factor is labeled Factor A, the second as Factor B, and so on.

The levels within a factor are often designated by the letter that corresponds to the factor and a number to differentiate the different levels.

Thus, the two levels within the first factor would be labeled A1 (A sub 1) and A2 (A sub 2).

How many Groups or Levels? Main effect

How many Groups or Levels? Main effect

A main effect refers to the sole effect of one IV in a factorial design.

Assigning Participants to Groups We have two options for this assignment –

independent groups or correlated groups.

Assigning Participants to Groups We have two options for this assignment –

independent groups or correlated groups. However, this question is not answered in such a simple

manner as in the two-group and multiple-group designs, each of which had only one IV.

Assigning Participants to Groups However, this question is not answered in such a simple

manner as in the two-group and multiple-group designs, each of which had only one IV.

All IV’s could have participants assigned randomly or in a correlated fashion, or we could have one IV with independent groups and one IV with correlated groups. This possibility is referred to as mixed assignment.

Assigning Participants to Groups Mixed assignment

Assigning Participants to Groups Mixed assignment

A factorial design that has a mixture of independent groups for one IV and correlated groups for another IV.

Assigning Participants to Groups Mixed assignment

A factorial design that has a mixture of independent groups for one IV and correlated groups for another IV.

In larger factorial designs, at least one IV has independent groups and at least one has correlated groups (also known as mixed groups).

Random Assignment to Groups Factorial designs in which both IV’s involve random

assignment may be called between-subjects factorial designs or completely randomized designs Random Assignment to Groups

Nonrandom Assignment to Groups In this section, we deal with factorial designs in which

participant groups for all IV’s have been formed through nonrandom assignment.

Nonrandom Assignment to Groups In this section, we deal with factorial designs in which

participant groups for all IV’s have been formed through nonrandom assignment.

We refer to such designs as completely within-groups

(or within-subjects) designs.

Nonrandom Assignment to Groups In this section, we deal with factorial designs in which

participant groups for all IV’s have been formed through nonrandom assignment.

We refer to such designs as completely within-groups (or within-subjects) designs.

We may want to resort to nonrandom assignment in order to assure the equality of participant groups before we conduct the experiment.

Nonrandom Assignment to Groups Matched Pairs or Sets.

Nonrandom Assignment to Groups Matched Pairs or Sets.

Matching can take place in either pairs or sets because factorial designs can use IV’s with two or more levels.

Nonrandom Assignment to Groups Matched Pairs or Sets.

Matching can take place in either pairs or sets because factorial designs can use IV’s with two or more levels.

The more levels an IV has, the more work matching for that variable takes.

Nonrandom Assignment to Groups Matched Pairs or Sets.

Matching can take place in either pairs or sets because factorial designs can use IV’s with two or more levels.

The more levels an IV has, the more work matching for that variable takes.

The more precise the match that is necessary, the more difficult matching becomes.

Nonrandom Assignment to Groups Repeated Measures.

Nonrandom Assignment to Groups Repeated Measures.

In a completely within-groups experiment using repeated measures, participants would take part fully and completely.

Nonrandom Assignment to Groups Repeated Measures.

In a completely within-groups experiment using repeated measures, participants would take part fully and completely.

Participants take part in every possible treatment combination.

Nonrandom Assignment to Groups Repeated Measures.

In a completely within-groups experiment using repeated measures, participants would take part fully and completely.

Participants take part in every possible treatment combination.

This requirement makes it difficult or impossible to conduct an experiment with repeated measures on multiple IV’s.

Nonrandom Assignment to Groups Repeated Measures.

In a completely within-groups experiment using repeated measures, participants would take part fully and completely.

Participants take part in every possible treatment combination.

This requirement makes it difficult or impossible to conduct an experiment with repeated measures on multiple IV’s.

The smaller the design, the more feasible it is to include all participants in all conditions of the experiment.

Nonrandom Assignment to Groups Natural Pairs or Sets.

Nonrandom Assignment to Groups Natural Pairs or Sets.

Using natural groups in a totally within-subjects design has the same difficulties as the matched pairs or sets variation of this design, but it would be even harder.

Nonrandom Assignment to Groups Natural Pairs or Sets.

Using natural groups in a totally within-subjects design has the same difficulties as the matched pairs or sets variation of this design, but it would be even harder.

The difficulty lies in being able to find an adequate number of naturally linked participants.

Nonrandom Assignment to Groups Mixed Assignment to Groups.

Nonrandom Assignment to Groups Mixed Assignment to Groups.

Mixed assignment designs involve a combination of random and nonrandom assignment, with at least one IV using each type of assignment to groups.

Nonrandom Assignment to Groups Mixed Assignment to Groups.

Mixed assignment designs involve a combination of random and nonrandom assignment, with at least one IV using each type of assignment to groups.

In a two-IV factorial design, mixed assignment involves one IV with random assignment and one IV with nonrandom assignment.

Nonrandom Assignment to Groups Mixed Assignment to Groups.

Mixed assignment designs involve a combination of random and nonrandom assignment, with at least one IV using each type of assignment to groups.

In a two-IV factorial design, mixed assignment involves one IV with random assignment and one IV with nonrandom assignment.

In such designs, the use of repeated measures is probably more likely than other types of nonrandom assignment.

Nonrandom Assignment to Groups Mixed Assignment to Groups.

Mixed assignment designs involve a combination of random and nonrandom assignment, with at least one IV using each type of assignment to groups.

Mixed designs combine the advantages of the two types of designs.

Nonrandom Assignment to Groups Mixed Assignment to Groups.

Mixed assignment designs involve a combination of random and nonrandom assignment, with at least one IV using each type of assignment to groups.

Mixed designs combine the advantages of the two types of designs.

The conservation of participants through the use of repeated measures for a between-subjects variable makes for a popular and powerful design.

Comparing the Factorial Design to Two-Group and Multiple-Group Designs Two-group designs are ideal for a preliminary

investigation of a particular IV in a presence-absence format.

Comparing the Factorial Design to Two-Group and Multiple-Group Designs In a similar fashion, 2 X 2 factorial designs may be used

for preliminary investigations of two IV’s.

Comparing the Factorial Design to Two-Group and Multiple-Group Designs The multiple-group design may be used to conduct

more in-depth investigations of an IV that interests us (chapter 10).

Comparing the Factorial Design to Two-Group and Multiple-Group Designs The multiple-group design may be used to conduct

more in-depth investigations of an IV that interests us. We took the basic two-group design (chapter 9) and

extended it to include more levels of our IV (chapter 10).

Comparing the Factorial Design to Two-Group and Multiple-Group Designs The multiple-group design may be used to conduct

more in-depth investigations of an IV that interests us. We took the basic two-group design and extended it to

include more levels of our IV. We can make the same type of extension with factorial

designs.

Comparing the Factorial Design to Two-Group and Multiple-Group Designs The multiple-group design may be used to conduct

more in-depth investigations of an IV that interests us. We took the basic two-group design and extended it to

include more levels of our IV. We can make the same type of extension with factorial

designs. Just as with the multiple-group design, there is no limit

to the number of levels for any IV in a factorial design.

Comparing the Factorial Design to Two-Group and Multiple-Group Designs The multiple-group design may be used to conduct

more in-depth investigations of an IV that interests us. We took the basic two-group design and extended it to

include more levels of our IV. We can make the same type of extension with factorial

designs. Just as with the multiple-group design, there is no limit

to the number of levels for any IV in a factorial design. The number of levels of the IV’s can be equal or

unequal.

Comparing the Factorial Design to Two-Group and Multiple-Group Designs The multiple-group design may be used to conduct more

in-depth investigations of an IV that interests us. We took the basic two-group design and extended it to

include more levels of our IV. We can make the same type of extension with factorial

designs. Just as with the multiple-group design, there is no limit to

the number of levels for any IV in a factorial design. Interaction effects must be interpreted in factorial designs

but not in two-group or multiple-group designs. A good rule of thumb to follow is to choose the simplest

research design that will adequately test your hypothesis.

Choosing a Factorial Design Experimental Questions

Experimental Questions Factorial designs provide considerable flexibility in

devising an experiment to answer your questions.

Experimental Questions Factorial designs provide considerable flexibility in

devising an experiment to answer your questions. The number of questions we can ask in a factorial

experiment increases dramatically, but….

Experimental Questions Factorial designs provide considerable flexibility in

devising an experiment to answer your questions. The number of questions we can ask in a factorial

experiment increases dramatically, but…. When we ask additional questions, we must make certain

that the questions coordinate with each other…experimental questions should not clash.

Experimental Questions Factorial designs provide considerable flexibility in

devising an experiment to answer your questions. The number of questions we can ask in a factorial

experiment increases dramatically, but…. When we ask additional questions, we must make certain

that the questions coordinate with each other…experimental questions should not clash.

(e.g., it would not make sense to propose an experiment to examine the effects of self-esteem and eye color on test performance)

Control Issues We need to consider independent versus correlated

groups in factorial designs.

Control Issues We need to consider independent versus correlated

groups in factorial designs. A complicating factor for factorial designs is that we

need to make this decision (independent vs. correlated groups) for each IV we include in an experiment.

Practical Considerations You are well advised to keep your experiment at the

bare minimum necessary to answer the question(s) that most interest(s) you.

Practical Considerations You are well advised to keep your experiment at the

bare minimum necessary to answer the question(s) that most interest(s) you.

Bear in mind that you are complicating matters when you add IV’s and levels.

Variations on Factorial Designs Comparing Different Amounts of an IV.

Variations on Factorial Designs Comparing Different Amounts of an IV.

When you add a level to an IV in a factorial design, you add several groups to your experiment because each new level must be added under each level of your other independent variable(s).

Comparing Different Amounts of an IV When you add a level to an IV in a factorial design, you

add several groups to your experiment because each new level must be added under each level of your other independent variable(s).

For example, expanding a 2 X 2 to a 3 X 2 design requires 6 groups rather than 4.

Comparing Different Amounts of an IV When you add a level to an IV in a factorial design, you

add several groups to your experiment because each new level must be added under each level of your other independent variable(s).

For example, expanding a 2 X 2 to a 3 X 2 design requires 6 groups rather than 4.

Adding levels in a factorial design increases groups in a multiplicative fashion.

Using Measured IV’s Using a measured rather than a manipulated IV results

in ex post facto research.

Using Measured IV’s Ex post facto research

A research approach in which the experimenter cannot directly manipulate the IV but can only classify, categorize, or measure the IV because it is predetermined in the participants (e.g., IV = sex).

Using Measured IV’s Using a measured rather than a manipulated IV results

in ex post facto research. Without the control that comes from directly causing an

IV to vary, we must exercise extreme caution in drawing conclusions from such studies.

Using Measured IV’s Using a measured rather than a manipulated IV results

in ex post facto research. Without the control that comes from directly causing an

IV to vary, we must exercise extreme caution in drawing conclusions from such studies.

We can develop an experiment that uses one manipulated IV and one measured IV at the same time.

Dealing with More than Two IV’s Designing an experiment with more than two IV’s is

probably the most important variation of the factorial

design.

Dealing with More than Two IV’s The simplest possible factorial design with three IV’s

(often referred to as a three-way design) has three IV’s, each with two levels.

Dealing with More than Two IV’s The simplest possible factorial design with three IV’s

(often referred to as a three-way design) has three IV’s, each with two levels.

This design represents a 2 X 2 X 2 experiment.

Dealing with More than Two IV’s The simplest possible factorial design with three IV’s

(often referred to as a three-way design) has three IV’s, each with two levels.

This design represents a 2 X 2 X 2 experiment. This design would require eight different groups if it is

planned as a completely between-groups design.

Statistical Analysis: What Do Your Data Show? Naming Factorial Designs

Labels you may hear that reflect the size of the design include:

Statistical Analysis: What Do Your Data Show? Naming Factorial Designs

Labels you may hear that reflect the size of the design include:

Factorial ANOVA

Statistical Analysis: What Do Your Data Show? Naming Factorial Designs

Labels you may hear that reflect the size of the design include:

Factorial ANOVA Two-way ANOVA

Statistical Analysis: What Do Your Data Show? Naming Factorial Designs

Labels you may hear that reflect the size of the design include:

Factorial ANOVA Two-way ANOVA Three-way ANOVA

Statistical Analysis: What Do Your Data Show? Naming Factorial Designs

Labels you may hear that reflect the size of the design include:

Factorial ANOVA Two-way ANOVA Three-way ANOVA X by Y

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Independent groups

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Independent groups Completely randomized

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Independent groups Completely randomized Completely between-subjects

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Independent groups Completely randomized Completely between-subjects Completely between-groups

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Independent groups Completely randomized Completely between-subjects Completely between-groups Totally between-subjects

Naming Factorial Designs For designs that use random assignment for all IV’s,

labels that describe how participants are assigned to groups might include:

Independent groups Completely randomized Completely between-subjects Completely between-groups Totally between-subjects Totally between-groups

Naming Factorial Designs Designs that use matching or repeated measures may

be called:

Naming Factorial Designs Designs that use matching or repeated measures may

be called: Randomized block

Naming Factorial Designs Designs that use matching or repeated measures may

be called: Randomized block Completely within-subjects

Naming Factorial Designs Designs that use matching or repeated measures may

be called: Randomized block Completely within-subjects Completely within-groups

Naming Factorial Designs Designs that use matching or repeated measures may

be called: Randomized block Completely within-subjects Completely within-groups Totally within-subjects

Naming Factorial Designs Designs that use matching or repeated measures may

be called: Randomized block Completely within-subjects Completely within-groups Totally within-subjects Totally within-groups

Naming Factorial Designs Designs that use a mixture of “between” and “within”

assignment procedures may be referred to as:

Naming Factorial Designs Designs that use a mixture of “between” and “within”

assignment procedures may be referred to as: Mixed factorial

Naming Factorial Designs Designs that use a mixture of “between” and “within”

assignment procedures may be referred to as: Mixed factorial Split-plot factorial

Planning the Statistical Analysis Suppose you are examining the data from the previous

(chapter 10) experiment and you think you detected an oddity in the data:

Planning the Statistical Analysis Suppose you are examining the data from the previous

(chapter 10) experiment and you think you detected an oddity in the data:

It appears that salesclerks may have responded differently to female and male customers in addition to the different styles of dress.

Planning the Statistical Analysis You decide to investigate this question in order to find

out whether both customer sex and dress affect salesclerks’ response times to customers.

Planning the Statistical Analysis You decide to investigate this question in order to find

out whether both customer sex and dress affect salesclerks’ response times to customers.

Because there was no difference between responses to customers in dressy and casual clothing (see chapter 10), you decide to use only casual and sloppy clothes.

Planning the Statistical Analysis Thus, you have designed a 2 X 2 experiment in which

the two IV’s are clothing style (casual and sloppy) and customer sex (male and female).

Rationale for ANOVA The rationale behind ANOVA for factorial designs is

basically the same as we saw in Chapter 10, with one major modification.

Rationale for ANOVA The rationale behind ANOVA for factorial designs is

basically the same as we saw in Chapter 10, with one major modification.

We still use ANOVA to partition (divide) the variability into two sources – treatment variability and error variability.

Rationale for ANOVA With factorial designs, the sources of treatment

variability increase.

Rationale for ANOVA With factorial designs, the sources of treatment

variability increase. Instead of having one IV as the sole source of treatment

variability, factorial designs have multiple IV’s and their interactions as sources of treatment variability.

Rationale for ANOVA With factorial designs, the sources of treatment

variability increase. Instead of having one IV as the sole source of treatment

variability, factorial designs have multiple IV’s and their interactions as sources of treatment variability.

Rationale for ANOVA The actual distribution of the variance among the

factors would depend, of course, on which effects were significant.

Rationale for ANOVA For a two-IV factorial design we use the following

equations:

Rationale for ANOVA For a two-IV factorial design we use the following

equations: Factor A = IV A variability error variability

Rationale for ANOVA For a two-IV factorial design we use the following

equations: Factor B = IV B variability error variability

Rationale for ANOVA For a two-IV factorial design we use the following

equations: Factor A by B = interaction variability error variability

Understanding Interactions When two variables interact, their joint effect may not

be obvious or predictable from examining their separate effects.

Understanding Interactions When two variables interact, their joint effect may not

be obvious or predictable from examining their separate effects.

For example, drinking a glass or two of wine may be a pleasurable and relaxing experience and driving may be a pleasurable and relaxing experience but is drinking wine and driving an extremely pleasurable and relaxing experience?

Understanding Interactions When two variables interact, their joint effect may not

be obvious or predictable from examining their separate effects.

For example, drinking a glass or two of wine may be a pleasurable and relaxing experience and driving may be a pleasurable and relaxing experience but is drinking wine and driving an extremely pleasurable and relaxing experience?

Of course not.

Understanding Interactions When two variables interact, their joint effect may not

be obvious or predictable from examining their separate effects.

Combinations of drugs, in particular, are likely to have synergistic effects so that a joint effect occurs that is not predictable from either drug alone.

Understanding Interactions Synergistic effects

Understanding Interactions Synergistic effects

Dramatic consequences that occur when you combine two or more substances, conditions, or organisms.

Understanding Interactions Synergistic effects

Dramatic consequences that occur when you combine two or more substances, conditions, or organisms.

The effects are greater than what is individually possible.

Understanding Interactions A significant interaction means that the effects of the

various IV’s are not straightforward and simple.

Understanding Interactions A significant interaction means that the effects of the

various IV’s are not straightforward and simple. For this reason, we virtually ignore our IV main effects

when we find a significant interaction.

Understanding Interactions A significant interaction means that the effects of the

various IV’s are not straightforward and simple. For this reason, we virtually ignore our IV main effects

when we find a significant interaction. Sometimes interactions are difficult to interpret,

particularly when we have more than two IV’s or many levels of an IV.

Understanding Interactions A strategy that often helps us to make sense of an

interaction is to graph it.

Understanding Interactions A strategy that often helps us to make sense of an

interaction is to graph it. By graphing your DV on the y axis and one IV on the x

axis, you can depict your other IV with lines on the graph (see Chapter 8).

Understanding Interactions A strategy that often helps us to make sense of an

interaction is to graph it. By graphing your DV on the y axis and one IV on the x

axis, you can depict your other IV with lines on the graph (see Chapter 8).

By studying such as graph, you can usually deduce what happened to cause a significant interaction.

Understanding Interactions When you graph a significant interaction, you will often

notice that the lines of the graph cross or converge.

Understanding Interactions When you graph a significant interaction, you will often

notice that the lines of the graph cross or converge. This pattern is a visual indication that the effects of one

IV change as the second IV is varied.

Understanding Interactions When you graph a significant interaction, you will often

notice that the lines of the graph cross or converge. This pattern is a visual indication that the effects of one

IV change as the second IV is varied. Nonsignificant interactions typically show lines that are

close to parallel.

Interpretation: Making Sense of Your Statistics Our statistical analyses of factorial designs will provide

us more information than we got from two-group or multiple-group designs.

Interpretation: Making Sense of Your Statistics Our statistical analyses of factorial designs will provide

us more information than we got from two-group or multiple-group designs.

The analyses are not necessarily more complicated than those we saw in Chapters 9 and 10, but they do provide more information because we have multiple IV’s and interaction effects to analyze.

Interpreting Computer Statistical Output We will deal with 2 X 2 analyses in these three different

categories to fit our clothing-by-customer-sex experiment:

Interpreting Computer Statistical Output We will deal with 2 X 2 analyses in these three different

categories to fit our clothing-by-customer-sex experiment:

Two-way ANOVA for independent samples

Interpreting Computer Statistical Output We will deal with 2 X 2 analyses in these three different

categories to fit our clothing-by-customer-sex experiment:

Two-way ANOVA for independent samples Two-way ANOVA for correlated samples

Interpreting Computer Statistical Output We will deal with 2 X 2 analyses in these three different

categories to fit our clothing-by-customer-sex experiment:

Two-way ANOVA for independent samples Two-way ANOVA for correlated samples Two-way ANOVA for mixed samples

Two-Way ANOVA for Independent Samples The two-way ANOVA for independent samples requires

that we have two IV’s (clothing style and customer sex) with independent groups.

Two-Way ANOVA for Independent Samples The two-way ANOVA for independent samples requires

that we have two IV’s (clothing style and customer sex) with independent groups.

To create this design we would use four different randomly assigned groups of salesclerks.

Two-Way ANOVA for Independent Samples The two-way ANOVA for independent samples requires

that we have two IV’s (clothing style and customer sex) with independent groups.

To create this design we would use four different randomly assigned groups of salesclerks.

The DV scores represent clerks’ response times in waiting on customers.

Two-Way ANOVA for Independent Samples Source Table

Two-Way ANOVA for Independent Samples Source Table

In the body of the source table, we want to examine only the effects of the two IV’s (clothing and customer sex) and their interaction.

Two-Way ANOVA for Independent Samples Source Table

In the body of the source table, we want to examine only the effects of the two IV’s (clothing and customer sex) and their interaction.

The remaining source (w. cell or Within) is the error term and is used to test the IV effects.

Two-Way ANOVA for Independent Samples Source Table

In the body of the source table, we want to examine only the effects of the two IV’s (clothing and customer sex) and their interaction.

The remaining source (w. cell or Within) is the error term and is used to test the IV effects.

Different statistical programs will use a variety of different names for the error term.

Two-Way ANOVA for Independent Samples Source Table

The effect of sex shows an F ratio of 3.70, with a probability of .07.

This IV shows marginal significance.

Two-Way ANOVA for Independent Samples Marginal significance

Marginal significance refers to statistical results with a probability of chance between 5% and 10% (almost significant but not quite).

Two-Way ANOVA for Independent Samples Marginal significance

Marginal significance refers to statistical results with a probability of chance between 5% and 10% (almost significant but not quite).

Researchers often talk about such results as if they reached the p = .05 level.

Two-Way ANOVA for Independent Samples Marginal significance

Marginal significance refers to statistical results with a probability of chance between 5% and 10% (almost significant but not quite).

Researchers often talk about such results as if they reached the p = .05 level.

Dealing with marginally significant results means you run an increased risk of making a Type I error (accepting the experimental hypothesis when the null hypothesis is true).

Two-Way ANOVA for Independent Samples Source Table

The effect of sex shows an F ratio of 3.70, with a probability of .07.

This IV shows marginal significance. The probability of “clothes” falls below .01 in the table.

Two-Way ANOVA for Independent Samples Source Table

The effect of sex shows an F ratio of 3.70, with a probability of .07.

This IV shows marginal significance. The probability of “clothes” falls below .01 in the table. The interaction between clothing and customer sex

produced an F ratio of 6.65 and has p = .02, therefore denoting significance.

Two-Way ANOVA for Independent Samples Source Table

A significant interaction renders the main effects moot because those main effects are qualified by the interaction and are not straightforward.

Two-Way ANOVA for Independent Samples Source Table

A significant interaction renders the main effects moot because those main effects are qualified by the interaction and are not straightforward.

The first step in interpreting an interaction is to draw a graph of the results from the descriptive statistics (from source table).

Two-Way ANOVA for Independent Samples Source Table

A significant interaction renders the main effects moot because those main effects are qualified by the interaction and are not straightforward.

The first step in interpreting an interaction is to draw a graph of the results from the descriptive statistics (from source table).

Two-Way ANOVA for Independent Samples Crossing lines, in conjunction with the low probability of

chance for the interaction term, denote a significant interaction.

Two-Way ANOVA for Independent Samples Crossing lines, in conjunction with the low probability of

chance for the interaction term, denote a significant interaction.

When we examine the figure, the point that seems to differ most represents the clerks’ response times to male customers in sloppy clothes.

Two-Way ANOVA for Independent Samples Crossing lines, in conjunction with the low probability of

chance for the interaction term, denote a significant interaction.

When we examine the figure, the point that seems to differ most represents the clerks’ response times to male customers in sloppy clothes.

This mean is considerably higher than the others.

Two-Way ANOVA for Independent Samples Thus, we would conclude that clerks take longer to wait

on men who are sloppily dressed than other customers.

Two-Way ANOVA for Independent Samples Thus, we would conclude that clerks take longer to wait

on men who are sloppily dressed than other customers. Notice that our explanation of an interaction effect must

include a reference to both IV’s in order to make sense.

Two-Way ANOVA for Independent Samples If you attempt to interpret the main effects in a

straightforward fashion when you have a significant interaction, you end up trying to make a gray situation into a black-and-white picture.

Two-Way ANOVA for Independent Samples If you attempt to interpret the main effects in a

straightforward fashion when you have a significant interaction, you end up trying to make a gray situation into a black-and-white picture.

In other words, you will be guilty of oversimplifying the results.

Two-Way ANOVA for Independent Samples Here is one way you could present the results from this

experiment:

Two-Way ANOVA for Independent Samples Here is one way you could present the results from this

experiment: The effect of the clothing on the clerks’ response times

was significant, F(1, 20) = 11.92, p = .003. The customer sex effect was marginally significant, F(1, 20) = 3.70, p = .069. However, the main effects were qualified by a significant interaction between clothing and customer sex, F(1, 20) = 6.65, p = .018. The proportion of the variance accounted for by the interaction was 0.25. The results of the interaction are graphed in Figure 1. Visual inspection of the graph shows that clerks’ response times for the sloppy clothes-male customer condition were higher than the other conditions.

Two-Way ANOVA for Correlated Samples The two-way ANOVA for correlated samples requires

that we have two IV’s with correlated groups for both IV’s.

Two-Way ANOVA for Correlated Samples The two-way ANOVA for correlated samples requires

that we have two IV’s with correlated groups for both IV’s.

Most often these correlated groups would be formed by matching or by using repeated measures.

Two-Way ANOVA for Correlated Samples In our example of the clothing-customer sex

experiment, repeated measures on both IV’s would be appropriate -

Two-Way ANOVA for Correlated Samples In our example of the clothing-customer sex

experiment, repeated measures on both IV’s would be appropriate –

We would merely get one sample of salesclerks and have them wait on customers of both sexes wearing each style of clothing.

Two-Way ANOVA for Correlated Samples Computer results

Two-Way ANOVA for Correlated Samples Computer results

The clothing effect is significant at the .001 level and the sex effect is significant at the .014 level.

Two-Way ANOVA for Correlated Samples Computer results

The clothing effect is significant at the .001 level and the sex effect is significant at the .014 level.

However, both main effects are qualified by the significant clothing-by-sex interaction (p = 0.0001).

Two-Way ANOVA for Correlated Samples Computer results

The clothing effect is significant at the .001 level and the sex effect is significant at the .014 level.

However, both main effects are qualified by the significant clothing-by-sex interaction (p = 0.0001).

To make sense of the interaction, we must plot the means for the combinations of clothing and customer sex.

Two-Way ANOVA for Correlated Samples Computer results

However, both main effects are qualified by the significant clothing-by-sex interaction (p = 0.0001).

To make sense of the interaction, we must plot the means for the combinations of clothing and customer sex.

Two-Way ANOVA for Correlated Samples One possible way of summarizing these results follows:

Two-Way ANOVA for Correlated Samples One possible way of summarizing these results follows:

Both the main effects of clothing and customer sex were significant, F(1, 5) = 24.69, p = .001 and F(1, 5) = 7.66, p = .014, respectively. However, the interaction of clothing and customer sex was also significant, F(1, 5) = 13.77, p = .001. The proportion of variance accounted for by the interaction was .78. This interaction appears in Figure 1. Salesclerks waiting on sloppily attired male customers were considerably slower than clerks with any other combination of customer sex and clothing.

Two-Way ANOVA for Correlated Samples One possible way of summarizing these results follows:

Both the main effects of clothing and customer sex were significant, F(1, 5) = 24.69, p = .001 and F(1, 5) = 7.66, p = .014, respectively. However, the interaction of clothing and customer sex was also significant, F(1, 5) = 13.77, p = .001. The proportion of variance accounted for by the interaction was .78. This interaction appears in Figure 1. Salesclerks waiting on sloppily attired male customers were considerably slower than clerks with any other combination of customer sex and clothing.

You would provide a fuller explanation and interpretation of this interaction in the discussion section of your experimental report.

Two-Way ANOVA for Mixed Samples The two-way ANOVA for mixed samples requires that we

have two IV’s with independent groups for one IV and correlated groups for the second IV.

Two-Way ANOVA for Mixed Samples The two-way ANOVA for mixed samples requires that we

have two IV’s with independent groups for one IV and correlated groups for the second IV.

One possible way to create this design in our clothing-customer sex experiment would be to use a different randomly assigned group of salesclerks for each customer sex.

Two-Way ANOVA for Mixed Samples The two-way ANOVA for mixed samples requires that we

have two IV’s with independent groups for one IV and correlated groups for the second IV.

One possible way to create this design in our clothing-customer sex experiment would be to use a different randomly assigned group of salesclerks for each customer sex.

Clerks waiting on each sex, however, would assist customers attired in both types of clothing.

Two-Way ANOVA for Mixed Samples Computer Results

Once again, the descriptive statistics did not change from our first and second analysis -

Two-Way ANOVA for Mixed Samples Computer Results

Once again, the descriptive statistics did not change from our first and second analysis –

We are still analyzing the same data.

Two-Way ANOVA for Mixed Samples Source Table

The source table appears at the bottom of Table 11-4 in your text.

As you can see from the headings, the between-subjects effects (independent groups) and the within-subjects effects (repeated measures) are divided in the source table.

Two-Way ANOVA for Mixed Samples Source Table

The source table appears at the bottom of Table 11-4 in your text.

As you can see from the headings, the between-subjects effects (independent groups) and the within-subjects effects (repeated measures) are divided in the source table.

This division is necessary because the between-subjects effects and the within-subjects effects use different error terms.

Two-Way ANOVA for Mixed Samples Source Table

This division is necessary because the between-subjects effects and the within-subjects effects use different error terms.

The interaction appears in the within-subjects portion of the table because it involves repeated measures across one of the variables involved.

Two-Way ANOVA for Mixed Samples Here’s one possibility for communicating the results of

this study in APA format:

Two-Way ANOVA for Mixed Samples Here’s one possibility for communicating the results of

this study in APA format: Results from the mixed factorial ANOVA showed no effect

on the customer sex, F(1, 10) = 2.42, p = .15. The clothing effect was significant, F(1, 10) = 25.21, p = .001. This main effect, however, was qualified by a significant customer-sex-by-clothing interaction, F(1, 10) = 14.06, p = .004. The proportion of variance accounted for by the significant interaction was .58. This interaction is shown in Figure 1, indicating that salesclerks who waited on sloppily dressed male customers were slower in responding than clerks who waited on casually dressed men or women dressed in either manner.

A Final Note Assuming that a significant main effect is not qualified

by an interaction, you need to calculate a set of post hoc tests to determine exactly where the significance of that IV occurred.

The Continuing Research Problem Pursuing a line of programmatic research is

challenging, invigorating, and interesting.

The Continuing Research Problem Pursuing a line of programmatic research is

challenging, invigorating, and interesting. Programmatic research refers to a series of experiments

that deal with a related topic or question.

The Continuing Research Problem Pursuing a line of programmatic research is

challenging, invigorating, and interesting. Programmatic research refers to a series of experiments

that deal with a related topic or question. Remember that pursuing such a line of research is how

most famous psychologists have made names for themselves.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: After our preliminary research in chapters 9 and 10, we

decided to use two IV’s (clothing and customer sex) in these experiments.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: After our preliminary research in chapters 9 and 10, we

decided to use two IV’s (clothing and customer sex) in these experiments.

Each IV had two levels (clothing casual, sloppy; customer sex men, women).

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: After our preliminary research in chapters 9 and 10, we

decided to use two IV’s (clothing and customer sex) in these experiments.

Each IV had two levels (clothing casual, sloppy; customer sex men, women).

This design allows us to determine the effects of the clothing, the effects of the customer sex, and the interaction between clothing and customer sex.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: The DV was the time it took salesclerks to respond to

customers.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: With large numbers of clerks, we randomly formed four

groups of clerks, with each waiting on one sex of customer in one type of clothing, resulting in a factorial between-groups design.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: With large numbers of clerks, we randomly formed four

groups of clerks, with each waiting on one sex of customer in one type of clothing, resulting in a factorial between-groups design.

We analyzed the response times using a factorial ANOVA for independent groups and found that clerks were slower to wait on male customers in sloppy clothing than all other customers.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: In a hypothetical situation with fewer clerks for the

experiment, we used repeated measures on both IV’s; that is, each salesclerk waited on both sexes of customers attired in both types of clothing, so that each clerk waited on four different customers.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: In a hypothetical situation with fewer clerks for the

experiment, we used repeated measures on both IV’s; that is, each salesclerk waited on both sexes of customers attired in both types of clothing, so that each clerk waited on four different customers.

Thus, this experiment used a factorial within-groups design.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: In a hypothetical situation with fewer clerks for the

experiment, we used repeated measures on both IV’s; that is, each salesclerk waited on both sexes of customers attired in both types of clothing, so that each clerk waited on four different customers.

Thus, this experiment used a factorial within-groups design.

We analyzed the data with a factorial ANOVA for correlated groups and found that clerks were slowest in waiting on sloppily dressed men.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: In a third hypothetical situation, we randomly assigned

salesclerks to the two customer sex groups but used repeated measures on the clothing IV so that clerks waited either on men in both types of clothing or women in both types of clothing.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: In a third hypothetical situation, we randomly assigned

salesclerks to the two customer sex groups but used repeated measures on the clothing IV so that clerks waited either on men in both types of clothing or women in both types of clothing.

This arrangement resulted in a factorial mixed-groups design (one IV using independent groups, one using correlated groups).

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: In a third hypothetical situation, we randomly assigned

salesclerks to the two customer sex groups but used repeated measures on the clothing IV so that clerks waited either on men in both types of clothing or women in both types of clothing.

This arrangement resulted in a factorial mixed-groups design (one IV using independent groups, one using correlated groups).

We analyzed the response times with a factorial ANOVA for mixed groups and found the slowest response times to male customers in sloppy clothes (see Table 11-4 and Figure 11-13).

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: We concluded that clothing and customer sex interacted to

affect salesclerks’ response times.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: We concluded that clothing and customer sex interacted to

affect salesclerks’ response times. Women received help quickly regardless of their attire, but

men received help quickly only if they were not sloppily dressed.

The Continuing Research Problem Let’s review the steps we took in designing the

experiments in this chapter: We concluded that clothing and customer sex interacted to

affect salesclerks’ response times. Women received help quickly regardless of their attire, but

men received help quickly only if they were not sloppily dressed.

Men attired in sloppy clothes had to wait longer for help than the other three groups.