What is Important in Study Design? And Why?

Roxy Peck

Cal Poly San Luis Obispo

rpeck@calpoly.edu

Study Design in the Common Core

7.SP.1

Understand that statistics can be used to gain information about a population by examining a sample of the population; generalizations about a population from a sample are valid only if the sample is representative of that population. Understand that random sampling tends to produce representative samples and support valid inferences.

S-IC.3

Recognize the purposes of and differences among sample surveys, experiments, and observational studies; explain how randomization relates to each.

S-IC.6

Evaluate reports based on data.

Important Theme

Our ability to draw conclusions from data depends on how the data are collected.

Two Types of Statistical Studies Observational Studies

Purpose: To learn about a population

Requires: Representative sample

To the Rescue: Random selection

Experiments

Purpose: To learn about the effect of an explanatory variable on some response variable

Requires: Comparable groups

To the Rescue: Random assignment

Why do we???

Why do we intentionally introduce “randomness” when we collect data?

Drawing conclusions in statistical studies usually involves answering the question

“Could this have happened by chance?”

When we introduce randomness in a planned way…

We know what “just by chance” looks like!

This allows us to recognize results that are “unusual” if chance is the only thing going on.

Random Selection in Sampling

Goal: Representative sample

Activity: Show Me the Money

(Thanks to Doug Tyson for sharing this activity and the next activity.)

Show Me the Money

Population: Top 200 Movies from 2011

1. Pick 10 movies from the list that you saw or wanted to see.

2. Record the title and the gross income for each of these 10 movies.

3. Calculate the mean for this sample of 10 movies.

Discussion Questions

1. Did everyone get the same sample mean?

2. Why?

Sampling variability: Different samples tend to produce different results.

Dotplot of Sample Means

Question

Is the way you selected the sample of 10 movies a good method?

Do you think that it resulted in a sample that is representative of the population?

Random Sampling

1. Use your calculator to generate 10 random numbers between 1 and 200.

2. Record the title and gross income for the 10 movies corresponding to these random numbers.

3. Calculate the mean for this sample of movies.

Dotplot of Sample Means from Random Samples

Discussion

And the point of this activity is…

Random Assignment in Experiments

Goal: Comparable groups

Activity: Does SpongeBob Make Kids Dumber?

Reference: “The Immediate Impact of Different Types of Television on Young Children’s Executive Function”, Pediatrics, 2011.

The Experiment Assigned 60 children to one of three groups:

1. Cartoon (SpongeBob)

2. Educational cartoon

3. Drawing After 9 minutes, children were given several

tests that measured executive function (self-regulation, working memory).

Findings: SpongeBob groups scored significantly lower than the other two groups.

Does it Matter??

Does it matter how the groups are created?

Let’s see. Group of 14 children from a list that is in

alphabetical order. Information for each subject—name, IQ,

gender

Creating groups

Create 2 groups SpongeBob Drawing

Put first seven children into SpongeBob group. What do you notice?

Why might this be a problem?

Now Try Random Assignment

1. Mix the cards and then divide into two groups of seven.

2. For each group, calculate the mean IQ and the proportion of females.

3. Calculate the difference in mean IQ for the two groups.

Dotplot of Difference in Mean IQ

So What is the Point?

What can we conclude about random assignment?

Implications for Drawing Conclusions from Statistical Studies

Random Selection Produces representative samples Allows us to generalize to population

Random Assignment Produces comparable groups Allows us to conclude that treatment is

cause of observed effect

Thank You!

Thank you for your participation in this session.

For copies of the slides, email to

rpeck@calpoly.edu