stats present sample size

7/29/2019 Stats Present Sample Size

1/38

Sample Size and PowerCalculations

Marcia A. Ciol

04/09/08


2/38

What resources do I need? How long will it take to conduct the study?

I need 50 participants in my study

About 5 individuals per year will be enrolled Therefore, it will take 10 years to finish the study

How much money do I need? I will follow a cohort of 500 individuals

A lab test that costs US$100 will be conducted foreach person

Therefore, I will need US$50,000 just for labtests


3/38

Am I going to reach my

objective? I have 2 years to finish my thesis, of

which one year is for data collection

I think I can get data on 50 people inthat year

Is 50 a sufficient number of people to

test my hypothesis with the significancelevel I want?


4/38

Why to calculate sample size

and power? To show that under certain conditions, the

hypothesis test has a good chance ofshowing a desired difference (if it exists)

To show to the funding agency that the studyhas a reasonable chance to obtain aconclusive result

To show that the necessary resources(human, monetary, time) will be minimizedand well utilized


5/38

What do I need to know to

calculate sample size? Most Important: sample size calculation is

an educated guess

It is appropriate for studies involvinghypothesis testing

There is no magic involved; only statisticaland mathematical logic and some algebra

Researchers need to know something aboutwhat they are measuring and how it variesin the population of interest


6/38

Factors related to the

sample size Population factor (cannot be controlled by

researcher)

Characteristics of the study design

Quantities related to the research question(defined by the researcher)


7/38

Where do we get this

knowledge? Previous published studies

Pilot studies

If information is lacking, there is nogood way to calculate the sample size!


8/38

Population factor Variance of the measure (outcome)

within the population


9/38

0.00

0.02

0.04

0.06

0.08

0.10

x-20 -10 0 10 20 30


10/38

0.00

0.02

0.04

0.06

0.08

0.10

x-20 -10 0 10 20 30


11/38

Study DesignType of response variable or outcome

Number of groups to be compared

Specific study designType of statistical analysis

In conjunction with the research question, thetype of outcome and study design willdetermine the statistical method of analysis


12/38

Quantities related to the research

question (defined by the researcher)

= Probability of rejecting H0 when H0 is true is called significance level of the test = Probability of not rejecting H0 when H0 isfalse

1- is called statistical powerof the test


13/38

Quantities related to the research

question (defined by the researcher) Size of the measure of interest to be detected

Difference between two or more means

Odds ratio Change in R2, etc

The magnitude of these values depend on

the research question and objective of thestudy (for example, clinical relevance)


14/38

Example: test of difference of

means in two populations Researcherfixes probabilities of type I and II

errors

Prob (type I error) = Prob (reject H0 when H0 is true) = Smaller error greater precision need more information

need larger sample size

Prob (type II error) = Prob (dont reject H0 when H0 isfalse) =

Power =1- More power smaller error need larger sample size


15/38

Example: test of difference of

means in two populations The equation for sample size is derived from

the equation for the statistical test

In a t-test the equation for the test ist = (x1 - x2) - (m1- m2)

(s12n + s2

2n)12

The derived equation for sample size is

n = (z1-/2+ z1-)2(s1

2+ s22)

(m1- m2)2


16/38

Using PASS: t-test example Question: does exercise help to decrease

body weight?

Study design: participants will be randomizedinto two groups (exercise and control)

Outcome: change in weight

Want to detect: a change of at least 15

pounds Known: from past studies, the standard

deviation varies between 10 and 15 pounds.


17/38


18/38


19/38


20/38


21/38

Example: One-way ANOVA

Number of Groups: 4Hypothesized means: 35, 20, 25, 18 (possibly

from a pilot study)Sample size pattern: same number in eachgroupSD of subjects: 18 (from a previous study)

= 0.01 and 0.05Find power for sample sizes from 5 to 30 pergroup (increments of 5)


22/38


23/38


24/38


25/38

Example: Linear Multiple

RegressionResearch Question: is depression score an importantfactor in explaining pain ratings, after adjusting for ageand sex?

Statistical question: does adding depression scoreincrease the explained variation of pain ratings, in alinear regression model that already has age and sex in

it and has R2 =.2?

Suppose I may have sample sizes of 20, 30, 50, 70,and 100. What is the minimum R2 change I can detect

with power .8?


26/38


27/38


28/38

Other Types of Hypothesis

Tests Different methods of data analysis

require different input for sample size

calculations


29/38

Cox Regression (Survival

analysis)


30/38

Logistic Regression


31/38

Repeated measures


32/38

Simple designs may not

require complex calculations Read chapter 2 ofStatistical Rules of

Thumb, by Gerald van Belle (2002,

John Wiley and Sons)

Using specialized software is useful if

many calculations will be performed


33/38

Important to remember Pilot studies do not need sample size

calculation!!!

There is no point in doing power analysis afterthe study is done

Sample size is an educated guess, and itworks only if: The study samples comes from the same or similar

populations to the pilot study populations

The population of interest is not changing over time

The difference or association being studied exists


34/38

How about Effect Size? Most common definition

E = m1

- m2

spooled

If we change de value of E, how do weknow what we changed in the formula?


35/38

Some situations I have

encountered Question: How many more people do I need to

enroll in the study (already in progress) to show

statistical significance? Answer: It depends If the two populations

have the same mean, increasing the samplesize will not help!

Since when is the objective of a study to find astatistically significant result??


36/38


encountered Researcher is interested in outcome A, which

differs very little for two treatments

Sample size needed is around 3000!!

Researchers changes the outcome to B, wheresample size is smaller

B does not answer the researchers question

and he needs to accept that his new treatmentis not really different (clinically speaking) fromthe already existent treatment


37/38


encountered Researcher is interested in comparing

two groups regarding prediction of

outcome A by using a regressionanalysis (using several variables)

He uses the only available formula fromhis statistical book (for a t-test)

Wrong! He should find a software thatcan calculate the sample sizeappropriately


38/38

Summary Define research question well

Consider study design, type of response

variable, and type of data analysis Decide on the type of difference or change you

want to detect (make sure it answers yourresearch question)

Choose and Use appropriate equation sample size

calculation

stats present sample size

Documents