Research Errors

Hashem Alhashemi

Objectives

• Hypothesis

• Hypothesis testing

• Types of error

• Confounding

• Bias

Hypothesis

An expected answer to a question

Do you have to start with a hypothesis all the time?

No

If you do not have enough background Info

Null hypothesis

A defendant is innocent until proven guilty

No relationship identified between variables tested

Alternative hypothesis

The opposite of Null hypothesis (your expectation).

Without torturing the Defendant (The Data).

𝛂 error/Type 1 error (P value)

• Set by agreement at 5%.

• To minimize the Probability of chance being the explanation for the findings.

• When Null hypothesis is true, we would tolerate the mistake of rejecting it 5 times, if the trial was repeated 100 times.

• The American error: America is Number 1.

• Finding a difference when there is no difference.

Confidence interval or margin of error (1-𝛂):

• The result of that is a 95% confidence interval.

• Means that if the trial was repeated 100 times the estimate would fall in the confidence range 95 times.

• The margin is a result of measurement error (what/How).

Hypothesis Testing

No Difference ?

Difference ! Reject null when True Find a Diff when ɸ Diff Type I error (American) (α) 0.05 due to Chance

Clinical Trial

No Difference Accept null when True Find ɸ Diff when ɸ Diff Confidence (1-α) 95% Truth CI 95%

Evidence of + effect True Effect =

Type 1 error

5%

?

𝛃 error/Type 2 error:

• Set by agreement at 20%.

• The British error: Blinding people about the difference when it actually exists (Palestinians').

Power of the study (1-𝛃):

• 𝛃 : When the alternative hypothesis is true, we would tolerate the mistake of rejecting it 20% of the time, if the trial was repeated 100 times.

• (1-𝛃): Means that if the trial was repeated 100 times the difference/effect would be found 80 times.

Difference

Hypothesis Testing

? Clinical Trial

No Difference ! Accept Null when False Find φ Diff when a Diff Type II Error (British) B = 20% (? Design)

Reject Null when False Find a Diff when a Diff Truth 1-B = 80% (Power)

Difference

lack of evidence lack of effect =

For thousands of years we did not have an evidence

about bacteria

(𝛃 error) Type 2 error

Lack of evidence did not prevent bacteria from killing

millions

• As you can see we are less tolerant to 𝛂 error because it may cause people to change their practice to a new way that is not superior (harmful?).

• On the other hand, we tolerate 𝛃 error more because its real side-effect is more tolerable than 𝛂 error.

• 𝛃 side effect is that the investigator may need to repeat the experiment under better conditions like larger sample size, or strict exclusion criteria.

𝛂 or 𝛃

Hypothesis Testing

No Difference ?

Difference ! Reject null when True Find a Diff when ɸ Diff Type I error (American) (α) 0.05 due to Chance

Clinical Trial

No Difference Accept null when True Find ɸ Diff when ɸ Diff Confidence (1-α) 95% Truth CI 95%

? Clinical Trial

No Difference ! Accept Null when False Find φ Diff when a Diff Type II Error (British) B = 20% (? Design)

Reject Null when False Find a Diff when a Diff Truth 1-B = 80% (Power)

Difference

Risk Matrix

Difference

Confounding

Is over estimation or under estimation of the relationship between two variables as a result of their relation to a third variable (confounder).

Confounding

C (Confounder i.e. smoking)

(Exposure i.e. coffee) E -- -- -- -- -- -- -- -- -- -- D (Disease i.e. lung cancer)

Confounding causes bias; however, bias is not limited to confounding.

Confounding in Reality

• We can adjust for it in the design and/or analysis.

• Design : Randomization, matching, & or restriction.

• Analysis : stratified analysis (MH), or multivariate analysis (Regression).

Unknown Confounders

known Confounders

Bias

Wrong/Invalid Conclusion

Bias

Any factor that can invalidate your conclusion by giving an alternative explanation for the results.

It can be due chance (external) or due systematic/design error (internal).

Bias

• an oblique or diagonal line of direction, especially across a woven fabric.

• a tendency or inclination, that prevents

someone from having a right judgment

(prejudice/wrong judgment).

Bias 101

Straight vs bias The bias runs at an angle to the straight and crossgrains. The true bias running at an exact 45-degree angle

Bias Bias

Truth

Bias Bias

Truth

Selection/sampling, Misclassification , Recall, Pt Response, Observer/assessor, loss of F/U, Publication, Language

Type of Bias

Misclassification Proper Case definition

Selection/sampling Randomization/ Allocation Concealment

Confounding Randomization& Regression

Patient Response Single blinding

Observer/assessor Double blinding

Crossover/contamination Intention to treat analysis

Recall Previous records & Documents

loss of F/U Visits, phone calls, e-mail

Questionnaire/interviewer Construct, pilot, & training

Non-response Restrict your population &

minimize non-response rate

Method of Prevention

Bias Prevention

Central Agency Next allocation

unknown Equal chance

each allocation

Regardless of intervention received Analysis by intention

not intervention

Bias

• General and design specific causes.

• Can not be accepted nor adjusted for.

• Better Avoided/prevented at any cost.

• Can happen in any stage of the study:

Design &, or conduct of the study (Pt selection,

Data Collection/entry).

Conclusion

Research Errors

• Type 1 & 2 we accept them as game roles from step one (sample size calculation).

• Confounding we can adjust/control for it in the design and/or analysis.

• Any thing else (we can not accept nor adjust for) is Bias. It should be avoided/prevented at any cost.

END

Conclusion

Sample size estimation

Statistical consideration

• Prevalence/treatment effect (from literature review)

• Maximum/margin of error

• Type I error 𝛂

• Type II error 𝛃

• Design effect

Practical consideration

• Personnel

• Resources

Being Wrong

Basics in Epidemiology & Biostatistics

Hashem Alhashemi MD, MPH, FRCPC Assistant Professor, KSAU-HS