Elements of Educational Research for the Academic Physician

Rebecca Henry, Ph.D.OMERAD

College of Human MedicineMichigan State University

Professional Roles In A Specialty

Practitioners

Teachers

Researchers

Skills Required of Research-focused Faculty

Content Knowledge• Uses a range of information searching tools (e.g., MEDLINE)• Critically appraises a variety of literature• Synthesizes theory into own research• Becomes expert in a body of knowledge• Informed about ethical conduct of researchMethodological Skills• Formulates researchable questions• Develops valid and reliable measures• Understands threats to internal and external validity• Understands at an advanced level the statistics relevant to one’s own research• Uses design and statistical consultants• Reports results accurately and can cite strengths and limitations of the study• Integrates findings into the existing literature

Skills Required of Research-focused Faculty (continued)

Management Skills• Develops research management plans• Prepares and submits required reports and other administrative documents• Adheres to university guidelines and regulations regarding the conduct of research• Locates appropriate funding sourcesSocialization• Establishes professional goals and priorities for a research career• Understands the promotion environment • Links to networks and associations• Identifies research mentors• Manages and balances competing professional obligations to achieve research goals.

Essential Skills of Teacher/Clinician and Teacher/Administrator

• Values the use of new knowledge as it applies to patient care

• Determines when the research literature can be used as an effective teaching tool

• Role models clinical practices based on evidence and not on tradition

• Incorporates evidence-based medicine into teaching strategies

• Interprets conflicting study results and their implications for patient care decisions

Skills Required of Teacher/Preceptors

• Critically assesses research literature, especially that relevant to patient care.• Is aware of issues emerging from discussions of practice guidelines, protocols and standards of care.• Articulates to students one’s own rationale for a particular practice strategy.• Participates in societies, conferences, and continuing education programs.• Models lifelong learning.• Is self-critiquing, especially when faced with challenging clinical problems.

Why Do We Engage in Research

• Satisfy Curiosity

• Describe Phenomena

• Evaluate Hypotheses

• Evaluate New Techniques or Methods

Types of Research Questions

Is there a relationship?Between the Henry Dexterity Assessment and the Clerkship OSCE?Between MCAT and our shelf exam?

Is there a difference?PBL vs Lecture?Direct observation vs simulated patient?

Can you predict?Who will fail the inservice exam? Who will practice in rural settings?

Can you describe? What are the most frequent diagnoses in a university based clinic?What is charted on domestic violence?

Most questions ultimately target:

Comparisons between groups on a dependent variable.“Does a mentoring program decrease drop-out rate for first year residents?”

Relationships among independent and dependent variables.Students performance on the Henry Dexterity Assessment is related to scores on the surgery OSCE.

Descriptions of responses to independent or dependent variables.“How do residents rate on problem solving skills?”“What are the practice choices of graduate general surgery residents?”

Developing Researchable Questions

Not Researchable

• Should I implement a PBL curriculum?

• What is the best way to learn suturing?

• Are some students unable to pass Step 1?

Researchable

• Does PBL or lecture have better test scores

• Does practice reduce errors in suturing?

• Will MCAT scores predict Step1 failures?

Dimensions of Research Design

I. Objective of the Study/Nature of the Question

ExploratoryNeed More Information to Sharpen QuestionCan’t Measure Variables

DescriptiveSeeks to Characterize Sets of VariablesAbout People or Phenomena

AnalyticSeeks to Determine Relationships(ultimately casual) Among Variables

Dimensions of Research Design (cont)

II. The Timeframe Under Investigation

RetrospectiveExamines Background of Residents Who Select Primary Care

Cross SectionalA “one shot” Survey Measuring a Variables(s) at one point in time

ProspectiveBegin in the Present and Follow Subjects Forward in Time

Dimensions of Research Design (cont)

III. Does the Investigator Influence an Outcome?

Observation

Watch and RECORD Information

Intervention

Introduces Some Agent to Impact an Outcome

Natural Experiments

Curriculum Reform

DESCRIPTIVECase SeriesEthnographiesNeeds Assessments

ANALYTIC

OBSERVATIONALCohortCase ControlCross Sectional

GOAL: Select a Design Which Can Give an Answer to Your Question in the Most PRACTICAL Way.

RESEARCH DESIGNS

INTERVENTIONALRandomizedControlledExperimentsEducational/Health CareExperiments(Quasi experimental)

Document and communicate experience:share ideas, programs, treatments,unusual events and observations

Begin search for explanations

Examples:

Case report or series• Rash developing while on drug• Cluster of cases of vaginal cancer

Clinical series• Treatment of 50 hernias by laparoscope technique

Population• Diagnosis seen in family practice• Community survey of needs of elderly

Basic Study DesignsDESCRIPTIVE

EXPERIMENTAL

EXPLANATORY

OBSERVATIONAL

Examine etiology, cause, efficacy using the strategy of comparisons

Seek causes, etiologies predictors, better diagnosis

Investigator observes nature

Examples:

Clinical trial• Compare two antidepressant drugs• Surgical vs. medical management of angina

Educational intervention• Self-Instruction vs. lecture on anemia

Health-care trial• Nurse practitioner vs. physician care

Examples:

Case Control• Diets of toxemic vs. nontoxemic patients

Follow-up• Development of surgical complications of inguinal hernias

Cross-sectional• Prevalence of dental caries in bottle fed children

Evaluate efficacy of therapeutic, educational, administrative interventions.

Investigator controls allocation

Levels of Evidence

Type 1 Evidence

• Randomized Controlled Trial

• Quasi-Randomized Controlled Trial

Type 2 Evidence• Cohort Study

• Before - After Study

Type 3 Evidence• Case Control

• Cross-Sectional

Type 4 Evidence• Descriptive Study

Characteristics of 353 Research Articles on Undergraduate Medical Education, 1975-1994*

Research environment Controlled Naturalistic Data not provided/not applicable

Research strategy Evaluation Comparison Survey Qualitative Experiment Other

Data collection method Observation/testing Questionnaire Historical analysis Interview Discussion

22 (6.2)245 (69.4)

86 (24.4)

135 (38.2)80 (22.7)52 (14.7)42 (11.9)29 (8.2)15 (4.3)

237 (67.1)97 (27.5)13 (3.7)

4 (1.1)2 (0.6)

CharacteristicsNo. of

Research Articles (%)

Characteristics of 353 Research Articles on Undergraduate Medical Education, 1975-1994*

Data analysis method Inferential Quantitative Non-quantitative Analytic Other

Report of external funding No funding reported Yes, non-U.S. government Yes, U.S. Public Health Service Yes, other U.S. government

200 (56.7)103 (29.2)46 (13.0)1 (0.3)3 (0.9)

273 (77.3)49 (13.9)25 (7.1)

6 (1.7)

CharacteristicsNo. of

Research Articles (%)

* Of a randomly selected sample of 773 articles, 353 were classified as reports of research activity (I.e., they used specific methods to ascertain new facts, concepts, or ideas).