Scholarship in Scholarship in EducationEducation

Rose van Zuilen, PhDRose van Zuilen, PhD

Director of Faculty DevelopmentDirector of Faculty Development

Co-Director of Longitudinal Curriculum in Co-Director of Longitudinal Curriculum in Geriatrics, Pain Management, and Geriatrics, Pain Management, and

Palliative CarePalliative Care

Division of Gerontology and Geriatric Division of Gerontology and Geriatric MedicineMedicine

University of Miami Miller School of University of Miami Miller School of MedicineMedicine

Learning ObjectivesLearning ObjectivesDefine educational scholarship and

distinguish it from other forms of scholarship

Describe a broad range of activities that constitute educational scholarship

Determine existing outlets to publish educational products

Utilize effective ways to capture and document the impact and quality of your scholarly work

Department of Medicine, Thomas Jefferson University

Scholarship of

Research

Clinical CareTeaching

Typical Model of Scholarship

Service

Academic Advancement is often Academic Advancement is often Slower for Clinician EducatorsSlower for Clinician Educators

Study at Johns Hopkins School of Medicine

A single track for promotionsThe odds of being at a higher rank were

85% less for academic clinicians and 69% less for teacher clinicians than for basic researchers

Rigor of promotion process affected by the paucity of valid evaluation methods

Thomas PA et al.(2004) Acad Med 79(3): 258‐264

Expanding the Definition of Expanding the Definition of ScholarshipScholarship

“What we need, then, in higher education is a reward system that reflects the diversity of our institutions and the breadth of scholarship, as well. The challenge is to strike a balance among teaching, research, and service, a position supported by two-thirds of today’s faculty who conclude that, ‘‘at my institution, we need better ways, besides publication, to evaluate scholarly performance of faculty.’’ Boyer, 1990

Expanding the Definition of Expanding the Definition of ScholarshipScholarship

Discovery Build new knowledge through traditional research

Integration

Make connections across disciplines and place specialties in a larger context

Application

The bridge between theory and practical use. Aid society and professions in addressing problems

Teaching Effective communication of knowledge to learners

Glassick’s Scholarship Glassick’s Scholarship AssessedAssessed1) Clear goals – the educator explicitly states the

basic purposes for the work, and defines realistic, achievable objectives, including desired goals and outcomes.

2) Adequate preparation – the educator shows an understanding of relevant existing scholarship and has skills and resources drawn from this research and from prior experience to advance the project.

3) Appropriate methods – in conjunction with the material and the context, the educator chooses, applies and, if necessary, modifies methods wisely.

Glassick’s Scholarship Glassick’s Scholarship AssessedAssessed4) Significant results – the educator achieves

the goals, and contributes notably to the field in a manner that invites further exploration.

5) Effective presentation – the educator uses a suitable style and organization to present the work with clarity and integrity in appropriate forums to reach the intended audience.

6) Reflective critique – the educator thoughtfully assesses the work him/herself and uses the resulting perceptions, along with reviews and critique from others, to refine, enhance or expand the original concept.

Association of American Medical Association of American Medical Colleges Consensus ConferenceColleges Consensus ConferenceGroup on Educational

Affairs (GEA) met in 2006

Criteria for educational scholarship

Five categories◦ Teaching◦ Learner assessment◦ Curriculum development◦ Mentoring and advising◦ Educational leadership and

administration

Issues Addressed by the GEAIssues Addressed by the GEA1. What educator activities should be

included in each category?2. What types of evidence should be

included for review in the academic promotion process?

3. How should a faculty member present the activities and associated evidence in a promotion/educator portfolio?

Issues Addressed by the GEAIssues Addressed by the GEAEducators’ contributions can be judged

through the effective presentation of evidence pertaining to the quantity, quality, and engagement with the educational community

Evidence of such engagements requires educators to draw upon the literature and best practices (scholarly approach) or contribute to the medical education field (scholarship).

Category 1 - Teaching Category 1 - Teaching Any activity that fosters learning,

including direct teaching and the creation of associated learning materials

Lectures, small groups, precepting, role modeling, formative assessments and feedback, online courses

Creation of learner materials strictly for enhancing one’s own teaching

Teaching Teaching

Documenting QuantityDocument the frequency and duration

of teaching with a description of one’s role at all levels (UME, GME, CME)

Provide a list of instructional materials authored with a brief description of their purpose, format, and length

Teaching Teaching

Documenting QualityLearner’s evaluations of teaching

performancePeer evaluations of teaching Teaching awards and honors

◦Describe the selection process and criteriaEvidence of learning

◦Pre-post assessments of learner performance◦Self-reported learning

Teaching Teaching Documenting a Scholarly ApproachDescribe how one’s teaching is informed by

the literature (e.g., competence based education)

Provide a comparative analysis of one’s own teaching and teaching materials with “best practices” in the field

Reflect critically on one’s own teaching and on feedback received by others noting the impact on subsequent teaching activities

Document participation in personal learning

Teaching Teaching

Documenting ScholarshipEvaluations from conference

presentationsInclusion of educational products in a peer-reviewed repository

Data demonstrating adoption by other faculty

References or citations to the product in other peer-reviewed materials

Category 2 - Learner Category 2 - Learner Assessment Assessment All activities associated with measuring

learner’s knowledge, skills and attitudesDevelopment: Identifying and creating

assessment process and toolsImplementation: Collecting dataAnalysis: Comparing data with correct

answer key or performance standardsSynthesis and interpretation:

Interpreting and reporting data to stakeholders

Learner AssessmentLearner Assessment

Documenting QuantityProvide information about the size and

scope of the assessment activity, one’s role, and the number and level of learners assessed

Documenting Quality and a Scholarly Approach

Provide evidence that the assessment tool is reliable, valid and informed by the existing educational measurement field.

Learner Assessment Learner Assessment Documenting ScholarshipPresentations on the assessment process

or outcomes to local audiencesPeer reviewed or invited presentations and

workshops at professional meetingsAcceptance of the assessment tool in a

peer-reviewed repositoryAssessment research presented at

national conferences or publishedInvitations to review assessment tools

Category 3 – Category 3 – CurriculumCurriculum DevelopmentDevelopmentCurriculum is defined as a longitudinal

set of designed educational activities that includes evaluation

More than one teaching session or presentation

Can occur at any level of trainingBasic science course, clerkship, theme,

rotation, fellowship, CME course

CurriculumCurriculum Development DevelopmentDocumenting Quantity and QualityFor each curricular piece authored, provide

a description of the purpose, intended audience, duration, design and evaluation.

Note specific contributions (if co-authored)Provide learner reactions and ratingsShow outcomes with graphic displays of

improvements over time when possible◦Course examinations, NBME subject scores,

OSCEs, performance observations

CurriculumCurriculum Development Development

Documenting a Scholarly ApproachDemonstrate that the curriculum

design was informed by the literature and “best practices”◦a curriculum designed to target nationally

accepted competencies◦The use of a well-accepted approach to

curriculum development (e.g., ADDIE)

CurriculumCurriculum Development DevelopmentDocumenting ScholarshipPeer review at the local level (curriculum

committee, accreditation reviewers)Peer reviewed or invited presentationsAcceptance of curricular material to a peer-

reviewed repositoryList of institutions that adapted the

curriculumRequests to provide consultation Invitations to conduct peer-reviews

Peer Reviews – Getting Peer Reviews – Getting InvolvedInvolvedPoster sessions (local and national)

◦Opportunities through colleagues and societiesConference abstracts and papers

◦Invited as a society member, referral, and volunteer

Journal articles◦Published author or recommended content expert◦Solicited or unsolicited applications

Educational products (e.g., MedEdPORTAL)◦Solicited applications … so, volunteer

Grants and awards

Things to Consider Before Things to Consider Before Signing Up as a ReviewerSigning Up as a Reviewer

Make sure the journal is a good fitHow many manuscripts does the

journal expect you to review – in what time period?

Is there a specific format for reviewsDo reviewers receive feedback?Is the review system open or closed

Sylvia LM & Herbel JL.(2001) Pharmacotherapy, 21(4)

Category 4 – Mentoring and Category 4 – Mentoring and AdvisingAdvisingEducators often serve as mentors and

advisors for their trainees and colleaguesMentoring implies a sustained committed

relationship from which both parties obtain reciprocal benefit, whereas advising is usually more time limited and one directional

Documentation should describe the nature of the relationship and one’s effectiveness in helping mentees and advisees meet their goals

Mentoring and AdvisingMentoring and AdvisingDocumenting Quantity and QualityProvide a list of mentees with their position

and time invested (duration, hours)List mentees significant accomplishments

◦Publications, presentations, grants, awardsStandardized evaluation of advising

effectivenessNarrative comments or letters from

mentees◦Solicited or unsolicited – Keep records

Mentoring and AdvisingMentoring and Advising

Documenting a Scholarly ApproachParticipation in professional developmentUse of mentoring strategies informed by

literatureDesign of a program guided by current

evidenceLeading of initiatives to improve

institutional mentoring and advising practices

Mentoring and AdvisingMentoring and AdvisingDocumenting ScholarshipSecure competitive funding for program

development or innovative mentoring projectsConduct skills enhancement training Publish peer-reviewed mentoring guidesReceive mentoring awardsConvene a scholarly conference on mentoringInitiate a faculty learning community on

mentoringServe as a mentoring consultantConduct mentoring research

Category 5 – Educational Category 5 – Educational Leadership and AdministrationLeadership and AdministrationExceptional educational administrators

and leaders achieve transform organizations in their pursuit of excellence

Their work's value is demonstrated through ongoing evaluation, dissemination of results, and maximization of resources.

Leadership roles are varied◦Clerkship or course director, associate dean,

educational committee membership, educational grant PI, conference organizer

Educational Leadership and Educational Leadership and AdministrationAdministration

Documenting Quantity and QualityDescribe the quantity, nature and duration

of administrative and leadership rolesDescribe the change that occurred as a

result of your leadership (problems identified, goals established, action taken)

Provide outcomes (e.g., MCAT scores of applicants have increased, clerkship evaluations have improved, AAMC GQ ratings improved)

Educational Leadership and Educational Leadership and AdministrationAdministration

Documenting a Scholarly ApproachCreatively design and evaluate

improvements◦Show you are making revision based on local

feedback, best practices, external peer reviewDocument ongoing QIUse evaluation tools to measure outcomes

◦Pre-post assessments, accreditation surveys, cohort performance on national exams

Educational Leadership and Educational Leadership and AdministrationAdministration

Documenting a Scholarly ApproachDemonstrate attainment of objectives

or benchmarks (AAMC GQ, course evals)

Use a 360-degree evaluation with peer comparisons, benchmarks, or external review

Employ self-reflection informed by best practices in the field

Educational Leadership and Educational Leadership and AdministrationAdministration

Documenting ScholarshipList invited and peer-reviewed presentations

and visiting professorship presentationsDocument awards received (e.g., best paper)Provide a list of institutions that have adopted

an innovationCurriculum model published on MedEdPORTALList resources obtained

◦Foundation support, grants, internal awards

Funding Sources for Medical Funding Sources for Medical Education Research GrantsEducation Research Grants Agency for Healthcare Research and Quality (AHRQ) Grants

◦ http://www.ahrq.gov/fund/grantix.htm American Association of Neurology Educational Research Grants:

◦ http://www.aan.com/go/education/eduresearch Amgen Foundation

◦ http://www.amgen.com/citizenship/IME_overview.html Arnold P. Gold Foundation:

◦ http://humanism-in-medicine.org/index.php/programs_grants Arthur Vining Davis Foundations:

◦ http://www.avdf.org/FoundationsPrograms/HealthCare.aspx Association of Professors of Gynecology and Obstetrics (APGO)

◦ https://www.apgo.org/grants.html Association for Surgical Education Foundation - Center for Excellence in

Surgical Education, Research and Training.◦ http://www.surgicaleducation.com/mc/page.do?

sitePageId=28551&orgId=ase

Funding Sources for Medical Funding Sources for Medical Education Research Grants - Education Research Grants - ContinuedContinued AstraZeneca Medical Education Research Grants:

◦ http://www.astrazenecagrants.com/ Fund for the Improvement of Postsecondary Education (FIPSE):

◦ http://www2.ed.gov/about/offices/list/ope/fipse/index.html The Henry J. Kaiser Family Foundation,

◦ http://www.kff.org/ HRSA-U. S. Department of Health and Human Services

◦ http://www.hrsa.gov/ Josiah Marcy, Jr. Foundation:

◦ http://www.macyfoundation.org/apply National Institutes of Health

◦ http://grants1.nih.gov/grants/index.cfm NBME Stemmler Medical Education Research Fund:

◦ http://www.nbme.org/research/stemmler.html

Funding Sources for Medical Funding Sources for Medical Education Research Grants - Education Research Grants - ContinuedContinued NSF Directorate for Education and Human Resources:

◦ http://www.nsf.gov/dir/index.jsp?org=EHR The PEW Charitable Trust:

◦ http://www.pewtrusts.com/ Pfizer Medical Education Grants:

◦ http://www.pfizer.com/responsibility/grants_contributions/medical_education_grants.jsp

RSNA Foundation Radiology Education Grants:◦ https://www.rsna.org/Education_Scholar_Grant.aspx◦ https://www.rsna.org/

Radiology_Education_Research_Development_Grant.aspx The Robert Wood Johnson Foundation:

◦ http://www.rwjf.org/index.jsp Society for Academic Continuing Medical Education, Research Grants in

Continuing Medical Education◦ http://www.sacme.org/SACME_grants

For Promotion…For Promotion…Educators can present evidence focused

on one category or in multiple categoriesTypes of evidence may vary but

documentation should be quantitative and qualitative

Breadth of engagement (local, regional, national, or international) varies by faculty rank

Display information in easily digestible format using tables, figures, graphs

Staying Current – New TrendsStaying Current – New TrendsMOOCS

◦ http://www.moocs.co/Higher_Education_MOOCs.html

Flipped classroomThe Khan Academy

https://www.khanacademy.orgUsing Social Media to Advance

Scholarship and Academic Careers

http://www.geripal.org/

http://www.eprognosis.org/

Making it CountMaking it CountWrite about what you doTwo-fers, Three-fersDisseminate your work in multiple

ways to multiple audiencesCreate alignments between your

activitiesBe an advocate for yourselfStart a mock portfolio (Adobe Acrobat)Choose wisely

Promoting Scholarship at Your Promoting Scholarship at Your InstitutionInstitutionDevelop methods to assess and

document scholarshipEducate stakeholders (faculty, chairs,

deans, P&T committee members)Start a workgroup (faculty learning

community) to develop policies to broaden the definition of educational scholarship

Be a leader and a mentor

Key PointsKey PointsAll faculty and administrators, particularly

those who serve on promotion and tenure committees, need to be aware of the principles of educational scholarship

Teaching can be a scholarly activity but dissemination, peer review, and reflective critique are needed for scholarship

Document quantity, quality, and engagement with the education community

Promote scholarship by getting involved

A Bit of Good NewsA Bit of Good NewsAMA MedEd Update May, 2013AMA MedEd Update May, 2013

Questions???Questions???