Individual Learning Plans (ILPs)& Learning Portfolios:New Age Educational Tools

Anne E. Burke, M.D.Dayton Children’s

Wright State University

Grace L. Caputo, M.D., M.P.H.Phoenix Children’s Hospital

University of Arizona

Introduction to ILPs: What They Are and Are Not

Individualized Learning Plans

Background-Why the shift? The History-Where did this come from? Adult Learners and ILPs Pediatric ILPs- What is that? Brief Literature Review

Background- Why the Shift? Broader concepts of CME Changes are:

1) New and evolving recertification process 2) Continuous Professional Development

(CPD) has supplanted CME3) Life long learning is recognized as crucial

(Practice Based Learning and Improvement-PBLI)

Background-Why the Shift? “Rather than assimilating a store of

largely irrelevant information, doctors now need to develop learning skills which enable them to sift out and acquire information as and when the need arises.”

Parsell G. Contract learning, clinical learning and clinicians. Postgrad Med J. 1996;72:284-289

The History-Where did this come from? Increasing knowledge and information Donald Schon: “Practice related

learning” “Reflective practice” Self-directed learning: identifying

learning needs, finding resources to meet those needs and evaluating their achievement

Donald Schon- Learning Cycle

Clinical problem Reflection-in-action Reflection-on-action Improvement in practice Level or Zone of expertise

Donald Schon. Educating the Reflective Practitioner. Jossey-Bass Publishers, 1987

The History-Where did this come from? “In the practice-learning environment, a

physician will begin an educational activity not by entering a conference room but by reflecting on his or her practice performance.”

Barnes BE. Creating the practice-learning environment:using information technology to support a new model of CME. Acad Med. 1999; 73:278-281

The History-Where did this come from? Continuous Professional Development

Not didactic More individualized Meets needs of adult learners

CPD: In Practice Seeing patients, asking clinical questions Searching the literature Teaching

Adult Learners: Principles 1. Adults are motivated to learn as they

experience needs and interests that learning will satisfy

2. Adults’ orientation to learning is life-centered; life situations, not subjects

3. Experience is the richest resource for adults’ learning; therefore the core methodology is analysis of experienceKnowles M. The Adult Learner: A Neglected Species. 4th Ed. 1990.

Adult Learners: Principles 4. Adults have a deep need to be self-

directing; therefore the role of the teacher is to engage the learner

5. Individual differences increase with age; therefore there must be optimal provision for differences in style, time, place, etc.

Adult Learners

“That is too important to be taught; it must be learned” - Carl Rogers

Adult Learners

“I’m always ready to learn, although I do not always like to be taught”

- Winston Churchill

Adult Learners

“The best way to learn about it, is to play about it!”

-Mister Rogers

Learning Contracts

“Without question the single most potent tool I have come across in my more than half-century of experience with adult education”

Knowles M. The Adult Learner: A Neglected Species. 4th Ed. 1990:139

Learning Contracts

“Without question the single most potent tool I have come across in my more than half-century of experience with adult education”

Knowles M. The Adult Learner: A Neglected Species. 4th Ed. 1990:139

Pediatric ILPs- What is that? Learning contract Self-assessment Exercise in self reflection Formulated by the individual (resident) Guided by Facilitator A requirement

Pros

“Lends focus. Stimulating more purposeful learning”

“Allows me to re-evaluate learning needs”

“It is good to have regular discussions about my goals”

Cons

“I often don’t know what to work on” “Never enough time” “Recommendations for specific goals

would be helpful” “I’m too tired or busy to focus on my

goals”

Pros and Cons

“Although the theoretical power of the ILP approach lies with its emphasis on self-direction and individualization of learning, both residents and faculty in our program wanted more guidance, standardization, and structure.”

Stuart et al. Are Residents Ready for Self-Directed Learning? A Pilot Program of ILPs in Continuity Clinic. Ambulatory Pediatrics. Vol 5, No. 5, Sept-Oct 2005, pp298-301.

Requirements

RRC

Requirements

A Requirement (January 2006) per The Pediatric RC “Documentation of an individual learning plan

for each resident must occur annually”

Requirements “Companion Document” gives some

guidance Defines ILP

Documented personal learning objectives Strategies to achieve them

ILP Components Define goals Self-assessment

Personal attributes Clinical competency

Summarize learning needs Define learning objectives and

strategies to accomplish them

Learning Contracts

Not set in stone Recognize individual differences Involves change of attitude Product vs. Process

Parsell and Bligh. Contract learning, clinical learning and clinicians. Postgraduate Medical Journal. 1996; 72:284-289

Learning Contracts: Medical Students Primary care clerkship: four weeks 187 3rd and 4th year students Produced 517 “learner-centered goals” 60% knowledge goals 37% skill goals 3% attitudinal goals

Learning Contracts: Medical Students

Viewed as “less useful” than the clinical experiences

“As useful” as lectures and seminars Utilized a simple form

McDermott M, et al.Use of learning contracts in an office-based primary care clerkship. Medical Education. 1999;33:374-381

Self-Assessment

Self-Assessment

Poor to modest correlations with other subjective and objective assessments

Over-assessment and under-assessment are not predictable

Relative ranking model may increase reliability

Gordon M. A review of the validity and accuracy of self-assessments in health professions training. Acad Med 1991; 66 762-769

Self-Assessment The value appears to be in its ability to

force the learner to reflect on their strengths and weaknesses

Recognize how these strengths and weaknesses may impact learning and performance

Stewart J et al. Clarifying the concepts of confidence and competence to produce appropriate self-evaluation measurement scales. Med Educ 2000; 34:903-909

Self-Assessment “ At present our assessment methods stem

from the reductionist philosophy that underpins our discipline, and we are, thus, trapped by our need to compare like to like….we will continue to struggle to measure the unmeasurable, and may end up measuring the irrelevant because it is easier.”

Snadden D. Portfolios-attempting to measure the unmeasureable? Medical Education 1999;33:478-479

ILPs are not to be confused with….

Portfolios Evaluations IEPs (in the traditional sense) In the literature they are made by the

Learner for the Learner to take control of Learner needs.

ILP Queries Is it personality based? Should they be done more frequently? Does the skill of the facilitator make a

difference? Can benefit only be seen after years? What did we do before ILPs?

What is a Portfolio?

An educational portfolio is a very personal collection of artifacts and reflections about one’s accomplishments, learning strengths, and best works.

“A professional development portfolio is a collection of material, made by a professional, that records and reflects on, key events and processes in that professional’s career.” (Hall 1992)

Educational Rationale

Knowledge of adult cognitive development

Promotion of learning Enhanced assessment Promote professional development

Educational Uses Professional development (Self-directed) learning Facilitate reflection Goal setting Mentorship Self-Assessment Curriculum, competency outcomes

Encourages autonomous and reflective learning

Based on real experiences of the learner (connection between theory and practice)

Provides evidence of learning in various settings

Assessment within framework of learning objectives (formative and summative)

Model for life-long learning and continuous professional development

Challis 1999

Benefits of Portfolio-based Learning

Practical Experiences ACGME Toolbox of Assessment Methods Competency-based assessment for

physician investigators (Dannefer 2007)

Professionalism in medical students (Kalet 2007)

Self-directed learning in OB residents (Fung 200)

Mixed success in Medical Education Need for instruments that assess

competence development 30 of 1939 papers, no meta-analysis Inter-rater reliability Factors for success:

Clear goals and procedures Integration with curriculum and assessment Support through mentoring Ease of use

Driessen 2007

Barriers to Using Portfolios Implementation

Incorporation into curriculum Training of Faculty Mentors

Standardizing content vs flexibility Ease of use

Web-based enhanced student motivation

Driessen 2007

Other Uses Faculty Development (Kuhn 2004) American Board of Pediatrics –

Maintenance of Certification Self-directed learning documentation

153 of 241 physicians 3,939 items submitted Positive perceived benefit suggested

(Campbell 1996)

What MUST we document in a resident’s portfolio? Procedure log Patient log EBM activity, teaching activities Evaluations from faculty, peers, patients,

nurses, staff Observed history, physical, counseling Demonstration of clinical competence ILP with follow-up In-service exam Demonstration of teaching skills

PRACTICE BASED LEARNING & IMPROVEMENT

a) Taking primary responsibility for lifelong learning to improve knowledge, skills, and practice performance through familiarity with general and rotation specific goals and objectives and attendance at conferences

A Quality Improvement Activity Must be Documented

The program must document that residents also acquire the skills needs to analyze and improve clinical practice

. . . accomplished by participation in a quality improvement project

PCH General PediatricsClinic QI Projects

Some recent projects include: Patient Discharge Instructions Form Time Study to Evaluate the Role of the

Triage Nurse Newborn Feeding Practices Dietary Regimens for the Management of

Constipation

Technology “The program must document the

residents’ ability to use the information technology systems within the clinical setting and apply [it] to clinical care”

“In addition, the program must evaluate the competence of residents performing an evidenced-based exercise.”

Evidenced Based ExercisesMust be Documented

d) locating, appraising, and assimilating evidence from scientific studies related to their patient’s health problems

e) using information technology to optimize lifelong learning

Resident/Fellow TeachingMust be Documented

f) actively participating in the education of patients, families, students, residents, and other health professionals which should be documented by evaluations of a resident’s teaching abilities by faculty and/or learners

Bi-Annual ReviewMust be Documented!

Document meetings between an individual resident and mentor or advisor for purposes of feedback and guidance must occur at least twice a year.

There Must be Annual ILP

Documentation of an individual learning plan for each resident must occur annually

Fellows Too!

Direct Observation The RRC requires direct observation

of resident performance to document competence.

“This must be accomplished through direct observation using a structured approach with different evaluators in different settings.”

Clinical Evaluation Form

Procedural Competence All residents need to log their procedure

experience on to reliable databases Currently all procedures MUST be logged

onto the ACGME website or an equally robust database

Additionally, JHACO requires documentation and availability of resident procedural competence

Continuity Clinic Logs Currently all procedures MUST be

logged on to the ACGME website or an equally robust database Be prepared to show complied data or

diagnoses, panel size, continuity of care

Feedback – it has to come from MANY sources – not just you!

You must document effective communication and resident behaviors by Patients Families Members of the health care team

Must ensure structurededucational experiences to address the following:

Patient advocacy Risk management Cost effectiveness, cost vs. quality Health care organization, finance,

management Business management, billing and

coding Error prevention

Brainstorming

What do we already do? What do others already do? What would we like to do? How will we know when we’ve done it?

Don’t Reinvent the Wheel Identify what is already available to you both

locally and globally What are the pieces of information that you

already have in separate files around your office? Does your hospital or other programs at your

institution already have a tracking process in use for its residents management, staff, etc.?

Take advantage of roadwork already accomplished by the APPD!

Use the Share Warehouse!

The APPD is there for YOU!

“Live as if you were to die tomorrow. Learn as if you were to live forever.”

Mahatma Ghandi