inspire. lead. engage. promoting educational research capacity through systematic evaluation of...
TRANSCRIPT
Inspire.
Lead.
Engage.
Promoting Educational Research Capacity through Systematic
Evaluation of Curricular Innovations
Janet Landeen, Lynn Martin, Charlotte Noesgaard, & Kirsten Culver, McMaster
Donna Carr, Conestoga
Nancy Matthew-Maich, Mohawk
Inspire.
Lead.
Engage.
Changing Perspectives
Inspire.
Lead.
Engage.
Overview of Approaches
Inspire.
Lead.
Engage.
Today’s Session Setting the Context-Curriculum Renewal Gathering the Research Team Identifying the Big Questions Using Interpretive Description to Organize
Approaches Narrowing the Questions, Finding Resources,
Inviting Others to the Dance Sharing Progress to Date
Inspire.
Lead.
Engage.
The Context
Inspire.
Lead.
Engage.
Designing the Kaleidoscope Curriculum
Experiences of the McMaster Mohawk Conestoga BScN Program
Inspire.
Lead.
Engage.
Honouring the many authors of the Kaleidoscope
Inspire.
Lead.
Engage.
Inspire.
Lead.
Engage.
Kaleidoscope…
Kalos…beautiful
Eidos…shape
Scopeo…to reflect on
Inspire.
Lead.
Engage.
Key Areas for Curricular Revision
PBL/PBLThinking Like
a Nurse
Ways of Knowing
Integrated Knowing/Acting
/Being
Retention of Program Philosophy & Goals
Inspire.
Lead.
Engage.
Inspire.
Lead.
Engage.
The Person in the Middle Encounter the person (individual,
family, group, community) Use of narrative to have person come
alive
Inspire.
Lead.
Engage.
Turn the Kaleidoscope to a different aspect
Inspire.
Lead.
Engage.
Clinical Reasoning & Judgment in Nursing
(Tanner, 2006, p.208)
Inspire.
Lead.
Engage.
Turn the Kaleidoscope to a different aspect
Inspire.
Lead.
Engage.
Pedagogical Approach: Ways of Knowing in Nursing
Empiric (scientific, evidence based) Ethical (moral component-Codes of Ethics) Personal (consistent with therapeutic use of
self) Aesthetic (art of nursing) Emancipatory (critical reflection & action based
on inequities)(Carper, 1978; Chinn & Kramer, 2008)
Inspire.
Lead.
Engage.
Turn the Kaleidoscope to a different aspect
Inspire.
Lead.
Engage.
Integrated Approaches to Learning
Pathophysiology & Evidence Informed Decision Making
Scaffolded across multiple courses versus stand-alone courses
(AACU, no date)
Inspire.
Lead.
Engage. So What?
OrWho Cares?
Inspire.
Lead.
Engage.
Gathering the Research Team Interest in educational reform and research Identification of key individuals-
– Chair of Program Evaluation Committee– Director of Nursing Education Research Unit– Leaders within Program
Individuals looking for career growth opportunities
Graduate Students Undergraduate Students
Inspire.
Lead.
Engage.
Identifying the Big Questions
Inspire.
Lead.
Engage.
Program Evaluation vs. Educational Research
Evaluation of all aspects of program
Real life/real time curriculum-as-lived versus curriculum-as-planned
Used to inform local decision-making
In-depth exploration of specific topics
Comparison group, before/after design, or systematic qualitative exploration of experience or process
Uncover broader knowledge relevant to others
Inspire.
Lead.
Engage.
CASN Accreditation Standard3.2.5 Ongoing comprehensive evaluation
continually improves program outcomes. Descriptor: The program is deliberative and
responsive in making timely improvements based on data from ongoing rigorous and ethical evaluations from faculty, learners, graduates, employers, and others, through the application of the Key Elements.
Key Element: The program and curriculum are monitored and evaluated to ensure currency and relevance to nursing practice.
Inspire.
Lead.
Engage.
Program Evaluation at McMaster Mohawk Conestoga
Comprehensive Program Evaluation Plan Use of Stufflebeam’s CIPP Model (Context,
Input, Process, & Product) to develop Evaluation Matrix
Matrix critiqued for congruence with program philosophy, Stake’s Responsive Evaluation Model, & Aoki’s suggestion of multiple perspectives
(Aoki 1991; Stake 2003; Stufflebeam & Shinkfield 2007)
Inspire.
Lead.
Engage.
Program Evaluation
Specific Questions
Situational Issues
Institutional Priorities
Accreditation Standards
Student Expectations
Inspire.
Lead.
Engage.
Program of Research
Specific Research Project
Informed by Program Evaluation
Funding Opportunities
Faculty Interest
Inspire.
Lead.
Engage.
Evaluating the Impact of the Kaleidoscope Curriculum
Have the educational innovations of the Kaleidoscope Curriculum made a difference in the graduates of the BScN Program?
Are there unintended consequences of the curricular changes?
Inspire.
Lead.
Engage.
Sub-questions Do students achieve the same or better
outcomes on standard measures of performance?
Do clinical faculty and clinical preceptors notice a difference in graduating (Level 4) students of the Kaleidoscope Curriculum? If so, are those differences consistent with the educational innovations?
Inspire.
Lead.
Engage.
Sub-questions cont’d. Do graduating students integrate knowledge of
pathophysiology and evidence informed decision making into their clinical reasoning and judgement?
Do stories of the educational experience that graduates and faculty tell focus on the person (individual, family, group, community)?
Do graduating students utilize ways of knowing in their approaches to nursing practice?
Inspire.
Lead.
Engage. Interpretive Description
Inspire.
Lead.
Engage.
Integrative description as organizing approach
When purpose is to gain knowledge that can be directly applied to enhance nursing education.
“requires an integrity of purpose deriving from two sources: (1) an actual practice goal, and (2) an understanding of what we do and don’t know on the basis of the available empirical evidence (from all sources).”
(Thorne, 2008, p. 35)
Inspire.
Lead.
Engage.
Developing the Approach
Data Source Data Analysis Strategy Rationale
Student performance on pathophysiology and EIDM examinations pre and post Kaleidoscope.
Per item analysis on equivalent or identical exam questions, and grouped by content areas.
Teaching methodology is different but overall content to be learned has remained the same. Where appropriate, direct comparisons will be exam on anonymized examination responses.
Do students achieve the same or better outcomes on standard measures of performance?
Inspire.
Lead.
Engage. Progress to Date
Inspire.
Lead.
Engage.
Evaluating the Impact of Pedagogical Innovations: Does the Kaleidoscope Curriculum Make a Difference in the
Clinical Practice of Final Year Undergraduate Nursing Students?
Kaleidoscope Research TeamFunded by Centre for Leadership in Learning,
McMaster University $8,000
Inspire.
Lead.
Engage.
Evaluating Clinical Learning Outcomes
Integrative description Focus groups or individual interviews with 30
Clinical Faculty who taught students pre and post Kaleidoscope
Individuals likely to be sensitive to change in students but least involved in curriculum discussions
Semi-structured interview questions Graduate student Research Coordinator Poised to begin recruitment
Inspire.
Lead.
Engage.
Understanding the Experiences of Students Who are Enrolled in
Joint College/University Collaborative Educational Programs
Nursing: J. Landeen (Mac), N. Mathew-Maich (Moh), M. Parzen (Moh), & L. Hagermann (Con)
Bachelor of Technology: L. Bolan (Mac) & D. Bender (Moh)
Medical Radiation Sciences: M. Faquharson (Mac) & L. Marshall (Moh)
Inspire.
Lead.
Engage.
“Dual Identity” Project McMaster President’s “Forward with Integrity” Fund
$5,000, matched by Dean of Engineering Uses interpretive description methodology 8 undergraduate students (2 per program & site)
paid to develop full proposal, recruit participants, conduct focus groups, analyze results, & present findings
Nursing student researchers receive course credit prior to assuming paid role
Interdisciplinary faculty team mentoring & teaching students
Inspire.
Lead.
Engage.
Examples of Other Research Projects
Evaluation of written clinical evaluation forms 2 years pre and 2 years post Kaleidoscope to assess degree of uptake of curriculum renewal-in proposal writing stage (Noesgaard et al)
Exploring the Meaning of Consistency in PBL Programs-study completed (Landeen & Jewiss)
What Makes an Effective Teacher? Listening to the Voice of Our Students- in data analysis (Matthew-Maich et al)
Inspire.
Lead.
Engage.
Barriers to conducting educational research
Work load of faculty Cultural & institutional differences between
universities & college partners Funding sources for educational research Perceptions of value of educational research
versus clinical research
Inspire.
Lead.
Engage.
Facilitators to Educational Research
Resurgence of interest in educational research Organizations such as International Society for
Teaching and Learning in Higher Education (ISSOTL)
Accreditation standards Multiplicity of venues for presenting and
publishing
Enriching the experiences of our students!
Inspire.
Lead.
Engage.
References Aoki, T. T. (1991). Layered understandings of orientation in
social studies program evaluation. In Pinar. W.F. & Irwin, R.L. (Ed.), (2005). Curriculum in a new key: The collected works of Ted. T. Aoki. .(pp.167-185) Mahwah, NJ: Lawrence Erlbaum..
Association of American Colleges & Universities. (no date) Integrative Learning VALUE Rubric. Retrieved from: http://www.aacu.org/value/rubrics/integrativelearning.cfm
Carper, B. (1978). Fundamental patterns of knowing in nursing. Advances in Nursing Science, 1(1), 13-23.
Chinn, P., & Kramer, M. (2008). Integrated theory and knowledge development in nursing (7th ed.). St. Louis: Mosby Elsevier.
Inspire.
Lead.
Engage.
References cont’d. Stake, R. (2003). Responsive evaluation. In T.
Kellaghan & D.L Stufflebeam (Eds). The international handbook of educational evaluation, . (pp. 63-68). Dordecht: Klower Academic Publishers.
Stufflebeam, D.L. & Shinkfield, A.J. (2007). Evaluation theory, models, and applications. San Francisco: Jossey-Bass
Tanner, C.A. (2006). Thinking like a nurse: A research-based model of clinical judgment in nursing. Journal of Nursing Education,45(6), 204-211
Thorne, S.E. 2008. Interpretive description. Walnut Creek, CA: Left Coast Press
