principles of electronic portfolios
DESCRIPTION
Principles of Electronic Portfolios. Simon Cotterill, Tony McDonald School of Medical Education Development University of Newcastle upon Tyne, UK. The workshop draws on the experiences of developing, implementing and evaluating Web-based ePortfolios, at the University of Newcastle, UK. - PowerPoint PPT PresentationTRANSCRIPT
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principles of Electronic PortfoliosPrinciples of Electronic Portfolios
Simon Cotterill, Tony McDonaldSchool of Medical Education DevelopmentUniversity of Newcastle upon Tyne, UK
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
The workshop draws on the experiences of developing, implementing and evaluating Web-based ePortfolios, at the University of Newcastle, UK.
ObjectivesObjectives
• To discuss some of the principles of electronic portfolios• To gain familiarity with the potential educational and technical
‘value-added’ features of using electronic media. • To consider approaches to the implementation of ePortfolios
Proposed StructureProposed Structure
• Presentation: Principles of ePortfolios • Exercise: ePortfolios in your context• Presentation: Case study: FDTL4 ePortfolio project• Exercise: Strengths and weaknesses• Plenary / discussion
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
UnstructuredUnstructured
FormativeFormative
SampleSample
Best workBest work
Factual / Factual / QuantitativeQuantitative
Learner OwnedLearner Owned
StructuredStructured
SummativeSummative
All workAll work
RepresentativeRepresentative
Reflective / InterpretiveReflective / Interpretive
Employer OwnedEmployer Owned
Introduction – What are Portfolios ?Introduction – What are Portfolios ?
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Introduction – What are Portfolios?Introduction – What are Portfolios?A collection of work or evidence on performanceA collection of work or evidence on performance
– Processes (the ‘journey’)Processes (the ‘journey’)– Learning Outcomes (the ‘destination’)Learning Outcomes (the ‘destination’)– Record volume and variety of experiencesRecord volume and variety of experiences
Essays, project work, logs of Essays, project work, logs of experiences/achievements, artwork, records of experiences/achievements, artwork, records of accomplished work, audits etc.accomplished work, audits etc.
Portfolios are defined by their PURPOSE(s)Portfolios are defined by their PURPOSE(s)– Employment: Builder gets commissions on the basis of Employment: Builder gets commissions on the basis of
his record of satisfied customershis record of satisfied customers– Portfolio for application: The researcher includes a Portfolio for application: The researcher includes a
personal profile in a grant application to show experience personal profile in a grant application to show experience – Professional requirements: Med/Law/Education -Professional requirements: Med/Law/Education -
evidence of essential experience and competency that evidence of essential experience and competency that can be auditedcan be audited
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
• A computer application which stores the user’s information on A computer application which stores the user’s information on a local drive.a local drive.
• A static Web page eg. CV on the InternetA static Web page eg. CV on the Internet
• A portfolio composed using ‘simple tools’ (e.g. Word, A portfolio composed using ‘simple tools’ (e.g. Word, Dreamwaever, Powerpoint etc.)Dreamwaever, Powerpoint etc.)
• A sophisticated database-driven system, typically accessed A sophisticated database-driven system, typically accessed over a network.over a network.
What might an What might an ElectronicElectronic Portfolio be? Portfolio be?
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Model presented by Jan van Tartwijk (Utrecht University) Presented at the ALT-SURF Spring Conference & Research Seminar,Edinburgh 2004
Overview(Planning / PDP)
Content(Evidence)
Discussion
3D Model3D Model
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
1. Should bring significant advantages over paper-based alternatives2. Should be considered in the context of wider (human) processes3. Clarity of purpose(s)4. One size does not fit all5. Should be learner-centric6. Should be an integral part of the learning experience7. Should support life-long learning8. Research / evaluation is essential
….not an exhaustive list !
Principles of Electronic PortfoliosPrinciples of Electronic Portfolios
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle 1. ePortfolios Principle 1. ePortfolios should bring significant should bring significant advantages over paper-based alternativesadvantages over paper-based alternatives
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
• Highly customisableHighly customisable• Multi-purpose Multi-purpose eg. formative & summative = reduced duplicationeg. formative & summative = reduced duplication
• Multiple structures / viewsMultiple structures / views• Easier cross-referencingEasier cross-referencing• Sharable / facilitate interactionSharable / facilitate interaction• TransportableTransportable• SearchableSearchable• Reduced adminReduced admin• Secure access from a range of locationsSecure access from a range of locations• Not left on the bus !Not left on the bus !
10 ‘value added’ features of an online approach to portfolios
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Potential Weaknesses of Potential Weaknesses of ePortfoliosePortfoliosThere may be barriers to effective use of electronic portfolios e.g. There may be barriers to effective use of electronic portfolios e.g.
• Limited access to computers / InternetLimited access to computers / Internet• Varied IT experience & Technophobia ! Varied IT experience & Technophobia ! • Reliability of IT systemsReliability of IT systems• Possible duplication / overlap with paper systemsPossible duplication / overlap with paper systems
• Research / Evaluation is essential (Principle 8)Research / Evaluation is essential (Principle 8)• Optimisation will take a number of yearsOptimisation will take a number of years
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle 2. ePortfolios Principle 2. ePortfolios should be considered should be considered in the context of wider (human) processesin the context of wider (human) processes
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Humans are important too !Humans are important too !• Need buy-in of key stakeholders:
Learners Clarity of purpose ? Sense of Ownership ?
Tutors & Admin staff Extra work ?
Senior Curriculum staffEmployers / Professional bodies
• Integration with existing workflow and/or change in practice and culture ?• Training and support (all)• Monitoring and evaluation
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle Principle 3. Clarity of purpose(s)3. Clarity of purpose(s)
• Choose / design your portfolio according to purpose
• Learners and assessors also need to be clear about purpose and requirements (motivation / fairness)
• ePortfolios can readily support multiple purposes
•Portfolio assessment
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Summative and Formative Summative and Formative PurposesPurposes• Portfolios are often used to evidence the achievement Portfolios are often used to evidence the achievement
of learning outcomes for of learning outcomes for summativesummative assessment. assessment.
• There may be potential problems if the portfolio There may be potential problems if the portfolio serves both serves both formativeformative and and summativesummative processes. processes.
• Reflection is less likely to be open and honest if the Reflection is less likely to be open and honest if the learner knows that the work will be assessed (loss of learner knows that the work will be assessed (loss of authenticity).authenticity).
• The production of a portfolio can itself be a formative The production of a portfolio can itself be a formative learning process ie. it is as much a ‘journey’ as an learning process ie. it is as much a ‘journey’ as an end-point for assessment.end-point for assessment.
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
• The use of electronic portfolios may potentially reduce The use of electronic portfolios may potentially reduce the tension between formative and summative the tension between formative and summative processes by supporting both assessed and private / processes by supporting both assessed and private / non-assessed content.non-assessed content.
• Students have the Students have the choicechoice to select which content is to select which content is private and which is made available to assessors, private and which is made available to assessors, appraisers, tutors, peers, and others involved in their appraisers, tutors, peers, and others involved in their education. education.
Summative and Formative Purposes Summative and Formative Purposes (2)(2)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
InstitutionalInstitutionalDataData
Portfolio for Presentation
Portfolio for Application
(job / promotion)
Portfolio for Assessment
Portfolio for Accreditation/Revalidation
PDP(shared)
PDP / Reflective(private)
Portfolio for Appraisal
Learner’sLearner’s‘‘repository’repository’
Central data:TranscriptMIS/ HR data
Programme data:Granular assessment dataOutcomes / skills sets
Supporting Multiple PurposesSupporting Multiple Purposes
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Portfolio AssessmentPortfolio Assessment
Key issues:Key issues:
• ValidityValidity
• ReliabilityReliability
• Attitudes & behavioursAttitudes & behaviours
• TimeTime demands demands
Baume D. A Briefing on the Assessment of Portfolios. Baume D. A Briefing on the Assessment of Portfolios. LTSN Generic Centre. Assessment Series No.6 (2001)LTSN Generic Centre. Assessment Series No.6 (2001)
Friedman M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJFriedman M, Davis MH, Harden RM, Howie PW, Ker J, Pippard MJAMEE Medical Education Guide No. 24: Portfolios as a method of AMEE Medical Education Guide No. 24: Portfolios as a method of student assessment. Medical Teacher. 2001; 23:535-51student assessment. Medical Teacher. 2001; 23:535-51
Where to beginbegin ?
KnowsKnows
Knows howKnows how
DoesDoes
Shows howShows how
Miller GE. Miller GE. Acad MedAcad Med 1990:S63-7 1990:S63-7
Assessing theAssessing thedevelopment ofdevelopment ofcompetenciescompetenciesover timeover time
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
ePortfolios and Assessment
--PotentialPotential benefits of ePortfolios benefits of ePortfolios::• Formative inputFormative input during during portfolio building (not just at end) portfolio building (not just at end)• Random sampling & ‘drilling down’ Random sampling & ‘drilling down’ • Peer Assessment (formative and summative)Peer Assessment (formative and summative)• Automatic inclusion of assessment results + feedbackAutomatic inclusion of assessment results + feedback• Automated summary information (quantitative)Automated summary information (quantitative)• Reduced monitoring / admin demandsReduced monitoring / admin demands
-Piloting essential before use in ‘high-stakes’ assessment-Piloting essential before use in ‘high-stakes’ assessment-Need for further research in this field !-Need for further research in this field !
ePortfolioePortfolioofof assessment (results / essays etc) assessment (results / essays etc)
forfor assessment assessment
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle 4. One size does not fit allPrinciple 4. One size does not fit all
Institutional Subject / context requirements vs. specific(standardisation) requirements
• ePortfolios should be highly flexible / customisable
• support diversity (accessibility, learning styles etc)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
The need for flexibility & customisation
• install from a set of ‘generic’ tools • create context-specific tools via simple Web forms
Course Admin view
Student view
e.g. selecting tools by course / year groups
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
e.g. configuring learning outcomes / skills sets
Course Admin view
Student view
The need for flexibility & customisation (2)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
• Personalisation / customisation
• Sense of ownership / privacy / sharing
• Constructivist theories. is eLearning contributing to ‘Paradigm shift’ ?
Student Independent Learner Teacher Facilitator Didactic Interactive + Explorative 1 location Distance learning 1 provider Multiple providers
Principle Principle 5. ePortfolios should be learner-centric5. ePortfolios should be learner-centric
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Students Students controlcontrol which parts of their portfolio can be which parts of their portfolio can be viewed for specific purposes or by specific peopleviewed for specific purposes or by specific people
Mixed-Ownership ModelMixed-Ownership Model
Corporate(HR) data
PersonalData
Tutor’sData
Emphasis on Learner Ownership / ControlEmphasis on Learner Ownership / Control
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Ability for Learners to Share their Portfolio Content
Users can control access to their portfolio
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Flexibility for the Learner to Add, Link & Cross-Reference ePortfolio Artefacts
Adding ‘child’ objects to an artefact
Cross-referenceAn artefact withA skill
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle 6. ePortfolios should be an Principle 6. ePortfolios should be an integral part of the learning experienceintegral part of the learning experience
• Educational (integration with the curriculum)
• Technical (integration within the MLE)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
• ePortfolio integrated with the VLE
• Tool properties include ‘module code’
= dynamic links to Study Guides + resources
• Learning outcomes linked to curriculum databases: ‘modules’ and units
• Links to other systems eg. SSC selection
Educationalintegration
• Should not be perceived as a ‘bolt-on’
• Support a holistic view of curriculum / outcomes (as well as specific components)
Technicalintegration(example from
Medicine atNewcastle)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle 7. ePortfolios shouldPrinciple 7. ePortfolios should support life-long learningsupport life-long learning
• Help develop life-long learning skills
• Continuity in LLL (transferability)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Towards a life-long learning recordTowards a life-long learning recordSchool School HE HE Continuing Development Continuing Development
InteroperabilityInteroperability
(technical standards to support the interaction and transfer of data between IT systems)
• IMS specifications /standards:Learner Information Package (LIP)Enterprise / Content PackagingePortfolio (early stage of
development)CETIS UK Learner Profile (based
on IMS-LIP)
• ePortfolio interaction with VLEs / MLEs
Continuity in LLL (transferability)Continuity in LLL (transferability)
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Preparation for professional requirements:
3. Independent learning skills4. Personal development planning5. Time-efficient evidencing of
performance / development for:- Appraisals- Assessment- Revalidation
Aiming to promote a reflective / deep-learning approach ….research
The ePortfolio process should promote The ePortfolio process should promote the develop life-long learning skillsthe develop life-long learning skills
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Principle 8. Research / Evaluation is Essential Principle 8. Research / Evaluation is Essential
Are the purpose(s) being fulfilled ?
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
‘‘A good fit happens over time with lots of use’A good fit happens over time with lots of use’
Spandel, 1997Spandel, 1997cited by Jan van Tartwijk (ALT-SURF Conference, 2004)
• Action Research approaches
• Incremental development and refinement
• Qualitative research methodologies(+some quantitative - not mutually exclusive!)
• Ethical and practical considerations
Evaluation and RefinementEvaluation and Refinement
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Research and Evaluation Research and Evaluation Does the ePortfolio improve learning ? How so ?Does the ePortfolio improve learning ? How so ?
Some evidence that PDP improves learning (more needed)Some evidence that PDP improves learning (more needed)A systematic map and synthesis review of the effectiveness of personal Development Planning A systematic map and synthesis review of the effectiveness of personal Development Planning for improving student learning. EPI Centre 2003 http://eppi.ioe.ac.ukfor improving student learning. EPI Centre 2003 http://eppi.ioe.ac.uk
Can ePortfolios foster a reflective approach to learning ?Can ePortfolios foster a reflective approach to learning ?• How defined ? How measured ?How defined ? How measured ?
How do particular groups of learners approach reflective portfolios ?How do particular groups of learners approach reflective portfolios ?• Learning StylesLearning Styles• Gender and AgeGender and Age• Cultural BackgroundCultural Background
John Mole. The geography of thinking. Clin Med 2002; 2:343-5 John Mole. The geography of thinking. Clin Med 2002; 2:343-5
Does the technology change the learning process ?Does the technology change the learning process ?
What are learner and staff perceptions / attitudes towards ePortfolios ?What are learner and staff perceptions / attitudes towards ePortfolios ?
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
1. Should bring significant advantages over paper-based alternatives2. Should be considered in the context of wider (human) processes3. Clarity of purpose(s)4. One size does not fit all5. Should be learner-centric6. Should be an integral part of the learning experience7. Should support life-long learning8. Research / evaluation is essential
…….the challenge is putting the principles into practice !.the challenge is putting the principles into practice !
Principles of Electronic PortfoliosPrinciples of Electronic Portfolios
© 2004, School of Medical Education Development, University of Newcastle upon Tyne
ePortfolios Ottawa Conference, 2004
Further information:Further information: [email protected]@ncl.ac.ukhttp://www.eportfolios.ac.uk