http:// eportfolios 2005 electronic portfolios simon cotterill school of medical education...

http://www.eportfolios.ac.uk

ePortfolios 2005

Electronic PortfoliosElectronic Portfolios

Simon CotterillSchool of Medical Education DevelopmentUniversity of Newcastle upon Tyne, UK

1. What are ePortfolios ?1. What are ePortfolios ?

2. FDTL-4 & related ePortfolio projects2. FDTL-4 & related ePortfolio projects

3. Evaluation 3. Evaluation

4. Key Issues with electronic portfolios / PDP4. Key Issues with electronic portfolios / PDP


ePortfolios 2005


2. FDTL-4 ePortfolio and related projects2. FDTL-4 ePortfolio and related projects

3. Evaluation3. Evaluation

4. Issues with electronic portfolios / PDP4. Issues with electronic portfolios / PDP


ePortfolios 2005

InstitutionalInstitutionalDataData

Portfolio for Presentation

Portfolio for Application

(job / promotion)

Portfolio for Assessment

Portfolio for Accreditation/Revalidation

PPD(shared)

PPD / Reflective(private)

Portfolio for Appraisal

Learner’sLearner’s‘‘repository’repository’

Central data:TranscriptMIS/ HR data

Programme data:Granular assessment dataOutcomes / skills sets

ePortfolios are learner-centricePortfolios are learner-centric& may serve multiple purposes& may serve multiple purposes


ePortfolios 2005

Model presented by Jan van Tartwijk (Utrecht University) Presented at the ALT-SURF Spring Conference & Research Seminar,Edinburgh 2004

Overview(Planning / PDP)

Content(Evidence)

Discussion

3D Model3D Model


ePortfolios 2005

• Sharable / facilitate interaction Sharable / facilitate interaction • TransportableTransportable - - supporting continuity in LLLsupporting continuity in LLL

• Highly customisableHighly customisable• Multi-purpose Multi-purpose eg. formative & summative = reduced duplicationeg. formative & summative = reduced duplication

• Multiple structures / viewsMultiple structures / views• Easier cross-referencingEasier cross-referencing• SearchableSearchable• Reduced adminReduced admin• Secure access from a range of locationsSecure access from a range of locations• Not left on the bus !Not left on the bus !

10 ‘value added’ features of an online approach to portfolios


ePortfolios 2005


2. FDTL-4 ePortfolio & related projects2. FDTL-4 ePortfolio & related projects


4. Issues with electronic portfolios / PDP 4. Issues with electronic portfolios / PDP


ePortfolios 2005

BackgroundBackground

2nd Newcastle-Nottingham PARs project, 2000-02


ePortfolios 2005

Managed Environments for Portfolio-based Reflective Learning. Integrated Support for Evidencing Outcomes.

An FDTL-4 project: Newcastle University (lead site) Leeds University Sheffield University St Andrew’s UniversityDundee University (for consultancy)

Developing Web based portfolios to support reflective approaches for evidencing the attainment of programme outcomes in undergraduate Medicine.

Integration with managed learning environments for Medicine.

Project funded by


ePortfolios 2005

NewcastleNewcastle


ePortfolios 2005

SheffieldSheffield

Reflect on Intensive Clinical Experiences (ICE)


ePortfolios 2005

LeedsLeeds


ePortfolios 2005

St Andrew’sSt Andrew’s


ePortfolios 2005


2. FDTL4 ePortfolio project2. FDTL4 ePortfolio project


4. Key Issues with electronic portfolios / PDP4. Key Issues with electronic portfolios / PDP


ePortfolios 2005

Evaluation of a Phase I Portfolio/Log BookEvaluation of a Phase I Portfolio/Log Book (paper and electronic versions).(paper and electronic versions).

Cotterill SJ, Sarma S, McDonald AM, Bradley PCotterill SJ, Sarma S, McDonald AM, Bradley P

Ethical approval / informed consent

Focus groups - facilitated by a 4th year student (Sean Sarma)

Questionnaires - based on findings of the focus groups

Key issues from the focus groups

• Generally well received but what motivation ?• not assessed• not viewed by tutors (staff interface completed later)

• Need better clarity of purpose• Learning diary - useful at first, less so over time• Good to browse the Learning Outcomes


ePortfolios 2005

Ethical approval / informed consent

Questionnaires (before and after using the ePortfolio)• change in awareness of intended learning outcomes ?• factors influencing use of the ePortfolio• attitudes and perceptions of using the ePortfolio• student perception of impact of using the ePortfolio• evaluation of technical features and ‘usability’

186 students completed the portfolio (~100% compliance)

157 questionnaires completed (84% of cohort) at 30/09/2004

Evaluation of an electronic portfolio to Evaluation of an electronic portfolio to facilitate reflective learning in stage 4 facilitate reflective learning in stage 4 medical students during their student medical students during their student

selected components.selected components.

Cotterill SJ, McDonald AM, Bradley P, Robinson R, Hammond GRCotterill SJ, McDonald AM, Bradley P, Robinson R, Hammond GR


ePortfolios 2005

Year 4 SSC Portfolio (Newcastle)Year 4 SSC Portfolio (Newcastle)


ePortfolios 2005

Building the ePortfolio was a useful learning experience

0 10 20 30 40 50

Strongly Disagree

Weakly Disagree

Agree

% responses

80% thought it wasa useful learning experience

83% felt they had recordedgood evidence

93% reflected on their learningafter the option

0 10 20 30 40 50

Strongly Disagree

Disagree

Weakly Disagree

Weakly Agree

Agree

Strongly Agree

% responses

Having clearly defined intended learning outcomes influenced the way in which I approached the option

72% felt that the LOsinfluenced their approach

n = 157


ePortfolios 2005

Evaluation - positive (SSC portfolio)

“It encouraged me to really give thought to what I wanted to achieve during the option, which was especially useful as this was my first option. As a result of the portfolio I think I got much more out of the option than I would have otherwise.”

“It made me concentrate on creating aims at the start of the option and allowed me to plan the option with my supervisor in a defined way. Overall it made my learning for the option more organised and focused.”

Planning and approach to learning:


ePortfolios 2005

Evaluation – negative (SSC portfolio)

“Most of the things I learned couldn't be 'measured/quantified' so I felt it was of little value to try and invent a way in which they could.”

“Access to computer as option took place in ------ Castle, records may have been more detailed if completed straight after meetings etc.”

Hard to measure:

Access to IT facilities:


ePortfolios 2005


2. FDTL4 ePortfolio project2. FDTL4 ePortfolio project


4. Key Issues with electronic portfolios / PPD4. Key Issues with electronic portfolios / PPD


ePortfolios 2005

Humans are important too !Humans are important too !

• Need buy-in of key stakeholders:

Learners Clarity of purpose ? Sense of Ownership ?

Tutors & Admin staff Extra work ? Resources ?

Senior Curriculum staff

Employers / Professional bodies

• Integration with existing workflow and/or change in practice and culture ?

• Training and support (all)

• Monitoring and evaluation


ePortfolios 2005

Software needs to match pedagogySoftware needs to match pedagogy(and not visa-versa !)(and not visa-versa !)

• Use or develop flexible solutions To meet both:

• Institutional requirements• Subject-specific requirements

eg. The FDTL4 ‘generic’ ePortfolio is component-based:

• High level of customisation eg. nomenclature

• Facility for course administrators to create context-specific tools via simple Web forms, without detailed technical skills.

• Local developers can develop more sophisticated tools using Open Source products. Share and contribute back to the ePortfolio community.


ePortfolios 2005

Getting the right balanceGetting the right balance

AssessedNecessary for motivation / engagement ?

““Either increase its importance or bin it - sitting on Either increase its importance or bin it - sitting on the fence is utterly pointless”the fence is utterly pointless”First year Medical student talking about a newly introduced non-First year Medical student talking about a newly introduced non-assessed paper log-book & portfolio (2004)assessed paper log-book & portfolio (2004)

Non-assessed Necessary for open-honest reflection


ePortfolios 2005

Towards a life-long learning recordTowards a life-long learning record

School School HE HE Continuing Development Continuing Development

InteroperabilityInteroperability

(technical standards to support the interaction and transfer of data between IT systems)

• IMS specifications /standards:Learner Information Package (LIP)Enterprise / Content PackagingePortfolio (early stage of

development)

CETIS UK Learner Profile (based on IMS-LIP)

• JISC ePET project to transfer data to/from the Newcastle ‘generic’ ePortfolio

Can I get my data out ?Can I get my data out ?


ePortfolios 2005

Research and Evaluation Research and Evaluation Does the ePortfolio improve learning ? How so ?

Some evidence that PDP improves learning (more needed)A systematic map and synthesis review of the effectiveness of personal Development Planning for improving student learning. EPI Centre 2003 http://eppi.ioe.ac.uk

Can ePortfolios foster a reflective approach to learning ?• How defined ? How measured ?

How do particular groups of learners approach reflective portfolios ?

• Learning Styles

• Gender and Age• Cultural Background

John Mole. The geography of thinking. Clin Med 2002; 2:343-5

Does the technology change the learning process ?

What are learner and staff perceptions / attitudes towards ePortfolios ?


ePortfolios 2005

Current & Future DevelopmentsCurrent & Future Developments FDTL4 Transferability Project

(with St Georges Hospital Medical School, London)

Involvement in 4 JISC regional ePortfolio projects (FE – HE)

ePortfolio support for 2 CETLs in Health (incl. use of mobile devices)

Dentistry undergraduate + vocational training

ePortfolios in other contexts: Postgraduate Research Students Contract Research Staff Undergraduate Bioscience


ePortfolios 2005

Further Information:Further Information: http://www.eportfolios.ac.ukhttp://www.eportfolios.ac.uk

[email protected]

Cotterill SJ., McDonald AM., Drummond P., Hammond GR. Design, implementation and evaluation of a ‘generic’ e-portfolio: the Newcastle experience (Proc ePortfolios 2004)

Hennessy S., Howes A. Using ePortfolios to assess the Reflective Capabilities of Medical Students (Proc. ePortfolio 2004)

Papers and bibliography available at:http://www.eportfolios.ac.uk/


ePortfolios 2005

Context-specific tools

‘Generic’ tools

ePortfolioframework Common contents structure

• sharing• cross-referencing• searchable• integrated action planning

Customisation• select tools by course/year• outcomes / skills sets• nomenclature• graphics / layout• simple tool editor

• CV• Outcomes / skills log• PDP• Reflective learning diary • Meetings log• SWOT

Integrated into thecommon contents structure

Flexible component-based architectureFlexible component-based architecture

Developed with Open Source products: Zope, MySQL


ePortfolios 2005

Ability for Learners to Share their Portfolio Content

Users can control access to their portfolio


ePortfolios 2005

Integration with a VLE Integration with a VLE (via a ‘tab’ in Blackboard)(via a ‘tab’ in Blackboard)

http:// eportfolios 2005 electronic portfolios simon cotterill school of medical education...

Documents