using eportfolios for evidence- based research julie hatcher, director of undergraduate programs,...
DESCRIPTION
Agenda 1:00-1:10 Introductions & General questions 1:10-1:25 Overview of ePortfolios 1:25-1:45 Activity (Review example ePortfolios) 1:45-1:55 Tools to assess the evidence 1:55-2:25 Activity (Read/Watch Call of Service) 2:25-2:40 Overview of Digital Storytelling Break (10-15 min) 3:00-3:30 Research, Reflection, Assessment 3:30-3:45 Choice Points 3:45-4:00 Take Aways & Next StepsTRANSCRIPT
Using ePortfolios for Evidence-Based ResearchJulie Hatcher,Director of Undergraduate Programs, Associate Professor of Philanthropic Studies
Kristin Norris, Instructional Technology Specialist, Center for Service & Learning
Kathy Steinberg, Assessment Specialist, Center for Service & Learning
Special Thanks to our Sponsor, Epsilen
General questions
• How familiar is your campus with e-portfolios?▫ Very ….. ….. … Not at all
• How familiar are you with e-portfolios and service learning?
• How familiar are you with e-portfolios and civic learning outcomes?
• How familiar are you with advising others on how to use e-portfolios in service learning research?
• What are a few KEY issues to address?
Agenda•1:00-1:10 Introductions & General questions•1:10-1:25 Overview of ePortfolios•1:25-1:45 Activity (Review example ePortfolios)•1:45-1:55 Tools to assess the evidence•1:55-2:25 Activity (Read/Watch Call of Service)•2:25-2:40 Overview of Digital Storytelling•Break (10-15 min)•3:00-3:30 Research, Reflection, Assessment•3:30-3:45 Choice Points•3:45-4:00 Take Aways & Next Steps
Overview of ePortfoliosHarness the power of the pedagogy
ePortfolios is both a process and product
• Process: a series of events (time & effort) to produce a result▫ Portfolio as a workspace▫ Working Portfolio (digital archive, repository of
artifacts, collaboration space, reflective journaling)▫ Primary purpose: learning or reflection▫ Organization: chronological
• Product: the outcome/results of an activity/process▫ Portfolio as a showcase▫ Presentation Portfolios (the “Story” or narrative,
multiple views, varied audiences & purpose)▫ Primary purpose: Accountability or showcase for
employment▫ Organization: thematic
Barrett, Eifel, July 2011(www.slideshare.net/eportfolios)
Processes
Portfolios Social Networking TechnologyCollection Connect (‘friending’) ArchivingSelection Listening (reading) Linking/ThinkingReflection Responding
(commenting)Digital Storytelling
Direction/Goals Share (linking/tagging)
Publishing
PresentationFeedback
Example of a Product ePortfolio
Example of a Process ePortfolio
What is an ePortfolio?•“A portfolio tells a story. It is the story of
knowing. Knowing about things…Knowing oneself…Knowing an audience….Portfolios are students’ own stories of what they know, why they believe they know it, and why others should be of the same opinion.” (Paulson & Paulson, 1991, p. 2)
Power of the Pedagogy
“Stories help us organize our experience and define our sense of ourselves” (Roger Schank, Tell Me a Story)
Why ePortfolios?• Reflecting• Celebrate learning• Personal planning & goal setting• Employment applications• Accountability (prove what you have learned)• Capture and store evidence (repository)• Give and receive feedback• Collaborate• Present what you know to an audience• Exploring your personal and professional
identity
ePortfolios as a Purpose
the “overarching purpose of ePortfolios is to create a sense of personal ownership
over one’s accomplishments, because ownership engenders feelings of pride,
responsibility, and dedication.”
Paris & Ayers (1994)
Managing Oneself
• What are my strengths?• How do I perform?• What are my values?• Where do I belong?• What should I contribute?• Responsibility for Relationships?
“Success in the knowledge economy comes to those who know themselves – their strengths, their values, and how best they perform” –Peter Drucker (2005, Harvard Business Review)
Why should you use ePortfolios•Facilitate Reflection•Enable students to create something that
demonstrates their knowledge, skills, abilities, dispositions – often times the things they have yet to articulate in any other way
•Gather evidence of student knowledge (and maybe growth)
•Because it is of value to the students
Choosing an ePortfolio Platform
https://sites.google.com/site/choosingeportsoftware/
Latest Blog by Trent Batson
http://campustechnology.com/articles/2011/10/12/a-survey-of-the-electronic-portfolio-market-sector.aspx#
Activity: Examine sample ePortfolios
•Pay attention to structure (tabs, headings, organization)•What evidence of civic learning do you see?•Key elements to look for – ‘About me’, artifacts vs reflections (is it ‘critical reflection’?), intended audience•What does an ePortfolio allow you to see that a post-test or a written reflection would not?
Examples• https://stonybrook.digication.com/egimenez/How
_did_the_Soup_Kitchen_influence_my_life
• https://stonybrook.digication.com/americorps_justin_thomas/Home
• https://oncourse.iu.edu/osp-presentation-tool/viewPresentation.osp?id=5DCDA2A93B4FE85E93F80F4DAE621672
Assessing the EvidenceIs there a magic rubric?
Goal of Exercise• Introduction to VALUE – Civic
Engagement Rubric▫ AAC&U purposes of meta rubrics with e-
portfolios▫ What are the domains of civic learning?▫ Does the CE Rubric work? What’s missing?▫ Does the CE Rubric work for different
types of evidence (reflection paper, digital story)?
▫ Keep in mind issues that may surface in your own context, adaptations
Public Purposes of Higher Education
• AAC&U LEAP – Personal and Social Responsibility VALUE project – Civic Engagement Rubric Bridging Theory to Practice
• AASC&U American Democracy Project
• Carnegie Foundation Voluntary Classification for “Community
Engagement”• Professional Associations• Accreditation
AAC&U Essential Learning Outcomes
• Personal and social responsibility• Civic knowledge and engagement- local and
global• Intercultural knowledge and competence• Ethical reasoning and action• Skills for lifelong learning
“Anchored through active involvement with diverse communities and real world challenges.”
AAC&U VALUE Initiative
• What learning should “look like”; 15 rubrics
• Meta rubric – adaptable to campus context
• Institutional assessment for e-portfolios
• Based on existing rubrics …. However….
Civic Engagement
• Few existing rubrics in civic engagement• Convening of “experts” in the field, by
phone• Common readings• Shared experiences – “what does a
student look like”• Definitional terms
▫ AAC&U ▫ AASC&U▫ Ehrlich definition – see rubric
VALUE Rubrics
• Simple language – used by variety of readers
• Progressively more “robust” learning
• Levels of sophistication; zero can be used
• Single, most vital performance in each cell
• Core, shared dimensions of learning, for all students across all majors
Dimensions of CE Rubric
• Diversity of communities and cultures
• Analysis of knowledge
• Civic-identity and commitment
• Civic communication
• Civic action and reflection
• Civic contexts/structures
Activity: Types of Evidence1. In groups of 3, read a student paper2. Using the AAC&U VALUE Rubric for
Civic Engagement, rate the student submission
3. Together, we will watch a digital story by the same student, rate using the rubric
4. Repeat the exercise for the 2nd paper and digital story
Digital StorytellingThe power of voice
Power of Voice
When the writer is engaged personally with a topic, they impart a personal tone and flavor to the piece that is unmistakably his/hers alone. It is the heart and soul of the writing, the magic, and wit, the feeling, the life and breath.
(http://educationnorthwest.org/resource/503)
“When words are infused by the human voice, they come alive” – Maya Angelou
What is a digital story?
•An illustration of learning•A way of documenting an experience(s)•One way to facilitate the reflection
process▫2-4 minute digital video clip▫First person narrative▫Told in your own voice▫Illustrated (mostly) by still images▫Additional music added to evoke emotions
Examples of Digital Stories• Using www.screenr.com
▫http://screenr.com/6ASs• Using PhotoStory3 (Windows)
▫http://www.youtube.com/watch?v=nqDu4uYtNJM&feature=feedu
• ISL example▫http://streaming.osu.edu/knowledgebank/
digstory08/JacksonFINAL/flash/f.htm• Using iMovie (created by 5th graders)
▫http://www.infotoday.com/MMSchools/jan02/digitalplace.mov
What makes for a great digital story?• Overall purpose of the story• Narrator’s point of view (emotional content)• A dramatic question• Choice of content• Clarity of voice• Pacing of the narrative• Meaningful soundtrack• Quality of the images• Economy of the story detail• Good grammar and language usage
Steps to develop a digital story
1. Write a script: write, get feedback, rewrite, and work with others (maybe in a group) to develop ideas
2. Capture and process the images to further illustrate the story
3. Record the author reading the story4. Combine audio and images onto a
timeline, add music 5. Present/publish
Example StoryboardScript/Sound Effect/Music Image/videoMary had a little lamb
(sound – birds singing, girl humming the tune
Drawing of Mary with her lamb in a field of flowers
Whose fleece was white as snow
(Sounds – girl continues to hum the tune)
Drawing of a snowflake
And everywhere that Mary went
(Sound – girl continues to hum the tune)
Drawing of Mary Walking into the mall
Copyright Laws • These days, almost all things are copyrighted the
moment they are written, and no copyright notice is required.
• Copyright is still violated whether you charged money or not, only damages are affected by that.
• Fair use is a complex doctrine meant to allow certain valuable social purposes. Ask yourself why you are republishing what you are posting and why you couldn't have just rewritten it in your own words.
• Copyright is not lost because you don't defend it. • Fan fiction and other work derived from copyrighted
works is a copyright violation. • Don't rationalize that you are helping the copyright
holder; often it's not that hard to ask permission. http://www.templetons.com/brad/copymyths.html
Just remember…• copyright has two main purposes, namely the
protection of the author's right to obtain commercial benefit from valuable work, and more recently the protection of the author's general right to control how a work is used.
• While copyright law makes it technically illegal to reproduce almost any new creative work (other than under fair use) without permission, if the work is unregistered and has no real commercial value, it gets very little protection. The author in this case can sue for an injunction against the publication, actual damages from a violation, and possibly court costs. Actual damages means actual money potentially lost by the author due to publication, plus any money gained by the defendant.
So what can you use without worrying about Copyright issues?
IMAGES• Flickr (Narrow search to
‘The Commons’)• Google Images (Labeled for
reuse)• Your own!• www.iupui.edu/galleries/• Photos.iupui.edu (need an
account – I’d be more than happy to share – not the same as my CAS login)
MUSIC• http://freeplaymusic.com• www.jamendo.com• Podsafe Audio
(www.podsafeadio.com)• Creative Commons
(http://creativecommons.org/legalmusicforvideos/)
• Yahoo Music (http://new.music.yahoo.com)
• Your own!
Putting it all together•Import all images, video, voice recording,
and musical elements•Lay your narration track onto the timeline•Add your images to match your narration•Create an initial rough cut before adding
transitions or special effects•Add titles, transitions, special effects
sparingly▫Note: these steps may vary depending upon
the program you are using.
Tools (Putting it all together)Mac Web 2.0 WindowsWrite Script Google Docs Write Script: WordRecord Audio: Audacity/GarageBand
Myna (Aviary) Record Audio: Audacity
Edit images: iPhoto Aviary Tools Edit Images: Picasa3Edit Video: iMovie Animoto or Voice
Thread PreziEdit Video: Photostory3 or Windows Movie Maker
But, there are 50+ free tools available and capable of doing something just slightly different. Check these blogs to learn more about the various tools and their capabilitieshttp://cogdogroo.wikispaces.com/StoryTools (5-+ Web 2.0 Ways to Tell a Story)http://50ways.wikispaces.com/ (More recent version of the site above)
Share/Publish
•Motionbox (http://www.motionsbox.com)•YouTube (http://www.youtube.com/)•TeacherTube (http://teachertube.com/)•SchoolTube (http://schooltube.com/)•Blip.tv (http://blip.tv) •Vimeo (http://www.vimeo.com) •Facebook video•Your ePortfolio!
BREAK Come back at 3:00
ePortfolios for Research
Assessment vs. EvaluationStudent Assessment
• Individual or group of learners
• Understand learner through performance of a specific learning task/standard
• Provide feedback to students
• Diagnostic tool for instruction
• Formative• Ex: Reflections
Student Evaluation• Individual/group of learners
• Understand learner through performance of a specific learning task/standard
AND• Judge the quality or worth of
the assessment results• Provide feedback to students• Based upon multiple sources
of assessment information. • Formative/Summative
Research vs. Evaluation
Research, especially fundamental or basic research, differs from evaluation in that its primary purpose is to generate or test theory and contribute to knowledge for the sake of knowledge. Such knowledge, and the theories that undergird knowledge, may subsequently inform action and evaluation, but action is not the primary purpose of fundamental research. (Patton, 2002, pp. 10-11)
Value of ePortfolios for Service Learning
•Most assessment tools are self-report instruments (nationally and locally)
•Eportfolios provide “authentic” assessment evidence/data
•Draw on strengths of Service Learning ▫critical reflection
•Eportfolios can be used for research▫also for course use and program assessment▫designs can be simple or complex
Important Considerations
•Confidentiality (FERPA)•Ownership of materials; permissions•Access to materials and platform after
student leaves or graduates•IRB Issues
▫Informed consent
Examples of Research on Service Learning using ePortfolios
• Project 1: Civic development of students in a Service Learning Assistant program
• Project 2: Development of civic
learning in freshmen taking service learning classes versus non-service learning classes
Project 1Service Learning Assistant Program Overview
• Scholarships recognize IUPUI students selected by faculty or professional staff to:▫assist in the implementation of a service learning
class, ▫collaborate with faculty in their community-based
research, ▫expand the capacity of campus departments to
increase the number of students who participate in service-learning, or
▫complete a service project in and with the community.
• Faculty development focus makes it unique.
Guiding Research Questions
•To what extent do ePortfolios support the civic development of students?
•What types of ePortfolios best facilitate student civic learning and assessment?
•What factors contribute to student civic development as a result of participation in a service-based scholarship program? (Kristin’s research)
Program Specific Research Questions
•To what extent do students who participate in service scholarship programs (compared to their SL peers or non-SL peers):▫develop a greater capacity/ability to
articulate an integrated sense of their personal, civic, professional identity?
▫develop enhanced civic learning KSAs, compared to other students who do not participate in these types of interventions?
Using ePortflios to gather evidence•Civic-Minded Graduate Scale•Complete reflection on a professional
development activity•Complete end-of-award period reflection
(CMG Narrative/Scale)•Faculty mentor rates the end-of-award
reflection (CMG Narrative Rubric)
Project 2
•Development of civic learning in freshmen taking service learning classes versus non-service learning classes
•High-impact practices: ▫First-year seminars▫Themed Learning Communities
•ePortfolios are both a high-impact practice and a tool for gathering evidence
•Evaluate ePortfolios using rubrics
Workshop Reflection:Choice points
• What ePortfolio platforms are available?• What type of ePortfolio do you want?
▫ Free-form vs. structured (how?)▫ Process/assessment purposes (matrix style) vs.
student development (presentation style)• What types of evidence do you intend to gain, and how?
▫ Are you wanting to use a ePortfolio as a repository of the student’s work -OR- to help facilitate reflection after having completed their assignment?
• How might you structure your course or program (learning outcomes, assignments, activities) to gather ePortfolio evidence?▫ ePortfolios CANNOT be an add-on
• How will you evaluate the evidence?▫ Are you grading an assignment, then evaluating their
ePortfolio?
Take-Aways & Next Steps•What is one thing you will take away from this workshop today?
•What is your next step(s)?
Questions? Comments?Feel free to contact us!Julie Hatcher ([email protected]) Kristin Norris ([email protected])Kathy Steinberg ([email protected])
Thank you Epsilen!