merlot: the peer review of digital scholarship professor cathy owens swift georgia southern...
TRANSCRIPT
![Page 1: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/1.jpg)
MERLOT: The Peer Review of Digital Scholarship
Professor Cathy Owens Swift
Georgia Southern University
Professor Susan M. Moncada
Indiana State University
Professor Theresa B. Flaherty
James Madison University
AACSB (April 26, 2003)
![Page 2: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/2.jpg)
Presentation
• What is MERLOT? – Swift
• The Peer Review Process – Moncada
• Benefits of Submitting Online Learning Materials to MERLOT & Peer Reviewing – Flaherty
• Benefits to You (Dean) - Swift• Questions or Comments
![Page 3: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/3.jpg)
What is MERLOT?
Cathy Owens Swift
Georgia Southern University
![Page 4: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/4.jpg)
Common Issues
- Quality Control- Dissemination- Sustainability
• Institutions invest in developing instructional software without provisions for:
• Lack of quality, interactive, web-based learning materials
• Faculty often work in isolation while developing online courses
![Page 5: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/5.jpg)
Evaluation of Digital Scholarship
• Chronicle of Higher Education (6/2/2000) “Counting Digital Scholarship & Teaching in Faculty Evaluations.” – Need peer review of teaching innovations
• Syllabus, (1/2001) “Faculty Rewards in Digital Instructional Environments”
• Carnegie Teaching Academy:– Scholarship of teaching definition includes “peer review”
• Physics Model of Scholarship of Teaching:– Discussions on “review and evaluation by acknowledged
national experts”
![Page 6: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/6.jpg)
• Faculty should remain in control of the teaching/learning process.
• Peer-reviews will contribute to expanded use and effectiveness of digital learning materials.
Assumptions
• Sharing of materials will maximize everyone’s investments.
• Faculty need mechanisms to document teaching and learning contributions.
![Page 7: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/7.jpg)
Vision & Mission
• MERLOT’S VISION is to be the place where faculty from around the world will share teaching-learning materials and pedagogy.
• MERLOT’S MISSION is to improve the effectiveness of teaching & learning by expanding the quantity and quality of peer-reviewed online learning materials that can be easily incorporated into faculty designed courses.
![Page 8: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/8.jpg)
Aspects of MERLOT• MERLOT is a COOPERATIVE
• Institutional Partners• Organizational Affiliates• Individual Members
• MERLOT is a SET OF PROCESSES• Building, organizing, reviewing, and developing applications of
online teaching-learning materials • Building and sustaining online academic communities.
• MERLOT is SOFTWARE• A user-centered, searchable database of online learning
materials, pedagogical support, and people.
![Page 9: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/9.jpg)
MERLOT follows the model of peer review of scholarship
IndividualReview #1
IndividualReview #2
CompositeReview
![Page 10: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/10.jpg)
MERLOT Alliance Partners
•National Science Foundation (NSF) Grant • $400,000• Online peer review training module
• NLII of Educause• AAHE – American Association of Higher Education•Education.au Limited•IMS Global Learning Consortium, Inc•Health Education Assets Library•NEEDS – Digital Library for Engineering•SMETE.org
![Page 11: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/11.jpg)
Discipline Communities
Business
Biology
Chemistry
Engineering
Health Sciences
History
Info Tech
Music
Math
Physics
Psych
Teacher Ed
Teachingw/Tech
World Languages
![Page 12: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/12.jpg)
Parties Involved in MERLOT
• Administration Team (12)• System Partners (22)• Campus Partners (2)• Discipline Editorial Boards (14)
– Editors/Co-Editors (25)– Editorial Review Board Members (150)
• Peer Reviewers (100+)
• Others – Authors, Submitters, Users
![Page 13: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/13.jpg)
Institutional Partner Commitments
• Faculty Reviewers– 6-8 faculty to serve as experts
• Faculty Development & Academic Technology Personnel
• Travel– Training, presentations, meetings
• Project Director (10+%)– Planning, coordinating, budgeting, etc.
• Participation Fee: $25K
![Page 14: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/14.jpg)
MERLOT: The Peer Review Process
Susan M. Moncada
Indiana State University
![Page 15: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/15.jpg)
Editorial Boards
Support development of discipline communities (14)
Editorial Review Board Members (150)– Editors/Co-Editors (25)– Associate and Assistant Editors– Peer Reviewers (100+)
Support development of discipline communities (14)
Editorial Review Board Members (150)– Editors/Co-Editors (25)– Associate and Assistant Editors– Peer Reviewers (100+)
![Page 16: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/16.jpg)
Board Qualifications
Expertise in scholarship of their field Excellence in teaching Experience in using technology in
teaching and learning Connections to professional
organizations Experience in conducting peer reviews
of online learning resources
Expertise in scholarship of their field Excellence in teaching Experience in using technology in
teaching and learning Connections to professional
organizations Experience in conducting peer reviews
of online learning resources
![Page 17: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/17.jpg)
Board Responsibilities
Expand and manage the collection Implement the peer review process Post peer reviews Recruit and train peer reviewers Education and outreach to the
community of educators
Expand and manage the collection Implement the peer review process Post peer reviews Recruit and train peer reviewers Education and outreach to the
community of educators
![Page 18: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/18.jpg)
MERLOT’s Business Discipline
• Accounting• Business Law• Economics• E-commerce• Finance• General
• Accounting• Business Law• Economics• E-commerce• Finance• General
• Information Systems
• International Business
• Management• Marketing
• Information Systems
• International Business
• Management• Marketing
![Page 19: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/19.jpg)
TYPES OF MODULES
• Simulations
• Tutorials
• Animations
• Drills & Practice
• Simulations
• Tutorials
• Animations
• Drills & Practice • Quiz/Tests
• Lecture/Presentations
• Collections
• Reference materials
• Quiz/Tests
• Lecture/Presentations
• Collections
• Reference materials
![Page 20: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/20.jpg)
Profile of the MERLOT Learning Module
Title, author and affiliation Peer Review Link User Comments Link Type of learning material Location (URL for the module) Subject classification Description Submitter Audience
Title, author and affiliation Peer Review Link User Comments Link Type of learning material Location (URL for the module) Subject classification Description Submitter Audience
![Page 21: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/21.jpg)
Editor’s Evaluation Process
• Stage 1:– Cursory review to identify worthy modules– Post triage comments and triage value online
• Stage 2:– Editor assigns “worthy” materials to reviewers– Reviewers apply MERLOT standards to write
reviews.
• Stage 1:– Cursory review to identify worthy modules– Post triage comments and triage value online
• Stage 2:– Editor assigns “worthy” materials to reviewers– Reviewers apply MERLOT standards to write
reviews.
![Page 22: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/22.jpg)
MERLOT follows the model of peer review of scholarship
IndividualReview #1
IndividualReview #2
CompositeReview
![Page 23: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/23.jpg)
Editor’s Evaluation Process
Stage 3:Editor sends review to author for feedback and
permission to post– Authors can elect to modify materials and
request review be modified– Authors can request module be pulled from the
repository– Authors can request 2 letters from MERLOT
summarizing peer review process and report to 2 people of their choice.
Stage 4:Peer review is posted
Stage 3:Editor sends review to author for feedback and
permission to post– Authors can elect to modify materials and
request review be modified– Authors can request module be pulled from the
repository– Authors can request 2 letters from MERLOT
summarizing peer review process and report to 2 people of their choice.
Stage 4:Peer review is posted
![Page 24: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/24.jpg)
Standard Evaluation Criteria (Strengths & Concerns)
1. Quality of Content
1. Quality of Content
2. PotentialEffectiveness
2. PotentialEffectiveness
3. Ease ofUse
3. Ease ofUse
![Page 25: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/25.jpg)
1. Quality of Content
Current and relevant Accurate information Clear and concise Informed by scholarship Completely demonstrates concepts Flexibility Integrates/summarizes concept well
Current and relevant Accurate information Clear and concise Informed by scholarship Completely demonstrates concepts Flexibility Integrates/summarizes concept well
![Page 26: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/26.jpg)
2. Potential Effectiveness
• Specifies learning objectives• Identifies prerequisite knowledge• Is very efficient • Reinforces concepts progressively• Builds on prior concepts• Demonstrates relationships between
concepts
• Specifies learning objectives• Identifies prerequisite knowledge• Is very efficient • Reinforces concepts progressively• Builds on prior concepts• Demonstrates relationships between
concepts
![Page 27: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/27.jpg)
3. Ease of Use
• Is easy to use• Has clear instructions• Is engaging• Has visual appeal• Is Interactive• Uses effective navigation techniques• All elements work as intended
• Is easy to use• Has clear instructions• Is engaging• Has visual appeal• Is Interactive• Uses effective navigation techniques• All elements work as intended
![Page 28: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/28.jpg)
Star Rating System
Excellent all around
Very good w/few minor concerns (4.0-4.9)
Meets/exceeds standards with some significant concerns (3.0-3.9)
Standards not met, some limited value (2.0-2.9)
Not worth using at all (1.0-1.9)
![Page 29: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/29.jpg)
Standards for Scholarly Work*
• Endeavors require high level of discipline expertise
• Breaks new ground and is innovative• Is of significance• Can be replicated or elaborated upon• Can be documented• Has the potential to be peer reviewed * (Merton, 1973)
• Endeavors require high level of discipline expertise
• Breaks new ground and is innovative• Is of significance• Can be replicated or elaborated upon• Can be documented• Has the potential to be peer reviewed * (Merton, 1973)
![Page 30: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/30.jpg)
Benefits of Submitting Online Learning Materials to MERLOT & Peer Reviewing
Theresa B. Flaherty
James Madison University
![Page 31: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/31.jpg)
Who contributes materials to MERLOT?
• MERLOT materials are added by people who have joined MERLOT (MERLOT members). – Anyone may join MERLOT, and there is no
cost or other obligation.
• Materials may be added by:– the people who created them (author), or – any member who finds a great resource to
share with others (submitter).
![Page 32: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/32.jpg)
How to Contribute Modules to MERLOT as an Author
Step 1: Develop your Module
![Page 33: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/33.jpg)
Step 2: Fill out a shortform to contributeyour material. Can send a note to Editorto request peer review.
![Page 34: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/34.jpg)
![Page 35: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/35.jpg)
The Peer Review of this Module
![Page 36: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/36.jpg)
Avoids “reinventing the wheel”
![Page 37: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/37.jpg)
Documents Teaching Efforts
![Page 38: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/38.jpg)
Provides Networking
Opportunities
![Page 39: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/39.jpg)
Search for members by discipline
![Page 40: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/40.jpg)
Peer Reviews are developed through online
workspace
![Page 41: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/41.jpg)
The peer review can be completed in different segments
![Page 42: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/42.jpg)
After completion of the review, the Peer Reviewer is recognized for his/her contribution by a letter from the Editor. This letter can be sent to two other people as well.
![Page 43: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/43.jpg)
To become a Peer Reviewer, an individual must:
• be an instructor at an institution of higher learning
• demonstrate expertise in the discipline• be recognized for excellence in teaching• have experience using technology in
teaching, and • have participated in the activities of the
discipline.
![Page 44: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/44.jpg)
Benefits of MERLOT(Dean’s Perspective)
• Qualitative & Quantitative Dimensions to Teaching Efforts– Peer Review is Available for General Public
• Assist in Evaluation of Faculty Efforts (P&T)• Encourage Faculty to Develop Digital Materials
that benefit multiple disciplines• Encourage Faculty to Provide Service to their
Disciplines as Peer Reviewer– Training Provided by MERLOT
• Encourage Faculty to Use Digital Materials
![Page 45: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/45.jpg)
Your Participation in MERLOT • System Partner• Campus Partner• Inform Faculty of Opportunities
– Author– Submitter– User– Peer Reviewer
• Conduct Faculty Training• Recognize MERLOT Contributions of Faculty• Join MERLOT
![Page 46: MERLOT: The Peer Review of Digital Scholarship Professor Cathy Owens Swift Georgia Southern University Professor Susan M. Moncada Indiana State University](https://reader035.vdocuments.net/reader035/viewer/2022070410/56649eb65503460f94bbf521/html5/thumbnails/46.jpg)
Questions or Comments?