Teaching in a Web-Based Distance Learning Environment: An Evaluation Summary Based on Four

Courses

Charles Graham, Joni M. Craner,

Byung-ro Lim, & Kursat Cagiltay,

Thomas M. Duffy

Center for Research on Learning and Technology Indiana University

Evaluation Scenario

You have just been asked by the VP of your institution to evaluate its online courses.

She asks you if there is any difference in how you would evaluate an online course compared to a face-to-face course.

What do you tell her?

Center for Research on Learning and Technology, Indiana University

It happened to us . . .

A similar scenario happened to the director of the Center for Research on Learning and Technology at Indiana University. . .

Center for Research on Learning and Technology, Indiana University

Context for the Evaluation

The assistant Dean in a large academic program in mid-west requested an evaluation of their online program

Their evaluation objectives:Assess the quality of online instruction and provide recommendations for improvement to program directorsProvide feedback to the instructors about the quality of their courses and provide recommendations for improvement

Center for Research on Learning and Technology, Indiana University

Evaluation Background

What was being evaluated? We evaluated 4 online courses

Who was conducting the evaluations? A team of 4 evaluators from the

Center for Research on Learning and Technology under the leadership of Tom Duffy

Center for Research on Learning and Technology, Indiana University

Evaluation Criteria

What Criteria was used? They use the 7 principles of Good Practice in

Undergraduate Education (Chickering & Gamson, 1987) to guide their on-campus courses

We decided to try and use the same criteria for online evaluations

Center for Research on Learning and Technology, Indiana University

The Challenge

The 7 Principles seemed to apply to both online and face-to-face courses.

BUT . . .

Specific evaluation strategies provided for each principle in a f2f context often did not apply to the online environment.

Center for Research on Learning and Technology, Indiana University

The 7 Principles of Good Practice(American Association of Higher Education, 1987)

Good practice:1. Encourages student-faculty contact2. Encourages cooperation among students3. Encourages active learning4. Gives prompt feedback5. Emphasizes time on task6. Communicates high expectations7. Respects diversity

Center for Research on Learning and Technology, Indiana University

Data Collection

We evaluated the courses looking for examples and non-examples of the 7 principles of Good PracticeWe formulated a set of guidelines for each principle based on our findingsHow was data collected? Literature review Interviews, and document analyses (course materials,

discussion forums, etc.)

Center for Research on Learning and Technology, Indiana University

Findings

These were our findings . . .

Center for Research on Learning and Technology, Indiana University

Online Guidelines1. Encourage Student-Faculty Contact

Instructors should:Contact students (by phone or private email) who are not participating regularly to keep students engagedClearly communicate email response policies (set student expectations)

Center for Research on Learning and Technology, Indiana University

Online Guidelines1. Encourage Student-Faculty Contact

Encourage students to contact instructor through private email messages with learning problemsInvite students to have on- or off-line appointments with instructorBe respectful to students in all interactions (listserv reply)

Center for Research on Learning and Technology, Indiana University

Online Guidelines2. Encourage Cooperation Among Students

Begin course with a structured ice-breaking activity where students share interests/backgrounds/etc.Develop assignments that require meaningful peer interaction that fosters a sense of community/connectednessEncourage students to share their life experiences

Center for Research on Learning and Technology, Indiana University

Online Guidelines2. Encourage Cooperation Among Students

Require and teach students how to evaluate each other’s work according to specific guidelines (Critical peers)Encourage students to praise and challenge each otherRequire group assignments and provide a mechanism for assessing individual contributions

Center for Research on Learning and Technology, Indiana University

Online Guidelines3. Encourage Active Learning

Ask students to relate course content to real-world issues/experiencesRequire students to do authentic projects and assignments Require students to present their work to the class for peer feedback

Center for Research on Learning and Technology, Indiana University

Online Guidelines3. Encourage Active Learning

Structure assignments and bulletin board discussions activities to require students to challenge the ideas of the instructor, of other students, or those presented in the readings or other course materialsProvide a mechanism for students to add relevant resources for the class

Center for Research on Learning and Technology, Indiana University

Online Guidelines4. Give Prompt Feedback

Provide information feedback on assignments and projects according to the written policyProvide acknowledgement feedback to students within 24 hoursMonitor group bulletin boards regularly and give specific information feedback to students

Center for Research on Learning and Technology, Indiana University

Online Guidelines5. Emphasize Time on Task

Structure assignments with deadlines that are spread throughout the semesterRequire students to participate in the online forum on a regular basisEmphasize the importance of regular work, self-directed learning, sound self-pacing, and scheduling.

Center for Research on Learning and Technology, Indiana University

Online Guidelines5. Emphasize Time on Task

Use easily accessible resources and prepare sufficient resources. Avoid unnecessary time spent on finding materials.

Center for Research on Learning and Technology, Indiana University

Online Guidelines6. Communicate High Expectations

Clearly list expectations from the students on the course Web site Provide examples of exemplary performance to students. Publicly call attention to excellent performance by the studentsReinforce expectations during the duration of the course

Center for Research on Learning and Technology, Indiana University

Online Guidelines6. Communicate High Expectations

Periodically discuss how well the class and individual groups are doing during the course of the semester Develop a mechanism for evaluating individual participation and contribution to group projects.

Center for Research on Learning and Technology, Indiana University

Online Guidelines7. Respect Diverse Talents and Ways of Learning

Learn about the different backgrounds and interests of the studentsAllow students to shape their own coursework by choosing project topicsEncourage students to express their diverse points of view

Center for Research on Learning and Technology, Indiana University

Human Computer Interaction Recommendations

Consistency of Web site layout and designClear Organization and Presentation of InformationConsistent and easy-to-use navigationAesthetically Pleasing Design and Graphics

Center for Research on Learning and Technology, Indiana University

Instructor Sharing and Collaboration

Problem

Asynchronous conferencing requires different methods and strategies from face-to-face conferencing.

Center for Research on Learning and Technology, Indiana University

Guidelines for using Asynchronous Conferencing

Effectively

Require student participation (make grade dependent on it)Students need a task to focus their discussion The task should have a product or something that brings closure The task must engage the learner in the content The discussion structure should be carefully thought out

Center for Research on Learning and Technology, Indiana University

Guidelines for using Asynchronous Conferencing

Effectively (cont’)

Discussion must be evaluated based on quality of content and not length of posting or number of postings Instructors should post examples of expectations for discussions Students must get some kind of feedback on the discussions Discussion groups should remain relatively small

Center for Research on Learning and Technology, Indiana University

Course Management

Problem

Managing a course online was much different from managing a face-to-face course. Instructors start to burn out trying to manage interactions in the same way they might in the classroom.

Center for Research on Learning and Technology, Indiana University

Online Course Management Strategies

Increase emphasis on peer evaluation and feedback. Increase the emphasis on group work. Selectively evaluate discussions. Set clear feedback expectations.

Center for Research on Learning and Technology, Indiana University

Questions?

Center for Research on Learning and Technology, Indiana University

Contact Information

Presentation Online at: http://php.indiana.edu/~crgraham/dl2000.ppt

Paper Online at:http://crlt.indiana.edu/crlt00-13.pdf

Email Contact:Charles Graham: crgraham@indiana.edu Byung-Ro Lim: byunlim@indiana.eduJoni Craner: jcraner@indiana.edu

Center for Research on Learning and Technology, Indiana University