stephen jones, david monson, & scott howell brigham young university educause, october 4, 2002...

Stephen Jones, David Monson, & Scott HowellBrigham Young University

Educause, October 4, 2002

Atlanta, Georgia

Copyright Statement

Copyright Scott L. Howell, Stephen M. Jones and David W. Monson, 2002. This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Three Presentation Objectives/Questions

1. Stephen Jones - Introduction

2. Scott Howell What happens pedagogically to teaching and learning when a large-scale hybrid course redesign comes to campus? (Pew)

3. David Monson What literally happens to teaching and learning when a CMS system comes to campus?

4. Stephen Jones What are faculty and students saying about the impact of the CMS upon their teaching and learning?

Stephen Jones

Off CampusOff Campus

On CampusOn Campus

Independent LargeStudy Course Redesign


On CampusOn Campus

Stephen Jones

Strategy—leverage technology-enhanced learning for campus, improve GE courses, experiment with cost/space savings

Strategy—”extend the blessings of learning,” anywhere, anytime350 Courses, 120 web-based25K enrollments

(Semester Online)

Independent learning model Inexpensive, scalable Content is of high quality Award winning Faculty can’t edit Doesn’t fit well on campus

Hybrid learning model Expensive Media rich, interactive Award winning Faculty can’t edit Doesn’t fit well off campus Long production cycles



On CampusOn Campus

Stephen Jones

(Semester Online)

Faculty-developedFaculty-developedusing Blackboardusing Blackboard


On CampusOn Campus

Stephen Jones


(Semester Online)

Hybrid model Inexpensive Faculty controlled Developed through iteration Leverages existing content Emphasizes student learning activities

over expensive media development



On CampusOn Campus

Stephen Jones

ResultFaculty gain confidence in teaching with technology, develop and maintain their own courses, and leverage the power of the CMS for improved learning.



(Semester Online)

Stephen Jones


(Semester Online)


On CampusOn Campus

The content faculty create can be further The content faculty create can be further developed for on and off campus use.developed for on and off campus use.


Scott Howell

“The move toward a hybrid education has been quieter than much-hyped efforts to create completely virtual programs. Graham B. Spanier, president of Pennsylvania State University, calls the convergence of online and resident instruction "the single-greatest unrecognized trend in higher education today," and he touted it as part of the vision for his university in a speech last year.”

“'Hybrid' Teaching Seeks to End the Divide Between Traditional and Online Instruction By blending approaches, colleges hope to save money and meet students' needs”

Jeffery R. YoungChronicle of Higher Education, 3/22/02

“Within five years, you'll see a very significant number of classes that are available in a hybrid fashion," says John R. Bourne, a professor of electrical and computer engineering at Franklin W. Olin College of Engineering who is editor of the Journal of Asynchronous Learning Networks. "I would guess that somewhere in the 80- to 90-percent range of classes could sometime become hybrid." And he says he expects to see more students choose to take online courses even if they live on campus.”

“"A strong case is beginning to be made on the basis of research evidence that many students learn better online than face-to-face, and therefore a mixture is the best way," says Mr. Dede. "What proportion that mixture should be would vary from course to course.”

The Pew Program in Course Resign

A $6 million award program conducted by the Center for Academic Transformation at Rensselaer Polytechnic Institute with support from the Pew Charitable Trusts.

Encourages colleges and universities to redesign their instructional approaches using technology to achieve cost savings as well as quality enhancements.

Focuses on large-enrollment, introductory courses, which have the potential of impacting significant student numbers and generating substantial cost savings.

The Pew Program in Course Resign

BYU Awarded $200,000 in Round 3 (2001) to REDESIGN English 115: Freshman English Composition Course

Show PEW Video (5 min 45 seconds)

Instructor Quality andCurriculum Varied Narrowly

Before Redesign After Redesign 42 Classroom Meetings

170 Classrooms(7140 classroom hrs/yr)

2 Faculty/Student Conferences/Yr(15 Minutes)

154 Graduate Instructors/Yr(Classroom Size = 20)

0% Cost Savings

Instructor Quality and Curriculum Varied Widely

Instructor Time Significantly Higher

10 Classroom Meetings

45 Classrooms (1360 classroom hrs/yr – 74% Reduction)

120 Graduate Instructors/Yr(Classroom Size = 25)

40% Cost Savings

Instructor Time Significantly Lower

4 Faculty/Student Conferences/Yr(30 Minutes – 400% Increase)

English 115 (per year)

25% reduction in instructor timeCategory OnlineTraditional

Preparation 101.9 min/week117.56 min/week

Instruction 38.5 min/week132.82 min/week

Grading 146.25 min/week164.87 min/week

Office Hours 0 min/week147.69 min/week

Email 110.38 min/week17.31 min/week

Conferencing 75.63 min/week73.08 min/week

Development 27.38 min/week39.49 min/week

Total 500 min/week692.82 min/week

Two recent findings: First Finding

Two recent findings: Second Finding

“Based on our initial pilot we are encouraged that overall paper quality is higher in the online versus the traditional version of the course. We are also encouraged that there was a significantly higher rating for the introductions and conclusions and focus and organizations. Because these are global writing skills that were a focus in the online modules we developed, we feel the online portions of the course helped improve student writing.”

David Monson

Course Management System Study

Conducted in the summer of 1999

Research team from the McKay School of Education’s Instructional Psychology and Technology Department

Email survey of 270 randomly selected faculty

12 In-depth Interviews

Usability testing of selected commercial products

Purpose of the Study

Identify a suite of online course management tools that:

• collectively meet the instructional needs of faculty

at Brigham Young University,

• address the administrative needs that support

online instruction, and

• are compatible with present and future university

technology standards for online instruction.

Study Goals

Determine present and future needs of BYU faculty for

online course management through a campus-wide

survey and in-depth interviews of faculty.

Use data about faculty needs to identify an ideal feature

set for a suite of tools and support for online course

management.

Identify what is being done at BYU to provide faculty with

online course management support.

Study Goals

Identify what is being done at top universities to provide

faculty with online course management support.

Evaluate commercial online course management products

using the feature set as criteria.

Identify the top products and conduct usability testing with

faculty.

Recommend to the university possible courses of action

for providing faculty with online course management tools

and support.

Functionality Ranked by Current Use

Communication Syllabi Announcements Instructional resources in the classroom Feedback to students Faculty-to-student collaboration Learning resources from other sources Research resources

Findings

Surveys and interviews showed the primary concern of faculty:

• lack of time to use the tools.

• lack of funds that might be used to pay students to do to the work for them, saving faculty time.

• university training and support of the tools provided.

• ease of use.

Usability Tests

Usability tests with BYU faculty from each college and school on-campus were conducted on the top three commercial products:

• Blackboard CourseInfo

• WBTSystems TopClass

• Educational Technologies WebCT

Results: all faculty ranked CourseInfo first for product

ease of use.

Blackboard Adoption and Support

50

450

600

920

716

1409

-190

10

210

410

610

810

1010

1210

1410

Winter '00 Fall '00 Winter '01 Fall '01 Winter '02 Fall '02

Co

urs

es

• Hardware-Software• Support• Marketing

23.9% of sections are using Bb (2,003 of 8,365)

43.4% of enrollments are also Bb enrollments (82,820 of 190,767)

81% of students are enrolled in a Bb course (29,563 of 36,492)

65% of faculty are using Bb (1,626 of 2,492)

Blackboard Adoption and Support

Stephen Jones

Growth of Hybrid Courses

80% of BYU students took one or more courses during the last year that were a mix of F2F and online.

19

36

22

11

6 7

20

29

12

4

12

22

0

5

10

15

20

25

30

35

40

0 1 2 3 4 4+

Per

cent

of R

espo

nden

ts

Freshman Seniors

Faculty User Types

716 courses created, Winter 2002

228 (32%) were empty course “shells”

488 had active users 72 “Power” Users 14.8%

180 Moderate Users 36.9%

236 Light Users 48.4%

(Winter 2002)

BlackboardTM Survey, Winter 2002

• Faculty• 100 most active faculty users

• 60 responses• Students

• All students in those classes • 1697 responses

BlackboardTM Feature Use Data

Announcements 78.3% Course Documents 76.0% Course Information 61.7% Assignments 49.6% Staff Information 45.5% External Links 30.3% Quizzes Online 24.0% Discussion Board 19.3% Groups 2.3% Virtual Classroom 0.6%

(Winter 2002)

Students Faculty

Improve access to course materials 83% 85%

Improve class communication (one and two way) 70% 83%

Provide more flexibility for instructor and students 63% 76%

Instructor just trying out BlackboardTM 26% 27%

Academic dept. encourages instructor to use it 20% 20%

Student demand 16% 4%

Why is Blackboard being used? (Check all that apply)

Student / Faculty Comparison

Yes 78.0%

No 17.3%

Undecided 4.7%

Overall, would you prefer instructors use BlackboardTM in their courses?

BlackboardTM Student Survey

BlackboardTM Instructor Survey

“Many students like BlackboardTM courses. They enjoy working at their own pace, any time, day or night. They like not having to attend class. But many miss the face to face interaction and discussion and the discipline required of a traditional class environment. “ – Audiology & Speech Pathology Professor

How do you think BlackboardTM has been received by your students?

(Winter 2002)


Yes 57%

No 41%

Unanswered 2%

Do you think BlackboardTM is changing the way instructors teach their courses?

(Winter 2002)

“It creates a more encompassing learning atmosphere. The teachers can refer the students to much more material and in a much more interactive environment. Blackboard is awesome.”

“Materials are more readily available. Communication is easier. Quizzes online save valuable class time. More time is spent learning in class and not being tested.”

“In a positive manner, it allows for more discussion among classmates. We can share ideas, and then learn from the ideas of others. On a negative side, some teachers have put so much on the BlackboardTM, that it is assumed that we should find all of the answers.”



87% indicated a change in their practice

13% indicated no change

What is the most significant teaching practice you have (or will) change now that this tool is available?

(Winter 2002)


Post materials online - 40%

Student comments

“I can access grades, quizzes, tests and many other teaching materials online that were otherwise unavailable to me before.”

“I can do it at any computer on campus. I don’t have to carry around tons of papers, and I know I won’t lose it.”


(Winter 2002)


Increase faculty-to-student and student-to-student collaboration and communication - 32%

Student comments “I can contact other students in the class much

easier through Blackboard. Group projects are much smoother when we have the use of Blackboard.”

“Using Blackboard has increased my team member skills.”

Faculty comment “I communicate with my students more frequently

now via Blackboard than I had in the past.”


(Winter 2002)


Provide quizzes and homework online - 10% Use to increase pre-class preparation - 5%


(Winter 2002)

1. Group problem-based learning activities with online collaboration. 2. Organizing out-of-class time. 3. Preparing students for class discussion--they post reactions to reading before coming to class.” – Political Science Professor

I am able to encourage students to learn from each other and to share at a deeper levels. My goal with this course has always been to encourage the use of critical thinking skills. Using BlackboardTM has allowed me to facilitate this type of interaction on a regular basis. Also, I am able to adjust course content to meet the current needs of the student interns. -- Family Science Professor



Are you meeting less often in class and more often online?

StudentsFaculty

Yes 26% 18%

No 74% 82%

Student / Faculty Comparison


“Instead of three times a week, we meet once. But this is great, I have no complaints. I'm learning just as much if not more. I can go at my own speed.”

“My Org. Behavior class has all of our assignments and quizzes on BlackboardTM , with optional classroom lectures six times throughout the semester. I liked it a lot, I could complete my quizzes and things on my own time.”

Student comments

(Winter 2002)


“We'll be in class two days a week instead of three. One day a week is an 'e-day.‘” – Political Science Professor

“Spring Term 2002 I will use the virtual classroom in three 1/2 hour periods during the week and allot additional time for electronic communication with students.” – Zoology Professor

“Yes, I meet more often online. Monday we meet in person, Wednesday in the virtual classroom and Friday my students post to the Discussion Board. – English Instructor

(Winter 2002)

Faculty comments


Yes 13%

Not changed 78%

Decreased 7%

No response 2%

(Winter 2002)

Overall, is your workload as a student changed by an instructor’s use of BlackboardTM ?


“I believe that they think it's easier for us to get more done so all of my teachers who use it pile more work into it.”

“I do more in my on-line class then my friend does in the same class, because there are always tests to make sure that we understand the material and are staying caught up.”

“I believe that with BlackboardTM , I am able to better manage my homework situation; therefore, I have been able to decrease my workload.”

“It is easier when I can use BlackboardTM. It saves my time a lot!”

(Winter 2002)

Student comments

StudentsFaculty

Positive comments 68% 78%

Negative comments 2% 12%

Neutral 1% 0%

No response 29% 10%

Can you think of at least one example of BlackboardTM use that has helped your learning / teaching experience?

(Winter 2002)Student / Faculty Comparison


“I like being able to read other's responses to what we discussed in class.”

“It allows me to access information when I need it. Timing is very important... If I'm working on a project, but am behind a little I can access the information at my own pace.”

“It has made communicating with teachers, TAs, and classmates MUCH easier. Specifically when we are doing group projects BlackboardTM allows us to have a common meeting place.”

“It allows me to take quizzes at my convenience.”

Student comments

(Winter 2002)


“I think I'm doing a better job of focusing the students' learning experience and delivering a specific and meaningful learning outcome.” – Organizational Leadership Professor

Faculty comments

(Winter 2002)


58% responded with at least one comment

42% did not respond

Can you think of at least one example of BlackboardTM use that has frustrated your learning experience?

(Winter 2002)


30.8% - technical problems

5% - Student dislike for online instruction

4.4% - Professors not updating courses enough to be helpful

2.9% - Teachers post assignments right before class

2.8% - Hard to get computer access

Top-ranking categories of negative responses

(Winter 2002)


“I prefer more interpersonal communication as opposed to the mass communication style of BlackboardTM when it comes to my education.”

“Sometimes instructors expect that if they post something on BlackboardTM right before class we will have time to look at it. It seems to encourage professors to procrastinate.”

“…sometimes navigation of the site isn't that easy. Many professors put things in different locations.”

“It's only frustrating when I can't get into it. Like, when…the system is down for a while...”

Can you think of at least one example of BlackboardTM use that has frustrated your learning experience?

(Winter 2002)


“Has provided ease in communicating with class through mass emails, tracking grades on-line, ect.” – Family Science Professor

“The last test… proved clearly that those who used the online materials did very well. Those who didn't had test scores MUCH lower.” – Italian Instructor

“It makes it so that I don't have to be personally available quite as often, which is definitely valuable when you are teaching 200 students in one course.” – Psychology Instructor

Has this tool helped create any efficiencies in managing your courses?

(Winter 2002)


“I, and my TA's, use the Communication link to contact students who have unusually high absences. We encourage them to come and see us. In a large class (166 students) it is way too easy to get lost and lose interest…” - Political Science Professor

“Hybrid is the way to go - part online for efficiency, part classroom for personal touches.” - Counseling & Special Ed Professor

General comments and observations about your BlackboardTM experience including the tool itself.

(Winter 2002)

Seven Principles of Good Practice

Good Practice Encourages Contacts Between Students and

Faculty Develops Reciprocity and Cooperation Among

Students Uses Active Learning Techniques Gives Prompt Feedback Emphasizes Time on Task Communicates High Expectations Respects Diverse Talents and Ways of Learning

Arthur Chickering and Stephen EhrmannAAHE Bulletin, October 1996

“My ASLP teacher has used the discussion board a lot, and I really think that helps in learning. I find it really beneficial to be able to see what other students think and what they are getting out of our assignments.”

Freshman

Contact, Cooperation, Active Learning, Feedback, Diverse Ways

Time on Task“It allowed me to listen in my religion class, and not get caught up in taking notes, . . .”

Freshman

“. . .we could go at our own pace and rate of understanding rather than having to keep up with the class (either being a faster or slower learner than the rest of the class.)

Senior

“. . .at least no time is wasted in class trying to figure out who has what grades.”

Senior

“The more contact I make with them through technology, the more my students seem to believe that I am readily accessible to help them…” – English Instructor

Virtual can = Available

Contact

“Helped me to become more technically literate.”

Freshman

Diverse Ways

“If I can’t remember if we have a homework assignment. No Worries. I don’t stress about it and cry, because I know that I can always go to Blackboard and look up what my assignment is. It has saved me a lot of useless tears.”

Freshman

Reduced Stress

Conclusions, Observations

The ability to extend the classroom using a virtual environment can create a seamless, connected experience for students that Helps organize their out of class time

Connects them to each other and the professor

Puts them in control, offers choice

Greatly improves their access to information

Conclusions, Observations

Teaching a class using a CMS causes faculty to Think reflectively about their practice Align content, assessment, and learning activities Think about the type and quality of student-to-

student interaction needed outside of class Use assessment more strategically and make

grades (and rubrics) more public

Next Steps, Future Opportunities

Move from Uploading documents to creating and publishing

learning objects Online content (“Putting my course online”) to

communities of practice Individual, opportunistic to institutional, strategic

course development Tool training to pedagogical dialogue, improvement “Slide, slide, quiz” to adaptive assessment, intelligent

tutoring Move from hybrid / blended/ e- / online learning to

improving learning Focus away from the technology and toward the goal of

improvement

stephen jones, david monson, & scott howell brigham young university educause, october 4, 2002...

Documents

campus slide

campus use

blackboard slide

stephen jones introduction

scott howell slide

content faculty

georgia slide

campus long production