http://jw65/ how to run “successful” online discussion boards: some observations from the...

http://www.pages.drexel.edu/~jw65/

How to run “successful” online discussion

boards: Some observations from the

coalface

Jim WatersThe iSchool at Drexel

http://www.pages.drexel.edu/~jw65/

Jim Waters, Drexel University, 2009

But Asynchronous Discussion Boards are old hat !

• Surely Chat is faster and more interactive ?

• Reflection – knowledge building discourse is not a speed sport

• Richer peer interactions


Gauging Success

• Participation• Discourse• Knowledge Building • Consensus• Understanding

If you cannot measure it, you cannot improve it - Lord Kelvin


Participation

Posting messages – how many ?

All messages are not equal“ I agree with Fred”“ I Like cats”“ I see you are an eagles fan”“ yeah it’s week 10 !”“ LOL”“ It is nice to see you again”

My understanding of participation is not yours

Thread length – The K-Tel approach, 25 original hits unrelated messages

Think better not more often !


Discourse

Student-to-Student interactionsStudent-faculty interactions

Apple for teacher ?Appetite for knowledge ?

Deep threads Most participants in a threadIterations – most dyadsInteraction types


Knowledge BuildingIs any knowledge really new ?Schrödinger's cat Is an inference new knowledge ?Content AnalysisLeast measured outcome


Consensus

“So we are agreed , we will land the Cuban exiles in the Bay of Pigs “

“A mode of thinking that people engage in when they are deeply involved in a cohesive in-group, when the members' strivings for unanimity override their motivation to realistically appraise alternative courses of action – Irving L. Janis, Victims of Groupthink, 1972


Understanding

The Turing test ?ElizaInterpretationInferenceExemplificationAbstraction


Small Group Discussion

Cohesive groupsLess cliquey behaviorNon-participation is obviousManageable size for creative projectMixed ability

The strong leaderAn army of followersThe dysfunctional group


Research Context Analyzed online interactions in graduate

classes in information systems and information science

Course interactions via discussion board on Blackboard learning system.

15 online courses over a five year period 325 Students 21,000 messages 4000 message threads


Research Method Collected and analyzed student messages

posted to discussion board and small group discussion

Analyzed Thread depth, thread length Cognitive content of message Patterns of message sequences

Student interactions related to outcomes


Findings I

Are all participants created equal ? Dealing with the Divas ? Rewarding the loudest ? Leave-alone or interact ?

Harangue Encourage Flatter The dark side of rewards

Questions or self-questions ? Question design ? Small group reflections ? In what context Charlie ? How many questions ? Grading discussion board interactions , does it motivate ?


Findings II

Cognitive Outcomes Do Social posts mean less cognitive outcomes How many social posts is acceptable Dyads, cliques and committees – what level of

group interaction is most productive ? How to deal with cliques ? Breaking up the cartels ? How truthful to be in feedback/moderation


Participatory democracy vs. benevolent oligarchy Will an active core of students bring in

peripheral participants or exclude them ? How do we get peripheral participants to

join the circle ? Does it matter if some do not actively

participate ? I meant to ask that but…… Vicarious learners or lurkers ?

Should we moderate core participants dominating the discussion or encourage them ?


Moderated vs. Laissez-faireCase Study III

Two sections of an IS course delivered at the same time – same basic syllabus

Same number of students (25) Selected six “identical” questions on each

section Different Instructor approach Heavy moderation vs. lightweight moderation


Question Heavy Moderation Low Moderation

Systems Analyst as problem solver

69 74

Agile methods 96 97Project design 150 97

Requirements Analysis 96 83Fact Finding 85 90Data Modeling Practice 182 180Average 112 103

Questions and Approach (messages)


Systems Analyst as Problem solver

Heavy Moderation Low Moderation

Total Messages 69 74

Instructor – student messages

17(24%) 0

Deep sub-threads ( 4 levels or greater)

4 6

Student messages 52 74

Student-instructor messages

25 25

Student-student messages

27 52


Agile methods Heavy Moderation Low Moderation



16(17%) 0


10 8

Student messages 80 97Student-instructor

messages52 24


28 73

Deep thread messages (students)

65 44


Agile methods Heavy Moderation Low Moderation



2 8

Deep threads without instructor intervention


Project design Heavy Moderation Low Moderation



44(30%) 0

Deep Threads 18 9

Student messages 106 97Student-instructor

messages53 23


53 74

Deep thread messages (students)

78 46


Goals objectives and scope Heavy Moderation Low Moderation



3 9

Deep threads without instructor intervention


So does providing feedback help ? Moderated Un-moderated

Agile methods (Words) 8823 21203

Agile methods (Messages) 80 97

Project design (Words) 12227 20211

Project Design (Messages) 106 97

SA problem solver (Words) 5220 15714

SA problem solver (messages) 52 74

Tot messages 238 268

Tot words 26270 57128

Average words/student message 110.38 213.16

What about question design ?


Posing questions - general Sometimes a committed , motivated and

interested group with decent moderation will be inert.

Even within the same niche of the same domain some questions just work better than others

Good question design is not trivial even for domain experts

Does the question connect to student experience, real or vicarious

Is the question relevant to the course Does the question represent a well-structured

single knowledge domain goal


Good, bad or average?

I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]

Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]

I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]


And the Winner is

I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]


I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]


GoodI want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]

150 posts (2nd best thread) Several sub-threads extremely deep (7 or 8 levels) Question is level 130% of messages were Instructor to Student29% of messages were student to instructorCritique, feedback, support and facilitationWell-placed faculty moderation, nudges rather than cattle prodsWell-bounded but open-ended problem, students define problemInitial high level of misunderstanding of task (goals vs. objectives) despite material having been formally covered alreadyStudents negotiated the meaning of the task collaborativelyDeliberate pitching as cooperative


AverageI would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]

85 posts (Mean was 91)

Moderate sub-thread depth (mostly 3 or 4 levels)

31% were messages from Instructor to students

20% were messages from students to instructor

Well-placed faculty moderation, focus on challenging assumptions.

Reasonably open-ended problem

Far less cooperative inter-student activity

Not pitched as a cooperative activity

Students not answering a common question, but question is defined


Bad


46 posts

Limited sub-thread depth - mostly 2 (question then single response)

45% were messages from Instructor to students

37% were messages from students to instructor

18% were student-student messages

Faculty intervention much more critical (didactic)

Five questions in one, one open-ended 4 bounded

Very little cooperative inter-student activity

Not pitched as a cooperative activity

One overarching common question


Final Words: Value of online discussion

this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions

I was moved to comment on how refreshing the lack of competition in the Communications for the online classes seemed to me. It was a discussion and a sharing of experiences

Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback

No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to..I've never done this before.

I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.


Worrying or not ?I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.

Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback

Stopped posting after week 2Little opportunity to get drawn inWhat do you do if someone will not contribute ?

Concerned about how posts interpretedMost posts were responded toMost posts positive feedback but no added contentMany posts Social rather than task-orientedRarely posted thread starters


Conclusions I Tentative findings Deep engagement can be encouraged by

thought-leaders Useful thought-leader behaviors highly

context dependent Interaction behaviors interact Faculty feedback improves some forms of engagement but perhaps at the expense of student-student interactions ?

Finite tank of engagement ?


Conclusions II Question design can be crucial to

engagement Discussion needs to be framed as

collaborative not competitive Students recognize core participants,

should we alter our behaviors to fit ? Committed engagement is possible So is isolation and inertia ! Sense of community is not automatic


Related References Waters, J. '

Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008.

Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007.

Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006.

Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot

Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Manua, Hawaii, January 2005, 2005, p. 3b.

A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm

http://www.pages.drexel.edu/~jw65/HICSS-2008-paper.pdf


http://www.pages.drexel.edu/~jw65/HICSS-2007-Workshop-Presentation.ppt

http://www.pages.drexel.edu/~jw65/HICSS-2007-Workshop-Presentation.ppt

http://www.pages.drexel.edu/~jw65/ICIS-2006-paper.pdf

http://www.pages.drexel.edu/~jw65/SLOAN-C-2006-presentation.ppt

http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot


http://www.pages.drexel.edu/~jw65/publications.htm


About Me I am a Doctoral Candidate at the iSchool at Drexel University, Philadelphia

(USA), graduating Summer 2009. My principal research interests lie in

Online Collaborative Knowledge Building, Technology-Supported Learning, Student Role-Behavior in Online Learning Communities and HCI. I am currently employed (2005 - 2008) as a Research Assistant on an IMLS funded project "Toward a Model Curriculum for the Management of Digital Information".

I received a BA in Psychology at Warwick University, UK (1979), an MSc in Occupational Psychology at the University of Hertfordshire, UK (1991) and a MS in Information Systems at the College of IS&T at Drexel University (2002). Prior to my advanced academic studies I enjoyed a substantial career in Systems Design, Management and IS Consultancy.

Email: [email protected]



http://www.ischool.drexel.edu/faculty/csaunders/html/IMLS.html

mailto:[email protected]

http://jw65/ how to run “successful” online discussion boards: some observations from the...

Documents