http://jw65/ engagement and participatory democracy in online communities of inquiry jim waters the...
TRANSCRIPT
http://www.pages.drexel.edu/~jw65/
Engagement and participatory democracy
in online communities of inquiry
Jim WatersThe iSchool at Drexel
Jim Waters, Drexel University, 2008
What is the BIG problem ?
How do we get students to engage in a committed collaborative knowledge building enterprise in an online learning setting ?
Engaging students is hard enough when you can see them in front of you … … When they are remote, it is even harder.
Jim Waters, Drexel University, 2008
What do we know about online learning ?
Rapid growth in online education (Allen and Seaman 2007;
Griffiths and Gatien 1999) Efficiency gains, flexibility for consumers and the potential
for more effective learning (Benbunan-Fich and Hiltz 1999; Benbunan-Fich, Hiltz and Turoff 2001; Hiltz and Wellman 1997)
Longer preparation times (Burgess and Strong 2003), Anecdotal nature of many positive results (McNaught,
Kenny, Kennedy and Lord 1999) Lack of satisfaction with the process experienced by many
learners (Ocker and Yaverbaum 1999).
Jim Waters, Drexel University, 2008
Dissatisfaction may be related to our traditional model of teaching and learning
Jim Waters, Drexel University, 2008
So what else do we know ? Collaborative activity important in constructing knowledge
(Stahl 2006; Lave and Wenger 1991; Jonassen, Mayes and McAleese 1993)
Online students exhibit the behaviors found in a Social Network (Garton, Haythornthwaite and Wellman 1997; McDonald, Noakes, Stuckey and Nyrop 2005) Weak Ties (Granovetter, 1973)
Even without formal roles being assigned , leadership behaviors emerge in online settings (Heckman and Misiolek 2005; Carte, Chidambaram, and Becker 2006)
Students display different levels of engagement, depending on psychological involvement (Kappelman & McLean 1992)
Jim Waters, Drexel University, 2008
Alternative models emphasize the social construction of knowledge
Jim Waters, Drexel University, 2008
Newer models view learning as taking place within a community of inquiry
Stahl 2006
Garrison et al 2001
Jim Waters, Drexel University, 2008
The community of inquiry concept is not new …
There is a community engaged in inquiry. Inquiry is an open-ended process with positive feedback.
Dewey (1916)
Jim Waters, Drexel University, 2008
But engagement with social construction in learning adds new dimensions
Cognitive Artifacts
Learning
Consensus
Product
Knowledge
Iteration
Collaboration
Internalization
Externalization
A Community of Inquiry (Garrison et al 2001)
Material
Jim Waters, Drexel University, 2008
Research Question
How may we encourage deep, constructivist learning within an online community of inquiry?
Jim Waters, Drexel University, 2008
Research Context Analyzed online interactions in graduate classes
in information systems and information science Graduate students are mostly experienced professionals,
who are motivated to engage with education Findings can inform how we implement “best practice”
for such students Course interactions via discussion board on
Blackboard learning system Most common channel of communication for online
courses Discussion boards used as substitute for in-class debate Issue is how to ensure greatest engagement with
learning via this channel of communication.
Jim Waters, Drexel University, 2008
Research Method Initial study: 2 ten-week graduate courses Collected and analyzed student messages posted
to discussion board Analyzed on a per-message level, to understand online
interaction behaviors Content analysis also performed, to understand degree
of engagement Analyzed
Type of message, Patterns of message sequences
Threads and sub-thread interactions Social interactions
Student behaviors related to learning outcomes
Jim Waters, Drexel University, 2008
Posted message From To Behavior
The value chain model doesn't fit today's business model… The correct sequence should put marketing and sales in the first place. Manufacturing, storage, distribution should rely on customer orders. …
S4 All Contributor
The value chain bugged me, too, when I first read it. On reading it again the light dawned. The text is only showing one version of the value chain, when in fact there are several. Which one is applicable to a business depends on their business model. … <elaborates>
S11 S4 Complicator
I think it is also difficult to understand the value chain because we are reading it from a static standpoint. Depending on what phase the product is in, i.e. is it new and the first batch is being processed, has it been around for a while. I would certainly like to see #4, Marketing & Sales to be first or second. … <suggests alternatives>
S12 S11 Complicator
I agree with S12. Product maturity is not so relevant in an ETO world because most end items are built only for one sale. But in a typical retail world I can see how there would be differences based on product maturity.; The text's value chain almost looks like it is for a brand new product.
S11 S12 VicariousAcknow-ledger
I have to go the other way. To me, the text's value chain seems best suited for an established product. As others have observed, there is no initial step where the need for the product is determined. That seems to imply a known demand that is being met. … S11 made some great points about different delivery processes. I think this model could be used for all of those.
S13 S11 Facilitator
I've been doing some thinking in this area. Education is different in many respects from manufacturing. … I got the sense that there is a kind of chicken/egg thing going on with the value-chain model. Without sales there is no need for inputs, but without manufacturing, etc. there is nothing to sell. Unless products are marketed, there is no sales. So it’s circular.
S9 All Closer
Jim Waters, Drexel University, 2008
Learning Interaction Behaviors Analysis revealed 8 types of interaction behaviors
Silence (Vicarious learner) Contractual Obligation (Contributor) Social connector (Initiator) I agree with Fred (Vicarious Acknowledger) How do I ? (Knowledge elicitor) Active Facilitation (facilitator) Reframing (complicator) Synthesis (closer)
Behaviors are fluid and interact (feedback loop) Some interactions are more productive than others It is possible to learn actively though invisibly.
Jim Waters, Drexel University, 2008
Initial Research Findings The most attended to participants posted
more facilitation messages (39% of messages posted)
The least attended to participants typically posted far fewer facilitation messages. (23% of messages posted).
Students quickly identify valuable contributors
Student-Student interactions become more frequent over time compared with Student-Instructor interactions
Jim Waters, Drexel University, 2008
Detailed Research Questions
1. Do various patterns of behavior lead to different levels of student engagement with learning?
2. Do various types of student interaction lead to different levels of student engagement with learning?
3. How can we encourage patterns of behavior or types of student interaction that lead to deeper levels of student engagement with learning?
Jim Waters, Drexel University, 2008
Three Levels of Engagement
Waters & Gasson (2006)
Level Form of Activity Predicted Outcomes
Participation Observable behavior that denotes interaction with course materials through passive activity.
Superficial learning, resulting from acquisition of terminology and content or domain definitions of knowledge (Externalization)
Involvement Behavior that indicates psychological state of identification with course objects.
Contextually-situated learning, results from active construction of knowledge within learning community. (Internalization)
Social Engagement
Behavior indicating commitment to the facilitation and direction of sustained learning.
Deep learning, that results from the active co-construction of knowledge with peer learners. (Objectivation through cycles of internalization & externalization.)
Jim Waters, Drexel University, 2008
Earlier research 2004 - 2007
Social network behaviors (Haythornthwaite 1999) In-degree and out-degree (centrality) Centrality correlates with status (a bit) Interaction behavior correlates strongly with
centrality Cliquing over time
In-degree and out-degree decline universally
Jim Waters, Drexel University, 2008
Earlier research 2004 - 2007
Social engagement Levels of engagement
Deep Iterative, collaborative knowledge building Identification with community and commitment
to group learning Problem framing important to engagement Strong Core individuals (Thought-leaders)
relate a community understanding to professional expertise or domain-knowledge
deeper levels of course engagement than other students
Drive engagement process for peers
Jim Waters, Drexel University, 2008
Thought-Leaders in Online Debate
Provide impetus to discussion Quality not volume Facilitate and complicate Responders and Inspirers (Inspirational responders) Critique, refine, reframe Encourage, moderate, enthuse
Jim Waters, Drexel University, 2008
Thought-leaders encouraging discussion
Jim Waters, Drexel University, 2008
Thought-leaders as exclusionary ?
Jim Waters, Drexel University, 2008
Weaving threads into a Cloth To what extent do interaction behaviors affect quality of
online engagement, learning outcomes and learning satisfaction ?
How do we recognize and encourage engagement ? How do we recognize a breakdown in engagement ? To what extent do interaction behaviors concord with
perceptions of students as thought-leaders ? Does the presence of thought-leaders affect quality of
online discussion ? Are there any common factors that identify thought-leaders
? Are thought leaders a good idea ?
Participatory democracy (Dewey, 1916) vs. benevolent oligarchy
Jim Waters, Drexel University, 2008
Participatory democracy vs. benevolent oligarchy Will an active core of students bring in
peripheral participants or exclude them ? How do we get peripheral participants to
join the circle ? Does it matter if some do not actively
participate ? I meant to ask that but…… Vicarious learners or lurkers ?
Should we moderate core participants dominating the discussion or encourage them ?
Jim Waters, Drexel University, 2008
Data 11 Drexel Online Graduate Courses Mix of IS and Library Science 240 Students Open-ended and Procedural material Blackboard™ Weekly Online Discussion Board Faculty committed to collaborative
learning approach
Jim Waters, Drexel University, 2008
The discussion board as a tool for engagement
Asynchronous discussion boards Guided collaborative debate Allow participants time to read prior posts
and reflect on them before making their own contributions (English and Yazadani 1999; Lipponen 2001)
Problem-based approach
How do students feel about this ?
Jim Waters, Drexel University, 2008
The Glass is Half-Full?Absolutely! This is the core of online learning
I enjoy the blackboard discussions because it gives a sense of community to an online course.
I do enjoy online collaboration and feel that this is a wonderful opportunity to learn from professionals with varied experiences and backgrounds
Yes, I like collaborating in online discussions. Different perspectives add value to the discussion
The exchange of ideas and the associated discussion is at the heart of the academic experience
I find that the discussions are more productive and that people make more substantial and better considered contributions when writing to the discussion boards. In addition, it is easier for me because I have time to read, think, and digest the material before crafting a response
Jim Waters, Drexel University, 2008
Or is it half empty ?I do not enjoy discussion boards at all. I would much prefer figuring out the content of the course by myself and emailing the professor for assistance when need be, or posting my question on the discussion board. I would rather not have to endure discussion board participation.
About seventy percent of the time I think that it’s an annoyance. … Of all the aspects of the online experience, the discussion board is my least favorite. So far I haven’t felt much like I’ve made any personal connections with my fellow students.
With the discussion boards, you have to make an effort to read the other students’ posts and sometimes it’s tempting not to.
Generally, no. From what I have seen so far here at Drexel, the discussions tend to get a bit grandstanding-ish and there’s always the one student who has to answer everyone and be first to get to everything and has cutsie messages for the teacher… same as in an onsite classroom, but more annoying online because you have to read everything to be sure you’re not missing anything important.
Jim Waters, Drexel University, 2008
Who are thought-Leaders?Case Study I
Lib Science course, 25 students Surveyed students opinions Strong agreement on leaders (2) S6 (85%) S13 (85%) How are they different, if at all
Jim Waters, Drexel University, 2008
Post CountStudent Posts Student-student Student-Inst
S13 19 12 7
S6 30 21 9
Average 13.24 9.72 3.52
S10 35 26 9
S24 21 18 3
S11 17 14 3
Jim Waters, Drexel University, 2008
So what is different about them ?
S6 Technical writer Teaching experience Limited Domain Knowledge Limited online learning experience Extensive work experience Discussion board skeptic
Generally, no. From what I have seen so far here at Drexel, the discussions tend to get a bit grandstanding-ish and there’s always the one student who has to answer everyone and be first to get to everything and has cutsie messages for the teacher… same as in an onsite classroom, but more annoying online because you have to read everything to be sure you’re not missing anything important
Jim Waters, Drexel University, 2008
So what is different about them ? S13 New graduate (Arts) Limited Domain Knowledge No online learning experience No teaching experience No opinion on discussion boards
Jim Waters, Drexel University, 2008
How do they behave ?
Not particularly social Neither attempts to build a social network Both task oriented Both top inspirers (posts elicit responses) Average responders More frequently read – NO Did they talk to each other – not really
Jim Waters, Drexel University, 2008
The Thought-leaders’ posts Facilitate
S23, your question stirred up my thoughts on librarianship. Have any of you had any practical experiences here? Any ideas on how to handle this? Interesting!
Encourage It's interesting that you said "reference hat" because it
reminded me of something… Reframe
the problem of inadequate users is completely different from solving the problem of hostile users because the former has problems of illiteracy and inability to use services, while the latter are well-educated but underestimate the usefulness of libraries. Using the same techniques for each of these non-users would be very ineffective
Jim Waters, Drexel University, 2008
IS course 25 students Surveyed student opinion Two strong thought-leaders S15 & S18
Who are thought-Leaders?Case Study II
Jim Waters, Drexel University, 2008
S15 & S18Similar backgroundsStrongest domain knowledgeStrongest practical experienceNot great inspirersAverage responders
Jim Waters, Drexel University, 2008
S18
Authoritative Posts Opinion supported by external sources Real world examples (own and 3rd party) Synthesizes arguments Reframes debate, challenges question Some positive feedback Long detailed posts, but few posts Limited involvement in deep threads A Closer – attempts definitive answer
Jim Waters, Drexel University, 2008
S15
Collaborative (collegial) tone Opinion supported by external sources,
though not as much as S18 Hypothetical examples “suppose they…” Strong positive feedback for peers Short posts Frequent posts Strong involvement in deep threads
Jim Waters, Drexel University, 2008
How does this help ? Context is crucial No approach has universal utility Domain differences ? Experience not always crucial ! Collaborative spirit vs. Authority It depends
So, if peer behavior is important to engagement, to what extent can faculty intervention encourage engagement ?
Jim Waters, Drexel University, 2008
Moderated vs. Laissez-faireCase Study III
Two sections of an IS course delivered at the same time – same basic syllabus
Same number of students (25) Selected six “identical” questions on each
section Different Instructor approach Heavy moderation vs. lightweight moderation
Jim Waters, Drexel University, 2008
Question Heavy Moderation Low Moderation
Systems Analyst as problem solver
69 74
Agile methods 96 97Project design 150 97
Requirements Analysis 96 83Fact Finding 85 90Data Modeling Practice 182 180Average 112 103
Questions and Approach (messages)
Jim Waters, Drexel University, 2008
Systems Analyst as Problem solver
Heavy Moderation Low Moderation
Total Messages 69 74
Instructor – student messages
17(24%) 0
Deep sub-threads ( 4 levels or greater)
4 6
Student messages 52 74
Student-instructor messages
25 25
Student-student messages
27 52
Jim Waters, Drexel University, 2008
Agile methods Heavy Moderation Low Moderation
Total Messages 96 97
Instructor – student messages
16(17%) 0
Deep sub-threads ( 4 levels or greater)
10 8
Student messages 80 97Student-instructor
messages52 24
Student-student messages
28 73
Deep thread messages (students)
65 44
Jim Waters, Drexel University, 2008
Agile methods Heavy Moderation Low Moderation
Total Messages 95 97
Deep sub-threads ( 4 levels or greater)
2 8
Deep threads without instructor intervention
Jim Waters, Drexel University, 2008
Project design Heavy Moderation Low Moderation
Total Messages 150 97
Instructor – student messages
44(30%) 0
Deep Threads 18 9
Student messages 106 97Student-instructor
messages53 23
Student-student messages
53 74
Deep thread messages (students)
78 46
Jim Waters, Drexel University, 2008
Goals objectives and scope Heavy Moderation Low Moderation
Total Messages 150 97
Deep sub-threads ( 4 levels or greater)
3 9
Deep threads without instructor intervention
Jim Waters, Drexel University, 2008
So does providing feedback help ? Moderated Un-moderated
Agile methods (Words) 8823 21203
Agile methods (Messages) 80 97
Project design (Words) 12227 20211
Project Design (Messages) 106 97
SA problem solver (Words) 5220 15714
SA problem solver (messages) 52 74
Tot messages 238 268
Tot words 26270 57128
Average words/student message 110.38 213.16
What about question design ?
Jim Waters, Drexel University, 2008
Posing questions - general Sometimes a committed , motivated and
interested group with decent moderation will be inert.
Even within the same niche of the same domain some questions just work better than others
Good question design is not trivial even for domain experts
Does the question connect to student experience, real or vicarious
Is the question relevant to the course Does the question represent a well-structured
single knowledge domain goal
Jim Waters, Drexel University, 2008
Good, bad or average?
I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]
I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]
Jim Waters, Drexel University, 2008
And the Winner is
I would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]
I want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]
Jim Waters, Drexel University, 2008
GoodI want you to cook up a systems development project (real or imagined). Describe the goal(s), the objective(s) of the project and the scope of the work the systems analyst for the project. Post your goals, objectives and scope by around Thursday of this week. I'd then like each of you to comment a bit on each other's work. [cooking up a new project]
150 posts (2nd best thread) Several sub-threads extremely deep (7 or 8 levels) Question is level 130% of messages were Instructor to Student29% of messages were student to instructorCritique, feedback, support and facilitationWell-placed faculty moderation, nudges rather than cattle prodsWell-bounded but open-ended problem, students define problemInitial high level of misunderstanding of task (goals vs. objectives) despite material having been formally covered alreadyStudents negotiated the meaning of the task collaborativelyDeliberate pitching as cooperative
Jim Waters, Drexel University, 2008
AverageI would like each of you to initially focus on one fact finding technique, your contribution should be a critical (but brief) examination of that technique within the domain of systems analysis. [fact-finding]
85 posts (Mean was 91)
Moderate sub-thread depth (mostly 3 or 4 levels)
31% were messages from Instructor to students
20% were messages from students to instructor
Well-placed faculty moderation, focus on challenging assumptions.
Reasonably open-ended problem
Far less cooperative inter-student activity
Not pitched as a cooperative activity
Students not answering a common question, but question is defined
Jim Waters, Drexel University, 2008
Bad
Critically evaluate the author's FAST approach. Is it useful? Practical? What are some alternatives? Is this a "real" model that could be used on "real" projects? [Fast or Slow]
46 posts
Limited sub-thread depth - mostly 2 (question then single response)
45% were messages from Instructor to students
37% were messages from students to instructor
18% were student-student messages
Faculty intervention much more critical (didactic)
Five questions in one, one open-ended 4 bounded
Very little cooperative inter-student activity
Not pitched as a cooperative activity
One overarching common question
Jim Waters, Drexel University, 2008
Final Words: Value of online discussion
this was so helpful because often I was struggling with the same thing so I could learn from their errors and gain new information from the answers to their questions
I was moved to comment on how refreshing the lack of competition in the Communications for the online classes seemed to me. It was a discussion and a sharing of experiences
Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback
No question that the on line discussion was critical to getting me through the class. There were mostly questions about how to..I've never done this before.
I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.
Jim Waters, Drexel University, 2008
Worrying or not ?I felt lost and inexperienced most of the time. I have no real full time work experience and I felt I had nothing much to contribute and compared to the rest of the posts mine would feel really insignificant.
Honestly, in the second half of the course, I have felt like I must be a pariah. Apart from the professor, I can't get anyone to respond to my posts- a very lonely feeling. I have posted to the the weekly board with little feedback
Stopped posting after week 2Little opportunity to get drawn inWhat do you do if someone will not contribute ?
Concerned about how posts interpretedMost posts were responded toMost posts positive feedback but no added contentMany posts Social rather than task-orientedRarely posted thread starters
Jim Waters, Drexel University, 2008
The problem with traditional teaching
“Mr. Osborne, may I be excused - my brain is full.”
Jim Waters, Drexel University, 2008
Solving limitations of the traditional model
Midway through the exam, Allen pulls out a bigger brain
Jim Waters, Drexel University, 2008
Conclusions I Tentative findings Deep engagement can be encouraged by
thought-leaders Useful thought-leader behaviors highly
context dependent Interaction behaviors interact Faculty feedback improves some forms of
engagement but perhaps at the expense of student-student interactions ?
Finite tank of engagement ?
Jim Waters, Drexel University, 2008
Conclusions II Question design can be crucial to
engagement Discussion needs to be framed as
collaborative not competitive Students recognize core participants,
should we alter our behaviors to fit ? Committed engagement is possible So is isolation and inertia ! Sense of community is not automatic
Jim Waters, Drexel University, 2008
What next ?
Much more data crunching Establish if patterns of behavior are recurrent Deeper analysis of message content and its
impact on interactions Digging into thought-leader issues Frame findings as positive contributions to
course design and management
Jim Waters, Drexel University, 2008
Related References Waters, J. '
Social Network Behavior, Thought-Leaders and Knowledge Building In An Online Learning Community', Proceedings of Hawaii Intl. Conference on System Sciences (HICSS-41), Knowledge Management Track, Jan. 2008.
Gasson, S. and Waters, J. “How (not) to construct ALN course questions that encourage student participation in peer collaboration and knowledge construction,” 40th Hawaii International Conference on System Sciences, Hawaii, January 2007.
Waters, J., and Gasson, S. "Social Engagement in an Online Community of Inquiry," 27th International Conference on Information Systems (ICIS), Milwaukee WI, 2006.
Waters, J. “Determinants of Engagement in an Online Community of Inquiry,” The 12th Sloan-C International Conference on Online Learning, November 2006, http://www.sloanconsortium.org/conference/proceedings/2006/ppt/1162852287092.pot
Waters, J., and Gasson, S. "Strategies Employed By Participants In Virtual Learning Communities," Hawaii Intl. Conference on System Sciences (HICSS-38), Collaboration Systems and Technology track, IEEE Software Society, Manua, Hawaii, January 2005, 2005, p. 3b.
A full list of publications, with full copies of articles, is available at http://www.pages.drexel.edu/~jw65/publications.htm
Jim Waters, Drexel University, 2008
About Me I am a Doctoral Candidate at the iSchool at Drexel University, Philadelphia
(USA), graduating Summer 2009.
My principal research interests lie in Online Collaborative Knowledge Building, Technology-Supported Learning, Student Role-Behavior in Online Learning Communities and HCI. I am currently employed (2005 - 2008) as a Research Assistant on an IMLS funded project "Toward a Model Curriculum for the Management of Digital Information".
I received a BA in Psychology at Warwick University, UK (1979), an MSc in Occupational Psychology at the University of Hertfordshire, UK (1991) and a MS in Information Systems at the College of IS&T at Drexel University (2002). Prior to my advanced academic studies I enjoyed a substantial career in Systems Design, Management and IS Consultancy.
Email: [email protected]