communication differences in virtual design teams: findings from a multi-method analysis of high and...
Post on 21-Dec-2015
218 views
TRANSCRIPT
Communication Differences in Virtual Design Teams:Findings from a Multi-Method Analysis of High and Low Performing Experimental Teams
Rosalie J. OckerCollege of Information Sciences and TechnologyPennsylvania State University
Jerry FjermestadSchool of ManagementNew Jersey Institute of Technology
Accepted for Publication inData Base for the Advances in Information Systems
Requirements determination is a collaborative process
Involving the interaction of analysts with the development team
For the purposes of acquiring, sharing, and integrating knowledge
To develop a mutual shared understanding of the software goals
Research Motivation
Software that fails to meet user requirements is a serious problem
Competitive pressure requires more creative and innovative software
Time pressure requires shorter development cycles
Research Question
Are high performing virtual design teams distinguishable from low performing virtual design teams in terms of the content of task-related communication?
Relationship between Team Climate for Innovationand Design Team Performance
Design Team PerformanceCreativity of DesignQuality of Design
Team Climate for Innovation
enhances
Climate of Support
Increased Supportive Feedback
Increasedcommunication
Cross-fertilization of ideas
Increased creativity of team performance
Relationship between a Team Climate of Support and Performance Creativity
Climate of Excellence
Increased criticaldebate
Flawed ideasrejected
Increasedquality ofteam performance
Tolerance for diversity
Relationship between a Team Climate of Excellence and Performance Quality
Research Model
High Performing
VDTs
Low Performing
VDTs
Communication ContentSupportiveDebateDesignSummaryCoordinationOther
Communication ActivityNumber of wordsNumber of messages
Task
Computerized Post Office (CPO) groups develop initial high-level
requirements for the CPO submit requirements in the form of a
report report on functionality of CPO and
the interface one group = one report
Subjects & Groups
Subjects: graduate students in the CIS and IS majors at Eastern Universities
Most subjects had coursework and/or job experience related to systems design
group size: predominantly 4-5 subjects
Technology, Facilitation & Training
computer-mediated groups used asynchronous computer conferencing system Web-EIES each group had its own private
conference for communication or FirstClass all groups were minimally facilitated
provided technical assistance only all groups had training session all groups had a leader
Table 1. Comparison of Experiments A & B
Experiment A Experiment B
Communication condition Asynchronous Asynchronous
Length 14 days 17 days
Subjectsgraduate students from IS courses at
University Agraduate students from IS courses at
University B
Computer conferencing system Web-EIES FirstClass™
Training task Entertainment for Dutch Visitors Entertainment for Dutch Visitors
Training proceduresdetailed script of procedures
developed and followed
same script but with minor modifications to reflect difference in user interface
Experimental task Computerized Post Office Computerized Post Office
Experimental proceduresdetailed script of procedures
developed and followed
same script with minor modification to reflect 17 day experiment length
Rating of quality and creativity panel of judges panel of judges
Table 2. Coding Scheme
Code Explanation Example
Design
Initial suggestion, idea or assumption pertaining to the functionality and services, user interface, advantages and disadvantages, and 5 year plan (see task description).
Computerization should provide accessibility to existing communication procedures-including E-mail, EDI, and general document transfer over VANS
Coordination
Any reference to project management (i.e., schedules, responsibilities, status of work), problem solving approach, or an individual's intentions
Any ideas or comments you want included in Saturday's finished product please have transmitted to me by tomorrow at 6:00 PM. I'll check back later tonight to see how things are coming along.
Summary
Summarizes or reviews prior discussions, debates or decisions.
I've collected and listed all of the functions and services we have been discussing. They are…
Debate
Opposing argument or different perspective regarding an idea previously communicated; includes reasons for disagreement
I don't agree with e-mail as a part of our CPO service because everyone can send their e-mail using internet...
Supportive
Agreeing with members' comments, positive feedback; includes reasons for agreement
I agree with Vineet, I think that we should locate this at all train stations along with the post offices.
OtherCommunication that does not fit into
any other category
Dependent Variables
Panel of Expert judgesQuality of reportCreativity of report
Quality Measure: The quality of each team's CPO solution was judged based on a consideration of the (1) functionality (2) interface layout (3) coherence of these ideas
Results of Initial Experiments
Dependent VariablesQuality A>FtF
C>FtFCreativity A>FtF
C=FtF
Results: Creativity and Quality
Dependent VariablesQuality C=A
Creativity A>C
Results: Comparison of Combined vs. Asynchronous
Table 5A. Coding Results by Team (in words)
TEAM Design Coord. Summary Debate Supportive Other Total
1 (H) 2002 956 403 888 369 34 4652
2 (H) 1889 949 858 1960 520 328 6504
3 (H) 1397 2037 533 3622 231 111 7931
4 (H) 826 1012 428 759 322 42 3389
5 (L) 1656 744 209 344 335 75 3363
6 (L) 2940 626 0 505 289 13 4373
7 (L) 591 948 0 195 84 381 2199
8 (L) 1435 1092 120 319 131 14 3111
H- high performance, L – low performance
Table 5B. Coding Results by Team (in proportions)
TEAM Design Coord. Summary Debate Supportive Other
1 (H) 0.43 0.21 0.09 0.19 0.08 0.09
2 (H) 0.29 0.15 0.13 0.30 0.08 0.13
3 (H) 0.18 0.26 0.07 0.46 0.03 0.04
4 (H) 0.24 0.30 0.13 0.22 0.10 0.11
5 (L) 0.49 0.22 0.06 0.10 0.10 0.12
6 (L) 0.67 0.14 0.00 0.12 0.07 0.07
7 (L) 0.27 0.43 0.00 0.09 0.04 0.21
8 (L) 0.46 0.35 0.04 0.10 0.04 0.05
Table 6. Team Communication Activity
TEAM # Words # Messages
1 (H) 4652 48
2 (H) 6504 59
3 (H) 7931 72
4 (H) 3389 29
5 (L) 3363 34
6 (L) 4373 44
7 (L) 2199 52
8 (L) 3111 32
Table 7. Mean and Std. Deviation for Team Communication Activity
Performance Category N Mean Std. Dev.
# Words High 4 5621.0 2004.23
Low 4 3261.5 893.92
# Messages High 4 52.0 18.20
Low 4 40.5 9.29
Table 8. ANOVA Results for Team Communication Activity
Sum of Squares df F Sig.
# WordsBetween
Groups 11134481 1 4.64 0.08
Within
Groups 14448057 6
# MessagesBetween
Groups 264.50 1 1.27 0.30
Within
Groups 1253.00 6
Table 9A. Coding Mean and Standard Deviation (words)
Code Category
Performance Category N Mean Std. Dev.
Design High 4 1529 537
Low 4 1655 972
Coord. High 4 1239 533
Low 4 853 208
Summary High 4 556 209
Low 4 82 102
Debate High 4 1807 1324
Low 4 341 127
Supportive High 4 361 121
Low 4 210 121
Other High 4 129 137
Low 4 121 146
Table 9B. Coding Mean and Standard Deviation (proportions)
Code Category
Performance Category N Mean Std. Deviation
Design High 4 .2851 .1076
Low 4 .4737 .1653
Coordination High 4 .2267 .6600
Low 4 .2866 .1290
Summary High 4 .1030 .3120
Low 4 .0252 .3060
Debate High 4 .2931 .1184
Low 4 .1022 .1100
Supportive High 4 .0708 .2870
Low 4 .0615 .2820
Other High 4 .0918 .3700
Low 4 .1123 .7330
Table 10A. Coding ANOVA Results (words)
Sum of Squares df F Sig.
Design
Between Groups 32258 1
0.05 0.83Within Groups 3696682 6
Coord.
Between Groups 297992 1
1.82 0.23Within Groups 982076 6
Summary
Between Groups 447931 1
16.53 0.01Within Groups 162546 6
Debate
Between Groups 4301245 1
4.86 0.07Within Groups 5309104 6
Supportive
Between Groups 45451 1
3.11 0.13Within Groups 87748 6
Other
Between Groups 128 1
0.005 0.95Within Groups 149348 6
Table 10B. Coding ANOVA Results (proportions)
Sum of Squares df
Mean Square F Sig.
Design Between 0.071 1 0.071 3.659 0.10
Within 0.117 6 0.019
Coordination Between 0.007 1 0.007 0.685 0.44
Within 0.063 6 0.010
Summary Between 0.012 1 0.012 12.667 0.01
Within 0.006 6 0.001
Debate Between 0.073 1 0.073 10.309 0.02
Within 0.042 6 0.007
Supportive Between 0.000 1 0.000 0.214 0.66
Within 0.005 6 0.001
Other Between 0.001 1 0.001 0.247 0.64
Within 0.02 6 0.003
Table 11. Summary of Results
Category Relationship Significance
Communication Activity words High VDT > Low VDT Marginal
messages non-sig.
Design words non-sig.
proportions High VDT < Low VDT Marginal
Summaries words High VDT > Low VDT Significant
proportions High VDT > Low VDT Significant
Coordination words non-sig.
proportions non-sig.
Supportive words non-sig.
proportions non-sig.
Debate words High VDT > Low VDT Marginal
proportions High VDT > Low VDT Significant
Other words non-sig.
proportions non-sig.
Findings from the Qualitative Analysis
Design Compared to high performing VDTs, low performing
VDTs spent more communication on idea generation.
Low VDTs neglected to build upon or question others’ ideas, instead opting to contribute by suggesting more ideas.
Thus, endless brainstorming where members worked in parallel on the same activity rather than as a team, each espousing his or her own ideas with little questioning of, or integration with, the ideas of others.
Brainstorming, while good for generating many ideas, precluded an in-depth interactive discussion or critique concerning the merits of the ideas being contributed.
Findings from the Qualitative Analysis
Debate: High performing virtual teams continually
assessed their members’ contributions Constructive conflict and deliberation was the
norm as members actively participated in frequent, and often intense and direct debate of ideas and issues.
Critical examination resulted in the back and forth exchange of opposing ideas, which frequently spanned several or more days.
The critical argumentation of the high teams is in harsh disparity to the convergent behavior which was dominant in the low performing VDTs.
Findings from the Qualitative Analysis
Summarization: By High Performing VDTs Reviewing the knowledge repository created as
a result of their electronic communication, Summarizing content. A leader sifted through the team’s
communications in order to summarize discussion content on a given topic.
The summaries provided a structuring mechanism that organized the team’s work and progress-to-date on a topic.
The summaries also served to bring all members up-to-date and thus helped to keep a team on the same “virtual page.”
Findings from the Qualitative Analysis
Summarization: Low Performance VDTs Knowledge management activities were almost
non-existent in the low performing teams. Only one member summarized the team’s
discussion of functionality. In the remaining teams, summary comments were
either absent or only re-capped a single individual’s input.
Since the means of communication was asynchronous and spanned two weeks, the content of design communications was often disjointed and lacking in coherence.
When it was time to produce the final report deliverable, the low performing teams were at a disadvantage as there were few summaries from which to directly draw report content.
Decreased idea generation
Increasedcommunication
Increasedcritical debate
Cross-fertilizationof ideas
Review communicationrepository
Summarize resultsof debates
Review and amendsummaries
Incorporate summaries into Final Deliverable
Figure 5. Virtual Team Climate for Innovation
Conclusions & Contributions
High Performance VDTs were effective in the absence of any technological support designed to aid knowledge management
The act of summarizing appeared quite effective.
By attending to the management of knowledge, by designating the role of knowledge manager within the team, may be a simple means of reaping the benefits of knowledge management without increasing the complexities of the communication technology.
Communication and Leadership Differences in Virtual Design Teams: Why some teams do better than others
Jerry FjermestadRosalie J. OckerJournal of the Brazilian Computer Society, 3 (13), 2007, 37-50
AsynchronousAsynchronous
Combined
4 High Performing
Teams
4 LowPerforming
Teams
Computer Conferencing Transcripts
Computer Conferencing Transcripts
2 Highest
2 Lowest
2 Lowest
2 Highest
Figure 1: Research Method
1998 Study Current Study Content Coding Scheme
•Design•Functionality•Interface•Advantages•Disadvantages•Implementation
•Summary•Summary•Write-up
•Coordination•Goal•Management
•Other•Digression•Other
Figure 2 Team Leader's Communication Profile
05
10152025303540
Category of Communication
% S
um
ma
ry M
es
sa
ge
s
% High Performance
% Low Perfromance
Table 5T-test Results for Team Leaders Using Ratio of Leaders Messages to that of
the Team
Measure MeanHigh Performing Teams
(Std)
MeanLow Performing
Teams(Std)
t-statistic
Effect SizeEta-Squared
% Number of Messages
36.5 (9.5) 27.3 (19.4) Ns ~
% Message length
43.3 (8.5) 22.3 (8.3) 7.1** 0.89
% SummaryMessages
Length (No.of Lines)
45.9 (42.6) 0 5.0* 0.76
** = significant at α = 0.001; t(6) α=.001 where t crit = 5.2
* = significant at α = 0.05; t(6) α=.05 where t crit = 3.14
Theory—Distributed Cognition- Information processing
Group information processing is the degree to which information, ideas, or cognitive processes are shared
Information affects both individual and team outcomes Distributed cognition-information processing theory
further suggests that sharing relates to a process activity that takes place within team members’ working memory leading to a modification of shared knowledge.
In virtual teams the information is the messages and their content.
The sharing through distributed cognition is the assimilation, combining and understanding of the shared knowledge which leads to new knowledge or knowledge presented in a new way.
Theory—Distributed Cognition- Information processing
The distributed cognition theory provides a basic framework for how teams work.
Each team member brings to each activity a network of ideas representing the individual’s prior knowledge that is relevant to the task.
As the activities continue members of the team share some of their ideas about the task and they process ideas shared by others.
As these and other activities proceed the team may construct artifacts (models, diagrams, reports) of their interaction and develop a shared understanding.
A leader (assigned or emergent) may monitor and comment on the team’s performance further enabling the sharing of information.
Thus, the distributed cognition theory suggest that team performance is the interaction of team member’s ideas, the sharing and processing of these ideas, and a leader process of commenting and processing of the shared information.
Discussion
High VDTs differed from their low VDTs in terms of number of messages, message length and in the content profile of those messages.
The high VDTs had significantly more messages, longer messages as measured by the number of lines) and had a higher message length per message than the low VDTs
The leaders of the high VDTs had more messages and longer messages than the leaders of the low teams.
High VDTs communicated more regarding aspects of the design (especially functionality, interface design and implementation considerations).
Discussion
High VDTs had more messages and message length focused on summarizing their work and discussing the write-up.
Simply put, it takes more effort to communicate more, especially in virtual space.
High VDTs not only communicated more, but they communicated regarding key design aspects of the CPO project.
Through their increased communication, it is not hard to conceive that they generated a greater number of high quality and creative ideas.
Discussion
High VDTs spent time summarizing their work and sharing these summaries with their teammates.
This supports the distributed-cognition theory in that the summarization is the development of a shared understanding.
High and low teams did not differ with respect to the amount of messages concerning team management issues, the summaries served a coordination function by keeping members apprised of their teammates’ ideas and progress.
The summaries also appear to be a key when preparing the final design report.
The transcripts shows that much of the design reports came directly from the text of comments, many of which were summary comments.
Discussion
It is the leaders in the high VDTs teams that do the summarization. In three of the four high VDTs this was the case. In the fourth team, another team member did the summarization and thus was an emergent leader.
In the low VDTs the leaders did not do any summarization at all. It is plausible that this simple act of summarizing work, coupled with
the not-so-simple act of putting forth more effort, were key aspects of the success of the high VDTs.
Effective teams have effective leaders who actively facilitate the sharing of specific information.
The research supports observations where the leaders of the more effective teams took on the role of monitor and producer
In this study the leaders are organizing the ideas about the functionality and design of the task for the rest of the team.
This summary then becomes the cornerstone of their final report.
Contributions
There are indeed measurable differences (number of messages and message length) between high and low performing virtual teams.
The content of the communication is also different between high and low performing teams. The high performing teams communicate more on the task related issues (functionality, interface, and implementation as well as on summarizing their communications.
This study has shown it is the leaders who do much of the summarization and create the shared intelligence.