Process protocols for virtual team effectivenessChristopher Sean Cordes

November 1, 2013Iowa State University

1

HCI Path: work and academics

2007 Human Computer Interaction

2004 Educational Technology & Information science

1999 Human Factors & Technical Communication

2008-> Instructional Design

Design, manage, & teach interactive machine and human systems.

Research assistant for MoDOT, the HS Truman Library, and MU Provost

Assistant ProfessorInstructional Technology Librarian

Associate ProfessorInstruction Services Coordinator

2

Problem, purpose, contribution, questions

• Virtual teams: the fastest growing work unit• 97% said organizations planned to increase

virtual work options or keep them the same (Leonard, 2011).

• 43 percent of HR professionals predict a larger proportion of their workers will be telecommuting in five years (Lockwood, 2010).

3

VT ChallengesMore importantly…

A recent report study of nearly 30,000 multinational companies found virtual teams challenged by:• reading nonverbal cues (94%).• establishing rapport and trust (81%).• managing conflict (73%).• expressing opinions (64%).• decision making (69%) (Hastings, 2010).

4

The Problem

• Virtual teams are distinctly different, especially in terms of boundaries, spatial, temporal, relational

• Virtual team effectiveness study has grown; but questions remain about fit of communication media, process, and team level outcomes

• Alignment of interaction processes, technology, and surrounding conditions gives outcomes that are relatively favorable (DeSanctis and Poole, 1993).

• Bell and Kozlowski (2002) call for more research on interaction between communication technology and task type on team processes and effectiveness

• Process and display structures foster interaction leading to gains in attitudes and performance, and provide insight into how teams will behave in the future given their unique operating conditions.

5

Guiding Questions

• To what extent does action process structure influence perceptions of team work climate?

• To what extent does action process structure influence justice perceptions of procedures?

• To what extent does action process structure impact team decision performance?

• To what degree does technology affordance moderate performance and attitudes?

6

Review of the Literature• Defining virtual

teams• Conceptual basis of

the research• Framework of team

effectiveness

7

Team definition

• There is overlap between groups, teams, and virtual team definitions, especially in terms of outcomes and boundaries.

• In this study, virtual teams are groups of people who work interdependently across boundaries using technology to communicate, collaborate and reach a common goal.

8

Theories

9

The study uses five schools based on the idea that teams are mechanisms that:• have member influence on each other and the team

• use interaction patterns to communicate information about problems and solve them

• do tasks with resources, strategies, and interactions required for effective outcomes

• use cognitive processes to make sense of information

• represent a dynamic of members, tools, resources, task and technology

Model Framework• Input-Process-Output (McGrath, 1984)

10

PROCESSINPUT OUTPUT

Inputs are properties of group structure, the task, and properties of the environment

Process represent interdependent acts; convert inputs to outcomes through cognitive, verbal, and behavioral action directed toward taskwork to achieve collective goals

Outputs parallel input classes and represent changes process makes to input variables

Process and Display Structures

• Action Process Structure– Monitoring– Backup– Coordination

• Collaborative Display Structure– Collaborative text

editing and chat– Chat only

11

Team Climate• Team climate represents shared perception of types of

behaviors, practices, and procedures that are supported in a specific setting

• Four dimensions, participative safety, support

innovation, vision, task orientation

• When climate is strong, members are more likely to…• participate and contribute to the team• express support for new ideas and bring them forward• make practical efforts to make resources available • enable effective monitoring and review of team actions• share information and acquire knowledge

12

Procedural Justice• The six rules: consistency, bias suppression, accuracy,

correctability, representativeness, and ethicality. 

• Positive justice increases helping, attachment, trust, cooperation, decision commitment

• change to one rule can be offset by another• justice levels are positively related to role performance

and conflict perceptions• justice perceptions tend to align and interact• Dispersion creates uncertainty so that justice

perceptions are more salient

13

Performance

• Comparable to F2F teams• Quality and quantity• Tied to task• May take VT longer• Less interaction• Less information exchange• Less conflict resolution

14

Hidden ProfilesYou are member of a four person pilot job search committee.

D

CB

A

All your team members havepositive and negative information too. Some is the same. Some is different.

The study uses a hidden profile problem

15

B C

DAIf team members

share all the information between them the initial worst choice becomes the best.

Critical thinking, counterfactual mindset, listing

Study designInputs Process Outputs

Task Design

Shared and unshared information between members

Monitoring, Backup, & Coordination protocol vs ad hoc process

Information Display

Editable / collaborative document vs chat only

Action Process

• Decision Performance• Team Climate• Procedural Justice

Performance/Affective

The within-subject factor was the exclusive decision information set for each member.

The between-groups factors were two independent variables, action process structure and information display structure.

16

Methodology

• Population• Variables &

Measurements• Treatment conditions• Hypotheses

17

Population• Participants drawn from 2 universities

• Some participants received extra credit for class

• All participated in a drawing for a large prize, and a $20 bonus for the team with the best performance (time and accuracy).

• Demographics included information about diversity, gender, prior knowledge of each other, and prior work together before the study

• In addition, they were asked about their comfort level using internet tools.

18

Virtual Study Environment• Study administration, trials, and data collection

performed online.• Scheduling was done through the Doodle tool • Correspondence of participants through email• Task was done in 4 member teams in Google

documents using text-based chat and collaborative text editing

• Data was collected using Qualtrics survey software

19

Treatment Conditions

Action Process Structure by Information Display Structure

High APS/High IDS• APS, Discussion monitoring,

backup, coordination, leader• IDS, Can use chat and edit

document

Low APS/High IDSAPS, Discussion ad hoc team processIDS, Can use chat and edit document

High APS/Low IDS• APS, Discussion monitoring,

backup, coordination, leader• IDS, Can use chat, no

collaborative editing

Low APS/Low IDSAPS, Discussion ad hoc team processIDS, Can use chat, no collaborative editing

20

Variables & Measures

21

• Team ClimateDesigned by Anderson and West (1994), to measure the overall climate of the team experience: The inventory has subscales for four specific areas: group vision, task-orientation, support, and participative safety.

• Procedural JusticeMeasures perceptions of procedural fairness, amount of control over process and outcomes, and the degree of consistency, accuracy, correctibility, bias, and ethicality. Jason Colquitt, (2001).

• Decision performance• Decision accuracy-objective measure coded

1=correct, 0=incorrect• Decision Quality-suitability candidate prior to and after discussion,

Information Sharing-value of individual shared and unshared attributes

Hypothetical Model

Action process structure will increase decision performance, team climate, and procedural justice perceptions.

Ability to list and structure information is expected to moderate the influence of action process on outcomes.

22

Results

• Preliminary analysis• Primary analysis• Secondary analysis• Confirmatory factor

analysis

23

Reliability and AggregationConstruct Cronbach’s

alphaRwg(j

) ICC(1) ICC2(2)

Team Climate .861 .962 .251 .876Procedural Justice .725 .875 .180 .624

Information Sharing        

Shared Information .880 .711 .213 .837

Unshared Information .815 .784 .140 .790

Decision Accuracy

--- --- --- ---

Decision Quality

--- --- --- ---

24

Preliminary Analysis

• Control Variables-These variables for internet comfort level, age, and ethnicity were controlled for in the regression tests.

• Internet technology comfort level was positively correlated with team climate and procedural justice scores.

• Ethnicity was positively correlated with procedural justice

• Age was positively correlated with team climate.

25

Outcomes by Factor

26

Dependent Variable High Action process, High

Information Display

High Action process, Low Information Display

Low Action process,

High Information

Display

Low Communication

Display, Low Information

Display

Team Climate 3.95 (.561) 3.80 (.520) 3.68 (.573) 3.52 (.520)

Procedural Justice 4.00 (.590) 3.70 (.561) 3.55 (.672) 3.70 (.532)

Decision Accuracy .807 (.410) .673 (.473) .635 (.470) .462 (.480)

Decision Quality 3.80 (1.34) 3.50 (1.30) 3.50 (1.40) 3.27 (1.23)

Information Sharing Shared Unshared

3.70 (.512)

3.67 (.661)

4.01 (.433)

3.67 (.734)

3.70 (.620)

3.40 (.844)

3.94 (.388)

3.61 (.734)

Hypothesis Test Results

Hypothesis Tested Supported P-Value Effect SizeHypothesis 1: Yes .001 h2=.049Hypothesis 2: Yes .008 h2.=036Hypothesis 3a: Yes .003 h2=.045

Hypothesis 3b: No.582.108

h2=.001 h2=.012

Hypothesis 3c: Yes .006 OR=2.34Hypothesis 4: No .535 h2=.001Hypothesis 5: Yes .009 h2=.042Hypothesis 6: No .462 OR=.794

27

Factor AnalysisGoodness-of-Fit Measures (x2=201.94, df= 63, x2/df, 3.20, GFI = 0.88, CFI = 0.82, IFI =0.82, RMSEA =.103).

Factor LoadingsMost factors (11 of 13) loaded significantly (p < .001) on latent constructs with acceptable factor loadings above 0.50. Two indicators, PJ6 (ability to appeal) and PJ7 (ethical and moral standards) were non-significant).

ReliabilityR-Square values for all indicators but three were less than the level of 0.50. The range of variance explained across all 13 indicators extended from 0.01 or 1% (PJ7) to 0.64 or 64% (TC2).The data showed the model had relatively low explanatory power on most indicators.

28

Study-Conclusions, Discussion, and Recommendations

• Summary, Conclusions Discussion

• Limitations• Recommendations

29

Main Effects

• There is evidence monitoring, backup and coordination processes support team climate and justice perceptions, and accurate, higher quality decisions.

• Display structure also influenced climate independently, but not justice or decision accuracy

• But impact may be small as shown by the modest effect size indicators.

30

Moderating Effects

• Action process and information display structure had separate influence on team climate

• In addition, increased technology affordance was found to moderate action process impact on justice climate, but experimental groups had similar scores to ad hoc

• Finally, display affordance enhances action process structure so teams make more accurate decisions than when action process structure is used alone. But this effect is not interactive.

31

Factor analysis• Examination shows correcting of error

covariances with and between justice and climate indicators will improve model fit.

E1E3 (TC1PS and TC3 Vision)E1 E6 (TC1PS, and PJ2 Outcome control)

Model fit indices: (x2=71.70, df= 40; x2/df, 1.80; GFI=0.95; CFI =0.96; IFI=0.96; RMSEA=.062

32

Limitations• Factor separation (control groups ad hoc)• Constraints of chat • Outside collaboration • Figurative vs literal interpretations• Instrument length, fatigue• Technology usability

33

Recommendations

• Team designers and members can implement action process structuring to meet unique task, technology, and virtual environment conditions

• Exploration of team and justice climate facets in varying contexts and in longitudinal studies may provide better understanding of action process and display structuring impact

34