the influence of shared mental models on team process …the influence of shared mental models on...

Journal of Applied Psychology Copyright 2000 by the American Psychological Association, Inc. 2000, VoL 85, No. 2, 273-283 0021-9010/00/$5.00 DOI: 10.I037t/0021-9010.85.2.273

The Influence of Shared Mental Models on Team Process and Performance

John E. Mathieu Pennsylvania State University

Gerald F. Goodwin Pennsylvania State University

Tonia S. Heffner University of Tennessee at Chattanooga

E d u a r d o S a l a s a n d J a n i s A . C a n n o n - B o w e r s Naval Air Warfare Center

The influence of teammates' shared mental models on team processes and performance was tested using 56 undergraduate dyads who "flew" a series of missions on a personal-computer-based flight- combat simulation. The authors both conceptually and empirically distinguished between teammates' task- and team-based mental models and indexed their convergence or "sharedness" using individually completed paired-comparisons matrices analyzed using a network-based algorithm. The results illustrated that both shared-team- and task-based mental models related positively to subsequent team process and performance. Furthermore, team processes fully mediated the relationship between mental model convergence and team effectiveness. Results are discussed in terms of the role of shared cognitions in team effectiveness and the applicability of different interventions designed to achieve such convergence.

Increased technology has contributed to the complexity of many tasks performed in the workplace, making it difficult for employ- ees to complete their work independently. In response to the technological advances, many organizations have adopted a team approach to work. Teams are viewed as being more suitable for complex tasks because they allow members to share the workload, monitor the work behaviors of other members, and develop and contribute expertise on subtasks. An abundance of research has been conducted on the factors that contribute to high team performance (for reviews, see Gist, Locke, & Taylor, 1987; Salas, Dickinson, Converse, & Tannenbaum, 1992). One variable that has recently received much theoretical attention concerns the influence of team members' mental models on team-related processes and behaviors (Klimoski & Mohammed, 1994; Kraiger & Wenzel, 1997; Rentsch, Heffner, & Duffy, 1994; Stout, Salas, &

John E. Mathieu and Gerald F. Goodwin, Department of Psychology, Pennsylvania State University, University Park; Tonia S. Heffner, Depart- ment of Psychology, University of Tennessee at Chattanooga; Eduardo Salas and Janis A. Cannon-Bowers, Training Systems Division, Naval Air Warfare Center, Orlando, Florida.

Tonia S. Heffner is now at the Arm), Research Institute, Washington, DC. Gerald F. Goodwin is now at the American Institutes for Research, Washington, DC. Eduardo Salas is now at the Department of Psychology, University of Cenlral Florida, Orlando, Florida.

The views, opinions, and findings contained in this article are those of the authors and should not be construed as an official position, policy, or decision of their respective affiliations. We thank Dana Born, Gwen Fisher, Kirsten Hobson, Kara Ivory, Mike Trip, and Nikki Windefelder, who were instrumental in the running of the teams and data coding portions of this work.

Correspondence concerning this article should be addressed to John E. Mathieu, who is now at the School of Business Administration, Depart- ment of Management, University of Connecticut, 368 Fairfield Road, 6-41MG, Storrs, Connecticut 06269-2041. Electronic mail may be sent to jmathieu @ sba.uconn.edu.

273

Kraiger, 1996). The present research was designed to empirically examine the impact that teammates' mental models have on team process and performance in a dynamic and exciting laboratory flight simulation.

A team can be defined as "a distinguishable set of two or more people who interact, dynamically, interdependently, and adap- tively toward a common and valued goal/objective/mission, who have each been assigned specific roles or functions to perform, and who have a limited life-span of membership" (Salas et al., 1992, p. 4). Research has identified numerous factors that affect teams and has offered several models of team functioning (Guzzo & Dickson, 1996). Although these different models vary in details, they all share an input-process-outcome (I-P-O) framework. Inputs to such models are conditions that exist prior to a performance episode and may include member, team, and organizational characteristics. Performance episodes are defined as distinguishable periods of time over which performance accrues and feedback is available. Processes describe how team inputs are transformed into outputs, Outcomes are results and by-products of team activity that are valued by one or more constituencies. Hackman (1990) identified three primary types of outcomes: (a) performance-including quality and quantity, (b) team longevity, and (c) members' affec- five reactions. Although we recognize the importance of all three types, for our purposes we will concentrate on performance outcomes.

Empirical examinations of I-P-O models have demonstrated their utility (e.g., Campion, Medsker, & Higgs, 1993; Gladstein, 1984; Guzzo & Dickson, 1996). However, the large number of factors that influence outcomes has prohibited a comprehensive examination of the model. Many variables that have been proposed to influence team processes and thereby team performance have yet to receive much empirical examination. Included among these are members' knowledge and its organizational structure. This oversight has occurred despite acknowledgement of the importance of knowledge organization for individual and team performance (Cannon-Bowers & Salas, 1990; Cannon-Bowers, Salas, &

274 MATHIEU ET AL.

Converse, 1993; Klimoskl & Mohammed, 1994; Kralger & Wen- zel, 1997; Rentsch & Hall, 1994).

Shared Menta l Mode l Theory

The notion of shared mental models in teams has been used to help explain team functioning for several years. According to Cannon-Bowers, Salas, and Converse (1993), shared mental models help explain how teams are able to cope with difficult and changing task conditions. In fact, researchers have suggested that the ability to adapt is an important skill in high-performance teams (Cannon-Bowers, Tannenbaum, Salas, & Volpe, 1995; Mclntyre & Salas, 1995). The shared mental model theory offers an expla- nation of what the mechanisms of adaptability might be-- that is, how teams can quickly and efficiently adjust their strategy "on the fly." The following sections provide more details regarding shared mental model theory and its relationship to effective teamwork.

Mental Models

The term mental model has been used as an explanatory mechanism in a variety of disciplines over the years (see Wilson & Rutherford, 1989). Essentially, mental models are organized knowledge structures that allow individuals to interact with their environment. Specifically, mental models allow people to predict and explain the behavior of the world around them, to recognize and remember relationships among components of the environment, and to construct expectations for what is likely to occur next (see Rouse & Morris, 1986). Furthermore, mental models allow people to draw inferences, make predictions, understand phenom- ena, decide which actions to take, and experience events vicari- ously (Johnson-Laird, 1983). For the purpose of this article, we defined a mental model, in keeping with Rouse and Morris (1986), as a "mechanism whereby humans generate descriptions of system purpose and form, explanations of system functioning and observed system states, and predictions of future system states" (p. 360). Hence, mental models serve three crucial purposes: They help people to describe, explain, and predict events in their environment.

Shared Mental Models

As we noted above, Cannon-Bowers et al. (1993) suggested that teams that must adapt quickly to changing task demands might be drawing on shared or common mental models. The rationale behind Cannon-Bowers et al.'s assertion was that in order to adapt effectively, team members must predict what their teammates are going to do and what they are going to need in order to do it. Hence, the function of shared mental models is to allow team members to draw on their own well-structured knowledge as a basis for selecting actions that are consistent and coordinated with those of their teammates.

To refine the concept of shared mental models even further, Stout, Cannon-Bowers, and Salas (1996) suggested that the man- ner in which shared mental models operate is related to task demands. Specifically, these authors argued that under conditions that allow team members to freely communicate with one anoth- e r - t o strategize--shared mental models will not be very important. This is because the team can discuss its next moves and does

not need to rely on preexisting knowledge. However, under conditions in which communication is difficult--because of excessive workload, time pressure, or some other environmental feature-- teams are not able to engage in necessary strategizing. In this case, shared mental models become crucial to team functioning because they allow members to predict the information and resource requirements of their teammates. Hence, members are able to act on the basis of their understanding of the task demands and how these will affect their team's response. It is this ability to adapt quickly that enables teams in dynamic environments to be successful.

Types of Shared Mental Models

Cannon-Bowers et al. (1993) and others have argued that there is probably not a single mental model that must be shared among team members. In fact, Klimoski and Mohammed (1994) con- tended that "there can be (and probably would be) multiple mental models co-existing among team members at a given.point in time. These would include models of task/technology, of response rou- tines, of team work, etc." (p. 432). Rentsch and Hall (1994) advanced similar notions and argued that team members' schema similarity (a concept quite similar to mental models) could be described in terms of both team work and task work. According to Cannon-Bowers et al. (1993), complex tasks probably dictate that multiple mental models be shared among members. Table 1 de- scribes several of these mental models.

First, team members must understand the technology or equipment with which they are interacting. The dynamics and control of the technology and how it interacts with the input of other team members is particularly crucial for team functioning. Second, team members must hold shared job or task models. Such models describe and organize knowledge about how the task is accom- plished in terms of procedures, task strategies, likely contingencies or problems, and environmental conditions. Third, team members must hold shared conceptions of how the team interacts. These models describe the roles and responsibilities of team members, interaction patterns, information flow and communication channels, role interdependencies, and information sources. The final model that team members must share is the team member model. This model contains information that is specific to the member's teammates--their knowledge, skills, attitudes, preferences, strengths, weaknesses, tendencies, and so forth. According to Cannon-Bowers et al. (1995), such knowledge is crucial for team effectiveness because it allows team members to tailor their behavior in accordance with what they expect their teammates to do. The more knowledge team members have about one another (and the more accurate that information is), the more efficient and automatic this process can be.

Although the detailed breakdown of mental-model types depicted in Table 1 helps to clarify the nature of the underlying knowledge structures, operationalizing four different types of mental models becomes unwieldy in a single study. The four types of model described above can be viewed as reflecting two major content domains: (a) task-related features of the situation (e.g., the technology/equipment and job/task models) and (b) team-related aspects of the situation (e.g., the team interaction and team models). This division is also consistent with the idea that teams develop two tracks of behavior--a teamwork track and a task- work track (see Mclntyre & Salas, 1995; Morgan, Glickman,

SHARED MENTAL MODELS AND TEAM EFFECTIVENESS

Table 1 Types of Shared Mental Models in Teams

Type of model Knowledge content Comment

Technology/equipment Equipment functioning Operating procedures System limitations Likely failures

Job/task Task procedures Likely contingencies Likely scenarios Task strategies Environmental

constraints Task component

relationships

Team interaction Roles/responsibilities Information sources Interaction patterns Communication channels Role interdependencies Information flow

Team Teammates' knowledge Teammates' skills Teammates' attitudes Teammates' preferences Teammates' tendencies

Likely to be the most stable model in terms of content. Probably requires less to be shared across team members.

In highly proceduralized tasks, members will have shared task models. When tasks are more unpredictable, the value of shared task knowledge becomes more crucial.

Shared knowledge about team interactions drives how team members behave by creating expectations. Adaptable teams are those who understand well and can predict the nature of team interactions.

Team-specific knowledge of teammates helps members to better tailor their behavior to what they expect from teammates.

Note. Adapted from "Shared mental models in expert team decision making," by J. A. Cannon-Bowers, E. Salas, and S. A. Converse, 1993, in Individual and group decision making, by N. J. Castellan, Jr. (Ed.), Hillsdale, NJ: Erlhaum. Copyright 1993 by Erlbaum. Adapted with permission.

275

Woodard, Blaiwes, & Salas, 1986). These researchers argued that in order to be successful, team members not only need to perform task-related functions well but also must work well together as a team. Accordingly, we distinguished be tween- -and measured separately the extent to which teammates share- - two different types of mental models, task and team, and then related them to indices of team processes and performance. To our knowledge, empirical research has yet to relate multiple types of shared mental models to team effectiveness.

I -P -O F r a m e w o r k

The nature of teams and teamwork as represented by the I-P-O model suggests that the impact of mental models on team effectiveness may be indirect and mediated by team processes. For example, Kralger and Wenzel (1997) suggested that team processes are likely to be influenced by shared mental models. They argued that "the areas in which the greatest impact is expected are decision making and communication" (p. 79). Widely different mental models suggests that teammates work toward different objectives and predict different future system states, and therefore have difficulty coordinating their efforts. Alternatively, highly similar mental models would suggest that teammates work toward common objectives and have a shared vision of how their team will function. Thus, teammates with shared mental models will easily coordinate their actions and be "in sync," whereas differences in team mental models would likely result in greater process loss and ineffective team processes. Therefore, we hypothesized

that the convergence of both teammates' team and task mental models would relate positively to all three team processes described more fully below. Furthermore, we hypothesized that teammates would exhibit greater sharedness over time as they gained experience both with the task and with each other. For example, Rentsch et al. (1994) found that more experienced team members conceptualized teamwork more concisely and coherently than did less experienced members. Dyer (1984) advanced similar conclusions and added that with increased experience teams are able to better coordinate their efforts and therefore exhibit better team processes and performance.

Many research studies have focused on the relationship between team process and outcomes (e.g., Campion et al., 1993; Gladstein, 1984). This body of research has identified a myriad of variables that can fall under the broad title of process variables. Although some recent work has advanced different taxonomies of team processes (cf. Fleishman & Zaccaro, 1992; Prince, Brannick, Prince, & Salas, 1992), Klimoski and Mohammed (1994) have argued that at least the following are important for linking shared mental models with team performance: communication processes, strategy and coordinated use of resources, and interpersonal relations or cooperation.

Roberts and O'Reilly (1976) found that communication fre- quency was related to increased performance across a variety of tasks in fighter aircraft crews. Similarly, Foushee and Manos (1981) found that air crews who communicated more frequently performed better than crews who communicated less frequently in a simulator environment. Kanki and Foushee (1989) found that air

276 MATHIEU ET AL.

crews who had previously flown together had significantly fewer errors than air crews who had not, and the difference was attrib-

utable to the more seasoned air crews having better communications. Therefore, we hypothesized that team communication would relate positively to team performance. In effect, then, we hypoth-

esized that team communication serves as a mediator of the influence of shared mental models on team performance.

Kleinman, Luh, Pattipati, and Serfaty (1992) defined strategy as "the mechanisms by which the team attains its performance" (p. 184). Several researchers have noted the importance of strategy

formation for effective group performance (Gist et al., 1987; Gladstein, 1984), particularly for complex tasks (Hackman, Brous-

seau, & Weiss, 1976). For example, Hackman et al. (1976) studied the impact of strategy discussions on group performance and found

them to relate positively to performance on new tasks, yet negatively to performance on established tasks. They also found that groups without instructions were unlikely to discuss performance

strategies, even for new tasks. Accordingly, given the emergent

nature of the task used and the fact that we encouraged teams to strategize in the current study, we hypothesized that team strategy would be positively influenced by shared mental models and, in

turn, relate positively to team performance. The commonsense notion that members must get along and

cooperate with each other if a team is to be successful has proven to be difficult to validate. For example, reviews of the relationship

between interpersonally based team cohesion and performance consistently conclude that the empirical findings are mixed or equivocal (cf. Bettenhausen, 1991; Salas, Bowers, & Cannon-

Bowers, 1995; Tannenbaum, Beard, & Salas, 1992). At issue here is whether teammates' interpersonal relationships distract from or

contribute to team performance. Some interpersonal conflicts are inevitable in teams, particularly ones that are placed in complex challenging situations. Thus, while overly enhancing team mem-

bers ' interpersonal relationships may not enhance their task capabilities, the teams certainly will not be successful if members

refuse to work with one another. This is not to say that cooperative or happy teams will always be successful. Rather, we submit that

the most successful teams will likely be those that combine a true concern for one another with a collective commitment toward the team task (Bettenhausen, 1991). Thus, we hypothesized that team cooperation would serve as an important mediating role linking

shared mental models with team performance. In summary, we anticipated that team and task mental models

could be assessed and distinguished. Furthermore, we hypothesized that the convergence or sharedness of teammates' mental

models, both task and team, would relate significantly to the subsequent team processes they exhibited over a series of performance episodes. Moreover, we hypothesized that team processes would relate significantly to team performance, and thereby mediate the influence of shared mental models on team outcomes. Although we had hypothesized that team processes would fully mediate the influence of shared mental models on team performance, we also tested whether there were any direct effects. Finally, we hypothesized that with increased experience with each other and the task, teammates would exhibit greater convergence among their mental models as well as better team processes and performance.

M e t h o d

Participants

Participants were 52 male and 60 female undergraduate students en- rolled in psychology classes at Pennsylvania State University. Their ages ranged from 18 to 29 years (M = 20.96, SD = 2.02). They participated in the experiment in exchange for extra course credit and were randomly assigned to 56 two-person teams. There were no significant effects of the gender composition of teams on the relationships reported here, so we have pooled all teams into a single sample.

Task Apparatus

Weaver, Bowers, Salas, and Cannon-Bowers (1995) recently noted that "researchers interested in investigating team performance have begun to employ low-fidelity networked simulations to gain an increased understanding of the various factors that might impact team performance" (p. 13). They concluded, "Low fidelity simulation provides a particularly effective method for furthering empirical knowledge of theory related to team performance and processes" (Weaver et ai., 1995, p. 22). Brannick, Roach, and Salas (1993) and Stout, Cannon-Bowers, Salas, and Milanovich (in press) have successfully used such tasks to investigate a variety of inputs and processes related to team performance. Furthermore, Baker, Prince, Shrestha, Oser, and Salas (1993) reported results indicating that real military aviators believed that dynamic cockpit-crew processes could be well emulated using personal-computer-based simulations. Accordingly, we used a low-fidelity personal-computer-based flight simulator task (Fal- con 3.0; see Spectrum Holobyte, 1991) as our research platform.

The software is a computer-generated simulation program of the F-16 fighter fixed-wing aircraft. The program is extremely flexible and enables one to modify several parameters of the simulation. For example, various scenarios can be scripted and controlled in terms of flight plans, aircraft weapons loadouts, the number of enemy and allied planes present, and even the ability and temperament of computer-guided pilots. On one hand, as compared with actual flight or with military and commercial airline simulators, this task is an extremely low grade simulation. On the other hand, as compared with common video and computer games, this task is extremely detailed and complicated. The hardware included an IBM- compatible personal computer equipped with a joystick and keyboard and video and audio recording equipment. A divider separated players, and they communicated via microphone-equipped headsets.

Procedure

The experimental sessions ran between 2.5 and 3 hr and were conducted in three phases. First, participants received an overview of the task and watched an automated demonstration of the simulation. Second, they were guided through a structured hands-on training program, conducted by a highly skilled experimenter, that lasted approximately 45 min. One player was trained to operate a joystick and had the responsibility of flying and positioning the plane. The second teammate used a standard computer keyboard and had the responsibilities of setting airspeed, calling up different weapon systems, and gathering information. Both players were able to fire weapons.

The training program taught both individual task responsibilities and basic team processes (e.g., coordination of activities), following the guidelines for effective team training outlined by Swezey and Salas (1992). A three-step sequence was followed: Step 1 focused on how to fly the plane, reach waypoints and destinations, and interpret information from a heads-up display; Step 2 concentrated on training how to call up and fire different weapon systems, how to read and to interpret radar information, and how to use basic flight maneuvering tactics and strategies; and Step 3 had teams practice the skills they developed during Steps 1 and 2 and then had them attempt a practice version of the missions they would complete.

SHARED MENTAL MODELS AND TEAM EFFECTIVENESS 277

The third phase of the experiment started at the conclusion of training. Players completed an initial survey and then flew Missions 1 and 2, after which they received a detailed summary of their performance in terms of waypoints reached and enemy aircraft shot down. They then completed a second survey that repeated the earlier administered measures, and the sequence was then repeated for Missions 3 and 4 and for Missions 5 and 6.

M e a s u r e s

Performance. After training, teams completed three blocks of two missions each. Each mission lasted 10 min, or less time if the team was shot down or crashed. During each mission, the team had three objectives: (a) to survive--worth three points; (b) to fly a preset route that had four waypoints, or destination markers--worth two points each); and (c) to shoot down enemy planes--worth one point each. This scoring scheme presented a strategic dilemma for teams, as the three objectives were incompatible. For example, flying directly toward all waypoints placed the team in great risk and left little time for fighting enemy planes. Alterna- tively, actively engaging enemy planes left little time for reaching waypoints. Finally, maximizing survivability by staying away from enemy planes minimized the likelihood that they would reach waypoints, because that was where the enemy planes were typically circling.

The series of six missions was programmed in blocks of two such that although the specific aspects of each mission varied, the three blocks were equally difficult (as pilot testing confu'med). The total distance between the four waypoints was held constant across missions (although the distances from one to the next varied both within and across missions), and on average four enemy planes were present in each mission (specifically: 3, 5, 4, 4, 5, 3 for Missions 1-6, respectively). Furthermore, the types and relative hostility of the enemy planes were balanced across the two missions within blocks. In short, although each of the six missions represented a unique, complicated, and unpredictable sequence of events, all aspects of the task were balanced over time. Team performance for each block was measured simply by the points earned across two missions.

Team process. Team processes were rated independently by two ob- servers who watched videotapes of the experimental missions. Each rater watched two missions (one performance episode), after which he or she completed the process ratings. We took care not to show the coders the detailed mission feedback so as to minimize potential performance-cue effects (see Martell, Guzzo, & Willis, 1995). We developed 21 items intended to measure three dimensions: (a) strategy formation and coordination (6 items, e.g., "To what extend did the team plan together and coordinate its efforts?"); (b) cooperation (4 items, e.g., "To what extent did they cooperate well during the missions?"); and (c) communication ( l l items, e.g., "To what extent was information about important events and situations shared within the team?"). Each item was rated using a 5-point scale that ranged from 1 (not at all) to 5 (to a very great extenO.

The intercoder reliabilities (i.e., correlations) for these scales were all significant (p < .001) and ranged from .50 to .86 (M = .67). The aggregate-scale internal consistencies were quite acceptable for both coordination (as = .85, .83, and .86 for Times 1-3, respectively) and communication (t~s = .86, .82, and .81 for Times 1-3, respectively), but not for cooperation (as = .55, .40, and .59 for Times 1-3, respectively). Furthermore, although the three process dimensions are conceptually distinguishable, empirically they were highly correlated (Ms = .59, .71, and .59, ps < .001, for Times 1-3, respectively). This degree of correlation limits the extent to which unique effects can be observed in regression analyses. In fact, we attempted to use the three scales separately in a repeated measures multiple regression analysis (described more fully later) and encountered signs of multicollinearity problems (e.g., several high tolerances). Moreover, unrestricted maximum likelihood analyses of the process variables, conducted separately at each time, consistently revealed the existence of a single underlying dimension (p < .05). Consequently, we combined all 21 ratings in to an omnibus team-process variable at each time. The intercoder reliabilities were all significant and high (rs = .76,

.76, and .59, ps < .001, for Times 1-3, respectively) as were the aggregate alphas (as = .91, .90, and .89 for Times 1-3, respectively).

Mental models. Task and team mental models were assessed using teammates' individual ratings of the relationships between various attributes. For the task mental model, we first conducted a detailed task analysis (cf. Baker, Salas, & Cannon-Bowers, 1998; Tesluk, Zaccaro, Marks, & Mathieu, 1997) of the simulation using subject-matter experts and technical documentation (e.g., Powell & Basham, 1992; Spectrum Holobyte, 1991). This effort identified eight critical attributes for the task mental model: (a) diving versus climbing; (b) banking or turning; (c) choosing airspeed; (d) selecting and shooting weapons; (e) reading and interpreting radar; (f) intercepting the enemy; (g) escaping the enemy; and (h) dispensing chaff and flares. Although each of these dimensions can be distinguished from the others, many are highly related. For example, during training the participants were instructed to radically alter altitude and direction as well as to dispense chaff and flares to avoid incoming enemy missiles. To engage an enemy, participants were instructed to adjust airspeed and modify direction and altitude in order to position their plane behind the enemy, and also to select the appropriate type of weapon system to use given their flight situation. Both of these examples would also require the use of the radar to determine the existence of and distance from an enemy plane.

The seven attributes for the team mental models were selected based on a literature review of previous taxonomic research on teamwork dimensions (e.g., Fleishman & Zaccaro, 1992; Stout, Cannon-Bowers, Salas, & Morgan, 1990) with emphasis on the current simulation context. Specifi- cally, these attributes were: (a) amount of information, (b) quality of information, (c) coordination of action, (d) roles, (e) liking, (f) team spirit, and (g) cooperation. These particular attributes were selected because they represent concrete indicators of the three targeted process dimensions. Namely, the amount and quality of information were designed to tap the communication dimension, coordination of action and roles were directed at the strategy component, and the remaining three facets were directed at team cooperation.

Participants were provided with two matrices (one for each mental model) that listed the attributes along the top and side of the grid. The definitions for each attribute were present along the side, immediately following the attribute label. Respondents rated each attribute of the mental model in relation to all other attributes for that model using a 9-point scale that ranged from - 4 (negatively related, a high degree of one requires a low degree of the other) to 4 (positively related, a high degree of one requires a high degree of the other). The ratings were completed before each of the three blocks of performance.

Mental model centrality and convergence. Task and team mental models were analyzed individually using a network-analysis program (UCINET; Borgatti, Everett, & Freeman, 1992). Among a variety of available indices, UCINET provides an index of convergence (QAP correlation) between two matrices. This correlation was calculated between the teammates' task mental models at each time period. Similarly, a QAP correlation was calculated between the teammates' team mental models at each time period. Thus, two mental-model convergence indices, task and team, were produced for each team at each time. QAP correlations are essentially zero-order correlations between the identical elements of two mental-model matrices and therefore range from - 1 (complete disagree- ment or counter-sharedness) to 1 (complete sharedness). Moreover, like any correlation, their magnitudes and the stability of the estimates are complex products of the number of attributes rated and the diversity of underlying relationships (i.e., whether the true cell entries would be ho- mogeneous or variable).

R e s u l t s

The results are presented in two sections. The first section details the hypothes ized mean changes over t ime for each variable.

278 MATHIEU ET AL.

Table 2 Variable Correlations and Descriptive Statistics Per Time

Variable 1 2 3 4 5 6 7 8 9 10 11 12

Time t 1. Task mental model - - 2. Team mental model .15 - - 3. Team process .25 .27 - - 4. Performance .21 .09 .38 - -

Time 2 5. Task mental model .53 .28 .27 .05 - - 6. Team mental model .01 .38 .23 .24 -.08 - - 7. Team process .23 .18 .89 .42 .20 .29 - - 8. Performance .26 .05 .19 .54 -.09 .30 .31

Time 3 9. Task mental model .40 .10 .05 .09 .42 .01 .11

10. Team mental model .24 .28 .26 - .04 .30 .40 .24 1I. Team process .28 .I6 .83 .35 .20 .19 .93 12. Performance .38 .17 .16 .30 - .02 .21 .15

M 0.30 0.24 2.30 2 .55 0.35 0.20 2.37 SD 0.24 0 .27 0.50 1 .24 0.26 0.30 0.50

.10 - -

. t l .04 - -

.28 .14 .20 - -

.52 .12 .16 .20 - - 3.05 0.32 0 .23 2 .42 3.03 1.52 0.26 0.30 0.49 1.36

Note. N = 56 teams. If[r[ /> .26, p < . 0 5 ; i f [ r I ~>.35, p <.01.

The second section reviews the observed correlations between all variables, both within and across times, as well as the tests of the mediational framework. Descriptive statistics and intercorrelations for all variables over time are presented in Table 2.

Change Over Time

We hypothesized that each of the study variables should increase over time as a result of team experience. Repeated measures analyses of variance (ANOVAs) and one-tailed dependent t tests were used to test such differences. The ANOVA results indicated whether, in general, there were significant differences across time for each variable. Should any differences be evident, inspecting the mean contrasts across times would reveal the nature of such effects. Because we advanced a priori hypotheses concerning the nature of such effects, we were justified in conducting a priori mean contrasts even if the omnibus ANOVAs were not significant (see Winer, 1971).

Performance. Kolmogorov-Smirnov tests revealed that the observed performance distributions differed significantly (p < .01) from a normal distribution at each time. Therefore, we applied a square-root transformation to the scores to better approximate a normal distribution. The transformed means and standard deviations are presented in Table 2. The performance ANOVA did not indicate a significant change over time, F(2, 110) = 2.34, ns. However, the a priori t tests revealed a significant change from Block 1 to Block 2,

tBlock l--mock 2(55) = 2.23, p < .05, yet no significant differences between Blocks 1 and 3, tBlo~ k l--mock 3(55) = 1.40, ns, or between Blocks 2 and 3, tmock 2--mook 3(55) = .61 ns. Thus, the performance hypothesis is only partially supported, because it rose significantly from Time 1 to Time 2 but then tapered off to a level that did not differ from either of the first two times.

Process. The process ANOVA revealed a significant change over time, F(2, 1 I0) = 9.28, p < .001. The a priori contrasts revealed significant differences between all three time periods,

tmock l-m,,~k 2(55) = 2.18, p < .05; tmo~k l--BIo~k 3(55) = 3.72,

p < .001; and tmo~k 2--Block 3 ( 5 5 ) = 2.58, p < .01. As we hypothesized, these analyses demonstrated that process increased significantly from each time period to the next.

Task and team mental model convergences. The task mental model convergence ANOVA demonstrated no significant difference over time, F(2, 110) = .90, ns, and none of the a priori contrasts were significant. Similarly, team mental model convergence failed to show any significant differences over time, F(2, 110) = .38, ns, and none of the specific contrasts were significant. Therefore, contrary to our hypotheses, neither type of mental model exhibited greater convergence over time.

The Mediat ional Mode l

Within-episode correlations. An examination of the correlations presented in Table 2 illustrates several interesting relationships. First, the two mental-model-convergence indices failed to correlate significantly (p > .05) at any time (rs = .15, - . 08 , and .04 for Blocks 1-3, respectively). This provides evidence that the two types of mental models are tapping qualitatively different processes. Second, the correlations between the same mental- model-convergence indices over time were significant and mod- erately high for both task (r = .53, p < .001; r = .40, p < .01; and r = .42, p < .01, for Blocks 1-2, 1-3, and 2-3, respectively), and team (r = .38, p < .01; r = .28, p < .05; and r = .40, p < .01, for Times 1-2, 1-3, and 2-3, respectively). These results, in combination with the mean contrasts detailed above, suggest that teams that were more in sync initially continued to be so and that few mental-model-convergence changes were evident over time.

A third pattern was evident in that both team process levels (Time 1-Time 2 r = .89, p < .001; Time 2-Time 3 r = .93, p < .001) and team performance levels (Time 1-Time 2 r = .54, p < .01; Time 2-Time 3 r = .52, p < .01) exhibited high significant correlations over time. These findings, in combination with the results reviewed above, indicate that although team processes generally improved over time, team rank orders remained fairly consistent. Similarly, although teams showed some change in


performance over time, generally those that did better initially continued to be superior.

In terms of more substantive findings, the task-mental-model convergence failed to exhibit any significant correlations with either team process (r = .25, ns; r = .20, ns; and r = .14, ns, for Times 1-3, respectively) or team performance (r = .21, ns; r =

- . 09 , ns; and r = . 12, ns, for Times 1-3, respectively). In contrast, team mental model convergence exhibited significant correlations with team process at Times 1 and 2 but not at Time 3 (r = .27, p < .05; r = .29, p < .05; and r = .20, ns, respectively), and only with team performance at Time 2 (r = .09, ns; r = .30,p < .05; and r = .16, ns, for Times 1-3, respectively). Finally, team processes exhibited significant correlations with team performance during the first two episodes, although not during the third (r = .38, p < .05; r = .31, p < .05; and r = .20, ns, for Times 1-3, respectively).

R e p e a t e d m e a s u r e s f r a m e w o r k . Although the within-episode correlational results reviewed above are encouraging, they are based on the findings from only 56 teams per time. A more powerful way to analyze these data and to test the mediational model is to apply a repeated measures multiple regression (RMMR; see Cohen & Cohen, 1983). In brief, an RMMR regression analysis essentially "stacks" observations from each team over time to create a unified database, analyzes the relationships among variables using this stacked data and traditional multiple regression techniques, and then adjusts the results for the fact that the observations from the same teams over time are not indepen- dent. This analytic strategy provides a far more powerful alterna- tive than a series of within-time analyses, and also allows one to partition total variance into that which exists across teams and that which occurs within teams across time.

We used the RMMR analyses in a path-analysis framework to test the mediational model. We compared the hypothesized model against a saturated model (that includes nonhypothesized paths), which yields two indices of fit: Q and chi-square (Pedhazur, 1982). Q is a descriptive index that reveals how much of the total potential variance in a matrix is captured by the hypothesized model. It ranges from 0 to 1, with values greater than or equal to .90 generally viewed as acceptable. The chi-square tests the difference between the hypothesized and saturated models. It should be noted, however, that with a reasonable sample size (such as that yielded by our RMMR design), the chi-square test is extremely powerful and can reject even acceptable models. Therefore, Ped- hazur (1982) suggested that researchers should rely more heavily on the Q index. Notably, because the overall fit indices in path analysis reflect whether there are any significant omitted paths in the model, a lack of fit here would suggest that one or both of the shared-mental-model indices had a significant direct impact on team performance. Thus, by examining both the overall-model fit indices and the significance of the hypothesized paths, we can ascertain whether team processes mediate the influence of shared mental models on team performance.

The results for the hypothesized model are depicted in Figure 1. The direct effects of both mental-model-convergence indices on team process were significant, R 2 = .10, F(4, 108) = 3.30, p < .05;/3te~,~ = .26, p < .01; and/3t~ k = .31, p < .01. Furthermore, the direct effect of process on performance was significant, R E =

.09, F(2, 110) = 18.70, p < .01;/3p . . . . = .49, p < .0 l, yet adding the two mental model convergence indices to the equation failed to account for significant additional variance, and overall the model

Team Mental Model

Convergence

Task Mental Model

Convergence

I Team Process

.31 *

.90 .91 / /

.49 *

Figure 1. Results of hypothesized model. N = observations. * p < .05.

Teanq

Performance

56 teams, 168

yielded impressive fit indices, Q = .98 and )(2(166, N = 168 observations) = 2.20, ns. T h e significance of the chi-square was determined using the degrees of freedom from the RMMR because it is a more conservative estimate than the degrees of freedom associated with the sample size.

Finally, to more fully explore the nature of the underlying relationships between shared mental models and team performance, we regressed performance onto the two convergence indices without including the team process variable. To conclude that team processes mediated the influence of shared mental models on performance, one must demonstrate that mental models indeed exhibited significant relationships with performance when taken by themselves (see Baron & Kenny, 1986). This final analysis revealed that although the direct effect of team-mental-model convergence was significant (/3t~m = .87, p < .01), the task convergence index was not (/3t~ k = .39, ns). Thus, in summary, we found substantial support for the hypothesized model. Further- more, the impact of members' team mental convergence on team performance was fully mediated by team processes. The task- mental-model convergence effect, however, was less clear. Whereas task mental models did not show any significant relationship with team performance, they did have a positive influence on team processes, which in turn related significantly to performance. Technically, these results do not fulfill the requirements for me- diation (again, see Baron & Kenny, 1986), but they are indicative of an indirect effect.

Discuss ion

We had a number of objectives for this study. First, we sought to distinguish, both conceptually and empirically, between task and team mental models. Second, we sought to examine the influence of convergence, or sharedness, of team members' mental models as related to team processes and performance. And third, we were interested in whether sharedness, as well as team processes and performance, developed over time. Our results supported the distinction between task and team mental models and illustrated that they each had unique effects on subsequent team processes. In turn, team processes were related significantly to team performance. More detailed analyses revealed that team- mental-model sharedness related significantly to team performance, but that the relationship was fully mediated by team

280 MATHIEU ET AL.

processes. In contrast, task-mental-model sharedness did not correlate significantly with team performance but did have an indirect effect through its impact on team processes. These results offer empirical confirmation for the often inferred but yet to be empirically demonstrated impact of shared mental models on team effectiveness (Cannon-Bowers & Salas, 1990; Klimoski & Mo- hammed, 1994; Kraiger & Wenzel, 1997; Rentsch & Hall, 1994).

In terms of the hypothesized changes over time, neither mental- model convergence increased significantly. In contrast, team processes did improve significantly from each performance episode to the next. Team performance, however, rose significantly from Episode 1 to Episode 2, but then faded during Episode 3 to a level that did not differ significantly from either of the first two. One potential reason for the lack of increase in mental-model shared- hess may lie in the nature of the feedback we provided. Specifi- cally, although we provided outcome feedback to all teams following each performance episode, we did not offer any developmental feedback in terms of pointing out mistakes, making process suggestions, and so forth. In real-world settings the military conducts what are known as "after action reviews" (AARs) in which seasoned officers provide detailed analysis of and feedback on how the teams performed and discusses with them how they might improve. We intentionally avoided providing AAR-type feedback in order to maintain experimental control and to test whether experience alone would act to align members' mental models. Apparently, practice alone was sufficient to enhance how well the teams could coordinate their efforts, strategize, and cooperate, but it did not crystallize in terms of greater mental model sharedness. Thus, our results echo the old training adage that time on task alone is not enough--teams need guided experiences and developmental feedback if we expect them to learn.

Significance of Shared Mental Models in Teams

The findings of this study are particularly important because they contribute evidence that supports the construct validity of the shared-mental-model construct. Specifically, we have shown that the similarity of knowledge structures between two team members can predict the quality of team processes and performance. This evidence adds to similar findings that indicate that shared mental models among team members can be measured and that they are related to important team outcomes (see Stout et al., in press). The significance of these findings to team performance research is worth considering in more detail.

To begin with, all of the arguments we presented earlier regarding the mental model construct share one important assumption-- that it is the organization or structure of a person's knowledge that contributes to his or her performance (Goldsmith & Kraiger, 1997; Kraiger, Ford, & Salas, 1993). In fact, several studies have found that measuring an individual's knowledge structures can be effective in predicting performance in various domains (e.g., see Kralger, Salas, & Cannon-Bowers, 1995; Lesgold et al., 1989; McKeithen, Reitman, Rueter, & Hirtle, 1981) above and beyond more traditional measures. One can draw two rather obvious conclusions from this work. The first is that that if one knows something about how a person's knowledge is organized, one can predict performance outcomes. Second, and perhaps more important to the current discussion, the predictive ability of knowledge- organization measures is superior to traditional knowledge tests, at

least for some tasks. Hence, invoking the mental-model construct at the individual level is justified because it seems to describe something important about knowledge that is not captured in traditional knowledge measures (i.e., the organization of that knowledge).

Extending this line of thinking to the team level also has important implications. Specifically, the shared-mental-model construct suggests that it is not only the overlap of knowledge among team members that is predictive of team outcomes but also the synergy of the knowledge organizations. Hence, when it is shown (as we have here) that the sharedness of teammates' mental models predicts subsequent team-processes performance, the im- plication is that knowledge organization--and the relationship among the ways various team members organize their own task knowledge--is a crucial concept. Once again, we believe that this fact justifies the use of the shared mental construct as something above and beyond simple shared task knowledge (see also Stout et al., in press).

Generalizability Issues

This study examined newly formed teams of relative novices performing a complex low-fidelity simulation with no expectation of later interactions. Naturally this raises some questions about the generalizability of our findings. Notably, questions of generalizability, or external validity, hinge on making inferences about the applicability of the results of a given study to some other target population and setting. Important considerations in this regard include questions about the comparability of the tasks or situational demands, the sample populations, and time-related factors (see Cook & Campbell, 1979; Pedhazur & Schmelkin, 1991).

As for the comparability of our low-fidelity simulation with other task environments, the flight simulation task that we used was dynamic, interactive, interdependent, and allowed for objective measures of team performance. It was not too simplistic and mundane or too complex for this sample, and proved to be a very engaging "middle-ground task." We are not suggesting that our results would generalize directly to flight crews and combat situations. Obviously, physical factors such as gravitational forces and life-threatening elements, along with a myriad of other characteristics, make the latter settings quite different than our situation. The point is that we were interested in testing how teammates' mental models and team processes and performance develop in general and sought a research platform that would enable us to simulate, yet control, task elements while maintaining critical team features (e.g., member interdependence, dynamic interaction, des- ignated roles and duties, shared and valued objectives). In effect, we argue that while the use of low-fidelity simulations limits the point-to-point correspondence that is critical for direct generalizability to real-world settings, they do provide for a controlled examination of the critical factors influencing team processes and performance in dynamic environments. In this sense we are interested in studying the construct interrelationships and their applicability to teams in other situations. We believe that our simulation provides a useful vehicle fbr such tests (for similar views, see Baker et al., 1993; Driskell & Salas, 1992; Weaver et al., 1995).

Perhaps a more salient issue regarding generalizability relates to the participant sample in this study as compared with most real- world teams. Because we formed teams strictly for the basis of the


experiment and they had no expectation of future interactions, our results would probably best apply to real-world crews. Airline, military, and other real-world crews are often formed on a mo- ment 's notice, perform their activities, and then disband (see Ginnett, 1990). Therefore, there are no (or limited) influences of previous experience working with one another, role relationships need to be established, and members have relatively little concern for the long-term consequences of their activities with one another. This may well alter the underlying relationships among variables in the I-P-O framework. Although we recognize that the crew versus team distinction may be an important boundary condition for this and other studies that examine newly formed teams, we are not aware of any systematic research that has sought to examine this issue. We believe that exploration of this issue represents an important avenue for future research.

A final issue regarding generalizability pertains to the relative skill levels of our participants. For example, U.S. military teams (or crews) flying multimillion-dollar jets, operating complex com- mand and control networks, and functioning in complex and de- manding environments with life and death literally hanging in the balance are among the most highly skilled and trained teams in existence. They know what they are supposed to do and have demonstrated that they have the capabilities to be successful in countless hours of simulated and real-life engagements. Neverthe- less, history has shown that breakdowns occur and even these teams make mistakes. Can much be applied to these teams from the study of novices? At issue here is whether the underlying nature of the construct relationships that we examined would be isomorphic across team-competency levels. On one hand, it is possible that the fundamental nature of how novice teams perform may differ significantly from how expert teams perform. On the other hand, the differences between the two may be more a matter of degree than nature. While we submit that the construct relationships that we examined are likely to be fairly robust, here too we are unaware of systematic research that has addressed this issue. This offers yet another fruitful direction for future research.

Suggest ions f o r Future Research and Appl icat ion

The caveats regarding generalizability discussed above notwith- standing, our results highlight several important areas for future research and application. We found that greater mental-model convergence relates significantly to better team process and thereby performance. This suggests that efforts to increase teammates' sharedness might lead to greater team effectiveness. One strategy for doing so might be to investigate common underlying cognitive abilities or experiences that give rise to certain knowledge structures. In other words, if individual differences can be tied consistently with the development and use of particular mental models, then teams might be composed so as to enhance members' sharedness. Traditional human resources efforts such as selection, staffing, and placement could be used to achieve such matches. Alternatively, or perhaps in addition, there are a variety of inter- vention strategies that could help to develop shared mental models. Naturally, training applications immediately come to mind, but there may be other strategies, such as job rotations and feedback programs (e.g., AARs), that might also prove valuable. Finally, perhaps certain mental models naturally emerge from exposure to events, whether those events be merely working with a teammate

(for longer periods than were examined here) or having to confront a given environmental challenge. In any case, there appear to be a number of different avenues that can be pursued to help team members develop shared mental models.

A second finding from this research is that different types of mental models can be identified and assessed and that they have unique influences on team processes. This suggests that work on "the" team mental model may be short-sighted at best, and con- founded at worst. Our findings suggest that researchers and prac- titioners should conduct thorough team task analyses to identify the most critical knowledge requirements for a given situation and which of those knowledges must be shared. In this way, the human resources programs mentioned above could be best aligned to target the most critical knowledge levers for team effectiveness.

One final direction for future research warrants mentioning. High mental-model convergence, as operationalized here, does not imply that the models formed by the team members are appropriate. In other words, convergence does not equal quality--and teammates may share a common vision of their situation yet be wrong about the circumstances that they are confronting. One approach to indexing the quality of members' mental models is to compare them against experts' models. This would help to identify deficiencies in members' models as compared with a standard referent structure. A potential problem with this approach, however, is that it assumes that there is a single expert model. For complex constructs such as task work and team work, multiple "expert" mental models may exist. Thus, while we suggest that future research should also consider the quality of teammates' models, we believe that such efforts should go beyond the use of a single "expert" model as the gauge.

References

Baker, D., Prince, C., Shrestha, L., Oser, R., & Salas, E. (1993). Aviation computer games for crew resource management training. International Journal of Aviation Psychology, 3, 143-156.

Baker, D. P., Salas, E., & Cannon-Bowers, J. (1998). Team task analysis: Lost but hopefully not forgotten. Industrial-Organizational Psycholo- gist, 35, 79-83.

Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 1173- 1182.

Benenhausen, K. L. (1991). Five years of groups research: What we have learned and what needs to be addressed. Journal of Management, 17, 345-381.

Borgatti, S., Everett, M. G., & Freeman, L. C. (1992). UCINET IV network analysis software. Columbia, SC: Analytic Technologies.

Brannick, M. T., Roach, R. M., & Salas, E. (1993). Understanding team performance: A multimethod study. Human Performance, 6, 287-308.

Campion, M. A., Medsker, G. J., & Higgs, A. C. (1993). Relations between work group characteristics and effectiveness: Implications for designing effective work groups. Personnel Psychology, 46, 823-850.

Cannon-Bowers, J. A., & Salas, E. (1990, August). Cognitive psychology and team training: Shared mental models in complex systems. Paper presented at the Annual Meeting of the Society of Industrial and Orga- nizational Psychology, Miami, FL.

Cannon-Bowers, J. A., Salas, E., & Converse, S. A. (1993). Shared mental models in expert team decision making. In N. J. Castellan, Jr. (Ed.), Current issues in individual and group decision making (pp. 221-246). Hillsdale, N J: Erlbaum.

282 MATHIEU ET AL.

Cannon-Bowers, J. A., Tannenbaum, S. I., Salas, E., & Volpe, C. E. (1995). Defining team competencies and establishing team training requirements. In R. Guzzo & E. Salas (Eds.), Team effectiveness and decision making in organizations (pp. 333-380). San Francisco: Jossey-Bass.

Cohen, J., & Cohen, P. (1983). Applied multiple regression~correlation for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.

Driskell, J. E., & Salas, E. (1992). Can you study real teams in contrived settings? The value of small group research to understanding teams. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 101-126). Norwood, NJ: Ablex.

Dyer, J. L. (1984). Team research and team training: A state-of-the-art review. In F. A. Muckler (Ed.), Human factors review (pp. 285-323). Santa Monica, CA: Human Factors Society.

Fleishman, E. A., & Zaccaro, S. J, (1992). Toward a taxonomy of team performance functions. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 31-56). Norwood, NJ: Ablex.

Foushee, H. C., & Manos, K. L. (1981). Information transfer within the cockpit: Problems in intracockpit communications. In C. E. Billings & E. S. Cheaney (Eds.), Information transfer problems in the aviation system (NASA Report No. TP-1875). Moffett Field, CA: NASA-Ames Research Center. (NTIS No. N81-31162)

Ginnett, R. C. (1990). Airline cockpit crews. In J. H. Hackman (Ed.), Groups that work (and those that don't) (pp. 427-448). San Francisco: Jossey-Bass.

Gist, M. E., Locke, E. A., & Taylor, M. S. (1987). Organizational behavior: Group structure, process, and effectiveness. Journal of Management, 13, 237-257.

Gladstein, D. L. (1984). Groups in context: A model of task group effectiveness. Administrative Science Quarterly, 29, 499-517.

Goldsmith, T. E., & Kraiger, K. (1997). Applications of structural knowledge assessment to training evaluation. In J. K. Ford, S. W. S. Kozlow- ski, K. Kraiger, E. Salas, & M. Teachout (Eds.), Improving training effectiveness in work organizations (pp. 73-96). Mahwah, NJ: Erlbaum.

Guzzo, R. A., & Dickson, M. W. (1996). Teams in organizations: Recent research on performance and effectiveness. Annual Review of Psychol- ogy, 47, 307-338.

Hackman, J. R. (1990). Work teams in organizations: An orienting framework. In J. R. Hackman (Ed.), Groups that work (and those that don't) (pp. 1-14). San Francisco, CA: Jossey-Bass.

Hackman, J. R., Brousseau, K. R., & Weiss, J. A. (1976). The interaction of task design and group performance strategies in determining group effectiveness. Organizational Behavior and Human Performance, 16, 350-365.

Johnson-Laird, P. (1983). Mental models. Cambridge, MA: Harvard Uni- versity Press.

Kanki, M. A., & Foushee, H. C. (1989). Communication as group performance mediator of aircrew performance. Aviation, Space, and Environ- mental Medicine, 60, 402-410.

Kleinman, D. L., Luh, P.B., Pattipati, K. R., & Serfaty, D. (1992). Math- ematical models of team performance: A distributed decision-making approach. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 177-217). Norwood, NJ: Ablex.

Klimoski, R., & Mohammed, S. (1994). Team mental model: Construct or metaphor? Journal of Management, 20, 403-437.

Kraiger, K., Ford, J. K., & Salas, E. (1993). Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. Journal of Applied Psychology, 78, 311-328.

Kraiger, K., Salas, E., & Cannon-Bowers, J. A. (1995). Measuring knowledge organization as a method for assessing learning during training. Human Factors, 37, 804-816.

Kraiger, K., & Wenzel, L. H. (1997). A framework for understanding and measuring shared mental models of team performance and team effec-

tiveness. In E. Salas, M. T. Brarmick, & C. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications (pp. 63-84). HiUsdale, NJ: Erlbaum.

Lesgold, A., Rubinson, H., Feltovich, P., Glaser, R., Klopfer, D., & Wang, Y. (1989). Expertise in a complex skill: Diagnosing x-ray pictures. In M. T. H. Chi & R. Glaser (Eds.), The nature of expertise (pp. 311-342). Hillsdale, NJ: Erlbaum.

Martell, R. E., Guzzo, R. A., & Willis, C. E. (1995). A methodological and substantive note on the performance-cue effect in ratings of work-group behavior. Journal of Applied Psychology, 80, 191-195.

Mcintyre, R. M., & Salas, E. (1995), Measuring and managing for team performance: Emerging principles from complex environments. In R. Guzzo & E. Salas (Eds.), Team effectiveness and decision making in organizations (pp. 149-203). San Francisco: Jossey-Bass.

McKeithen, K. B., Reitman, J. S., Rueter, H. H., & Hirtle, S. C. (1981). Knowledge organization and skill differences in computer progranuners. Cognitive Psychology, 13, 307-325.

Morgan, B. B., Jr., Glickman, A. S., Woodard, E. A., Blaiwes, A. S., & Salas, E. (1986). Measurement of team behaviors in a Navy environment (NTSC Tech. Rep. No. 86-014). Orlando, FL: Naval Training Systems Center.

Pedhazur, E. J. (1982). Multiple regression in behavioral research. New York: Holt, Rinehart & Winston.

Pedhazur, E. J., & Schmelkin, L. P. (1991). Measurement, design, and analysis: An integrated approach. Hillsdale, NJ: Erlbaum.

Powell, J., & Basham, T. (1992). Falcon 3: The complete handbook. Corte Madera, CA: The Waite Group.

Prince, A., Brannick, M. T., Prince, C., & Salas, E. (1992). Team process measurement and the implications for training. Proceedings of the Hu- man Factors Society 36th annual meeting (pp. 1351-1355). Santa Monica, CA: Human Factors Society.

Rentsch, J. R., & Hall, R. J. (1994). Members of great teams think alike: A model of the effectiveness and schema similarity among team mem- bets. Advances in Interdisciplinary Studies of Work Teams, 1, 223-261.

Rentsch, L R., Heffner, T. S., & Dully, L. T. (1994). What you know is what you get from experience: Team experience related to teamwork schemas. Group and Organization Management, 19, 485-509.

Roberts, K. H., & O'Reilly, C. A. (1976). Interpersonal, work group, and organizational communication: A systematic approach to understanding organizations (Technical report, Contract No. N00014-69-A-0200- 1054). Berkeley, CA: Institute of Industrial Relations.

Rouse, W. B., & Morris, N. M. (1986). On looking into the black box: Prospects and limits in the search for mental models. Psychological Bulletin, 100, 349-363.

Salas, E., Bowers, C. A., & Cannon-Bowers, J. A. (1995). Military team research: 10 years of progress. Military Psychology, 7, 55-75.

Salas, E., Dickinson, T. L., Converse, S. A., & Tannenbaum, S. I. (1992). Toward an understanding of team performance and training. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 3-29). Norwood, NJ: Ablex.

Spectrum Holobyte. (1991). Falcon 3.0 flight manual. Alameda, CA: Author.

Stout, R. J., Cannon-Bowers, J. A., & Salas, E. (1996). The role of shared mental models in developing team situational awareness: Implications for training. Training Research Journal, 2, 85-116.

Stout, R. J., Cannon-Bowets, J. A., Salas, E., & Milanovich, D. M. (in press). Planning, shared mental models, and coordinated performance: An empirical link is established. Human Factors.

Stout, R. J., Cannon-Bowers, J. A., Salas, E., & Morgan, B. B. (1990). Does crew coordination behavior impact performance? Proceedings of the Human Factors Society 34th annual meeting (pp. 1382-1386). Santa Monica, CA: Human Factors Society.

Stout, R. J., Salas, E., & Kraiger, K. (1996). The role of trainee knowledge


structures in aviation team environments. International Journal of Avi- ation Psychology, 7, 235-250.

Swezey, R. W., & Salas, E. (1992). Guidelines for use in team-training development. In R. W. Swezey & E. Salas (Eds.), Teams: Their training and performance (pp. 219-246). Norwood, NJ: Ablex.

Tannenbaum, S. I., Beard, R. L., & Salas, E. (1992). Team building and its influence on team effectiveness: An examination of conceptual and empirical developments. In K. Kelley (Ed.), Issues, theory, and research in industrial~organizational psychology (pp. 117-153). New York: Elsevier Science.

Tesluk, P., Zaccaro, S. J., Marks, M. A., & Mathieu, J. E. (1997). Task and aggregation issues in analysis and assessment of team performance. In M. T. Brannick, E. Salas, & C. Prince (Eds.), Team performance assessment and measurement: Theory, methods, and applications (pp. 197-224). Mahwah, NJ: Erlbaum.

Weaver, J. L., Bowers, C. A., Salas, E., & Cannon-Bowers, J. A. (1995). Networked simulations: New paradigms for team performance research. Behavior Research Methods, Instruments and Com- puters, 27, 12-24.

Wilson, J. R., & Rutherford, A. (1989). Mental models: Theory and application in human factors. Human Factors, 31, 617-634.

Wirier, B. J. (1971). Statistical principles in experimental design (2nd ed.). New York: McGraw-Hill.

Received May 13, 1998

Revision received June 4, 1999

Accepted June 9, 1999 •

the influence of shared mental models on team process …the influence of shared mental models on...

Documents