effects of communication mode on social presence, virtual presence, and performance in collaborative...

Eva-Lotta SallnasInteraction and PresentationLaboratoryRoyal Institute of TechnologyStockholm, [email protected]

Presence, Vol. 14, No. 4, August 2005, 434-449

© 2005 by the Massachusetts Institute of Technology

Effects of Communication Modeon Social Presence, VirtualPresence, and Performance inCollaborative VirtualEnvironments

Abstract

How does communication mode affect people’s experience of social presence,presence, and performance, and how does it affect their actual collaboration in avirtual environment? In a first experiment, subjects communicated by text-chat, au-dio conference, or video conference in a desktop collaborative virtual environment(CVE). Both perceived social presence and presence were shown to be lower inthe text-chat condition than in the audio- and video-conference conditions. Peoplespent a longer time performing a decision-making task together, spoke fewer wordsin total, and also spoke fewer words per second in the text-chat environment. Fi-nally, more words per second were spoken in the audio-conference than in thevideo-conference condition. In a second experiment, collaboration in a CVE audio-and a CVE video condition was compared to collaboration in a Web audio-conference and a Web video-conference condition. Results showed that presencewas rated higher in the two video than in the two audio conditions and especiallyin the Web video condition. People spent more time in the video than in the audioconditions and more words per second were spoken in the Web than in the CVEconditions. In conclusion, it was found that both the communication media usedand the environment in which collaboration takes place (CVE or Web) make a dif-ference for how subjects experience interaction and for their communication be-havior.

1 Introduction

In desktop collaborative virtual environments (CVEs), the communica-tion mode is often text-chat, whereas audio or video channels are less oftenused. For this reason, it is interesting to compare the use of different kinds ofcommunication media in CVEs. In this paper, two experiments are presentedwhere such comparisons were performed. In the first experiment, subjectscompleted a task together in a CVE and used text-chat, audio conference, orvideo conference when communicating. In the second experiment, subjectscompleted the same task as in the first experiment in a Web environment withequivalent information content using audio conference or video conference.The Web audio and Web video data in the second experiment were comparedto the CVE audio and CVE video data from the first experiment. The reason

434 PRESENCE: VOLUME 14, NUMBER 4

for performing the second experiment was to comparecollaboration in a virtual environment with collabora-tion in a Web environment. The virtual environment isoften perceived as a 3D place in which people can navi-gate with an avatar, interact with other people, manipu-late objects, and obtain information. The question iswhether collaboration in a virtual environment is verydifferent from collaboration in a Web environment, andwhat role communication media play. In both experi-ments, the aims were to investigate the extent that sub-jects felt that they were present with another person inthe shared environments, the level of perceived presencein the different conditions, and how well the subjectsfelt they could complete the task. Finally, the aim was toexplore how long subjects spent completing the task,the extensiveness of the dialogue, and the subjects’communication tempo in different conditions.

Earlier research has shown that communicatingthrough audio is perceived to be important when col-laborating at a distance and that audio has significantadvantages in comparison to text-chat (Ochsman &Chapanis, 1974; Short, Williams, & Christie, 1976;Vaske & Grantham, 1989; Matsuura, Fujino, Okada, &Matsushita, 1993). Research has not found such uni-form results, however, regarding benefits from usingvideo conferencing. On the contrary, several studieshave shown that video does not make a significant dif-ference compared to audio (Olson, Olson, & Meader,1995; Anderson et al., 1996; Whittaker & O-Conaill,1997). In one study (Chapanis, 1975), various modesof interaction were examined ranging over audio, video,handwritten, typewritten, and face-to-face communica-tion. Results showed that people spoke more wordsthan they wrote and completed tasks faster when audiowas provided compared to when text was used. How-ever, it was found that in terms of time to completetasks and amount of words spoken, there was no differ-ence when a video-conference condition was comparedto an audio-conference condition. One study showedthat a person’s preference ratings for how much he likedother persons whom he met in an audio mode werelower than for those whom he met in a video mode orface-to-face (Williams, 1977). Daly-Jones, Monk, andWatts (1998) found that users were more aware of the

presence of their partner, could monitor their partner’sattentional status better, and felt that video communica-tion aided collaboration. Video has also been shown tosupport informal communication and relation building(Kraut, Fish, Root, & Chalfonte, 1990; Dourish & Bly,1992; Bly, Harrison, & Irwin, 1993).

2 Presence

The importance of the extent to which a personhas a feeling of being present in virtual-reality environ-ments, or other media such as films or books, is increas-ingly recognized (Hendrix & Barfield, 1996; Lombard& Ditton, 1997; Slater & Wilbur, 1997). Presence hasbeen defined by Witmer and Singer (1998) as “the sub-jective experience of being in one place or environment,even when one is physically situated in another.”

In order to maintain the individual’s focus and senseof presence, the information in the media environmentmust be meaningful to that individual. Involvementthus becomes important for the degree of perceivedpresence. Involvement depends on the degree of signifi-cance or meaning that individuals attach to stimuli, ac-tivities, or events (Witmer & Singer, 1998). As usersfocus more attention on the virtual-reality stimuli, theybecome more involved in the virtual-reality experience,which leads to an increased sense of presence. Further,distracting events in the physical locale must be limited,or it must be possible for the individual to integratethem into the virtual environment in a meaningful way.

The degree of immersion also affects perceived pres-ence. Immersion is a psychological state characterized byperceiving oneself to be enveloped by, included in, andinteracting with an environment that provides a contin-uous stream of stimuli and experiences (Witmer &Singer, 1998). Factors that affect immersion includeisolation from the physical environment, perception ofself-inclusion in the virtual environment, natural modesof interaction, and control and perception of self-movement. A virtual world that produces a greater senseof immersion increases the level of presence. Finally, ahigh level of immersion increases the level of involve-ment and vice versa.

Sallnas 435

3 Social Presence

It is also important to investigate the social dimen-sion of perceived presence. In virtual-reality research,the concepts of togetherness, copresence, and socialpresence are used in order to address issues of socialinteraction (Heeter, 1992; Durlach & Slater, 2000;Slater & Steed, 2002; McLaughlin, Sukhatme, Peng,Zhu, & Parks, 2003; Biocca, Harms, & Burgoon, sub-mitted). In social psychological studies of mediated in-teraction, interest in the capacity of media to transmitsocial information has focused on the interaction be-tween people through different communications mediasuch as e-mail, telephone, or video conference com-pared to face-to-face interaction. A general argument isthat the social richness of the communication medium(Katz & Tushman, 1978; Daft & Lengel, 1986; Rice,1993) or the level of perceived social presence (Short etal., 1976), has to be matched with the task in order forcollaborators to accomplish it satisfactorily.

Social-presence theory evolved through researchabout efficiency and satisfaction in the use of differenttelecommunications media. It was Short et al. (1976)who introduced and defined the term social presence as“the salience of the other in a mediated communicationand the consequent salience of their interpersonal inter-actions.” The construct social presence can be tracedback to Mehrabian’s (1969) concept of immediacy,which he defined as “those communication behaviorsthat enhance closeness to and nonverbal interactionwith another.” Social presence is regarded as a singledimension that represents a cognitive synthesis of sev-eral factors that naturally occur in face-to-face commu-nication. Among these are the capacity to transmit in-formation verbally and also by nonverbal cues such asgestures, facial expression, direction of gaze, and pos-ture. The inability of media to transmit nonverbal cuescan thus have negative effects on interpersonal commu-nication. The level of social presence is the extent towhich a medium is perceived as sociable, warm, sensi-tive, personal, or intimate when it is used to interactwith other people (Short et al.). The degree to whichpeople perceive a medium as supporting social presencerelates to the purpose of the interaction, and will influ-

ence the medium chosen by the individual who wishesto communicate. The order of media that is suggestedby social-presence theory is, from higher to lower: face-to-face, television (video), multispeaker audio, tele-phone (also speakerphone and monaural audio), and,finally, business letters (Short et al.).

4 Overview and Research Questions

A first experiment was conducted in order to in-vestigate the effects of communication medium on col-laboration in a desktop collaborative virtual environ-ment (CVE) mediated by text-chat, audio conference,or video conference. The aspects of collaboration inves-tigated were to what extent the medium affects the sub-jective perception of social presence, presence, and per-formance in a CVE. Furthermore, the experimentinvestigated to what extent the medium affects the timeit takes persons to complete a task together, the numberof words spoken, and finally, the number of words spo-ken per second during collaboration.

A second experiment was conducted in order to com-pare the results of the first experiment with results fromcollaboration in a Web environment. Subjects per-formed the same task in the Web environment contain-ing the same information as the CVE did (Figure 1). Inthe analysis, the results from the interaction in the CVEaudio and CVE video conditions in the first experimentwere compared to results from the interaction in theWeb audio and Web video conditions in the second ex-periment. A Web text-chat condition was not imple-mented, because it was assumed that it was the audioand video conditions that needed to be controlled for asthe text-chat would always be significantly differentfrom audio and video conditions.

5 Experiment 1

5.1 Method

5.1.1 Subjects. Sixty subjects participated in theexperiment. They were university students and adminis-trative personnel. The subjects were placed in different


rooms and they did not know each other prior to theexperiment. Subjects collaborated in pairs of 1 womanand 1 man when performing the experimental task inthe CVE. Thereafter, individual questionnaires weregiven to each participant, which they filled out indepen-dently.

5.1.2 Experimental Design. A between-subjectsdesign was used in order to avoid learning effects. Theindependent variable was the collaborative virtual envi-ronment with three communication-mode conditions:CVE video, CVE audio, and CVE text-chat. The con-cepts of interest in the experiment were social presence,presence, and perceived performance, and these weremeasured by three questionnaires. Other dependentvariables were time to finish the task, number of wordsused, and, finally, number of words used per second inthe dialogue between the subjects.

5.1.3 Apparatus. Two PowerBook PCs, net-worked through an ethernet, were used in the experi-ment. The CVE was constructed in Active Worlds withthe appearance of an exhibit with information points(Figure 2).

The information consisted of posters with picturesand QuickTime movie clips with video images and au-dio information. The video clips were shown indepen-dently (nonsimultaneously) when chosen by the subjectsand therefore subjects could see different video clips inparallel. Each subject was represented by a humanlikeavatar in Active Worlds, which the other person sawduring navigation in the virtual world. The female sub-jects had a female avatar with brown hair in a ponytailthat was wearing purple trousers and T-shirt. The malesubjects had a male avatar with black short hair that waswearing blue trousers and a white T-shirt.

In all three conditions, subjects had a PowerBookdirectly in front of them with the Active Worlds display.In the CVE video condition, each subject had a 21-inchtelevision monitor about 30 degrees on their left-handside. They also had headsets and talked to each other viaa telephone connection. In the condition with an audioconnection, two telephones with headsets were used. Inthe CVE text-chat condition, the subjects communi-cated via text-chat, which is a feature in Active Worlds.When a user writes text in this chat, the text is notshown to the receiver of the text until the composer of

Figure 1. Two people collaborating in the Web video environment.

The video conference setup was the same as the setup used in the

CVE video condition.

Figure 2. One subject’s perspective of the information in the CVE in

Active Worlds showing the other subject and posters. Blue signs with

text are links to QuickTime movie clips. These clips are running on the

right side, playing movies with verbal information about the different

car models.

Sallnas 437

the messages presses the Enter key. The text-chat washidden in the audio and video condition.

The interaction between subjects was video recordedvia a video splitter for later analysis. In that way, thevideo image of both subjects’ faces and upper bodieswere visible as well as their navigation in Active Worlds,and their dialogue was audible synchronously in thesame recording on the same screen.

5.1.4 Experimental Task. The subjects wereinstructed to perform a decision-making task in theCVE that was presented to them as a written scenario:

You have participated in a competition in whichyou and another participant performed equally well.You therefore have to share the first prize: a Volvo carof your choice with insurance and gas for one year.You will alternate in using the car every secondmonth. You will not be able to sell the car. The orga-nizers of the competition now want you to go into avirtual exhibition and choose the car you are going toshare. You should then decide together which one ofyou is going to have the car the first month.

The task in this experiment was formulated in such away that consensus had to be reached by the two sub-jects regarding both which car to choose and whoshould have it the first month. Subjects were instructedon how to navigate in the CVE, and kept a printed ver-sion of the instructions together with the written sce-nario during the whole experiment. All possible ways ofnavigation were tried once by the subjects before theexperiment: walking or running forward, backward, andsideways on the ground or turning around either left orright. Before the experiment, both users’ avatars wereplaced just outside the entrance of the virtual exhibi-tion. The experimenter explained to each of the subjectsthat they had one avatar, that the avatar standing nextto him/her was their collaborator’s avatar, and that theywere both going to go into the exhibition in front ofthem.

The subjects usually first discussed their assignmentand decided on a strategy for how to accomplish thetask. The most common strategy was to go together,usually to the left, and study each information point in

turn, discussing the information and personal prefer-ences. The two persons in many cases placed themselvesso that they saw the other avatar and the information atthe same time. When the subjects felt that they knewthe information, they usually stopped, sometimes facingthe other avatar, and tried to negotiate decisions. Insome cases, users went off on their own to different in-formation points after some discussion, in order tocheck a second time that they remembered things cor-rectly. Finally, both subjects often went to the car theyhad agreed upon and then said goodbye there.

5.1.5 Dependent Variables. Presence. In thisstudy, presence is defined as “the subjective experienceof being in one place or environment, even when one isphysically situated in another.” The presence question-naire (PQ) developed by Witmer and Singer (1998) formeasuring presence was used. Presence in this study wasmeasured by PQ Items 1, 2, 3, 5, 6, 7, 10, 12, 13, 14,15, 16, 18, 23, 25, 26, 27, 28, 29, and 30. These itemsmeasure the aspects addressed by the involved/control,natural, and interface quality PQ subscale items plusthree auditory items. The involved/control subscaleitems address perceived control of events in the virtualenvironment, responsiveness of the virtual environmentto a person’s actions, how involving the visual aspectswere, and, finally, to what extent the person becameinvolved in the experience. The natural subscale itemsmeasure how natural and how realistic was the interac-tion in the virtual environment and also how naturalwas the control of locomotion through the virtual envi-ronment. The interface quality subscale items addresswhether control devices or display devices interfere ordistract from task performance and the extent to whichthe participants could concentrate on the task.

Social Presence. The subjects’ perceived socialpresence was measured using a questionnaire with a bi-polar 7-point scale consisting of 13 questions (Appendix1). The definition of social presence in this experimentalstudy was “the feeling that one is present with anotherperson in a mediated environment.” The first 5 ques-tions are equivalent to the questions used by Short et al.(1976). The following 8 questions were derived from


social presence theory (Short et al.), and the aim was tomeasure to what extent persons felt they were sociallypresent, to what extent they understood the other per-son’s intentions, emotions, and reactions, and to whatextent they felt that their reactions were perceived bythe other person.

Perceived Task Performance. Perceived task per-formance was measured by a questionnaire with bipolar7-point scales. The questionnaire focused on the users’evaluations of their own task performance when usingthe system and also their skill level in using specific fea-tures in the system. The questionnaire consisted of fourquestions (Appendix 2).

Time. The time spent by collaborators to finishthe task was obtained by observations of video record-ings from the experiments. In other studies, time hasbeen used as a basic measure of efficiency and it is as-sumed that there is a relationship between the activitiesof collaboration and the speed at which a group per-forms a task (Gutwin & Greenberg, 1999). But whensubjects’ available time is unlimited, the interpretationof the measure of time spent gets more complex. Then,time spent is first a measure of how much time groupsneed in order to complete a task, and second, a measureof how much time subjects want to spend interactingwith each other in the environment (Short et al., 1976).

Number of Words Spoken. The dialogues weretranscribed from the video recordings, and data aboutthe total number of words spoken were obtained. Thedialogues from the text-chat condition were obtainedfrom the text logs saved after each experimental session.The amount of messages or words exchanged in a dia-logue can similarly be regarded as a measure of howefficiently information can be transmitted through amedium, and also to what extent a medium encouragesconversation (Chapanis, Ochsman, Parrish, & Weeks,1972).

Number of Words Spoken per Second. The numberof words used per second gives an indication of thetempo of the conversation and indirectly an indication

of the amount of pause time in the dialogue. The mea-sures time, number of words, and words used per sec-ond analyzed together tell more about the communica-tion process than each of them does alone. The measureof time taken to perform a decision-making task has tobe seen in relation to the quantity of words used in thedialogue in order to understand the meaning of the re-sults. This is also why the measure of words used persecond adds meaning to an analysis of communication,since it is an indication of dialogue tempo.

5.2 Results

The subject pairs performed the experimental tasktogether and therefore the unit of analysis is the pairs inthe analysis of the data of time to complete the task,number of words used, and words used per second. Forthe analysis of these measures, there were 8 pairs of sub-jects in each condition (missing data is due to technicalproblems). When the experimental task was finished,participants ended their conversation and individualquestionnaires were given to each participant, whichthey filled out independently of each other. The unit ofanalysis for the questionnaire data was the 60 individualparticipants’ ratings. The level of significance was set to0.05 for the analysis in this study.

5.2.1 Presence. The main effect of a one-wayanalysis of variance (ANOVA) regarding participants’perceived presence was significant (F2,57 � 3.10),power � 0.57. Items in the questionnaire were analyzedtogether as a total dimension. The least significant dif-ference (LSD) post hoc test showed that participantsrated presence significantly higher in the CVE videothan in the CVE text condition and also significantlyhigher in the CVE audio than in the CVE text condi-tion. The post hoc test did not show a significant differ-ence for presence between the CVE video and the CVEaudio condition. The subjects’ total mean rating ofpresence is shown in Table 1. As there were 20 ques-tions in the questionnaire measuring presence, the meanvalue for each question on the 7-point scale was 4.5 inthe CVE text-chat condition and 5.0 in the CVE videoand audio conditions.

Sallnas 439

5.2.2 Social Presence. The main effect of aone-way ANOVA showed that the three CVE condi-tions video, audio, and text-chat differed significantly(F2,57 � 3.57) regarding subjects’ perceived social pres-ence (power � 0.64). Items in the questionnaire wereanalyzed together as a total dimension. The post hoctest (LSD) showed that perceived social presence wassignificantly higher in the CVE video than in the CVEtext condition and also significantly higher in the CVEaudio than in the CVE text condition. No significantdifference was found between the CVE video and theCVE audio condition regarding social presence. Thetotal mean ratings of social presence for the CVE text-chat, CVE audio, and CVE video conditions are shownin Table 1. There were 13 questions, and the meanvalue for each question on the 7-point scale was 5.2 inthe CVE video condition, 5.1 in the CVE audio condi-tion, and 4.5 in the CVE text-chat condition.

5.2.3 Perceived Task Performance. No sig-nificant main effect was found for perceived task perfor-mance (F � 2.92, power � 0.55). Items in the ques-tionnaire were analyzed together as a total dimension.However, the post hoc analysis (LSD) showed that peo-ple perceived they performed the task significantly bet-ter in the CVE video condition than they did in theCVE text-chat condition. No other significant differ-ences were found between conditions. The total meanratings of the items in the dimension perceived perfor-mance for the CVE text-chat, CVE audio, and CVEvideo conditions are shown in Table 1. There were fourquestions, and the mean value for each question on the7-point scale was 5.5 in the CVE video condition, 5.0

in the CVE audio condition, and 4.5 in the CVE text-chat condition.

5.2.4 Time. A one-way ANOVA showed a sig-nificant main effect (F2,21 � 6.45) for the time spent byparticipants performing the experimental task (power �

0.86). The post hoc analysis (LSD) showed that partici-pants spent significantly longer performing the task inthe CVE text condition than in the CVE audio andCVE video conditions. No significant difference wasfound regarding task-completion time between theCVE audio and CVE video conditions. The mean val-ues of the total task-completion time are presented inTable 2.

5.2.5 Number of Words Spoken. Regardingnumber of words spoken, results from a one-wayANOVA showed a significant main effect (F2,21 � 6.21)between the conditions (power � 0.84). The post hocanalysis (LSD) showed that participants used signifi-cantly more words in their dialogue in the CVE videoand CVE audio conditions than they did in the CVEtext-chat condition. There was no significant differencein the number of words used between the CVE audioand CVE video condition. The mean values of the totalnumber of words used in dialogues in the three condi-tions are presented in Table 2.

5.2.6 Number of Words Spoken per Sec-ond. A one-way ANOVA showed a significant (F2,21 �

20.97) main effect for the number of words used persecond (power � 0.99). The post hoc analysis showedthat participants used significantly fewer words per sec-

Table 1. Results from the Analysis of Questionnaire Data Regarding Social Presence, Presence, and Perceived PerformanceShowing Mean Values, Standard Deviations (SD), and Sample Size (n)

Social presencemean (SD), n

Presence mean(SD), n

Perceived performancemean (SD), n

CVE—text 58 (15), 20 89 (16), 20 18 (6), 20CVE—audio 67 (12), 20 99 (10), 20 20 (5), 20CVE—video 68 (12), 20 99 (15), 20 22 (5), 20


ond in the CVE text-chat condition than they did in theCVE audio and CVE video conditions. Furthermore,the post hoc test also showed that participants used sig-nificantly more words per second in the CVE audiocondition than they did in the CVE video condition.The mean values of the total number of words used persecond in dialogues are presented in Table 2.

6 Experiment 2

In a second experimental study, the results fromthe collaboration in the 3D CVE in the first experimentwere compared with results from collaboration in a 2DWeb environment with the same information content.The purpose was to gain a better understanding ofthe specific effects of CVEs by comparing such anenvironment with a Web environment. Only twocommunication-mode conditions, Web audio confer-ence and Web video conference, were implemented,because it was assumed that it was the audio and videoconditions that needed to be controlled for as the text-chat would be significantly different from audio andvideo.

6.1 Method

6.1.1 Subjects. Twenty subjects participated inthe second experiment, completing the task in a Webaudio or a Web video condition. Subjects were univer-sity students and administrative personnel. The match-ing in pairs of 1 woman and 1 man, who did not know

each other prior to the study, was the same as used inthe first experiment.

6.1.2 Experimental Design. A between-subjectsdesign was used. In this follow-up experiment, data re-garding collaboration in a Web video and a Web audiocondition was compared with the data from the collabo-ration in the CVE audio and CVE video conditions inthe first experiment. The dependent variables were thesame as in the first experiment: social presence, pres-ence, perceived performance, task-completion time, to-tal number of words used in the dialogue, and the num-ber of words used per second. The experimental task,presented as a scenario, was the same as in the first ex-periment.

6.1.3 Apparatus. The Web environment wasdesigned as a Web site with the same information points(posters and QuickTime movie clips with audio) as usedin the CVE. In the Web conditions, the informationpoints were placed next to each other in one Web pagein a sequential fashion (Figure 3). The subjects had noavatars in the Web conditions and they could not seethe other subject’s cursor.

Subjects had a PowerBook directly in front of themwith the Web environment just as in the first experi-ment. In the Web video condition, subjects had a televi-sion monitor about 30 degrees to their left side (Figure1). They also had headsets and talked to each other via atelephone connection. In the Web audio condition, thesubjects communicated via a telephone connection us-

Table 2. Results from Analysis of Time to Complete the Task, Number of Words Used, and Number of Words Used perSecond in Dialogues, Showing Mean Values, Standard Deviations (SD), and Number of Subjects (n)

Time (s)mean (SD), n

Words mean(SD), n

Words/smean (SD), n

CVE—text 1729 (925), 8 342 (223), 8 0.2 (0.1), 8CVE—audio 608 (241), 8 1084 (452), 8 1.9 (0.7), 8CVE—video 978 (549), 8 1243 (798), 8 1.3 (0.5), 8

Sallnas 441

ing headsets as in the first experiment. The interactionbetween subjects was video recorded for later analysis.

6.2 Results

A two-way ANOVA was performed on the datafrom the CVE audio and CVE video condition from thefirst experiment and the Web audio and Web video con-dition from the second experiment. The unit of analysiswas individual participants for the questionnaire dataand pairs of collaborating participants for the measurestask-completion time, number of words spoken, andnumber of words spoken per second. The level of signif-icance was set to 0.05 for this analysis.

6.2.1 Presence. A two-way ANOVA did notshow a significant main effect (F3,56 � 2.63) regarding

presence (power � 0.61). The two-way ANOVA didshow, however, that participants’ perceived presencewas significantly higher in the two video conditions thanin the two audio conditions (F1,56 � 4.13). Participants’perceived presence was not affected by whether theycollaborated in a CVE or Web environment. However,a significant interaction effect (F1,56 � 4.07) shows thatwhen participants interacted in a CVE, there was noeffect of medium, but in the case of interaction in aWeb environment, presence was perceived to be in-creased by video conference. The subjects’ total meanratings on the presence dimension are shown in Table 3.

6.2.2 Social Presence. The two-way ANOVAshowed no significant main effect (F3,56 � 0.77,power � 0.21) regarding perceived social presence, andno significant differences were found either between

Figure 3. The Web environment with posters and QuickTime movie clips with audio.


audio and video conditions or between CVE and Webconditions. There was no significant interaction effectregarding social presence. The subjects’ total mean rat-ings on the social-presence dimension are shown in Ta-ble 3.

6.2.3 Perceived Task Performance. A two-way ANOVA did not show a significant main effect(F3,56 � 1.78) between conditions regarding perceivedperformance (power � 0.44). The subjects’ total meanratings for perceived task performance are shown in Ta-ble 3. The two-way ANOVA showed no significant dif-ferences regarding perceived task performance eitherbetween audio and video conditions or between CVEand Web conditions. A significant interaction effect(F1,56 � 4.03), however, showed that when participantsinteracted in a CVE, task performance was perceived tobe better in the video-conference condition, but taskperformance was perceived to be better with audio in aWeb environment.

6.2.4 Time. A two-way ANOVA did not show asignificant main effect (F3,22 � 2.76) regarding the ef-fects on total mean time (power � 0.59). The subjects’

total mean time values in seconds are shown in Table 4.The two-way ANOVA, however, showed that partici-pants spent significantly longer in the video conditionsthan in the audio conditions (F1,22 � 5.10), when thedata from the Web and CVE conditions were com-bined. The time spent by participants performing theexperimental task was not affected significantly bywhether they collaborated in a CVE or a Web environ-ment. The Web and CVE versus the audio and videoconditions did not interact in any significant way.

6.2.5 Number of Words Spoken. The two-way ANOVA showed no significant main effect(power � 0.21) between conditions regarding numberof words spoken in the dialogue and no significant ef-fects between communication media or between CVEand Web conditions. There was no significant interac-tion effect regarding number of words used in the dia-logue. The mean values of the number of words partici-pants used in the different conditions are shown inTable 4.

6.2.6 Number of Words Spoken per Second.A two-way ANOVA showed a significant main effect(F3,22 � 3.40) when conditions were compared regard-

Table 3. Summary Values of Questionnaire Data RegardingPresence, Social Presence, and Perceived Performance ShowingMean Values, Standard Deviations (SD), and Sample Size (n)

CVE mean(SD), n

Web mean(SD), n

PresenceAudio 99 (10), 20 96 (14), 10Video 99 (15), 20 110 (8), 10

Social presenceAudio 67 (12), 20 63 (10), 10Video 68 (12), 20 71 (9), 10

Perceivedperformance

Audio 20 (5), 20 21 (5), 10Video 22 (5), 20 18 (5), 10

Table 4. Summary Values of Data Regarding Time toComplete the Task, Number of Words Spoken, and Number ofWords Spoken per Second in Dialogues, Showing Mean Values,Standard Deviations (SD) and Number of Subjects (n)

CVE mean(SD), n

Web mean(SD), n

TimeAudio 608 (241), 8 365 (116), 5Video 978 (549), 8 701 (433), 5

WordsAudio 1084 (452), 8 819 (261), 5Video 1242 (798), 8 1463 (918), 5

Words/sAudio 1.9 (0.7), 8 2.2 (0.2), 5Video 1.3 (0.5), 8 2.0 (0.5), 5

Sallnas 443

ing the number of words spoken per second (power �

0.69). The total number of words used by the partici-pants per second in the different conditions are shownin Table 4. The two-way ANOVA showed that partici-pants used significantly more words per second (F1,22 �

4.96), in the Web conditions than in the CVE condi-tions, when the data from the audio and video condi-tions are combined. The number of words used per sec-ond by participants in their dialogues was not affectedsignificantly (F1,22 � 3.64) whether audio or video wasprovided. The Web and CVE versus the audio andvideo conditions did not interact in any significant way.

6.2.7 Reliability and Construct Validity. Theinternal consistency measure of reliability (Cronbach’salpha) was calculated for the three questionnaires, ondata from the total of 80 subjects who participated inthe first and second experiment presented in this paper.The reliability was good for social presence (alpha �

0.88), presence (alpha � 0.80), and perceived task per-formance (alpha � 0.81).

In order to investigate the construct validity of socialpresence, presence, and perceived performance mea-sured by the questionnaire items used in this study, anexplorative principal component analysis was performed,followed by a factor analysis. In the analysis, data wereused from the total of 80 subjects who participated inthe first and the second experiment presented in thispaper. The 20 items in the presence questionnaire, the13 items in the social-presence questionnaire, plus the 4items in the perceived-performance questionnaire wereanalyzed together. The aim of doing a principal compo-nent analysis was to establish the number of compo-nents that should be used for the factor analysis. It wasdecided that three was a reasonable number of compo-nents to use, based on the scree test (Figure 4).

The first component accounted for 21% of the vari-ance (eigenvalue � 18.92) and the following compo-nents accounted for 11% (eigenvalue � 10.40) and 10%(eigenvalue � 9.11) of the variance, respectively.

A factor analysis (principal axis factoring) of the datawith oblique rotation (promax) was performed. Theresult showed that the first factor was meaningful, as allitems of the social-presence questionnaire loaded high

in the first factor, whereas all other items had low loadingson the first factor. In the second factor, loadings werehigh for the four items in the perceived-performancequestionnaire and for five of the items in the presencequestionnaire, but low for all other presence and social-presence items. This result, that five presence itemsloaded high in the second factor together with the per-ceived-performance items, might be explained by thefact that these presence items are questions about thesubject’s performance in different ways in relation totheir interaction with the interface. In the third factor,all remaining items from the presence questionnaire hadhigh loadings, whereas all items that loaded high in thefirst and second factor loaded lower on Factor 3. Fi-nally, three items from the presence questionnaire didnot load high on any factor. From the factor correlationmatrix, it was found that the correlation was 0.37 be-tween Factor 1 and Factor 3; 0.23 between Factor 2and Factor 3, and 0.13 between Factor 1 and Factor 2.

The scatter plot (Figure 5) of the rotated factor load-ings shows that all social-presence items cluster on thelower right side, and that all four perceived-performance

Figure 4. Scree plot with eigenvalues from the principal component

analysis showing a clear break between the third and fourth

components.


items cluster to the upper left side, while the presenceitems are clustered mainly on the lower left side.

These results suggest that the items in the social-presence questionnaire measure a different constructthan do the presence and the perceived-performanceitems. There is an overlap between the items measuringperceived performance and a small number of presenceitems, but apart from that, presence items measure aseparate construct.

7 Discussion

The results from the first experiment show thatusing text-chat when collaborating in a virtual environ-ment is significantly different in terms of most measuredvariables in this experiment compared to using audio orvideo conference. When communicating in a CVE withtext-chat, people perceived their presence and socialpresence to be significantly lower compared to whenthey interacted in a CVE with audio or video confer-ence. Performance was not perceived to be significantlyworse in the text-chat than in the audio condition, butpeople perceived they performed significantly better inthe video condition than they did in the text-chat con-

dition. Making joint decisions in the CVE text-chatcondition took a significantly longer time, and dialogueswere significantly scarcer in text-chat. These resultsmake up a pattern consistent with Chapanis’s (1975)results that people speak more than they write, and thatpeople perform best in terms of time to complete taskswhen audio or video is provided, compared to whentext is provided. That significantly fewer words per sec-ond were used in text-chat than in both the CVE audioand CVE video conditions suggests that the tempo wasvery slow and indicates indirectly that there were morepauses in the dialogue, something that was also ob-served during the experimental sessions. These resultsshow that text-chat is very different in many respects,but that in some situations (for example, when a sensi-tive topic is discussed in a group) interaction can be per-ceived as more comfortable when text is used (Skinner& Allen, 1983).

The results regarding how audio conferences differfrom video conferences are more complicated to inter-pret. There were no differences regarding perceived per-formance, presence, or social presence between CVEvideo or CVE audio conditions when analyzed usingone-way ANOVA in the first experiment. When theCVE and Web conditions and communication mediawere compared in the two-way ANOVA, however, itwas found that video improved people’s perceived pres-ence significantly. Interestingly, a significant interactioneffect showed that this difference was very strong inWeb environments, but that it was not as important inCVE environments. The social-presence theory (Shortet al., 1976) predicts that access to nonverbal cues im-proves people’s feelings of social presence, but this pre-diction is not supported by the results in this study. Ifmean values of perceived presence and social presenceare studied, the pattern that the effect of communica-tion medium is larger in the Web than it is in the CVEis the same. One explanation for this result might bethat navigating a CVE with an avatar makes nonverbalcues from the video conference less important for thesubject’s feelings of presence in the environment and ofsocial presence with the other person. It could be ar-gued that certain nonverbal cues were actually transmit-ted in the CVE. Examples of such cues are interpersonal

Figure 5. Factor plot for loadings of the 37 variables constituting

the three questionnaires measuring social presence, presence, and

perceived performance.

Sallnas 445

distance, the direction of gaze, the distribution of pointof focus over time, and small movements with the avatarthat possibly could have been interpreted as moods suchas restlessness or calmness.

Results from the experiment showed that significantlyfewer words per second were used in CVE video(mean � 1.3) than in CVE audio (mean � 1.9). Thisfinding shows that the tempo was slower in the CVEvideo condition and might indirectly show that therewere more pauses. This fact suggests that the CVEvideo condition was in fact different from the CVE au-dio condition. The video condition might have been amore relaxed medium that encouraged conversation,whereas the audio condition was the most efficient forthis task. Mean values showed that collaborating inCVE audio was quite fast, about 10 minutes, whereaspeople spent more time in CVE video, about 16 min-utes. Furthermore, people had slightly less extensivedialogues in the CVE audio (1084 words) than in theCVE video condition (1243 words). The reason for thisresult could be that people did not feel as awkwardabout pauses in the CVE video environment. It was ob-served that people used the video channel to check theactions of the other person when there were long pausesin the dialogue and both subjects were, for example,investigating information.

The two-way ANOVA showed that it took signifi-cantly less time to complete the task in the audio than inthe video conditions, but no significant difference wasfound between the CVE and Web conditions. However,there was a significant difference in the number ofwords used per second between the CVE conditionsand the Web conditions. If mean values are compared,many words per second were used in the Web audio(mean � 2.2) and Web video conditions (mean � 2.0)and fewer were used in the CVE audio condition(mean � 1.9), whereas far fewer words per second wereused in the CVE video condition (mean � 1.3). Thissuggests that CVE video might have had unique quali-ties compared to the other conditions. The result thatpeople spoke significantly more words per second in theWeb environments than in the CVE, combined with theresult in the first experiment that people spoke signifi-cantly more words per second in the audio than in the

video condition, suggest that the pace of the conversa-tion was faster in Web than in CVE and was faster inaudio than in video.

The results from this study partly support the conclu-sions made in other studies where it was often foundthat video was not significantly different compared toaudio (Chapanis, 1975; Olson et al., 1995; Anderson etal., 1996; Whitaker & O-Conaill, 1997). The results inthis study, however, showed that the video medium in-deed makes a difference regarding the tempo of theconversation during collaboration, when people have toinvestigate information in order to solve a task. Peoplespoke significantly more words per second in the audiothan in the video condition in the first experiment. Fur-thermore, when collaboration in a CVE was comparedto a Web environment, it was found that people weresignificantly faster in audio than in video. It was alsoshown that video significantly improved perceived pres-ence compared to audio when collaboration took placein a Web environment but not when people collabo-rated in a CVE.

It is likely that mediated human-human interaction inhybrid media environments will be more common inthe future. It will then be important when designingsuch systems to understand how these different mediaaffect social interaction and people’s ability to solvetasks together. We do not know how people will usevirtual spaces in the future, but both the communica-tion medium and where the collaboration takes place (aCVE or a Web environment) have implications for peo-ple’s perception of the interaction and for their commu-nication behavior. Some of these effects interact, whichstresses the importance of carefully selecting what con-stellation to use in integrated systems.

Acknowledgments

Anders Hedman is gratefully acknowledged for his contri-butions to this study. I would also like to thank KerstinSeverinson-Eklundh for her valuable comments and sugges-tions. Finally, Bo Schenkman is gratefully acknowledged forhis comments and suggestions regarding the statistical anal-ysis.


References

Anderson, A., Newlands, A., Mullen, J., Flemming, A. M.,Doherty-Sneddon, G., & Van der Velden, J. (1996). Im-pact of video-mediated communication on simulated serviceencounters. Interacting with Computers, 8(2), 193–206.

Biocca, F., Harms, C., & Burgoon, J. (submitted). Criteriaand scope conditions for a theory and measure of socialpresence. Presence: Teleoperators and Virtual Environments.

Bly, S. A., Harrison, S. R., & Irwin, S. (1993). Media spaces:Bringing people together in a video, audio, and computingenvironment. Communications of the ACM, 36(1), 28–47.

Chapanis, A. (1975). Interactive human communication. Sci-entific American, 232, 36–42.

Chapanis, A., Ochsman, R. B., Parrish, R. N., & Weeks, G. D.(1972). Studies in interactive communication: The effects offour communication modes on the behavior of teams dur-ing a co-operative problem solving. Human Factors, 14,487–509.

Daft, R. L., & Lengel, R. (1986). Organizational informationrequirements, media richness, and structural design. Man-agement Science, 32, 554–571.

Daly-Jones, O., Monk, A., & Watts, L. (1998). Some advan-tages of video conferencing over high-quality audio confer-encing: Fluency and awareness of attentional focus. Interna-tional Journal of Human-Computer Studies, 49, 21–58.

Dourish, P., & Bly, S. (1992). Portholes: Supporting aware-ness in a distributed work group. Proceedings on HumanFactors in Computing Systems, 541–547.

Durlach, N., & Slater, M. (2000). Presence in shared virtualenvironments and virtual togetherness. Presence: Teleopera-tors and Virtual Environments, 9(2), 214–217.

Gutwin, C., & Greenberg, S. (1999). The effects of work-space awareness support on the usability of real-time distrib-uted groupware. ACM Transactions on Computer-HumanInteraction, 6 (3), 243–281.

Heeter, C. (1992). Being there: The subjective experience ofpresence. Presence: Teleoperators and Virtual Environments,1(2), 262–271.

Hendrix, C., & Barfield, W. (1996). The sense of presencewithin auditory virtual environments. Presence: Teleoperatorsand Virtual Environments, 5(3), 290–301.

Katz, R., & Tushman, M. (1978). Communication patterns,project performance, and task characteristics: An empiricalevaluation in an R&D setting. Organizational Behavior andHuman Performance, 23, 139–162.

Kraut, R. E., Fish, R. S., Root, R. W., & Chalfonte, B. L.

(1990). Informal communication in organizations: Form,function, and technology. In S. Oskamp & S. Spacapan(Eds.), Human Reactions to Technology: The Claremont Sym-posium on Applied Social Psychology (pp. 145–199). BeverlyHills, CA: Sage.

Lombard, M., & Ditton, T. (1997). At the heart of it all: Theconcept of presence. Journal of Computer-Mediated Com-munication, 3(2).

Matsuura, N., Fujino, G., Okada, K., & Matsushita, Y.(1993). An approach to encounters and interaction in a vir-tual environment. Proceedings of ACM Computer ScienceConference, Association for Computing Machinery, 298–303.

McLaughlin, M., Sukhatme, G. S., Peng, W., Zhu, W., &Parks, J. (2003). Performance and co-presence in heteroge-neous haptic collaboration. Proceedings of the Eleventh Sym-posium on Haptic Interfaces for Virtual Environment andTeleoperator Systems, IEEE Virtual Reality (VR) 2003.

Mehrabian, A. (1969). Some referents and measures of non-verbal behavior. Behavior Research Methods and Instrumen-tation, 1(6), 205–207.

Ochsman, R. B., & Chapanis, A. (1974). The effects of 10communication modes on the behavior of teams duringco-operative problem-solving. International Journal ofMan-Machine Studies, 6(5), 579–619.

Olson, J. S., Olson, G. M., & Meader, D. K. (1995). Whatmix of video and audio is useful for small groups doing re-mote real-time design work? Proceedings of the ACMCHI’95 Conference on Human Factors in Computing Sys-tems, 362–368.

Rice, R. E. (1993). Media appropriateness: Using social pres-ence theory to compare traditional and new organizationalmedia. Human Communication Research, 19(4), 451– 484.

Short, J., Williams, E., & Christie, B. (1976). The social psy-chology of telecommunications. London: Wiley.

Skinner, H., & Allen, B. (1983). Does the computer make adifference? Computerized versus face-to-face versus self-report assessment of alcohol, drug and tobacco use. Journalof Consulting and Clinical Psychology, 51(2), 267–275.

Slater, M., & Steed, A. (2002). Meeting people virtually: Ex-periments in shared virtual environments. In R. Schroeder(Ed.), The social life of avatars: Presence and interaction inshared virtual environments (pp. 146–171). London:Springer-Verlag.

Slater, M., & Wilbur, S. (1997). A framework for immersivevirtual environments (FIVE): Speculations on the role ofpresence in virtual environments. Presence: Teleoperators andVirtual Environments, 6(6), 603–616.

Sallnas 447

Vaske, J. J., & Grantham, C. E. (1989). Socializing the human-computer environment. Norwood, NJ: Ablex.

Whittaker, S., & O-Conaill, B. (1997). The role of vision inface-to-face and mediated communication. In E. E. Finn,A. J. Sellen, & S. B. Wilbur, (Eds.), Video-Mediated Com-munication (pp. 23–50). Mahwah, NJ: Erlbaum.

Williams, E. (1977). Experimental comparisons of face-to-faceand mediated communication: A review. Psychological Bulle-tin, 84(5), 963–976.

Witmer, B. G., & Singer, M. J. (1998). Measuring presence invirtual environments: A presence questionnaire. Presence:Teleoperators and Virtual Environments, 7(3), 225– 240.

Appendix 1

The questions measuring perceived social presence are shown below.The following pairs of words describe how you could have perceived the virtual communications

environment.Please mark with an X the alternative that corresponds with your impression.I perceived it to be:1. Personal � � � � � � � � Impersonal *2. Cold � � � � � � � � Warm3. Unsocial � � � � � � � � Social4. Pleasant � � � � � � � � Unpleasant *5. Negative � � � � � � � � Positive

The following statements describe how you could have perceived the social interaction between thecollaborators. Please mark with an X the alternative that corresponds with your impression.

6. I felt very much socially present � � � � � � � � I did not feel socially present at all7. Not many pauses at all � � � � � � � � Very many pauses *8. Easy to understand the other person’s intentions � � � � � � � � Very hard tounderstand other persons intentions

*

9. One easily understands the other person’s reactions *I totally agree � � � � � � � � I do not agree at all10. One easily understands how the other person reacts to one’s comments *I totally agree � � � � � � � � I do not agree at all11. The other person perceives how one reacts to her/his comments *I totally agree � � � � � � � � I do not agree at all12. It is easy to express emotions *I totally agree � � � � � � � � I do not agree at all13. It would be easy to know if the other person is angry or not in this environment *I totally agree � � � � � � � � I do not agree at all*should be reverse scored in order to contribute to the overall totals


Appendix 2

The questions measuring perceived task performance are shown below.The following questions consider how you perceived that you could handle the system that

you used in this experiment. Please mark with an X the alternative that corresponds withyour impression.

1. How do you think that you managed to do the task in the system?Very well � � � � � � � � Not well at all *2. How efficiently did you feel that you managed to do the task?Very efficiently � � � � � � � � Not efficiently at all *Was it hard to?3. Show the other person things in the environment?Not at all problematic � � � � � � � � Very problematic *4. Point out directions to the other person in the environment?Not at all problematic � � � � � � � � Very problematic **should be reverse scored in order to contribute to the overall totals

Sallnas 449

effects of communication mode on social presence, virtual presence, and performance in collaborative...

Documents