VACE Multimodal Meeting Corpus

Lei Chen1, R. Travis Rose2, Ying Qiao2, Irene Kimbara3, Fey Parrill3, HaleemaWelji3, Tony Xu Han4, Jilin Tu4, Zhongqiang Huang1, Mary Harper1, FrancisQuek2, Yingen Xiong2, David McNeill3, Ronald Tuttle5, and Thomas Huang4

1 School of Electrical Engineering, Purdue University, West Lafayette IN,2 CHCI, Department of Computer Science, Virginia Tech, Blacksburg, VA

quek@vt.edu3 Department of Psychology, University of Chicago, Chicago, IL

4 Beckman Institute, University of Illinois Urbana Champaign, Urbana, IL5 Air Force Institute of Technology, Dayton, OH

Abstract. In this paper, we report on the infrastructure we have de-veloped to support our research on multimodal cues for understandingmeetings. With our focus on multimodality, we investigate the interactionamong speech, gesture, posture, and gaze in meetings. For this purpose,a high quality multimodal corpus is being produced.

1 Introduction

Meetings are gatherings of humans for the purpose of communication. Suchcommunication may have various purposes: planning, conflict resolution, negoti-ation, collaboration, confrontation, etc. Understanding human multimodal com-municative behavior, and how witting or unwitting visual displays (e.g., shoulderorientations, gesture, head orientation, gaze) relate to spoken content (the wordsspoken, and the prosody of the utterances) and communicative acts is critical tothe analysis of such meetings. These multimodal behaviors may reveal static anddynamic social structuring of the meeting participants, the flow of topics beingdiscussed, the high level discourse units of individual speakers, the control of theflow of the meeting, among other phenomena. The collection of rich synchronizedmultimedia corpora that encompasses multiple calibrated video streams, audiochannels, motion tracking of the participants, and various rich annotations isnecessary to support research into these phenomena that occur at varying levelsof temporal resolution and conceptual abstraction.

From the perspective of the technology, meetings challenge current audioand video processing approaches. For example, there is a higher percentage ofcross-talk among audio channels in a six party meeting than in a two partydialog, and this could reduce the accuracy of current speech recognizers. Ina meeting setting, there may not be the ideal video image size or angle whenattempting to recognize a face. Recorded meetings can push forward multimodalsignal processing technologies.

To enable this research, we are assembling a planning meeting corpus that iscoded at multiple levels. Our research focuses not only on low level multimodalsignal processing, but also on high level meeting event interpretation. In partic-ular, we use low-level multimodal cues to interpret the high-level events relatedto meeting structure. To carry out this work, we require high quality multimodal

data to jointly support multimodal data processing, meeting analysis and cod-ing, as well as automatic event detection algorithms. Our corpus is designed tosupport research both in the understanding of the human communication andthe engineering efforts to meet the challenges of dealing with meeting audioand video content. This collection, which is supported by the ARDA VACE-IIprogram, is called the VACE meeting corpus in this paper.

We describe our efforts in collecting the VACE meeting corpus, the infras-tructure we have constructed to collect the data, and the tools we have developedto facilitate the collection, annotation, and analysis of the data. In particular,Section 2 describes the ongoing multimodal meeting corpus collection, Section3 describes audio and video data processing algorithms needed for corpus pro-duction, and Section 4 briefly summarizes some of the research that this corpusenables.

2 Meeting Data Collection

To investigate meetings, several corpora have already been collected, includingthe ISL audio corpus [1] from Interactive Systems Laboratory (ISL) of CMU, theICSI audio corpus [2], the NIST audio-visual corpus [3], and the MM4 audio-visual corpus [4] from Multimodal Meeting (MM4) project in Europe. Usingthese existing meeting data resources, a large body of research has already beenconducted, including automatic transcription of meetings [5], emotion detection[6], attention state [7], action tracking [8, 4], speaker identification [9], speechsegmentation [10, 11], and disfluency detection [12]. Most of this research hasfocused on low level processing (e.g., voice activity detection, speech recognition)or on elementary events and states. Research on the structure of a meeting or thedynamic interplay among participants in a meeting is only beginning to emerge.McCowan et al. [4] have used low level audio and video information to segmenta meeting into meeting actions using an HMM approach.

Our multimodal meeting data collection effort is depicted schematically inFigure 1. We next discuss three important aspects of this meeting data collectioneffort: (1) meeting room setup, (2) elicitation experiment design, and (3) dataprocessing.

2.1 Multimodal Meeting Room Setup

Under this research effort, Air Force Institute of Technology (AFIT) modified alecture room to collect multimodal, time-synchronized audio, video, and motiondata. In the middle of the room, up to 8 participants can sit around a rect-angular conference table. An overhead rail system permits the data acquisitiontechnician to position the 10 Canon GL2 camcorders in any configuration re-quired to capture all participants by at least two of the 10 camcorders. UsingS-video transfer, 10 Panasonic AG-DV2500 recorders capture video data fromthe camcorders. The rail system also supports the 9 Vicon MCam2 near-IR cam-eras and are driven by the Vicon V8i Data Station. The Vicon system recordstemporal position data. For audio recording, we utilize a setup similar to the

Fig. 1. VACE meeting corpus production

ICSI and NIST meeting rooms. In particular, participants wear CountrymanISOMAX Earset wireless microphones to record their individual sound tracks.Table-mounted wired microphones are used to record the audio of all participants(two to six XLR-3M connector microphones configured for the number of par-ticipants and scenario, including two cardioid Shure MX412 D/C microphonesand several types of low-profile boundary microphones (two hemispherical polarpattern Crown PZM-6D, one omni-directional Audio Technica AT841a, and onefour-channel cardioid Audio Technica AT854R). All audio signals are routed toa Yamaha MG32/14FX mixing console for gain and quality control. A TASCAMMX-2424 records the sound tracks from both the wireless and wired microphones.

There are some significant differences between our video recording setup andthose used by previous efforts. For example, in the NIST and MM4 collections,because stereo camera views are not used to record each participant, only 2D

tracking results can be obtained. For the VACE meeting corpus, each partic-ipant is recorded with a stereo calibrated camera pair. Ten video cameras areplaced facing different participants seated around the table as shown in Figure 1.To obtain the 3D tracking of 6 participants, 12 stereo camera pairs are setupto ensure that each participant is recorded by at least 2 cameras. This is im-portant because we wish to accurately track head, torso and hand positions in3D. We also utilize the Vicon system to obtain more accurate tracking results

to inform subsequent coding efforts, while also providing ground truth for ourvideo-tracking algorithms.

2.2 Elicitation ExperimentThe VACE meeting corpus involves meetings based on wargame scenarios andmilitary exercises. We have selected this domain because the planning activ-ity spans multiple military functional disciplines, the mission objectives are de-fined, the hierarchical relationships are known, and there is an underpinningdoctrine associated with the planning activity. Doctrine-based planning meet-ings draw upon tremendous expertise in scenario construction and implemen-tation. Wargames and military exercises provide real-world scenarios requiringinput from all functionals for plan construction and decision making. This elicitsrich multimodal behavior from participants, while permitting us to produce ahigh confidence coding of the meeting behavior. Examples of scenarios includeplanning a Delta II rocket launch, humanitarian assistance, foreign material ex-ploitation, and scholarship award selection.

2.3 Multimodal Data ProcessingAfter obtaining the audio and video recordings, we must process the data toobtain features to assist researchers with their coding efforts or to the train andevaluate automatic event detection algorithms. The computer vision researcherson our team from University of Illinois and Virginia Tech focus on video-basedtracking, in particular, body torso, head, and hand tracking. The VCM trackingapproach is to obtain 3D positions of the hands, which is important for obtain-ing a wide variety of gesture features, such as gesture hold and velocity. Thechange in position of the head, torso, and hands provide important cues for theanalysis of the meetings. Researchers on our team from Purdue handle the audioprocessing of the meetings. More detail on video and audio processing appear inthe next section.

Our meeting room corpus contains time synchronized audio and video record-ings, features derived by the visual trackers and Vicon tracker, audio featuressuch as pitch tracking and duration of words, and coding markups. Details onthe data types appear in an organized fashion in Table 1.

3 Multimodal Data Processing

3.1 Visual TrackingTorso Tracking Vision-based human body tracking is a very difficult problem.Given that joint-angle is a natural and complete way to describe human bodymotion and human joint-angles are not independent, we have been investigatingan approach to learn these latent constraints and then use them for articulatedbody tracking [13] After learning the constraints as potential functions, beliefpropagation is used to find the MAP of the body configuration on the MarkovRandom Field (MRF) to achieve globally optimal tracking. When tested on theVACE meeting corpus data, we have obtained tracking results that will be usedin future experiments. See Figure 2 for an example of torso tracking.

Table 1. Composition of the VACE meeting corpus

Video

Audio

Vicon

Visual Tracking

Audio Processing

Prosody

Gaze

Gesture

Metadata

MPEG4 Video from 10 cameras

AIFF Audio from all microphones

3D positions of Head, Torso, Shoulders and Hands

Head pose, Torso configuration, Hand positions

Speech segments, Transcripts, Alignments

Pitch, Word & Phone duration, Energy, etc.

Gaze target estimation

Gesture phase/phrase, Semiotic gesture coding, e.g., deictics, iconics

Language metadata, e.g., sentence boundaries, speech repairs, floor control change

Fig. 2. Torso tracking

Head Pose Tracking For the video analysis of human interactions, the headpose of the person being analyzed is very important for determining gaze direc-tion and the person being spoken to. In our meeting scenario, the resolution of aface is usually low. We therefore have developed a hybrid 2D/3D head pose track-ing framework. In this framework, a 2D head position tracking algorithm [14]tracks the head location and determines a coarse head orientation (such as thefrontal view of the face, the side view and the rear view of the head) based onthe appearance at that time. Once the frontal view of a face is detected, a 3Dhead pose tracking algorithm [15] is activated to track the 3D head orientation.For 2D head tracking, we have developed a meanshift tracking algorithm with anonline updating appearance generative mixture model. When doing meanshifttracking, our algorithm updates the appearance histogram online based on somekey features acquired before the tracking, allowing it to be more accurate androbust. The coarse head pose (such as frontal face) can be inferred by simplychecking the generative appearance model parameters. The 3D head pose track-ing algorithm acquires the facial texture from the video based on the 3D facemodel. The appearance likelihood is modelled by an incremental PCA subspace,and the 3D head pose is inferred using an annealed particle filtering technique.An example of a tracking result can be found in Figure 3.

Fig. 3. Head pose tracking

Hand Tracking In order to interpret gestures used by participants, exact 3D

hand positions are obtained using hand tracking algorithms developed by re-

searchers in the Vislab [16, 17]. See [18] for details on the Vector CoherenceMapping (VCM) approach that is being used. The algorithm is currently beingported to the Apple G5 platform with parallel processors in order to address thechallenge of tracking multiple participants in meetings. An example of a trackingresult can be found in Figure 4.

Fig. 4. VCM hand position tracking

3.2 Audio ProcessingA meeting involves multiple time synchronized audio channels, which increasesthe workload for transcribers [19]. Our goal is to produce high quality transcrip-tions that are time aligned as accurately as possible with the audio. Words, andtheir components, need to be synchronized with video features to support codingefforts and to extract prosodic and visual features used by our automatic meet-ing event detection algorithms. To achieve this goal, we need an effective wayto transcribe all audio channels and a highly accurate forced alignment system.As for the transcription convention, we are utilizing the Quick Transcription(QTR) methodology developed by LDC for the 2004 NIST Meeting RecognitionEvaluations to achieve a balanced tradeoff between the accuracy and speed oftranscription. Meeting audio includes multi-channel recordings with substantialcross-talk among the audio channels. These two aspects make the transcriptionprocess more challenging than for monologs and dialogs. We use several tools toimprove the effectiveness and efficiency of audio processing: a tool to pre-segmentaudio channels into transcribable and non-transcribable regions, tools to supportthe transcription of multiple channels, and tools to enable more accurate forcedalignments. These are described in more detail below.

Automatic Pre-Segmentation Since in meetings typically one speaker isspeaking at any given time, the resulting audio files contain significant portionsof audio that do not require transcription. Hence, if each channel of audio isautomatically segmented into transcribable and non-transcribable regions, thetranscribers only need to focus on the smaller pre-identified regions of speech,lowering the cognitive burden significantly compared with handling a large undif-ferentiated stream of audio. We perform audio segmentation based on the close-talking audio recordings using a novel automatic multi-step segmentation [20].The first step involves silence detection utilizing pitch and energy, followed byBIC-based Viterbi segmentation and energy based clustering. Information fromeach channel is employed to provide a rough preliminary segmentation. The sec-ond step makes use of the segment information obtained in the first step to traina Gaussian mixture model for each speech activity category, followed by decod-ing to refine the segmentation. A final post-processing step is applied to removeshort segments and pad silence to speech segments.

Meeting Transcription Tools For meetings, transcribers must utilize manyaudio channels and often jump back and forth among the channels to supporttranscription and coding efforts [19]. There are many transcription and anno-tation tools currently available [21]; however, most were designed for monologsand dialogs. To support multi-channel audio, researchers have either tailoredcurrently available tools for dialogs (e.g., the modification of Transcriber [22] formeeting transcription by ICSI (http://www.icsi.berkeley.edu/Speech/mr/channeltrans.html) or designed new tools specific for the meeting transcrip-tion and annotation process, (e.g., iTranscriber by ICSI).

We have evaluated ICSI’s modified Transcriber and the beta version of ICSI’siTranscriber currently under development. Although these tools can not be usedin their current form for our transcription and annotation process, they highlightimportant needs for transcribing meetings: transcibers need the ability to easilyload the multiple channels of a meeting and efficiently switch back and forthamong channels during transcription. Hence, we have developed two Praat [23]extensions to support multi-channel audio transcription and annotation. We havechosen to use Praat for the following reasons: 1) it is a widely used and supportedspeech analysis and annotation tool available for almost any platform, 2) it iseasy to use, 3) long sound supports the quick loading of multiple audio files, and4) it has a built-in script language for implementation of future extensions. Wehave added two menu options to the Praat interface: the first supports batchloading of all the audio files associated with a meeting, and the second enablestranscribers to switch easily among audio files based on the transcription tiers.

Improved Forced Alignment Forced alignment is used to obtain the start-ing and ending time of the words and phones in the audio. Since such timinginformation is widely used for multimodal fusion, we have investigated factorsfor achieving accurate alignments. In the first step, the forced alignment sys-

tem converts the words in the transcript to a phoneme sequence according tothe pronunciations in the dictionary. For out of vocabulary (OOV) words, thetypically used method is to use a special token, such as UNK, to replace theOOV words. However, this approach can significantly degrade forced alignmentaccuracy. Hence, we have created a script to identify all of the OOV words in thefinished transcription and have designed a Praat script to prompt the transcriberto provide an exact pronunciation for each OOV word given its audio and then tosubsequently update the dictionary with that word and pronunciation. Althoughthis approach is more time consuming, it provides a more accurate alignment re-sult important to our corpus production task. Based on a systematic study [24],we have also found more accurate forced alignments are obtained by having tran-scribers directly transcribe pre-identified segments of speech (supported by thepresegmentation tool described previously) and by using a sufficiently trained,genre matched speech recognizers to produce the alignments. For meeting roomalignments, we have been using ISIP’s ASR system with a triphone acousticmodel trained from more than 60 hours long spontaneous speech data [25] toforce align transcriptions provided for segments of speech identified in the pre-segmentation step. Alignments produced given this setup requires little handfixing. We are currently evaluating SONIC [26] for future use in our system.

4 Meeting Interaction Analysis Research

The VACE meeting corpus enables the analysis of meeting interactions at anumber of different levels. Using the visualization and annotation tool Macvis-

sta, developed by researchers at Virginia Tech, the features extracted from therecorded video and audio can be displayed to support psycholinguistic codingefforts of researchers on our team at University of Chicago. Some annotationsof interest to our team include: F-formation [27], dominant speaker, structuralevents (sentence boundary, interruption point), and floor control challenges andchange. Given the data and annotations in this corpus, we are carrying out mea-surement studies to investigate how visual and verbal cues combine to predictevents such as sentence or topic boundaries, interruption points in a speech re-pair, or floor control changes. With the rich set of features and annotations, weare also developing data-driven models for meeting room event detection alongthe lines of our research on multimodal models for detecting sentence boundaries[28].

Visualization of visual and verbal activities is an important first step fordeveloping a better understanding of how these modalities interact in humancommunication. The ability to add annotations of important verbal and visualevents further enriches this data. For example, annotation of gaze and gestureactivities is important for developing a better understanding of those activitiesin floor control. Hence, the availability of a high quality, flexible multimodalvisualization/annotation tool is quite important. To give a complete display ofa meeting, we need to display any of the multimodal signals and annotationsof all participants. The Vissta tool developed for multimodal dialog data has

been recently ported to the Mac OS X while being adapted for meeting roomdata [29]. Currently the tool supports the display of multiple angle view videos,as shown in Figure 5. This tool can display transcriptions and visual features,together with the spoken transcripts and a wide variety annotations. It has beenwidely used by our team and is continually being refined based on feedback.

Fig. 5. A snapshot of the MacVissta multimodal analysis tool with multiple videosshown on the left, gaze annotations and speech mark-ups shown on the right

Using MacVissta, researchers at the University of Chicago are currently fo-cusing on annotating gesture and gaze patterns in meetings. Gesture onset andoffset are coded, as well as the semiotic properties of the gesture as a whole,in relation to the accompanying speech. Because gesture is believed to be asrelevant to a person’s communicative behavior as speech, by coding gesture, weare attempting to capture this behavior in its totality. In addition, gaze is codedfor each speaker in terms of the object of that gaze (who or what gaze is di-rected at) for each moment. Instances of shared gaze (or ”F-formations”) arethen extractable from the transcript, which can inform a turn-taking analysis.More fine-grained analyses include the coding of mimicry (in both gesture andspeech), and the tracking of lexical co-reference and discourse cohesion, whichpermits us to capture moments where speakers are negotiating how to refer to

an object or event. These moments appear to be correlated with shared gaze.To date, we have finished annotation of one complete meeting, AFIT Jan 07,involving five participants lasting forty minutes. Video and annotations can beviewed in the MacVissta tool after download from the VACE site at VirginiaTech.

5 Conclusions

In this paper, we have reported on the infrastructure we have developed tosupport our research on multimodal cues for understanding meetings. With ourfocus on multimodality, we investigate the interaction among speech, gesture,posture, and gaze in meetings. For this purpose, a high quality multimodal corpusis being produced. Each participant is recorded with a pair of stereo calibratedcamera pairs so that 3D body tracking can be done. Also an advanced motiontracking system is utilized to provide ground truth. From recorded audio andvideo, research on audio processing and video tracking focus on improving qualityof low features that support higher level annotation and modeling efforts.

6 Acknowledgments

We thank all team members for efforts to produce the VACE meeting corpus:Bing Fang, and Dulan Wathugala from Virginia Tech, Dr. Sue Duncan, MattHeinrich, Whitney Goodrich, and Alexia Galati from University of Chicago,Dr. David Bunker, Jim Walker, Kevin Pope, Jeff Sitler from AFIT. This re-search has been supported by the Advanced Research and Development ActivityARDA VACEII grant 665661: From Video to Information: Cross-Model Analysis

of Planning Meetings. Part of this work was carried out while the tenth authorwas on leave at NSF. Any opinions, findings, and conclusions expressed in thispaper are those of the authors and do not necessarily reflect the views of NSFor ARDA.

References

[1] Burger, S., MacLaren, V., Yu, H.: The ISL meeting corpus: The impact of meetingtype on speech type. In: Proc. of Int. Conf. on Spoken Language Processing(ICSLP). (2002)

[2] Morgan, N., et al.: Meetings about meetings: Research at ICSI on speech inmultiparty conversations. In: Proc. of ICASSP. Volume 4., Hong Kong, HongKong (2003) 740–743

[3] Garofolo, J., Laprum, C., Michel, M., Stanford, V., Tabassi, E.: The NIST MeetingRoom Pilot Corpus. In: Proc. of Language Resource and Evaluation Conference.(2004)

[4] McCowan, I., Gatica-Perez, D., Bengio, S., Lathoud, G., Barnard, M., Zhang, D.:Automatic analysis of multimodal group actions in meetings. IEEE Trans. onPattern Analysis and Machine Intelligence 27 (2005) 305–317

[5] Schultz, T., Waibel, A., et al.: The ISL meeting room system. In: Proceedings ofthe Workshop on Hands-Free Speech Communication, Kyoto Japan (2001)

[6] Polzin, T.S., Waibel, A.: Detecting emotions in speech. In: Proc.of the CMC.(1998)

[7] Stiefelhagen, R.: Tracking focus of attention in meetings. In: Proc. of Int. Conf.on Multimodal Interface (ICMI), Pittsburg, PA (2002)

[8] Alfred, D., Renals, S.: Dynamic bayesian networks for meeting structuring. In:Proc. of ICASSP. Volume 5., Montreal, Que, Canada (2004) 629–632

[9] Gatica-Perez, D., Lathoud, G., McCowan, I., Odobez, J., Moore, D.: Audio-visualspeaker tracking with importance particle filters. In: Proc. of Int. Conf. on ImageProcessing (ICIP). Volume 3., Barcelona, Spain (2003) 25–28

[10] Renals, S., Ellis, D.: Audio information access from meeting rooms. In: Proc. ofICASSP. Volume 4., Hong Kong, Hong Kong (2003) 744–747

[11] Ajmera, J., Lathoud, G., McCowan, I.: Clustering and segmenting speakers andtheir locations in meetings. In: Proc. of ICASSP. Volume 1., Montreal, Que,Canada (2004) 605–608

[12] Moore, D., McCowan, I.: Microphone array speech recognition: Experiments onoverlapping speech in meetings. In: Proc. of ICASSP. Volume 5., Hong Kong,Hong Kong (2003) 497–500

[13] Han, T.X., Huang, T.S.: Articulated body tracking using dynamic belief propa-gation. In: Proc. IEEE International Workshop on Human-Computer Interaction.(2005)

[14] Tu, J., Huang, T.S.: Online updating appearance generative mixture model formeanshift tracking. In: Proc. of Int. Conf. on Computer Vision (ICCV). (2005)

[15] Tu, J., Tao, H., Forsyth, D., Huang, T.S.: Accurate head pose tracking in lowresolution video. In: Proc. of Int. Conf. on Computer Vision (ICCV). (2005)

[16] Quek, F., Bryll, R., Ma, X.F.: A parallel algorighm for dynamic gesture tracking.In: ICCV Workshop on RATFG-RTS, Gorfu,Greece (1999)

[17] Bryll, R.: A Robust Agent-Based Gesture Tracking System. PhD thesis, WrightState University (2004)

[18] Quek, F., Bryll, R., Qiao, Y., Rose, T.: Vector coherence mapping: Motion fieldextraction by exploiting multiple coherences. CVIU special issue on Spatial Co-herence in Visual Motion Analysis (submitted) (2005)

[19] Strassel, S., Glenn, M.: Shared linguistic resources for human language technologyin the meeting domain. In: Proceedings of ICASSP 2004 Meeting Workshop.(2004)

[20] Huang, Z., Harper, M.: Speech and non-speech detection in meeting audio fortranscription. In: MLMI-05 NIST RT-05S Workshop. (2005)

[21] Bird, S., Liberman, M.: (Linguistic Annotation: Survey by LDChttp://www.ldc.upenn.edu/annotation/)

[22] Barras, C., Geoffrois, D., Wu, Z., Liberman, W.: Transcriber : Development anduse of a tool for assisting speech corpora production. Speech Communication(2001)

[23] Boersma, P., Weeninck, D.: Praat, a system for doing phonetics by computer.Technical Report 132, University of Amsterdam, Inst. of Phonetic Sc. (1996)

[24] Chen, L., Liu, Y., Harper, M., Maia, E., McRoy, S.: Evaluating factors impactingthe accuracy of forced alignments in a multimodal corpus. In: Proc. of LanguageResource and Evaluation Conference, Lisbon, Portugal (2004)

[25] Sundaram, R., Ganapathiraju, A., Hamaker, J., Picone, J.: ISIP 2000 conversa-tional speech evaluation system. In: Speech Transcription Workshop 2001, CollegePark, Maryland (2000)

[26] Pellom, B.: SONIC: The University of Colorado continuous speech recognizer.Technical Report TR-CSLR-2001-01, University of Colorado (2001)

[27] Quek, F., McNeill, D., Rose, T., Shi, Y.: A coding tool for multimodal analysisof meeting video. In: NIST Meeting Room Workshop. (2003)

[28] Chen, L., Liu, Y., Harper, M., Shriberg, E.: Multimodal model integration forsentence unit detection. In: Proc. of Int. Conf. on Multimodal Interface (ICMI),University Park, PA (2004)

[29] Rose, T., Quek, F., Shi, Y.: Macvissta: A system for multimodal analysis. In:Proc. of Int. Conf. on Multimodal Interface (ICMI). (2004)