audiovisual phenomenal causality - biomotionlab

Perception amp Psychophysics2003 65 (5) 789-800

Imagine an object moving horizontally toward a sec-ond object and then stopping abruptly right in front of itAt this moment the target starts moving while the firstobject remains motionless When observing such a se-quence on a computer screen we tend to say that the launch-ing of the second object was caused by the first one Thiseffect is called phenomenal causality (Michotte 19461963) or perceived causality (Piaget 1967) and has oftenbeen replicated as has been reviewed by Scholl andTremoulet (2000) Michotte stressed the systematic rela-tionship between causality judgments and the spatialtemporal parameters of the displays The main variablecontributing to causality judgments in many fields is thetemporal contiguity between the observed events (Oester-meier amp Hesse 2000 White 1995) Temporal contiguitydoes not necessarily mean exact co-occurrence of thecause and its effect For instance Michotte observed ldquobestrdquocausality impressions in launching displays with delaysof about 30ndash40 msec between the end of the first motionand the start of the target motion He also claimed thatldquodirect launching is only reported constantly for intervalsup to 50 milliseconds though it may still be reported oc-casionally up to 100 millisecondsrdquo (Michotte amp Thinegraves19631991 p 70) Several experiments have shown thatthe delay may rise up to 200 msec and still produce reliableimpressions of causality Gruber Fink and Damm (1957)reported a ldquomedian temporal threshold of causalityrdquo of

95ndash98 msec in their pretests and of 150ndash200 msec in aseries of delay trials (0ndash250 msec) Scholl and Nakayama(2000) reported that the ldquocausal capture effectrdquomdashthat isfacilitating the causal interpretation of an ambiguousstreaming display by means of an additional launching dis-playmdashis confined to a 0- to 100-msec overlap betweenthe two displays Larger delays result in significant de-creases of judged causality

In most experiments in this area unimodal visual dis-plays have been used To our knowledge no publication re-porting the use of unimodal auditory displays has appearedin print until now but Michotte (19461963 pp 235ndash241)mentioned three audiovisual experimentsmdashthat is ex-periments in which sounds were linked to visual motionsIn all three experiments (Experiments 80ndash82) the soundwas generated by an electrically controlled hammer en-closed in a wooden box and occurred 20 msec after oneof the following visual events For instance in Experi-ment 80 a circle appeared disappeared changed itsbrightness moved and then stopped abruptly or movedand passed over another object which was stationary ToMichottersquos great surprise ldquoit was only in exceptionalcases that observers used expressions implying that oneevent had an influence on the otherrdquo (p 236) A few ofthe reported comments of the subjects point to the prob-ability that some subjects judged the acoustic event tocause the visual eventsmdasha causal relation that Michotteseems to have ldquocounted as negativerdquo (p 237) Experi-ment 81 used 6 subjects with the classic launching dis-play containing two circles and the sound occurring atthe time of the contact between the circles In some ofthe trials rebounding of the first disk took place ldquoThe

789 Copyright 2003 Psychonomic Society Inc

Correspondence concerning this article should be addressed toR Guski Department of Psychology Ruhr-Universitaumlt Bochum D-44780Bochum Germany (e-mail rainerguskiruhr-uni-bochumde)

Audiovisual phenomenal causality

RAINER GUSKI and NIKOLAUS F TROJERuhr-Universitaumlt Bochum Bochum Germany

We report three experiments in which visual or audiovisual displays depicted a surface (target) setinto motion shortly after one or more events occurred A visual motion was used as an initial event fol-lowed directly either by the target motion or by one of three marker events a collision sound a blinkof the target stimulus or the blink together with the sound The delay between the initial event and theonset of the target motion was varied systematically The subjects had to rate the degree of perceivedcausality between these events The results of the first experiment showed a systematic decline ofcausality judgments with an increasing time delay Causality judgments increased when additional au-ditory or visual information marked the onset of the target motion Visual blinks of the target and audi-tory clacks produced similar causality judgments The second experiment tested several models ofaudiovisual causal processing by varying the position of the sound within the visual delay period Nosystematic effect of the sound position occurred The third experiment showed a subjective shorten-ing of delays filled by a clack sound as compared with unfilled delays However this shortening can-not fully explain the increased tolerance for delays containing the clack sound Taken together the re-sults are consistent with the interpretation that the main source of the causality judgments in ourexperiments is the impression of a plausible unitary event and that perfect synchrony is not necessaryin this case

790 GUSKI AND TROJE

results showed considerable variation In some cases theevents seemed independent while in others the impactseemed to lsquoproducersquo the noise For present purposeshowever the most important point is that for more thanone observer the lsquoproductionrsquo of the noise was much moreclearly marked when rebounding took placerdquo (p 237f)Experiment 82 was identical to Experiment 81 exceptthat the second disk moved at a considerably slowerspeed after the collision ldquoTwenty-one people tookpart in this experiment twelve of them or 57 statedwhen we asked them specifically that the impact pro-duced the noiserdquo (p 238) Michotte seemed to be trou-bled by the fact that (1) virtually no subject spontaneouslymentioned the sound and (2) the sound was not synchro-nous with the contact between the experimental objectsFrom both aspects he concluded that the ldquoimpressionwas not really a causal onerdquo (p 240) and he stopped ex-perimenting on audiovisual displays mainly because oftechnical problems

In the light of more recent data Michotte (19461963)probably drew a false conclusion because audiovisualevents are judged to be most synchronous when the soundoccurs 50 msec after the visual event connected with thesound (Kohlrausch amp van de Par 2000 Rudloff 1997)In addition he seemed to believe that the audiovisualsystem is less tolerant with small delays between the endof the initial movement and the start of the auditoryevent than the visual system is with small delays betweenthe stop of the initial movement and the start of the ef-fect movement This is strange because it is generallyknown that sound waves travel at rather low velocitiesand that echoes are considerably behind their cause (eghand claps or shouts)

There are some historic papers that have been con-cerned with auditory influences on motion perceptionWe will mention them here briefly because their line ofreasoning is different from that of Michotte (19461963)For instance Zietz and Werner (1928) used pairs of draw-ings (eg a drawing of a bent arrow and a drawing of asmall circle) and presented them repeatedly one after theother with a delay of several hundred millisecondsWithout sound the majority of subjects seemed to per-ceive the two drawings in isolation but when the pre-sentation of each drawing was accompanied by a colli-sion sound the subjects tended to see the visual objectsin motionmdashjumping expanding or rotating in spaceMaass (1938) extended the experimental approach ofZietz and Werner to a visual display consisting of 12 tinyelectric bulbs which could be programmed in order torepresent rapidly changing geometric forms and threeelectric hammers which also could be programmed andproduced 64 different rhythms Without sound the opticdisplays produced a huge variety of movement descrip-tions and the author did not see a specific relation be-tween the visual stimuli and the movement descriptionsof the subjects When the optic displays were connectedwith sounds the movement descriptions the subjects

produced were less variable and the light points were ex-perienced as belonging more closely together

There are some recent papers that did not describeaudiovisual causality experiments but were essentiallyrelated to them

1 Sekuler Sekuler and Lau (1997) reported that anambiguous visual display is perceived as a collision if acollision sound is added When two surfaces move to-ward each other in a straight line coincide and thenmove apart this display is consistent with two differentinterpretations After coincidence either the two objectscould have continued to move in their original directionsor they could have collided and then bounced reversingdirections Adding (acoustic) clicks at the time of the vi-sual coincidence increased the perception of collisionsignificantly Watanabe (2000) reported that the bounce-inducing effect of 3-msec low-intensity clicks workedfrom 250 msec before to 150 msec after the visual coin-cidence and within a separation of up to 30ordm of visualangle between the sound and the visual display Watanabeand Shimojo (2001b) found that the bounce-inducing ef-fect of very brief clicks was attenuated when other iden-tical sounds (auditory flankers) were presented beforeand after the simultaneous sound The attenuation oc-curred only when the simultaneous sound and auditoryflankers had similar acoustic characteristics and the simul-taneous sound was not salient The authors conclude that(1) the bounce-inducing effect of the sound is a genuineaudiovisual effect and (2) there exists a saliency-assigningprocess that is context sensitive and can be utilized bythe visual system for resolving ambiguity In a second se-ries of experiments on the streamingbouncing displayusing a sound a flash or a pause after the coincidence ofthe two surfaces Watanabe and Shimojo (2001a) foundthat each of the additional events increased the bouncingjudgment but that ldquothe percentage of the bouncing per-cept did not reach its maximal level immediately afterthe coincidence but increased as a function of postcoin-cidence trajectory duration up to 150ndash200 msecrdquo (p 16)The authors hypothesized that the perceptual systemsneed a certain amount of time to integrate the (more orless conflicting) information into a unitary concept

2 Experiments on the perception of audiovisual syn-chrony (Dixon amp Spitz 1980 Kohlrausch amp van de Par2000 Lewkowicz 1996 Rudloff 1997 van de Par ampKohlrausch 2000) have shown that a sound may bejudged to be synchronous with a visual event even whenthe sound occurs after a short delay In contrast a pre-ceding sound is less likely to be judged as synchronousThe tolerance for synchronism depends on the content ofthe audiovisual events (eg speech or collisions betweenobjects) and lies in the area between ca 2130 msec(sound preceding) and +250 msec (sound following)Optimal synchronism between sound and image is per-ceived if the sound follows 80ndash100 msec after the visualevent (cf Rudloff 1997 van de Par amp Kohlrausch2000)

AUDIOVISUAL CAUSALITY 791

3 Lewald Ehrenstein and Guski (2001) varied thedegree of synchronism between moving visual and sta-tionary auditory pulse stimuli in an auditory localizationtask They reported that the auditory localization ofpulse stimuli with a 2-Hz repetition rate is most stronglyinfluenced by a visual stimulus that is 650 msec syn-chronous with an auditory stimulus

Taken together the results that have been reportedshow that auditory and visual events seem to be linked ifthey occur within a time frame of less than 200 msecWhite (1995 1999) discussed a similar time frame underthe heading of iconic processingmdashthat is he proposedthat most visual information has been lost after 250ndash300 msec and that iconic processing operates ldquoon mate-rial integrated over a span of not more than 250 msecrdquo(White 1995 p 40) Following this line of reasoning vi-sual events will not be causally linked together whenthey are more than 250 msec apart Crowder (1993) alsoreported 250 msec for echoic processingmdashwithout dis-cussing phenomenal causality When audiovisual eventsare considered it is an open question whether the causal-ity time span is (1) shorter than the shorter of both theechoic and the iconic memory stores (2) the shorter ofthe two or (3) longer than the longer of the two The em-pirical evidence presented so far suggests that visual andauditory information interact at least during the timeframe of about 100ndash200 msec in a way that would be ad-vantageous for audiovisual phenomenal causality and itseems to be even more favorable when the visual eventprecedes the auditory event

Several questions have been open until now especiallywith respect to the role of auditory information in visualevents associated with phenomenal causality For in-stance we do not know whether sudden auditory eventstimed in close contiguity with the ldquocollisionrdquo of the twovisual objects will enhance the causal impression or notFurthermore if they enhance the causal impression willthey simply increase the degree of judged causality orwill they expand the tolerated delay between the two vi-sual motions If auditory events enhance the impressionof causality in displays containing a delay between thetwo motions what is the optimal point in time for doingso Would this enhancement be due to the distraction ofattention from the visual display or to some other mech-anism Earlier experiments have shown that delays filledwith events often are judged to be shorter than unfilleddelays of the same duration (for a review see Ihle ampWilsoncroft 1983) This may also happen with delays inlaunching displays

The experiments to be described here relate mainly tothe question of whether the phenomenal causality of vi-sual launching displays can be altered by means of addi-tional impulsive sounds or additional visual impulsiveevents The expected increase in judged causality couldbe due simply to the temporal contiguity of visual andauditory events (a temporal cross-modal informationlink) Alternatively it could be due to associations withreal-life collisions because real collisions between two

objects usually cause impulsive sounds (a ldquorealityrdquo link)A third hypothesis relates to the assumption of unity inthe sense proposed by Welch and Warren (1980) Thediscrepancy of cross-modal information may be partiallyignored when structural factors (eg amount of stimu-lus discrepancy) and cognitive factors (eg instructions)support the impression of a common cause of the cross-modal information There is a fourth hypothesis The en-hancement of causality impressions is due neither tocross-modal nor to ldquorealisticrdquo information but is causedsimply by an increase of the total available informationabout sudden changes in a stream of events Thereforethe effect of a sudden color changemdashwhich is not a typ-ical effect of real collisionsmdashinstead of the sound wastested in Experiment 1

In Experiment 1 we also varied the time delay be-tween the end of the first movement and the start of thesecond one (delay) In Experiment 2 we varied the timedelay between the end of the first visual movement andthe start of the auditory event In Experiment 3 we useda two-alternative forced choice (2AFC) time comparisontask and tested the hypothesis that delays filled by aclack sound are subjectively shorter as compared withunfilled delays (See Figure 1 for a schematic display)

Causality judgments were recorded by means of con-tinuous probability ratingsmdashthat is subjects rated theprobability that the target motion was caused by a pre-ceding perceptual event We did not try to distinguish be-tween judged and perceived causality as proposed bySchlottmann and Shanks (1992) because we were con-vinced that the second and each of the following reactionsof subjects in a repeated measures design would not re-flect immediate impressions any more Rather they mustreflect judgmental processesmdashfor example compar-isons between experimental stimuli (cf Parducci 1974Sarris 2000) and in our case additional comparisonsbetween experimental stimuli and memorized collisionevents

Generally we expected causality judgments of visualcollision events to increase if synchronous collision soundsor visual events were present This increase could be either

Figure 1 Schematic display of the motions in the animations ofthe launching type (prestudy and Experiments 1ndash3) The back-ground color was green the left disk was yellow (prestudy) orblue (Experiments 1ndash3) and the right disk was red Arrows thevertical bar and numerals were not displayed in the original

792 GUSKI AND TROJE

additive or multiplicative In the first case the increasein judged causality would be constant and therefore in-dependent of other experimental variables In the secondcase the increase induced by the additional marker eventwould depend on the judged causality without themarker event If judged causality was already high (egbecause the delay was short) adding a marker eventwould have less effect than it would in a case in whichthe initial causality judgment was low We further ex-pected that causality judgments would be greater whenthe collision sound occurred in the middle of the visualdelay as compared with the margins of the delay andthat the increase in causality judgments with additionalsound would be due partly to the subjective shortening oftime delays filled with sound as compared with unfilleddelays

PrestudyIn a classroom experiment a series of animations was

presented by means of an LCD projector most of themdepicting the classic Michotte (19461963) launchingdisplay (Several other displays were used in the samestudy but they will not be reported here) The two disksmoved with a constant speed of 16 cmsec both beforeand after the ldquocollisionrdquo The delay between the stop ofthe first movement and the start of the target movementvaried systematically between 0 and 400 msec in stepsof 80 msec A small gap separated the disks even at theclosest position (It had been shown earlier that a smallgap does not decrease causality judgments see eg Mi-chotte 19461963 p 99 Yela 1952) Half of the trialscontained collision sounds 10-msec clack sounds in the

middle of the visual delay recorded while a wood blockwas hit with a drumstick (peak frequency = 880 Hz) andemitted by means of active loudspeakers in front of theprojection screen

A total number of 30 subjects (students) were naivewith respect to the experimental questions They wereshown several examples of the movies and received sep-arate answer sheets for each video clip each containingthe same question ldquoHow probable is it that the move-ment of the red object (disk or ball) is caused by a per-ceivable event immediately beforerdquo This questionleaves the kind of causal event (eg sound or motion)open to the subjects The numeric answer scale com-prised 11 steps from 0 to 10 and both of the extremeswere marked not at all (0) and very probable (10)

Results As was expected the median scores of causal-ity judgments decreased with an increasing time delaybetween the two movements in both conditions (Figure 2)The median scores in the sound condition (aggregatedover all delays) were significantly above those withoutsound (Wilcoxon Z = 4192 p lt 001) The medians at alldelays above 0 msec in the sound condition lay withinthe 95 confidence interval of the no-sound conditionif the sound effect was subtracted Considering half of thefull range of the response scale (0 10 = 5) to be thecritical value for causality judgments we computed thetolerated visual delay as the point at which a quadraticregression of the medians intersected with the criticalcausality value As a result 180 msec were tolerated inthe condition without sound whereas 340 msec couldpass in the condition with sound without a decreasebelow the critical value

Figure 2 Median causality judgments with Michotte launching displays(prestudy) The term visual delay denotes the interval between the stop of thefirst disk and the start of the target motion The solid lines show the quadraticregressions of the medians


EXPERIMENT 1

Parts of Experiment 1 served as a reliability test of theprestudy with a refined technique In addition we askedwhether the observed increase of causality judgments inthe sound conditions of the prestudy was due to a spe-cific acoustic event that might easily be attributed to acollision or whether it was due more generally to an un-specific sudden event Therefore we used a blink (ie ashort color change of the target from red to white andback to red) in one of the experimental conditions

There were three reasons for using blinks as an alter-native to clacks First it was nonacoustic informationwith a similar impulse character like clacks and secondit would probably be less related to preexperimentalknowledge of ldquorealrdquo collisions than clack sounds wouldbe Whereas clack sounds especially hitting wood mighteasily be associated with collisions of billiard balls asudden flash of one of the balls would not be expectedThird Watanabe and Shimojo (1998 Experiment 1)used blinks in the middle of the streamingbouncingbistable display (similar to that of Sekuler et al 1997)and found enhanced bouncing impressions as comparedwith control situations (no blinks) and nonsynchronousblinks In contrast Schlottmann and Shanks (1992 p 321)reported that ldquosubjectsrsquo ratings of the degree of per-ceived causality were not affected by the color changerdquo

MethodSubjects Ten undergraduate students of psychology (6 women

and 4 men mean age = 228 years) served as subjects for part of theircourse credit All were naive as to the purposes and hypotheses mo-tivating the study They reported normal or corrected-to-normal vi-sion and hearing

Stimuli The classic Michotte (19461963) launching displaywas used (as in the prestudy) The visual stimuli were 26 mm in di-ameter (189ordm of visual angle at viewing distance) their velocitywas 180 mmsec (132 degsec both before and after the ldquocolli-sionrdquo) and the gap size was 25 mm (0176ordm) The visual stimuli(blue and red disks) were displayed on a 21-in Silicon Graphicsscreen The delay between the stop of the first diskrsquos movement andthe start of the second diskrsquos movement was varied between 0 and400 msec in nine steps Right in the middle of this delay an addi-tional marker event could occur the same 10-msec clack sound asin the prestudy (sound condition) a blink event during which the(red) target disk changed its color to white for 50 msec before re-turning to the red color (blink condition) or a combined sound plusblink event with both the visual and the acoustic markers occurringat the same time (sound + blink condition) The acoustic back-ground level in the laboratory was 45 dB(A) The auditory stimuliwere presented monophonically via headphones (Sennheiser HD450 II) Their peak level was 75 dB(A) measured by means of anartificial ear (Bruumlel amp Kjaer 4152 position ldquofastrdquo) and the mea-suring amplifier (Bruumlel amp Kjaer 2609)

Procedure and Design In contrast to the prestudy Experiment 1was run with individual subjects using a Silicon Graphics work-station The stimuli were computed and displayed in real time witha precision of about 1 msec The distance from the subjectsrsquo eyes tothe screen was 76 cm The head was supported by a chinrest Theinstructions were the same as those in the prestudy but a 9-point re-sponse scale (1 to 9) was used and the subjects reported the causal-ity ratings verbally In four experimental conditions the kind of ad-

ditional events was varied In the control condition no additionalevent was used In the sound condition a 10-msec clack sound waspresented In the blink condition the target disk changed its colorfor 50 msec In the sound + blink condition the clack sound waspresented together with the blink of the target disk The additionalsound blink occurred exactly in the middle of the delay betweenthe stop of the first diskrsquos motion and the beginning of the targetdiskrsquos motion The delay varied in nine steps of 50 msec between 0and 400 msec

At the beginning of the experiment the subjects were presentedwith 12 demonstration trials 3 from each condition (containingminimum maximum and intermediate delays respectively) Afterthe verbal instructions the order of the trials was randomized indi-vidually for each subject Each condition and each delay was mea-sured three times resulting in a total of 108 trials (4 conditions 39 delays 3 3 repetitions) The repetitions were evenly distributedwithin the four quarters of the total individual session At the endof the session each subject answered several questions abouthisher experiences during the experiment The questions we con-sider relevant to be discussed here were (1) whether the subjects hadthe impression that the disks were ldquorealrdquo objects and (2) whether theythought that the motions of the two disks belonged to the same event

ResultsAs in the prestudy the median causality ratings de-

clined with an increasing delay between the motions ofthe disks (Figure 3) and the lowest causality ratings wereobtained in the control conditionmdashthat is without anyadditional (sound or blink) information Because ofmany skewed frequency distributions of the causalityjudgments we decided to perform nonparametric statis-tical analyses The Friedman test with causality ratingsaggregated for all the delays tested showed a significanteffect of the experimental conditions (c 2 = 234 N = 10df = 3 p lt 001) with that of the sound + blink conditionbeing highest and that of the control condition lowestSeparate post hoc analyses showed the sound and the blinkconditions not to be significantly different from eachother with respect to causality ratings (Wilcoxon Z = 065p = 5) whereas their respective differences from thecontrol condition were significant (Wilcoxon Z of about275 p lt 01) The differences between aggregated causal-ity ratings in the sound + blink condition and the respec-tive sound and blink conditions were statistically signif-icant (Wilcoxon Z for sound vs sound + blink = 267p lt 01 Z for blink vs sound + blink = 199 p lt 05)

The regression lines of the median causality ratingsover the 9 delays showed significant linear trends mainly(r 2 = 95ndash97) except for the control condition whichshowed a marked quadratic trend (r 2 = 99) Unlike theprestudy all the noncontrol conditions showed some-what shallower slopesmdashthat is the majority of the sub-jects kept their causality judgments above the midpointof the scale (1 9 = 5) for all conditions with additionalevents up to delays of 280 msec In the sound + blinkcondition they even tolerated 390 msec The general ef-fect was independent of the particular choice of the cri-terion A more strict criterion for causality would useonly median ratings exceeding two thirds of the responsescale (ie 7) With this criterion our subjects did not tol-

794 GUSKI AND TROJE

erate a 50-msec delay in the control condition but theytolerated up to a 100-msec delay in the soundblink con-ditions and 150 msec in the sound + blink condition

We also tested for learning effects during this experi-ment For analytic purposes we divided the whole ses-sion into four quarters and compared the median causal-ity ratings between them It turned out that neither theoverall test nor specific tests between each of the possi-ble pairs produced any significant result (Wilcoxon Z be-tween 01 and 077 p between 9 and 4 respectively) Inaddition we hypothesized that causality judgments wouldbe greater for trials that occurred after control conditiontrials as compared with trials occurring before We testedthis by using pairs of causality judgments from noncontrol-type trials before control-type trials against judgmentsfrom the same type of trials after control-type trialsContrary to the hypothesis the median causality judg-ments seemed to be somewhat lower after control-type tri-als than before but statistical tests did not show conclu-sive results (Friedman c2 = 36 p = 06 Wilcoxon Z =22 p lt 05)

The postexperimental interview produced two usefulresults Eight of the 10 subjects said in response toQuestion 1 (real objects or not) that they perceived ldquo(bil-liard) balls coins or bowlsrdquo Only 1 person spontaneouslymentioned the blink She was reminded of an explosionAnswering Question 2 (inquiring whether one or twoevents were perceived) 7 of the subjects explained thatthey sometimes perceived two events Elaborating onthis topic 4 of these 7 people mentioned either the delaybetween the two motions or ldquotwo independent motionsrdquoand the other 3 mentioned the asynchrony betweensound and motion We will come back to this issue

DiscussionThe results of Experiment 1 and the prestudy both

showed a clear continuous decrease of causality judg-ments with an increasing delay between the two visualmovements involved Furthermore they both showedthat an additional perceivable event in the middle of thevisual delay enhanced the impression of causality withMichotte-type launching displays Considering the con-tinuous monotonic decrease of the causality ratings withincreasing delays in each of the experimental conditionsof Experiment 1 and the prestudy we tend to believe thatwith respect to experimental launching events there isno clear-cut temporal causality time span Instead it seemsthat different aspects of the stimulus information (egthe delay the additional impulsive sound event the ad-ditional impulsive visual event the size of the disks therelative speed of the disks and directions) all contributemore or less to a continuous causality judgment aboutartificial stimuli that remind one of collisions (cf An-derson 1990 pp 161f)

Experiment 1 also tested the hypothesis that the in-creased tolerance for delays in the case of audiovisual dis-plays is a genuine audiovisual phenomenon We found thata visual event has the same effect as an auditory eventSekuler et al (1997) suggested that the increase of bounc-ing interpretations in an ambiguous visual displaymdashwhich is seen mostly as two motions crossing each othermdashis due to ldquoan acoustic event that signals a collision be-tween moving objectsrdquo (p 308) In the same paper the au-thors reported that a visual changemdasha pause in the visualmotion at the moment of crossingmdashenhances the bounc-ing impression In the case of launching events the ma-jority of our subjects agreed with the causal interpretation

9

8

7

6

5

4

3

2

10 100 200 300 400

Visual Delay (msec)

Med

ian

Cau

salit

y (1

9)

Sound + BlinkBlinkSoundControl

Figure 3 Median causality judgments in Experiment 1 The x-axis shows thedelay between the stop of the first disk and the onset of the target motion


as long as the delay between the two motions was shortThe addition of a sudden sound or a sudden blink duringthe delay helped to increase the causal interpretationand it turned out that there was virtually no difference in causality judgments between displays using a briefclack sound and displays using a brief blink of the targetThis demonstrates that the increase in the duration of delays tolerated is no genuine cross-modal effect it maybe due merely to attention distraction or to the amount of information connected with launching or collision sit-uations

In summary the tolerated delays can be stretched if asudden impulsive event is inserted in the middle of thedelay between the two visual events At a 340-msec vi-sual delay the subjects still consider the events to becausally related Although this seems to be a clear effectwe do not know the reasons for its occurrence Three dif-ferent lines of thought may apply

1 Watanabe and Shimojo (1998) suggested that theincrease of bouncing judgments in their streamingbouncing display with the addition of a blink at the timeand location of the overlap was due to attention distrac-tion This interpretation may apply to our results too InExperiment 2 we tested some of their assumptions

2 It is uncertain which events are causally related inthe audiovisual condition Most of the stimuli used in theprevious experiment consisted of three parts the move-ment of the first disk the marker event and the movementof the second disk It is uncertain which of these eventswere causally related It is conceivable that the subjectsdid not perceive a unitary collision event but rather oneof the three possible dyadic causendasheffect relations(1) the launching of the target was caused by movement(and stopping) of the first disk (2) the launching of thetarget was caused by the marker event and (3) the markerevent was caused by the movement of the first disk (cfFigure 4) The reported distribution of answers to Ques-tion 2 reflects two different aspects (1) the range ofstimuli used during the whole session contained severalscenes that did not produce a clear impression of causalityand (2) in some cases the subjects may not have experi-enced a direct causal link between the first motion themarker and the second motion Experiment 2 explicitlyaccounted for this question

3 It is well known that information presented duringan interval may decrease the judged duration of the in-terval (eg Ihle amp Wilsoncroft 1983) Therefore a clackor a blink presented during the delay between the twomotions in the launching display may decrease the func-tional delay and therefore result in increased causalityjudgments Experiment 3 explored this interpretation

Taken together the results of Experiment 1 showed acontinuous decrease of causality ratings with increasingdelays between cause and effect The majority of the sub-jects rejected the notion of causality with visual launch-ing displays using delays of more than 100ndash160 msec inthe control condition The decrease of causality ratingswith an increasing delay could be partially compensated

for by additional (auditory or visual) information Evendelays of more than 300 msec were tolerated if sounds orblinks were added to the classic visual launching infor-mation Experiment 1 did not analyze the specific con-tributions of specific components of the total informa-tion to the global causality judgments This was the aimof the following experiments

EXPERIMENT 2

In the prestudy and Experiment 1 the marker event(clack sound andor color change) always occurred inthe middle of the visual delay We assumed that this con-dition provided a maximum temporal link between thestop of the first movement and the start of the secondone at least in the range of delays used (up to 400 msec)In Experiment 2 we asked whether the position of themarker event within the delay period has a significant in-fluence on the causality ratings Since Experiment 1showed that a visual event has the same effect as anacoustic event we used only the latter in Experiment 2

In the case of a unitary collision event we would ex-pect no effect of the exact timing of the marker event (seeFigure 4) The three possible dyadic causendasheffect rela-tions however should all be influenced by changing theexact time of the marker event within the delay periodIf the predominant causendasheffect relation is the one be-tween the stop of the first movement and the onset of thesecond movement (a in Figure 4) we hypothesize that amaximal temporal link between the two events isachieved if the marker event occurs right in the middle ofthe delay period This hypothesis is based on the as-sumption that the causal impression decreases with anincreasing delay the decrease being almost complete atabout 200 msec The causality impression can be par-tially refreshed by means of additional information con-nected to collisions After refreshing the impression thedecrease starts again

If the important causendasheffect relation is the one be-tween the marker event and the launching of the seconddisk (b in Figure 4) causality judgments should increasethe closer these two events are connected in time Weshould expect low causality ratings if the marker eventoccurs at the beginning of the delay period and high rat-ings if it occurs at the end of the delay period In contrastif the main relation is the one between the movement ofthe first disk and the marker event (c in Figure 4) a highcausality rating would be expected if the marker eventoccurs early and therefore temporally close to the stop-ping of the first disk and a low rating would be expectedif it occurs late

There is another line of reasoning quite different fromthe one just discussed Extrapolating the results of Wata-nabe and Shimojo (1998) it can be hypothesized thatsound attracts attention and that the momentary distrac-tion hinders the complete pickup of visual informationthus leading to a greater contribution of inference pro-cesses at the cost of accurate perception The effect

796 GUSKI AND TROJE

should be greatest with very short delays and within thedelay it should be greatest at the beginning and smallestat the end of long delays This hypothesis makes predic-tions similar to those of Hypothesis c in Figure 4

MethodThe experiment was performed in two parts In Part A delays of

100 200 and 300 msec were used in Part B delays of 200 300and 400 msec were used

Subjects A total number of 21 undergraduate psychology stu-dents took part in Experiment 2 (Part A 6 women and 5 men meanage = 259 years Part B 5 women and 5 men mean age = 258 years)as part of their course credits None of them had participated in theprestudy or the preceding experiment and all were naive as to the

purposes and hypotheses motivating the study They reported normalor corrected-to-normal vision and hearing

Stimuli The stimuli (disks motions and sound) were the sameas those in the sound condition of Experiment 1 except for the vi-sual delays and the variable position of the sound Only three visualdelays were selected in each part (100 200 and 300 msec in Ex-periment 2A and 200 300 and 400 msec in Experiment 2B) Therelative position of the sound within each visual delay was variedin 7 steps For example in the 100-msec delay condition the clackmaximum was at 0 17 33 50 67 83 or 100 msec after the stop ofthe first visual movement

Procedure and Design The procedure was the same as that inExperiment 1 Each of the 21 conditions was measured three timesmaking a total of 63 trials in each part The order of the trials wasrandomized individually for each subject

MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c

Marker positionwithin delay

unitary

bac

Cau

salit

y

Figure 4 Models of unitary and dyadic causendasheffect relations (left andashc) together with the ex-pected effects of the marker positions on causality ratings (right)

Figure 5 Effect of clack position in Experiment 2A


ResultsAs is shown in Figures 5 and 6 the causality ratings at

each of the four visual delays were similar to those of thecomparable conditions in Experiment 1 In Experiment2A the clack position had no clear effect on the causal-ity ratings There was no significant linear quadratic orcubic regression with a 100-msec delay (r 2 = 05 05and 62 respectively p gt 05 each) and no significantlinear or quadratic regression with a 200-msec delay(r 2 = 46 and 51 respectively) but there was a signifi-cant cubic regression (r 2 = 83 p lt 05) With a 300-msecdelay there was no significant linear regression (r 2 = 59p gt 05) but there were significant quadratic and cubicregressions (r 2 = 86 and 95 respectively p lt 01 each)However in single-position tests even the decrease ofthe causality ratings at the end of the 300-msec delay wasnot significantly different from that at the next earlierposition (250 msec Wilcoxon Z = 19 p = 058) Becausewe still felt that the position of the clack sound mighthave an effect in combination with long visual delaysExperiment 2B was conducted with 200- 300- and400-msec delays This time there was a weak cubic re-gression with the 200-msec delay (r 2 = 71 p = 05) but thesignificant nonlinear trends with a 300-msec delay whichwere shown in Experiment 2A were not confirmed in Ex-periment 2B (r 2 = 39 and 65 respectively p gt 05 each)Also the causality judgments at the 400-msec delay didnot show any systematic covariation with the clack posi-tion at all (r 2 = 06 46 and 54 for linear quadratic andcubic regression respectively p gt 05 each see Figure 6)

DiscussionThe similarity of causality ratings at the four visual

delays used in Experiment 2 with the comparable soundconditions of Experiment 1 showed that the subjectswere sensitive to the clack sound However no consistent

effect of the clack position within the visual delays of100ndash400 msec was found The cubic trend of the clackposition within the 200-msec delay was almost statisti-cally consistent but the form of the trend differed be-tween Experiments 2A and 2B We conclude that the pre-cise forms of the curves in Figures 5 and 6 are notreliable and that the subjects were not sensible to the po-sition of the sound during the visual delay The pattern ofresults falsif ies the presumption that the effect of thesound on the causality ratings in Experiment 1 was duemainly to the fact that the sound occurred in the middleof the visual delay thereby ldquostretchingrdquo the causalitytime span by a certain amount of time The results alsocast doubt on the presumption that the effect of the soundon the causality ratings is due mainly to attention dis-traction because we would expect the greatest distrac-tion effect at the beginning of short delays which did notoccur It seems that the effect of the sound is due mainlyto the addition of sudden information within the delaythereby increasing the causality impression by a certainquantity as long as the additional information occurswithin the visual delay However this does not mean thatthe position of the additional information is completelyirrelevant since previous studies have shown that syn-chrony judgments between vision and hearing are high-est when the sound occurs a few milliseconds later thanthe respective visual information (see the introductionabove) But within the range of visual delays tested herethe position of the sound does not seem to matter arguingfor an impression of a unitary collision event rather thana predominance of some dyadic causendasheffect relation

EXPERIMENT 3

It is still an open question why the clack sound increasesthe causality judgments in Michotte-type launching dis-

Figure 6 Effect of clack position in Experiment 2B

798 GUSKI AND TROJE

plays using visual delays We hypothesized that thesound shortened the perceived duration of the delay aneffect that has been reported several times in the litera-ture for short durations (eg the so-called OppelndashKundtillusion in the temporal domain see also Boltz 1991Ihle amp Wilsoncroft 1983 Underwood amp Swain 1973Wearden Edwards Fakhri amp Percival 1998) Howeverit should be noted that the opposite effect also occursunder certain conditions (eg rather long time periods)We designed an experiment that compared no-sound de-lays with sound delays With a 2AFC procedure the sub-jects had to decide which of the two launching stimulihad the longer delay

MethodSubjects Ten psychology students (5 women and 5 men mean

age = 255 years) took part in Experiment 3 for course credit Noneof them had participated in one of the previous experiments and allwere naive as to the purposes and hypotheses motivating the studyThey reported normal or corrected-to-normal vision and hearing

Stimuli The Michotte-type no-sound launching display of Ex-periment 1 with a 200-msec delay between the stop of the first andthe start of the second movement of the disks was used as the ref-erence stimulus and displays with variable delays (40ndash 440 msec in11 steps) containing the clack sound in the middle of the delay pe-riod were used as comparisons

Procedure and Design Each trial consisted of the successivepresentation of two launching stimuli separated by an interstimulusinterval of 1 sec One of the displays was the reference stimulus theother was the comparison stimulus Each of the 11 possible com-parisons was presented five times resulting in a total of 55 trialsThe order of reference and comparison was counterbalanced Theorder of the trials was randomized The subjects were asked to in-dicate which of the two delays between the respective stop of thefirst and the start of the second disk movements appeared longer

The clack was not mentioned in the instructions The verbal re-sponses were noted down by the experimenter All other proceduraldetails were the same as those in Experiment 1

ResultsAs was expected the overall effect of the filled dura-

tions on the judgments was highly significant [withinsubjects F(110) = 2935 p lt 001] and the linear modelfits were best [F(1) = 59 p lt 001] The cubic term wasalso significant [F(1) = 142 p lt 005] The point of sub-jective equality for display durations containing the clacksound was 2595 msec as compared with 200 msec with-out the sound (Figure 7) In other words the durationsfilled with clack sound appeared to be about 60 msecshorter than the unfilled durations This difference isoutside the 95 confidence interval of 200-msec dura-tion judgments

DiscussionThe results were consistent with previous studies

showing a subjective shortening of short filled delays incomparison with unfilled ones Our results showed a dif-ference of 60 msec between visually defined intervals of200-msec duration containing no event and visually de-fined intervals containing an auditory event It is possiblethat cross-modal effects are less dramatic with respect toduration judgments than are unimodal auditory effectsDifferences of 60 msec in cross-modal judgments do notexplain the difference between the causality ratings ofsound and no-sound launching displays As is shown inFigures 3 and 4 the tolerated delay in the no-sound con-dition of Experiment 1 was 110 msec and in the sound

Figure 7 Comparisons of durations of constant 200-msec delays containingno sound with variable delays (40ndash440 msec) containing a clack sound in themiddle of the delay The y-axis shows the probability with which subjects reportthat the duration seems to be longer when it contains a sound as opposed towhen it does not contain a sound Depicted are means (filled circles) standarderrors and 95 confidence intervals


condition it was 280 msec Although the subjects proba-bly had experienced shorter visual delays when the de-lays were filled with a sound this effect seems too smallto fully explain the long delays tolerated

GENERAL DISCUSSION

Our experiments used the classic launching-type dis-play with varying delays between the stop of the firstdisk and the start of the second one Some of the delayswere filled with a clack sound some with a visual blinkA number of results occurred that have not been reportedbefore (1) There was a continuous decrease of causalityratings with an increasing delay (2) part of the delaycould be compensated for by additional sudden markerevents (eg clack sounds or blinks) within the delay(3) the nature of the marker event did not seem to play alarge rolemdashclack sounds that could be easily interpretedas part of the collision event had the same effect as blinkevents that had no obvious semantic relation to a colli-sion event (4) the exact temporal location of the markerevent within the delay did not affect the causality ratingsand (5) although the additional marker event had a smalleffect on the perceived duration of the delay period thisapparent extension could not account for the effects ob-served in Experiment 1

Several hypotheses have been discussed and tested inorder to explain the increase of causality ratings bymeans of additional marker events that occur during thedelay between the stop of the first disk and the start ofthe second one (see the introduction section above)

1 The increase in causality judgments is due to thetemporal proximity of visual and auditory events and isno true cross-modal phenomenon This temporal link hy-pothesis has been supported by showing that the sameincrease occurs when blinks are used instead of clacksin close temporal proximity to the visual motions

2 The increase is due to an increase of the total avail-able information about sudden changes in a stream ofevents This interpretation (information link) cannot beruled out because all the stimuli used in the noncontrolconditions of the experiments contained additional in-formation about sudden changes as compared with thecontrol conditions

3 The increase is due to the increased similarity of thesimulated collisions to real collisions Although all thestimuli in the noncontrol conditions contained informa-tion about sudden changes this reality link hypothesis isnot very probable because blinks or color changes rarelyoccur during real collisions

4 The increase is due to attention distraction The in-sertion of an additional stimulus in the delay distracts at-tention from the delay and observers are less likely to pro-cess the causality-contradicting information provided bythe delay Such an effect should be greatest with very shortdelays and with the additional marker event positioned atthe beginning of the delay The results of our experimentsdo not fit this hypothesis because the contribution of an

additional stimulus to the mean causality ratings wasgreatest with medium and long delays (Experiment 1) andthe position of the clack sound within the delay did not in-fluence causality judgments (Experiment 2)

5 The increase is due to the subjective shortening offilled delays as compared with unfilled ones Althoughsubjective shortening was shown in Experiment 3 thesize of this effect (about 60 msec for 200-msec delays)cannot explain the size of the contribution of additionalstimuli to the causality ratings (about 180 msec for meantolerated delays)

We conclude that the subjects in our experiments pro-cessed most of the audiovisual and visualndashvisual stimulias being unitary events Intuitive logical reasoning mayhave contributed to the subjectsrsquo judgments Accidentaltemporal and spatial co-occurrence of two events (egthe stop of the first disk in the vicinity of the second diskand the start of the second disk) is rather unlikely butaccidental co-occurrence of three events (eg the twoevents described above together with a marker event) iseven more unlikely It seems that the probability of spa-tial and temporal proximity of the events is sufficient toinduce causality impressions irrespective of the percep-tual modality

Until now it has been an open question as to which ofthe many informational aspects of a clack sound (or ablink) in the context of visual launching events con-tribute most to the increase of causality ratings Possiblecandidates are (1) the sudden change of the stimulationas such irrespective of the dimension and modality ofchange (2) the specific sudden change of intensity at thebeginning of the soundblink (3) the maximum inten-sity of the changed stimulation (sound or light) (4) thesudden return of the intensity (sound or light) to initiallevels (5) the spatial closeness of the events and (6) thecontent of the events which may be associated withevents in the real world In the case of the clack soundthe informational aspects (1ndash6) covaried in our experi-ments and they do so in most real-world collisions Inthe case of the blink this covariation is rarely found inthe real world although one of the subjects sponta-neously mentioned ldquoexplosionrdquo as an association to theblink Future experiments will have to test the informa-tion aspects necessary and sufficient to enhance multi-modal causality judgments

Although the causality judgments of the sound and theblink conditions in Experiment 1 produced comparableresults we would like to stress qualitative aspects of thelaunching display because 80 of our subjects judgedthe disks to be ldquorealrdquo objectsmdashfor example colliding ballsThis may have contributed to the assumption of unity asproposed by Welch and Warren (1980)mdashthat is a cogni-tive factor supporting the assumption of a common causeof the separate stimulus events In the terms used byJones and Boltz (1989) and Boltz (1991) the scenes con-taining colliding balls may be seen as a ldquohigher orderrdquocoherent structured information sequence This allowsobservers to switch between attending to the global struc-

800 GUSKI AND TROJE

ture (eg collision events) and details of the eventrsquosstructure (eg the velocity of motions the sound sourceetc) When observers are attending to the global struc-ture certain violations of physical principles of colli-sions (eg a short delay or a sudden color change) willnot be detrimental for causal judgments especially in thecase of redundant information (eg with sound + blink)This perspective is compatible with our results in Exper-iment 1 and with the general observation that causaljudgments are not necessarily qualitative yesno responsesthey can be ordered on a continuous scale ranging fromnot at all to extremely The more coherent informationthat fits a physical causation principle is available thehigher the causal judgment will be

REFERENCES

Anderson J R (1990) The adaptive character of thought HillsdaleNJ Erlbaum

Boltz M (1991) Time estimation and attentional perspective Per-ception amp Psychophysics 49 422-433

Crowder R G (1993) Auditory memory In S McAdams amp E Bigand(Eds) Thinking in sound The cognitive psychology of human audi-tion (pp 113-145) Oxford Oxford University Press ClarendonPress

Dixon N F amp Spitz L (1980) The detection of auditory visual de-synchrony Perception 9 719-721

Gruber H E Fink C D amp Damm V (1957) Effects of experienceon perception of causality Journal of Experimental Psychology 5389-93

Ihle R C amp Wilsoncroft W E (1983) The filled-duration illu-sion Limits of duration of interval and auditory fillers Perceptual ampMotor Skills 56 655-660

Jones M R amp Boltz M (1989) Dynamic attending and responsesto time Psychological Review 96 459-491

Kohlrausch A amp van de Par S (2000) Experimente zur Wahrnehm-barkeit von Asynchronie in audio-visuellen Stimuli [Experiments onthe perception of asynchrony with audio-visual stimuli] In Fortschritteder Akustik (DAGA 2000 pp 316-317) Oldenburg DEGA Geschaumlfts-stelle

Lewald J Ehrenstein W H amp Guski R (2001) Spatio-temporalconstraints for auditory-visual integration Behavioural Brain Re-search 121 69-79

Lewkowicz D J (1996) Perception of auditoryndashvisual temporal syn-chrony in human infants Journal of Experimental PsychologyHuman Perception amp Performance 22 1094-1106

Maass H (1938) Uumlber den Einfluss akustischer Rhythmen auf optischeBewegungsgestalten [On the influence of acoustic rhythms on visualmovement gestalts] Archiv fuumlr die Gesamte Psychologie 100 424-464

Michotte A (1946) La perception de la causaliteacute Louvain Publica-tions Universitaires [English translation The perception of causalityLondon Methuen 1963]

Michotte A amp Thinegraves G (1963) La causaliteacute perceptive Journalde Psychologie Normale et Pathologique 60 9-36 [English transla-tion ldquoPerceived causalityrdquo in G Thinegraves A Costall amp G Butter-worth (Eds) Michottersquos experimental phenomenology of perception(pp 66-87) Hillsdale NJ Erlbaum 1991]

Oestermeier U amp Hesse F W (2000) Verbal and visual causal ar-guments Cognition 14 65-104

Parducci A (1974) Context effects A range-frequency analysis InE C Carterette amp M P Friedman (Eds) Handbook of perception(Vol 2 pp 127-141) New York Academic Press

Piaget J (1967) Le deacuteveloppement des perceptions en fonction delrsquoage In J Piaget P Fraisse E Vurpillot amp R Francegraves (Eds) Traiteacutede psychologie expeacuterimentale Vol 4 La perception (pp 1-62) ParisPresses Universitaires de France

Rudloff I (1997) Untersuchungen zur wahrgenommenen Syn-chronitaumlt von Bild und Ton bei Film und Fernsehen [Experiments onthe perceived synchrony between pictures and sound with movies andtelevision] Unpublished diploma thesis Ruhr-Universitaumlt BochumFakultaumlt fuumlr Psychologie

Sarris V (2000) Perception and judgment in psychophysics An intro-duction into the frame of reference theories In A Schick M Meisamp C Reckhardt (Eds) Contributions to psychological acoustics Re-sults of the 8th Oldenburg Symposium on Psychological Acoustics(pp 39-62) Oldenburg Bibliotheks- und Informationssystem derUniversitaumlt

Schlottmann A amp Shanks D R (1992) Evidence for a distinctionbetween judged and perceived causality Quarterly Journal of Ex-perimental Psychology 44A 321-342

Scholl B J amp Nakayama K (2000 November) Contextual effectson the perception of causality Poster presented at the annual meet-ing of the Psychonomic Society New Orleans [Abstract published inAbstracts of the Psychonomic Society 5 91]

Scholl B J amp Tremoulet P D (2000) Perceptual causality and an-imacy Trends in Cognitive Sciences 4 299-309

Sekuler R Sekuler A B amp Lau R (1997) Sound alters visualmotion perception Nature 385 308

Underwood G amp Swain R A (1973) Selectivity of attention andthe perception of duration Perception 2 101-105

van de Par S amp Kohlrausch A (2000) Sensitivity to auditory-visual asynchrony and to jitter in auditory-visual timing In B E Ro-gowitz amp T N Pappas (Eds) Human vision and electronic imaging(Proceedings of SPIE Vol 3952 pp 234-242) Bellingham WASPIE Press

Watanabe K (2000) When sound affects vision Effect of auditorysaliency on visual motion perception and its relation to crossmodalattention Abstracts of Sloan Meeting 2000 New York New YorkUniversity Center for Neural Science

Watanabe K amp Shimojo S (1998) Attentional modulation in per-ception of visual motion events Perception 27 1041-1054

Watanabe K amp Shimojo S (2001a) Postcoincidence trajectory du-ration affects motion event perception Perception amp Psychophysics63 16-28

Watanabe K amp Shimojo S (2001b) When sound affects vision Ef-fects of auditory grouping on visual motion perception Psychologi-cal Science 12 109-116

Wearden J H Edwards H Fakhri M amp Percival A (1998)Why ldquosounds are judged longer than lightsrdquo Application of a modelof the internal clock in humans Quarterly Journal of ExperimentalPsychology 51B 97-120

Welch R B amp Warren D H (1980) Immediate perceptual re-sponse to intersensory discrepancy Psychological Bulletin 88 638-667

White P A (1995) The understanding of causation and the produc-tion of action From infancy to adulthood Hove UK Erlbaum

White P A (1999) Toward a causal realist account of causal under-standing American Journal of Psychology 112 605-642

Yela M (1952) Phenomenal causation at a distance Quarterly Jour-nal of Experimental Psychology 4 139-154

Zietz K amp Werner H (1928) Uumlber die dynamische Struktur derBewegung [On the dynamic structure of motion] Zeitschrift fuumlr Psy-chologie 105 226-249

(Manuscript received August 28 2001revision accepted for publication January 29 2003)

790 GUSKI AND TROJE

results showed considerable variation In some cases theevents seemed independent while in others the impactseemed to lsquoproducersquo the noise For present purposeshowever the most important point is that for more thanone observer the lsquoproductionrsquo of the noise was much moreclearly marked when rebounding took placerdquo (p 237f)Experiment 82 was identical to Experiment 81 exceptthat the second disk moved at a considerably slowerspeed after the collision ldquoTwenty-one people tookpart in this experiment twelve of them or 57 statedwhen we asked them specifically that the impact pro-duced the noiserdquo (p 238) Michotte seemed to be trou-bled by the fact that (1) virtually no subject spontaneouslymentioned the sound and (2) the sound was not synchro-nous with the contact between the experimental objectsFrom both aspects he concluded that the ldquoimpressionwas not really a causal onerdquo (p 240) and he stopped ex-perimenting on audiovisual displays mainly because oftechnical problems

In the light of more recent data Michotte (19461963)probably drew a false conclusion because audiovisualevents are judged to be most synchronous when the soundoccurs 50 msec after the visual event connected with thesound (Kohlrausch amp van de Par 2000 Rudloff 1997)In addition he seemed to believe that the audiovisualsystem is less tolerant with small delays between the endof the initial movement and the start of the auditoryevent than the visual system is with small delays betweenthe stop of the initial movement and the start of the ef-fect movement This is strange because it is generallyknown that sound waves travel at rather low velocitiesand that echoes are considerably behind their cause (eghand claps or shouts)

There are some historic papers that have been con-cerned with auditory influences on motion perceptionWe will mention them here briefly because their line ofreasoning is different from that of Michotte (19461963)For instance Zietz and Werner (1928) used pairs of draw-ings (eg a drawing of a bent arrow and a drawing of asmall circle) and presented them repeatedly one after theother with a delay of several hundred millisecondsWithout sound the majority of subjects seemed to per-ceive the two drawings in isolation but when the pre-sentation of each drawing was accompanied by a colli-sion sound the subjects tended to see the visual objectsin motionmdashjumping expanding or rotating in spaceMaass (1938) extended the experimental approach ofZietz and Werner to a visual display consisting of 12 tinyelectric bulbs which could be programmed in order torepresent rapidly changing geometric forms and threeelectric hammers which also could be programmed andproduced 64 different rhythms Without sound the opticdisplays produced a huge variety of movement descrip-tions and the author did not see a specific relation be-tween the visual stimuli and the movement descriptionsof the subjects When the optic displays were connectedwith sounds the movement descriptions the subjects

produced were less variable and the light points were ex-perienced as belonging more closely together

There are some recent papers that did not describeaudiovisual causality experiments but were essentiallyrelated to them

1 Sekuler Sekuler and Lau (1997) reported that anambiguous visual display is perceived as a collision if acollision sound is added When two surfaces move to-ward each other in a straight line coincide and thenmove apart this display is consistent with two differentinterpretations After coincidence either the two objectscould have continued to move in their original directionsor they could have collided and then bounced reversingdirections Adding (acoustic) clicks at the time of the vi-sual coincidence increased the perception of collisionsignificantly Watanabe (2000) reported that the bounce-inducing effect of 3-msec low-intensity clicks workedfrom 250 msec before to 150 msec after the visual coin-cidence and within a separation of up to 30ordm of visualangle between the sound and the visual display Watanabeand Shimojo (2001b) found that the bounce-inducing ef-fect of very brief clicks was attenuated when other iden-tical sounds (auditory flankers) were presented beforeand after the simultaneous sound The attenuation oc-curred only when the simultaneous sound and auditoryflankers had similar acoustic characteristics and the simul-taneous sound was not salient The authors conclude that(1) the bounce-inducing effect of the sound is a genuineaudiovisual effect and (2) there exists a saliency-assigningprocess that is context sensitive and can be utilized bythe visual system for resolving ambiguity In a second se-ries of experiments on the streamingbouncing displayusing a sound a flash or a pause after the coincidence ofthe two surfaces Watanabe and Shimojo (2001a) foundthat each of the additional events increased the bouncingjudgment but that ldquothe percentage of the bouncing per-cept did not reach its maximal level immediately afterthe coincidence but increased as a function of postcoin-cidence trajectory duration up to 150ndash200 msecrdquo (p 16)The authors hypothesized that the perceptual systemsneed a certain amount of time to integrate the (more orless conflicting) information into a unitary concept

2 Experiments on the perception of audiovisual syn-chrony (Dixon amp Spitz 1980 Kohlrausch amp van de Par2000 Lewkowicz 1996 Rudloff 1997 van de Par ampKohlrausch 2000) have shown that a sound may bejudged to be synchronous with a visual event even whenthe sound occurs after a short delay In contrast a pre-ceding sound is less likely to be judged as synchronousThe tolerance for synchronism depends on the content ofthe audiovisual events (eg speech or collisions betweenobjects) and lies in the area between ca 2130 msec(sound preceding) and +250 msec (sound following)Optimal synchronism between sound and image is per-ceived if the sound follows 80ndash100 msec after the visualevent (cf Rudloff 1997 van de Par amp Kohlrausch2000)











792 GUSKI AND TROJE









EXPERIMENT 1












794 GUSKI AND TROJE







9

8

7

6

5

4

3

2

10 100 200 300 400

Visual Delay (msec)

Med

ian

Cau

salit

y (1

9)











EXPERIMENT 2





796 GUSKI AND TROJE








MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c


unitary

bac

Cau

salit

y









EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES














































792 GUSKI AND TROJE









EXPERIMENT 1












794 GUSKI AND TROJE







9

8

7

6

5

4

3

2

10 100 200 300 400

Visual Delay (msec)

Med

ian

Cau

salit

y (1

9)











EXPERIMENT 2





796 GUSKI AND TROJE








MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c


unitary

bac

Cau

salit

y









EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES





































EXPERIMENT 1












794 GUSKI AND TROJE







9

8

7

6

5

4

3

2

10 100 200 300 400

Visual Delay (msec)

Med

ian

Cau

salit

y (1

9)











EXPERIMENT 2





796 GUSKI AND TROJE








MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c


unitary

bac

Cau

salit

y









EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES




































794 GUSKI AND TROJE







9

8

7

6

5

4

3

2

10 100 200 300 400

Visual Delay (msec)

Med

ian

Cau

salit

y (1

9)











EXPERIMENT 2





796 GUSKI AND TROJE








MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c


unitary

bac

Cau

salit

y









EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES












































EXPERIMENT 2





796 GUSKI AND TROJE








MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c


unitary

bac

Cau

salit

y









EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES




































796 GUSKI AND TROJE








MarkerEvent

MotionDisk 1

MotionDisk 2

unitary

b

a

c


unitary

bac

Cau

salit

y









EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES










































EXPERIMENT 3



798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES




































798 GUSKI AND TROJE














GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES






































GENERAL DISCUSSION












800 GUSKI AND TROJE


REFERENCES




































800 GUSKI AND TROJE


REFERENCES




































audiovisual phenomenal causality - biomotionlab

Documents