|
Bouncing/streaming
Double-flash
-
Illusory Flashing Visual Percept Induced by Sound
L. B. Shams, Y. Kamitani, S. Shimojo
Purpose: Various studies have shown the modulation of perceived
intensity, detectability, spatial and temporal localization of visual
stimuli by an accessory auditory stimulus (AAS). AAS is an auditory
stimulus that is presented in close temporal proximity of the visual
stimulus and which is not to be responded to. We investigated whether the
visual percept itself can be modified by such auditory stimulation.
Methods: In each trial, a disk is flashed for 13.3ms in the center
of a computer screen (where the subjects fixate). In one condition, the
flash is accompanied by a brief sound stimulus which consists of 1 to 3
beeps (with the number and pitch of the beeps randomized across trials),
and in another condition, the visual stimulus is not accompanied by any
auditory signal. The subjects' task is to judge the number of flashes
displayed on the screen in each trial.
Results: Although the visual stimulus is identical across all
trials (a single flash), the percept of the visual stimulus varies
drastically across trials. Subjects perceive the single flash as more than
one flash in the condition where the flash is accompanied with auditory
beeps, and not otherwise. In other words, the sound beeps induce the
perception of "visual beeps." Moreover, the number of flashes perceived in
each trial is correlated significantly with the number of beeps in the same
trial. A single flash is reported in trials with single beep. The results
are robust to many parameters namely the duration of the visual flash, the
flashing visual pattern, the size of the visual pattern, etc. and spatial
concordance between the auditory and visual stimuli is not necessary.
Conclusion: AAS can alter the visual percept qualitatively and
significantly. The observed illusory flash effect does not seem to be due
to general attentional enhancement caused by auditory stimulation, as there
is no illusory flash elicited by a single beep. The illusion does not
appear to be a result of eye movements, as the effect is stronger with
shorter flash durations, persists with very large disk size, and degrades
with decrease in disk contrast. The observed illusion (as well as its
non-requirement of spatial concordance between the auditory and visual
stimuli) can be explained based on prior behavioral findings relating the
audio-visual sensory interactions.
-
When sound affects vision: Audiovisual attention and visual motion perception
K. Watanabe , S. Shimojo
Two identical visual targets moving across each other can be
perceived either to bounce off or to stream through each other (Metzger,
1934). A brief sound at the moment the targets coincide biases perception
toward bouncing (Sekuler et al., 1997). By using this ambiguous motion
display and the bounce-inducing effect, we determined an interaction range
for audiovisual event classification.
We found that a sound can bias the perception even when it was
presented from 250-ms before to 150-ms after the visual coincidence,
suggesting that the temporal interaction range is rather long. Likewise,
the spatial interaction range is wide: With a separation up to 30-deg of
visual angle between the sound and the visual display, the bounce-inducing
effect was virtually unchanged. Subsequent experiments showed that the
bounce-inducing effect depends on auditory context. For example, the effect
was attenuated when the simultaneous sound was embedded in other identical
sounds (auditory flankers) with an sound-onset-asynchrony of about 300 ms.
The attenuation occurred only when the simultaneous sound and auditory
flankers has similar acoustic characteristics and the simultaneous sound
was not salient. These results suggest that there is an aspect of
auditory-grouping (saliency-assigning) processes which is context-sensitive
and can be utilized by the visual system for ambiguity solving. Based on
the finding that the saliency of sound is crucial for the bounce-inducing
effect, I hypothesized that an allocation process of crossmodal attention
may be involved in this audiovisual effect. If the motion mechanism
responsible for the dominance of the streaming percept requires attention
and a salient sound attracts attention automatically, the bounce-inducing
effect would results. This was confirmed by the experiments where a sudden
visual flash (exogenous attention distraction) or an additional task
(endogenous attention distraction) at the moment of the visual coincidence
also enhanced the bouncing percept.
Thus the present study shows (1) the spatial-temporal interaction
range for audiovisual event classification, (2) the dependency of
audiovisual interaction on auditory context, and (3) its possible relation
to crossmodal attention.
References
Metzger, W. (1934). Psychologische Forschung, 19, 1-60.
Sekuler, R., Sekuler, A. B., & Lau, R. (1997). Nature, 385, 308
-
Illusory Flashing Visual Percept Induced by Sound
L. B. Shams, Y. Kamitani, S. Shimojo
Purpose: Various studies have shown the modulation of perceived
intensity, detectability, spatial and temporal localization of visual
stimuli by an accessory auditory stimulus (AAS). AAS is an auditory
stimulus that is presented in close temporal proximity of the visual
stimulus and which is not to be responded to. We investigated whether the
visual percept itself can be modified by such auditory stimulation.
Methods: In each trial, a disk is flashed for 13.3ms in the center
of a computer screen (where the subjects fixate). In one condition, the
flash is accompanied by a brief sound stimulus that consists of 1 to 3
beeps (with the number and pitch of the beeps randomized across trials),
and in another condition, the visual stimulus is not accompanied by any
auditory signal. The subjects' task is to judge the number of flashes
displayed on the screen in each trial.
Results: Although the visual stimulus is identical across all
trials (a single flash), the percept of the visual stimulus varies
drastically across trials. Subjects perceive the single flash as more than
one flash in the condition where the flash is accompanied with auditory beeps, and not otherwise. In other words, the sound beeps induce the perception of
"visual beeps." Moreover, the number of flashes perceived in each trial is
correlated significantly with the number of beeps in the same trial. A
single flash is reported in trials with single beep. The results are robust
to many parameters namely the duration of the visual flash, the flashing
visual pattern, the size of the visual pattern, etc. and spatial
concordance between the auditory and visual stimuli is not necessary.
Conclusion: AAS can alter the visual percept qualitatively and
significantly. The observed illusory flash effect does not seem to be due
to general attentional enhancement caused by auditory stimulation, as there
is no illusory flash elicited by a single beep. The illusion does not
appear to be a result of eye movements, as the effect is stronger with
shorter flash durations, persists with very large disk size, and degrades
with decrease in disk contrast. The observed illusion (as well as its
non-requirement of spatial concordance between the auditory and visual
stimuli) can be explained based on prior behavioral findings relating the
audio-visual sensory interactions.
-
Sound-induced Visual "Rabbit"
Y. Kamitani & S. Shimojo
Purpose:
We previously reported that a single visual flash presented with
multiple beeps with short intervals appears to be multiple flashes
(Shams, Kamitani, & Shimojo, Nature 2000). Do the beeps produce mere
impression of brightness fluctuation, or rather independent visual
tokens? To demonstrate that the illusory flashes can be perceived
independently at different spatial locations, a visual apparent motion
display was combined with beeps.
Methods:
Two vertical bars (13 ms duration each) were flashed 3 deg horizontally
apart with an interval of 106 ms, creating clear apparent motion. Each
bar was accompanied by a beep (10 ms duration, simultaneous onset with
the bar). Another beep was presented at a various timing between the
flashes/beeps. In a condition without sound, a real additional bar was
flashed at an inbetween timing at the same position as either of the two
bars. Subjects reported the perceived location of the illusory bar
associated with the second beep (if any), or that of the additional real
bar. The direction of apparent motion was randomized across trials.
Results:
All subjects (3) reported that an illusory bar was perceived with the
second beep at a location between the real bars (difference from the
real bars, p < .05). This is analogous to the cutaneous "rabbit"
illusion where trains of successive cutaneous pulses delivered at a few
widely separated locations produce sensations at many inbetween points.
The illusory bar appeared closer to the first (second) bar, as the
second beep was presented temporally closer to the first (third) beep.
The perceived location of the additional real bar was not significantly
different from the actual location, thus purely visual "rabbit" was not
observed.
Conclusion:
The sound-induced visual flashing creates visual tokens that can be
spatially displaced, indicating visual proper, as opposed to cognitive,
alteration, which nontheless is uniquely induced by sound.
Simultaneity
|
