When transcranial magnetic stimulation (TMS) modulates feature integration
Abstract
How the brain integrates visual information across time into coherent percepts is an open question. Here, we presented two verniers with opposite offset directions one after the other. A vernier consists of two vertical bars that are horizontally offset. When the two verniers are separated by a blank screen (interstimulus interval, ISI), the two verniers are perceived either as two separate entities or as one vernier with the offset moving from one side to the other depending on the ISI. In both cases, their offsets can be reported independently. Transcranial magnet stimulation (TMS) over the occipital cortex does not interfere with the offset discrimination of either vernier. When a grating, instead of the ISI, is presented, the two verniers are not perceived separately anymore, but as ‘one’ vernier with ‘one’ fused vernier offset. TMS strongly modulates the percept of the fused vernier offset even though the spatio-temporal position of the verniers is identical in the ISI and grating conditions. We suggest that the grating suppresses the termination signal of the first vernier and the onset signal of the second vernier. As a consequence, perception of the individual verniers is suppressed. Neural representations of the vernier and second vernier inhibit each other, which renders them vulnerable to TMS for at least 300 ms, even though stimulus presentation was only 100 ms. Our data suggest that stimulus features can be flexibly integrated in the occipital cortex, mediated by neural interactions with outlast stimulus presentations by far.
Introduction
Even after a century of research, it is still a mystery how the human visual system groups elements across space and time into coherent objects. Spatial grouping has been extensively studied (for review, see Palmer et al., 2003). Less is known about how the visual system integrates elements across time. Here, we used a feature fusion paradigm combined with transcranial magnetic stimulation (TMS) to shed further light on temporal integration. In feature fusion, two elements with different features are presented in rapid succession at the same retinotopic location. For example, if a red and a green disk are presented, ‘one’ yellow disk is perceived, i.e. diskness and colours fuse (Efron, 1967; Yund et al., 1983). Likewise, if verniers with opposite offset directions are presented immediately one after the other, the verniers are not perceived individually but as one fused vernier. The fused vernier appears to be almost aligned because the vernier offsets integrate and cancel each other (Fig. 1A). TMS modulates the fusion process of the two verniers for more than 370 ms even though verniers are presented for 60 ms only (Fig. 1B; Scharnowski et al., 2009). Within the first 120 ms after stimulus presentation, TMS increases responses in accordance with the second vernier. From 120 to 400 ms TMS increases the response in accordance with the first vernier (Scharnowski et al., 2009). We suggest that during fusion the neural representations weaken each other, thus making them vulnerable to TMS. Hence, if verniers do not fuse, TMS should not have an impact on vernier offset discrimination. This prediction was tested here.
Feature Fusion and TMS. (A) A vernier, a pair of vertical bars that are horizontally offset, is followed by a second vernier with an offset in the opposite direction of the first vernier. For example, if the first vernier is offset to the right, the second vernier is offset to the left. Because of its opposite offset direction, the second vernier is referred to as the anti-vernier. Because of the short durations, the verniers are perceived as one fused vernier only. The fused vernier appears to be almost straight because the opposite vernier offsets are integrated and, thus, almost cancel each other out. Still, a small offset of the fused vernier is perceived on each individual trial. If vernier and anti-vernier are of the same duration, the anti-vernier dominates performance on average, i.e. observers predominantly perceive the fused vernier to be offset in the same direction as the anti-vernier (Herzog et al., 2003; Scharnowski et al., 2007a,b). (B) For the TMS experiment, the offset size of the vernier was adjusted such that performance was at 50% on average, i.e. both verniers contributed equally to performance (dashed line). Then, TMS was applied at different times after the onset of the vernier (solid line). For onset asynchronies ranging from 45 to 95 ms, the anti-vernier dominated performance. For TMS onset asynchronies greater 145 ms, the vernier dominated up to a surprisingly long time of 370 ms (Scharnowski et al., 2009). Vernier and anti-vernier presentation are indicated by the small depictions in the graph. Performance was quantified as dominance, i.e. the percentage of responses in which the perceived offset direction of the fused vernier corresponded to that of the first vernier. Figure adapted from Scharnowski et al. (2009) with permission from ARVO.
Here, we show that when an interstimulus interval (ISI) is inserted between the verniers the vernier offsets become individually visible either as apparent motion or as two consecutively flashed stimuli. In this situation, the vernier offsets do not fuse and TMS indeed cannot modulate the offset discrimination of either the first or second vernier. However, when the ISI is replaced by a grating with the same duration as the ISI, the verniers are not perceived individually anymore and their offsets fuse. This fusion process can be modulated by TMS the very same way as if the verniers were following each other immediately in accordance with the prediction.
Materials and methods
Experimental design
In Experiment 1, we presented two verniers with opposite offset directions. We show that the offsets can be discriminated individually when the verniers are separated by an ISI, but not when they are separated by a grating of the same duration as the ISI. In Experiment 2, we show that the lack of discriminability of the two offsets is not due to masking of the grating, but due to the integration of the verniers. In the third, the main experiment, we show that TMS can modulate the fusion process when a grating is inserted between the vernier and anti-vernier. In Experiment 4, we show that this TMS modulation is not caused by interaction of the grating mask with the verniers, and Experiment 5 demonstrates that offset discrimination is not impaired by TMS if the stimuli are perceived independently. In Experiment 6, we investigated whether the termination of the vernier or the onset of the anti-vernier determines whether the offsets are integrated. In Experiment 6, we presented an additional ISI either between the first vernier and the grating, or the second vernier and the grating, to show that masking the onset transient of the second vernier causes the verniers to fuse.
Observers
A total of 23 observers (aged 21–29 years, nine female) participated in the study and gave written informed consent approved by the local ethics committee in accordance with the Declaration of Helsinki. Participants had normal or corrected-to-normal visual acuity as measured by the Freiburg visual acuity test (Bach, 1996). Paid observers were students from the local universities and naïve towards the purpose of the experiment.
Stimulus presentation
Stimuli were presented on an X–Y display controlled by a PC via a fast 16-bit DA converter. In the experiments involving TMS, a HP 1332A X–Y display equipped with a P4 phosphor has been used. In the remaining experiments, either a HP 1332A X–Y display or a Tektronix 608 X–Y display, both equipped with P11 phosphor, were used. Stimuli were presented at 80 cd/m2, a 1-MHz dot rate, a 200-Hz refresh rate and a dot pitch of 200 μm. Viewing distance was 2 m. The room was dimly illuminated by a background light (∼0.5 lx) to prevent adaptation to scotopic vision. Stimulus contrast was close to 1.0. All stimuli were presented centred at the screen. Stimuli were preceded by a fixation dot 400 ms before stimulus presentation. A vernier consists of two vertical segments. Each segment was 10′ (arc minutes) long, 0.5′ wide and separated by a vertical gap of 1′. A small horizontal offset was inserted between the upper and the lower segment (Fig. 1A). The grating consisted of 25 grating elements. Each grating element was an aligned vernier with otherwise the same spatial layout as the offset vernier. Horizontal spacing between the grating elements was 3.3′, resulting in a horizontal extension of 80′. The central element of the grating appeared at the same position as the vernier.
The offset direction of the (first) vernier was chosen randomly (left or right) for each trial. The second vernier, if presented, always had an offset direction opposite to the first vernier. If, for example, the first vernier was offset to the left, the second vernier was offset to the right. Because of its opposite offset direction, the second vernier is referred to as the anti-vernier.
Performance was measured blockwise in two runs of 60 trials per observer. The order of blocks was randomized across participants. Blocks were repeated in the reversed order to counteract practice and fatigue effects in the averaged data. Within each block of 60 trials, a different pseudorandom sequence of left and right vernier offset directions was presented.
Experiment 1
A vernier was presented for 10 ms followed by an anti-vernier for 10 ms. The horizontal offset of the vernier was 40″ (arc seconds), the anti-vernier offset was 32″. All other parameters of the two verniers were identical. First, an ISI of variable duration was presented between the vernier and the anti-vernier (Fig. 2A). Observers were asked to indicate whether the vernier or the anti-vernier was offset to the right. Second, a grating was presented instead of an ISI (Fig. 2B). The central grating element and the vernier and anti-vernier fused and appeared as one brighter central element, standing out from the remaining elements of the grating. The stimulus sequence was explained to the observers prior to the experiment and observers were asked to indicate whether the vernier or the anti-vernier was offset to the right. Five observers, naïve to design and purpose of the experiment participated (two female, age range 21–29 years).
Experiment 1. (A) A left or right offset vernier was presented for 10 ms followed by a blank screen, i.e. an interstimulus interval (ISI), of a variable duration and, then, by an anti-vernier for 10 ms. Observers were asked to indicate whether the vernier or the anti-vernier was offset to the right. (B) Instead of the ISI, a grating of 25 aligned verniers was presented between vernier and anti-vernier. (C) In the ISI condition (A), performance improves quickly with increasing ISI, reaching ceiling performance at 50 ms (solid line). In the grating condition (B), performance does not improve and stays close to chance level for all values tested (dashed line). Error bars indicate 95% confidence interval based on the bootstrap analysis. Note the axis break in the abscissa.
Experiment 2
First, a vernier was presented for 10 ms followed by a grating of variable duration and an anti-vernier for 10 ms (Fig. 3A). In the second condition, the grating was followed by an aligned vernier, i.e. a vernier without a horizontal offset, instead of the anti-vernier (Fig. 3B). In the third condition, the first vernier was replaced by an aligned vernier, followed by a grating and an anti-vernier (Fig. 3C). Vernier and anti-vernier offset sizes were as in Experiment 1. Observers were asked to report the offset of the central element. Vernier dominance (see below) was determined in all three conditions. Five observers participated in this experiment, of which all but one observer (the first author) were naïve to the design and purpose of the experiment (four female, aged 23–28 years).
Experiment 2. (A) In the first condition, a vernier was presented for 10 ms followed by a grating of 25 aligned verniers of a variable duration and an anti-vernier of 10 ms duration. (B and C) Either the anti-vernier (B) or the vernier (C) was replaced by an aligned vernier. In all three conditions, observers were asked to discriminate the offset direction of the central element. (D) In the first condition, performance was close to 50% for all durations as in Fig. 2C (dashed line). The replacement of the anti-vernier by an aligned vernier (B) leads to a clear dominance of the vernier (dashed line). Similarly, the replacement of the vernier (C) leads to a clear dominance of the anti-vernier (dotted line). Dominance diminishes when mask duration increases in both conditions, i.e. performance approaches 50%. The performance in the vernier and anti-vernier condition (A) can be predicted well by the dominance levels in the vernier-only (B) and anti-vernier-only (C) condition (A*; empty circles; see text for details). Error bars indicate 95% confidence interval based on the bootstrap analysis.
Experiment 3
A vernier was followed by a grating and an anti-vernier. The duration of the grating was 50 ms. The vernier duration was 30 ms and the duration of the anti-vernier was 20 ms (Fig. 4). We used longer vernier (and anti-vernier) durations of 30 ms (and 20 ms) than in Experiments 1 and 2 because longer verniers yield stronger TMS modulations (unpublished data). Stimulus offset sizes for vernier and anti-vernier were individually determined prior to the experiment to obtain 50% dominance, i.e. both vernier and anti-vernier contributed equally to performance on average. Still, observers perceived a small offset in each trial, either in accordance with the vernier or anti-vernier. Offset sizes ranged from 28″ to 40″. Observers were asked to indicate the offset direction of the fused central element. All offset sizes were at least twice the individual offset discrimination threshold for a single vernier for each subject. Single-pulse TMS was applied at 14 onset asynchronies. Four observers, naïve to design and purpose of the experiment participated (one female, aged 21–27 years).
Experiment 3. A vernier was presented for 30 ms, followed by a grating of 50 ms duration, and an anti-vernier of 20 ms duration (see also inlet). Observers were asked to indicate the offset direction of the fused element. Transcranial magnetic stimulation (TMS) was applied over the occipital cortex at different onset asynchronies. We adjusted the offset size of the first vernier such that performance was about 50%, i.e. on average, both verniers contributed equally to performance (dashed line). TMS leads to a modulation of the dominance depending on the onset asynchrony. For onset asynchronies ranging from 70 to 145 ms, the anti-vernier dominates performance. For TMS onset asynchronies of more than 145 ms, the vernier dominates for up to 320 ms after the onset of the first vernier. Error bars indicate 95% confidence interval based on a bootstrap analysis (see Materials and methods).
Experiment 4
In the first part of the experiment, a vernier of 30 ms duration or the anti-vernier of 20 ms duration was presented alone. Observers were asked to report its offset direction. Offset discrimination thresholds for the vernier or anti-vernier were determined to avoid ceiling performance, which would have occurred otherwise with the offset sizes used in Experiment 3 (28″–40″). Thresholds were determined by the adaptive PEST procedure (Taylor & Creelman, 1967). Second, a grating was added immediately following the vernier or directly preceding the anti-vernier. As in Experiment 3, the grating duration was 50 ms. Third, in addition, TMS was applied at several onset asynchronies ranging from 20 to 270 ms. Three new observers, naïve to the design and purpose of the experiment participated (two male, aged 22–26 years).
Experiment 5
First, a vernier of 30 ms duration was followed by a grating of 50 ms duration and an anti-vernier of 20 ms duration. The dominance was balanced at 50% by adjusting vernier and anti-vernier offset sizes individually. Vernier offset sizes ranged from 40″ to 60″, anti-vernier offset sizes from 20″ to 30″ (Fig. 6B, dashed line). Next, we removed the grating and replaced it by an ISI. Observers were asked to report whether the vernier or the anti-vernier was offset to the right (Fig. 6A). TMS was applied at different onset asynchronies ranging from 0 to 320 ms. Finally, the grating was again inserted in-between vernier and anti-vernier. TMS was applied at the onset asynchronies for which maximal and minimal dominance was found in Experiment 3 to show that TMS modulates dominance for the new observers (Fig. 6B). Three observers, naïve to the purpose of the experiment participated (one female, aged 22–26 years).
Experiment 5. (A) A vernier of 30 ms duration was followed by an ISI of 50 ms and an anti-vernier of 20 ms duration. Observers were asked to indicate whether the vernier or the anti-vernier was offset to the right. Observers can reliably discriminate whether the vernier or the anti-vernier was offset to the right as determined by the percentage of correct responses (dashed line). Transcranial magnetic stimulation (TMS) at various onset asynchronies does not impair performance (black line). (B) Because new observers participated in this experiment, we repeated the conditions of Experiment 3, where TMS onset asynchronies showed minimal (95 ms) or maximal dominance (220 ms) to show that these observers show TMS modulation. Without TMS, vernier and anti-vernier cancel each other out (dashed line). As in Experiment 3, TMS applied at 95 ms yields anti-vernier dominance, TMS at 220 ms vernier dominance. The ordinate indicates vernier dominance. Error bars indicate 95% confidence interval based on the bootstrap analysis.
Experiment 6
A vernier presented for 10 ms was followed by a grating of 50 ms duration and an anti-vernier of 10 ms duration. An ISI of variable duration was inserted between the grating and the anti-vernier (Fig. 7A), between the vernier and the grating (Fig. 7B), or between both (Fig. 7C). Observers were asked whether the vernier or the anti-vernier was offset to the right. Six observers participated (six male, aged 21–29 years). All but one observer (the first author) were naïve to the design and purpose of the experiment.
Experiment 6. (A) A vernier of 10 ms duration was followed by a grating of 50 ms duration, an ISI of variable duration, and an anti-vernier of 10 ms duration. The duration of the ISIs varied from 0 to 300 ms. (B) A vernier of 10 ms duration was followed by an ISI of variable duration, a grating of 50 ms duration, and an anti-vernier of 10 ms duration. (C) A vernier of 10 ms duration was followed by an ISI of variable duration, a grating of 50 ms duration, an additional ISI of the same duration as the first one, and an anti-vernier of 10 ms duration. In all three conditions, observers were asked to indicate whether the vernier or the anti-vernier was offset to the right. (D) In the first condition (A), performance starts to improve from ISIs of 20 ms on, reaching ceiling performance at about 150 ms (solid line). In the second condition (B), performance immediately improves, reaching ceiling performance at about 75 ms (dashed line). In the third condition (C), performance improves quickly as in the second condition, reaching ceiling performance at 75 ms (dotted line). Hence, observers could not use the additional ISI in the third condition (C) to improve performance. The percentage of correct responses is plotted on the ordinate. Error bars indicate 95% confidence interval based on the bootstrap analysis.
TMS
Magnetic stimulation was applied to the occipital pole with a Medtronic MagPro X100 stimulator (MagVenture, Farum, Denmark) with a MCF125 circular coil in biphasic single-pulse mode. The coil was placed with its lower rim 1.5–2 cm above the inion to interfere with visual processing (Amassian et al., 1989; Walsh & Cowey, 1998; Corthout et al., 1999). For each observer, stimulator output was maximized to the highest output level while preventing eye blinking and muscle contractions. Across participants, outputs ranged from 70 to 85% of maximum stimulator output. The experimenters monitored eye-blinks and muscle artefacts induced by TMS throughout the experiments. Blocks containing TMS-induced eye-blinks were not included in the analysis. Four out of the nine participants reported seeing phosphenes at the centre of their visual field. The remaining observers did not report perceiving phosphenes.
Performance measures
In feature fusion, observers perceive only one fused vernier. Observers were asked to report the offset of the lower segment of the fused vernier with respect to that of the upper segment by pressing one of two push buttons. Dominance was computed as the proportion of trials on which this response matched the offset direction of the first vernier. Thus, values above 50% indicate dominance of the first vernier; values below 50% indicate dominance of the anti-vernier (e.g. Fig. 1B). Fifty percent is the point of subjective equality, where vernier and anti-vernier equally contribute to performance. We also quantified offset discrimination thresholds for a single vernier. By applying the adaptive PEST procedure (Taylor & Creelman, 1967), the threshold and slope of the psychometric function (cumulative Gaussian) were estimated by means of a maximum likelihood analysis (Wichmann & Hill, 2001). The guessing rate was set to 50% and the maximum lapsing rate was set to 3%.
Regardless of the task, observers were always instructed to be as accurate as possible. No feedback about performance was given.
Bootstrap analysis
For statistical analysis, we performed a bootstrap analysis (Efron & Tibshirani, 1993). For each observer, we drew 120 data points with replacement from the observer’s original data and determined the percentage of correct responses or vernier dominance depending on the task. Next, we computed a mean across observers. We repeated this procedure 10 000 times to obtain an estimate of the distribution from which our experimental data was sampled. Based on this distribution, we could determine the outmost 2.5% on both sides of the distribution, thereby providing an estimate of the 95% confidence interval. Error bars derived from this analysis no longer need to be symmetric.
Interference statistics between two data points were computed by simultaneously bootstrapping both performances for each observer and determining the difference. Next, we computed the mean difference across observers in each repetition. This way, we obtained an estimate of the distribution of the differences. The null hypothesis of equal performance in both conditions was rejected if the 95% confidence interval of the distribution of differences did not contain 0.0. This procedure, carried out for each observer separately, allows one to compute differences on a single observer level.
Results
Experiment 1
A vernier was followed by an ISI of variable duration and an anti-vernier. Observers were asked to indicate whether the vernier or anti-vernier was offset to the right. Performance quickly improved, reaching ceiling performance at an ISI of 50 ms (Fig. 2C, solid line). Observers perceived the two verniers either as apparent motion with the offset direction shifting from the vernier direction to the anti-vernier direction, or as two consecutively flashed stimuli, depending on the ISI. When a grating instead of the ISI was inserted between the vernier and anti-vernier, observers were unable to identify the vernier or anti-vernier offset (Fig. 2C, dashed line). The vernier, the central grating element and the anti-vernier fuse.
Experiment 2
In the previous experiment we found that vernier offsets cannot be identified if a grating is between vernier and anti-vernier. Performance was at 50%. This result may be due to masking of the vernier offsets by the grating. Here, we tested this hypothesis by replacing either the vernier or the anti-vernier by an aligned vernier. When both vernier and anti-vernier were presented, neither of them dominated performance. Observers performed at about 50% for all grating durations (Fig. 3D, solid line). In contrast, when either the vernier or the anti-vernier was replaced by an aligned vernier, performance clearly differed from 50%. When the anti-vernier was replaced by an aligned vernier, the vernier offset dominated performance (Fig. 3D, dashed line). When the vernier was replaced by an aligned vernier, the anti-vernier dominated performance (Fig. 3D, dotted line). With increasing grating durations, the dominance of either the vernier or the anti-vernier slowly decreased, but remained different from 50% for durations of up to 150 ms. A bootstrap analysis revealed significant differences between all three conditions for grating durations up to 150 ms. For the 50-ms grating duration, which will be used in the following experiments, performance between all three conditions differed significantly for each observer (individual observer data are plotted in the Supporting Information Fig. S1).
The dominances in the vernier-only condition and the anti-vernier-only condition were of the same magnitude. The dominance in the two verniers condition can be well predicted by a combination of the dominances in the vernier-only and the anti-vernier-only conditions [i.e. 50% + (DomV−50%) − (50%−DomAV); Fig. 3D, empty circles]. Hence, the two dominance levels cancel each other out. Thus, if vernier, grating and anti-vernier are presented in succession, the vernier offset and the anti-vernier offset fuse.
Experiment 3
In the main experiment, a vernier was followed by a grating and an anti-vernier. Observers were asked to report the offset of the central element. As aimed for, performance without TMS was about 50% vernier dominance [Fig. 4, dashed line; mean (M) = 51.6%; SEM = 1.9]. When TMS was applied between 70 and 145 ms after the onset of the vernier, the anti-vernier dominated performance (M95 = 33.2%; SEM = 4). For later onset asynchronies of 170 ms up to 320 ms, the pattern reversed and the vernier dominated performance (Fig. 4, solid line; M220 = 68.3%; SEM = 7.3). Hence, TMS strongly modulated vernier dominance.
Experiment 4
In this control experiment, to test the impact of the grating on the vernier and anti-vernier, first, the vernier or the anti-vernier was presented alone. Due to its longer duration, the vernier could be discriminated slightly better than the anti-vernier (Fig. 5A and B, dotted lines; MV = 8.9″; SEM = 1.6; MAV = 10.3″; SEM = 0.7). Second, discrimination thresholds were about 26″ when the vernier was backward masked by the grating and about 13″ when the anti-vernier was forward masked (Fig. 5A and B, dashed line). TMS did not increase thresholds further in both cases (Fig. 5A and B, solid line).
Experiment 4. (A) First, a vernier of 30 ms duration was presented alone (dotted line). Second, the vernier was followed by a grating of 50 ms duration (dashed line). Third, transcranial magnetic stimulation (TMS) was applied over the occipital cortex at different asynchronies after the onset of the vernier (solid line). Observers were asked to indicate the offset direction of the vernier. TMS does not increase thresholds. (B) An anti-vernier of 20 ms duration was presented alone (dotted line) or preceded by a grating of 50 ms (dashed line). TMS was applied at the same asynchronies as in (A). TMS did not increase masking (solid line). Backward masking was considerably stronger than forward masking. The threshold in the forward masking condition was lower than in the backward masking condition, even though vernier durations are larger than anti-vernier durations. Error bars indicate SEM.
Hence, neither backward nor forward masking per se renders offset discrimination susceptible to TMS.
Experiment 5
In this control experiment, the vernier was followed by an ISI and an anti-vernier. Vernier offsets did not fuse and observers were able to discriminate whether the vernier or anti-vernier was offset to the right (Fig. 6A, dashed line; M = 79; 21%; SEM = 8.93). In this condition, TMS had no impact on performance (Fig. 6A, black line).
When vernier and anti-vernier were separated by a grating, observers were asked to indicate the offset direction of the central element. As aimed for, dominance was about 50% (Fig. 6B, dashed line; M = 51.2%; SEM = 3.1). As in Experiment 3, TMS applied at 95 ms led to anti-vernier dominance, whereas TMS applied at 220 ms led to a dominance of the vernier (Fig. 6B; M95 = 36.4%; SEM = 3.85; M220 = 64.9%; SEM = 1.8). Hence, TMS does not impair offset discrimination of verniers that are perceived individually.
Experiment 6
A vernier was followed by a grating and an anti-vernier. In addition, either between the vernier or the anti-vernier and the grating an ISI was inserted. Observers were asked which of the two vernier offsets was to the right. Performance improved quickly when an ISI was inserted between the anti-vernier and the grating, reaching ceiling performance at an ISI of 75 ms duration (Fig. 7D, dashed line). When the ISI was inserted between the vernier and the grating, performance improved slower, reaching ceiling performance at an ISI of 150 ms duration (Fig. 7D, solid line). When an ISI was inserted between both the vernier and the grating and between the anti-vernier and the grating, performance was comparable to the condition in which only one ISI was inserted between the grating and the anti-vernier (Fig. 7D, dotted line). Hence, the additional ISI in-between the vernier and the grating does not further improve performance. When the vernier is separated from the grating performance differs significantly from the other two conditions for all ISI durations exceeding 5 ms as determined by bootstrap analysis. Hence, it seems to be the onset signal of the anti-vernier that predominantly allows to separate the vernier and anti-vernier. Masking the onset of the anti-vernier allows the offsets of the vernier and anti-vernier to fuse.
Discussion
Verniers presented in rapid succession are perceived as one single vernier. The fused vernier offset is a combination of both the vernier and the anti-vernier offset, of which each cannot be accessed individually (1, 2). Fusion also occurs if vernier and anti-vernier are separated by a grating (Fig. 3D). In both cases, TMS strongly modulates the fusion process (1, 4). In contrast, when vernier and anti-vernier are separated by an ISI, instead of the grating, observers can well identify whether the vernier or the anti-vernier is offset to the right (Fig. 2C). The verniers do not fuse but are perceived as two consecutively flashed stimuli or in apparent motion, depending on the ISI. The temporal presentation of the vernier and anti-vernier is identical in the ISI and the grating conditions, e.g. the vernier is presented from 0 to 30 ms and the anti-vernier from 80 to 100 ms in both cases. Why is performance so qualitatively different in the two conditions?
A neuron elicits a strong transient response at stimulus onset and stimulus termination (e.g. Adrian & Matthews, 1927; Hubel & Wiesel, 1959). It is often proposed that the individual visibility of a stimulus is mediated by these transient responses (e.g. Breitmeyer & Ritter, 1986; Macknik & Livingstone, 1998; Macknik et al., 2000). We suggest that the transient signals of both the vernier and anti-vernier are unaffected when they are separated by ISIs longer than 20 ms. The verniers are treated as two objects and are well visible (or are perceived as one object in apparent motion with two appearances; Fig. 2). In the grating condition, the termination signal of the vernier and the onset signal of the anti-vernier are suppressed and, hence, the two are treated as one event. In this case, the offsets fuse because they belong to the same object. This is in line with the subjective experience. The verniers appear as ‘one’ bright element superimposed on the grating.
In this case, we suggest that the neural representations of the ‘conflicting’ vernier and anti-vernier offsets are mutually inhibiting each other (Scharnowski et al., 2009). This weakening makes the representations vulnerable to TMS (Fig. 4). Likewise, offset discrimination of low-contrast verniers (2 cd/m2) can be deteriorated by TMS (Kammer et al., 2003), but not for high-contrast verniers (80 cd/m2) where offset discrimination is not impaired by TMS (Fig. 6; see also Scharnowski et al., 2009). Equally, TMS cannot deteriorate performance when the vernier and anti-vernier are separated by an ISI because the intact transient signals hinder neural interaction.
The offset of the central grating element is not integrated with the vernier offsets, even though the central element is presented at the same retinotopic location as the verniers. We suggest the central grating element belongs to a different object, namely the grating, and is not integrated with the vernier or anti-vernier because of grouping. The central grating element is grouped with the other grating elements, which are well visible because of their intact onset and termination signals. It seems that the human brain integrates features very carefully even when elements are presented rapidly (see also Hermens & Herzog, 2007; Hermens et al., 2009b; Herzog & Brand, 2009).
In terms of the object file metaphor (Kahneman et al., 1992; Wolfe & Bennett, 1997; Mitroff et al., 2004), we suggest that transient signals open and close object files. TMS can affect only interacting features ‘within’ one object file. Experiment 6 suggests that the onset transient is more important to open and close an object file than the termination transient, because vernier and anti-vernier discrimination deteriorates less when the termination transient of the vernier is masked, compared with when the onset transient of the anti-vernier is masked (Fig. 7D).
Interestingly, TMS modulates feature fusion for up to 320 ms, outlasting the stimulus duration for more than 200 ms (Fig. 4). A duration of more than 300 ms is surprising as observers can rapidly categorize natural scenes in < 300 ms and show differential neural activity already after 150 ms (Vanrullen & Thorpe, 2001). During integration, the interaction of the neural representations closely follows the temporal order of the stimulus. The peak of anti-vernier dominance is at 95 ms regardless of whether the vernier is followed by a grating and then the anti-vernier or by the vernier immediately. The peak of the vernier dominance, however, is at 220 ms, when the anti-vernier immediately follows the vernier, and at 270 ms when vernier and anti-vernier are separated by the grating. The time course is exactly shifted by the duration of the grating (50 ms; Figs 1 and 4).
Feature fusion is a special type of visual masking in which the single verniers are not the primary targets but their fused offset (in this sense feature fusion is also different from mutual masking in which the features of both targets have to be judged individually, which is impossible when verniers are fused; Bachmann & Allik, 1976; for a masking review, see Breitmeyer & Ögmen, 2006). As with typical masking, also in feature fusion, backward masking is stronger than forward masking (Fig. 2; Scharnowski et al., 2005, 2007a, 2009; Hermens & Ernst, 2007; Hermens & Herzog, 2007; Herzog, 2007; Herzog et al., 2007; Hermens et al., 2009a,b). Feature fusion is a case of integration masking where target and mask spatially overlap and make up a composite. Integration masking is supposed to be different from interruption masking where target and mask do not overlap, still leading to strong suppression. This suppression is often related to a neural interaction of transient magnocellular channels on sustained parvocellular channels (Breitmeyer & Ganz, 1976). We suggest that the transient signals suppress the individual visibility of the target also in integration masking. This suppression yields fusion and makes the vernier offsets susceptible to TMS interference.
We can only speculate about the cortical substrate involved in feature fusion. Because we used a round coil placed 1.5–2 cm above the inion, V2 is the most likely site of interaction of TMS with visual processing (Thielscher et al., 2010). Still, subcortical TMS effects due to antidromic stimulation or effects in higher cortical areas involved in visual temporal processing, like V5 (Bueti et al., 2008), the posterior parietal cortex (Alexander et al., 2005) and the temporal parietal junction cannot be excluded (for a review, see Lamme & Roelfsema, 2000; Battelli et al., 2008). However, in the current study, TMS has its initial peak in the modulatory effects at about 95 ms, which is consistent with re-entrant processing in early visual cortices (Bullier, 2001; Ro et al., 2003). Furthermore, in Scharnowski et al. (2009) results did not change regardless whether a focal eight-figure coil or a round coil was used.
Acknowledgements
We would like to thank Marc Repnow for excellent technical support and Guillaume Guex for help with the data collection. We also would like to thank Frank Scharnowski for fruitful discussions during the planning of the experiment. This work was funded by the Swiss National Science Foundation (SNF) project ‘The dynamics of feature integration’.
Abbreviations
-
- ISI
-
- interstimulus interval
-
- TMS
-
- transcranial magnetic stimulation
