Detailed spatiotemporal brain mapping of chromatic vision combining high-resolution VEP with fMRI and retinotopy
Funding information: University of Rome “Foro Italico,” the IRCCS Santa Lucia Foundation, and the Italian Ministry of Education, University and Research
Abstract
Neuroimaging studies have identified so far, several color-sensitive visual areas in the human brain, and the temporal dynamics of these activities have been separately investigated using the visual-evoked potentials (VEPs). In the present study, we combined electrophysiological and neuroimaging methods to determine a detailed spatiotemporal profile of chromatic VEP and to localize its neural generators. The accuracy of the present co-registration study was obtained by combining standard fMRI data with retinotopic and motion mapping data at the individual level. We found a sequence of occipito activities more complex than that typically reported for chromatic VEPs, including feed-forward and reentrant feedback. Results showed that chromatic human perception arises by the combined activity of at the least five parieto-occipital areas including V1, LOC, V8/VO, and the motion-sensitive dorsal region MT+. However, the contribution of V1 and V8/VO seems dominant because the re-entrant activity in these areas was present more than once (twice in V8/VO and thrice in V1). This feedforward and feedback chromatic processing appears delayed compared with the luminance processing. Associating VEPs and neuroimaging measures, we showed for the first time a complex spatiotemporal pattern of activity, confirming that chromatic stimuli produce intricate interactions of many different brain dorsal and ventral areas.
1 INTRODUCTION
The perception of colors is a central component of primate vision. Chromatic perception has been extensively studied, and brain correlates of this function have been investigated mainly using neuroimaging and electrophysiological techniques. While neuroimaging studies have finely mapped brain areas involved in human perception of chromatic stimuli (mainly foveal representation of V1, V2, and V8/VO—a distinctive patch of color-selective activation in the fusiform gyrus (e.g., Brewer, Liu, Wade, & Wandell, 2005; Hadjikhani, Liu, Dale, Cavanagh, & Tootell, 1998), the timing of these activities has been investigated using the visual-evoked potentials (VEPs).
As typically reported, chromatic-onset VEPs have two main components called N1 and P2 peaking approximately at 120 and 180 ms after the stimulus onset (reviewed in Celesia, 2005). This spatiotemporal structure appears much simpler when compared with other luminance VEP stimulation modalities such as pattern-onset, which are described by at least six components (Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2002; Di Russo et al., 2005). The reason for this apparent simplicity may be due to the size and position of the stimulus in the visual field used so far in the literature. In fact, the stimulus is usually a large full field encompassing 10 or more degrees of visual angle (e.g., Xing et al., 2015). This large stimulation may yield multiple activities of differed polarity which could cancel each other out. Evidence in support to this hypothesis comes from a previous study from our group (Di Russo, Spinelli, & Morrone, 2001) where we used luminance pattern-onset VEPs combined with functional magnetic resonance imaging (fMRI) and retinotopic mapping. There we showed that small stimuli encompassing one visual quadrant could better separate VEP components allowing to fully express the spatiotemporal complexity of VEPs. Thus, it is possible that an adequate stimulation of the four quadrants of the visual field may reveal a more complex spatiotemporal structure of the chromatic-onset VEPs than that found so far. For example, it seems unlikely that the chromatic-onset VEPs lacks early components which originate from the striate cortex, like C1. Many studies confirmed that the VEPs correlate of primary visual area (V1) activity is the well-known C1 component peaking 70–90 ms after the onset of a luminance stimulus. The association between V1 and C1 comes from the original observation of Jeffreys and Axford (1972), then confirmed many years later by Clark, Fan, and Hillyard (1994) in a retinotopic VEP study and finally assessed by a combined VEP-fMRI recording in Di Russo et al. (2002, 2005, 2012) and in Pitzalis, Fattori, and Galletti (2012). In the specific case of chromatic stimulation, there is no direct evidence that VEP also reflects V1 activity, although it can be expected based on the fMRI literature that assigns to V1 a central role in color perception. So far there is only some indirect evidence of V1 contribution coming from the presence of activity within medial occipital scalp (Kulikowski, Robson, & Murray, 2002; Nakamura, Kakigi, Okusa, Hoshiyama, & Watanabe, 2000; Onofrj et al., 1995) and from the lack of attentional effects (Highsmith & Crognale, 2010) measured by low-resolution VEP only. However, Foxe et al. (2008) reported a C1 component for a particular red–green equiluminant stimulus, but this was a centrally-presented, elongated, rectangular shape subtending 14° in width and 0.12° in height at a very low spatial frequency (0.007 cpd). This stimulus, although equiluminant, was displayed with square-wave temporal and spatial envelopes, which are known to elicit activity in magnocellular cells too. In addition, stimuli extending on more than one visual quadrant produce a combined bilateral activity from the dorsal and ventral banks of V1 that may render ambiguous the C1 detection and labeling (Di Russo et al., 2002, 2005, 2007, 2012; Di Russo, Martínez, & Hillyard, 2003; Di Russo & Pitzalis, 2014). For recent consideration on this topic see also Baumgartner, Graulty, Hillyard, and Pitts (2017).
In addition, although prior neuroimaging studies revealed the neural substrate for the perception of color experience (e.g., Hadjikhani et al., 1998), given the lack of information regarding the source of the chromatic VEP it is still not clear whether fMRI and VEP responses share physiological substrate.
The main purpose of the present study was to determine a detailed spatiotemporal profile of red-green chromatic VEP and to localize its neural generators. Furthermore, we aimed to clarify the relationship between the two most important color brain centers as V1 and V8/VO and to study the possible association of these areas with independent VEP components. Finally, we addressed another debated question concerning the role played by the dorsal motion region termed middle-temporal complex (or MT+) in color processing (Conway et al., 2014). Some evidence has led to the notion that MT+ is color blind (e.g., Zeki, 1983c), some others suggest that MT+ is not entirely unresponsive to chromatic signals (e.g., Gegenfurtner & Kiper, 2003). To this end, we used combined VEP-fMRI technique and stimulation paradigms (i.e., size and position of the visual stimulus) that was developed and used by our group in many previous studies (Di Russo & Pitzalis, 2014; Di Russo et al., 2002, 2003, 2005, 2007, 2012, 2016; Pitzalis, Strappini, De Gasperis, Bultrini, & Di Russo, 2012; Pitzalis, Bozzacchi, et al., 2013; Sulpizio et al., 2017). This method can be defined “fMRI-informed EEG analysis” because it aims at moderating the spatial EEG inverse problem by guiding electromagnetic source imaging using results obtained from fMRI (Huster, Debener, Eichele, & Herrmann, 2012). Briefly, we used a dense electrode array and chromatic stimulation within each of the visual quadrants. Then, time-courses of cortical sources were identified using source modeling based on a realistic head model, taking into account the loci of cortical activation revealed by fMRI in response to the same stimuli. These sources were also localized on flat maps with respect to visual cortical areas (V1, V2, V3, V3A, V7, V6, V6Av, VP, V4v, V8/VO) identified in each subject by an independent fMRI wide-field retinotopic mapping experiment (e.g., Pitzalis et al., 2006). In addition, the typical lateral motion-sensitive cortical region MT+ was individually mapped using a dedicated functional localizer (Tootell et al., 1995).
2 METHODS
2.1 Subjects
Thirty paid volunteer subjects (mean age 24.2, range 20–35 years, 16 females) participated in the main VEPs experiment. A subset of 12 subjects (mean age 24.8, range 22–32 years, 7 females) also received structural MRI and fMRI scanning. All subjects were right-handed and had normal or corrected-to-normal vision. Gross color deficiencies were screened by interview and using the Ishihara test. Each subject participated to one VEP session and up to three scanning sessions.
All participants gave written informed consent prior to both electrophysiological and neuroimaging measures. All procedures followed the Declaration of Helsinki compliance and were approved by the independent ethic committee of the IRCCS Santa Lucia Foundation of Rome.
2.2 VEPs experiment
2.2.1 Stimuli
Stimuli were optimized to stimulate chromatic perception through parvocellular pathways. Therefore, they consisted in a circular Gabor pattern modulated vertically in chromaticity (red-green) subtending a foveal visual angle of 3° in diameter (pattern cutoff), with 2 cycles/deg of spatial frequency and a maximum Michelson contrast of 50%. Red-green patterns were obtained by superimposing out of phase iso-chromatic red-black and green-black gratings of identical contrast. The stimulus appeared for 400 ms (200 ms to reach the maximum intensity and 200 ms to disappear) following a sinusoidal envelope to avoid magnocellular contribution produced by abrupt stimulus onset. The average luminance was 22 cd/m2 and the C.i.e. coordinates at Vλ equiluminance were x = 0.61, y = 0.35, and x = 0.28, y = 0.60 for the red and green phosphors respectively. The ratio of the red luminance over the sum of the red and green luminance (R/[R + G]) was used to measure the subjective equiluminant point of the subjects, evaluated by standard flicker photometry. Equiluminant points varied between 0.46 and 0.56 for between subjects. As control, we used a circular Gabor pattern modulated vertically in luminance (yellow-black) and obtained by superimposing red-black and green-black gratings of identical contrast in phase. Importantly, we defined our stimuli based on the color space and (x,y,z) coordinate. The yellow/black and the red/green had the same average chromaticity (same coordinate x,y,z), but had modulation along the luminance or R/G axis. In other words, both stimuli were equated in the colorimetric space. Therefore, the fact that the neuronal mechanism (probably at IT level or V8/VO) will be differently adapted by the yellow and R/G stimulus is a mere consequence of the selective chromatic neuronal mechanisms. To stimulate quadrants one at a time, the circular Gabor was divided into four sectors (quadrants) leaving a small space of 0.1° along the meridians (Figure 1). Each quadrant appeared in random order, while the other three remained off. The stimulus onset asynchrony varied between 650 and 1,000 ms. The background luminance was equiluminant to the mean luminance of the pattern. Visual stimulation was displayed on a 21″ CRT monitor at a refresh rate of 144 Hz using the Presentation software (Neurobehavioral Systems, Inc. Albany, CA).
Stimuli were four Gabor sectors presented once at time corresponding with the four visual quadrants. Gratings were modulated in chromaticity in both VEP and fMRI experiments (left panel) and in luminance in the VEP study [Color figure can be viewed at wileyonlinelibrary.com]
2.2.2 Procedure
During the electroencephalogram (EEG) recordings, subjects were comfortably seated in a dimly lit, sound-attenuated and electrically shielded room while stimuli were presented in binocular vision on a video monitor at a viewing distance of 114 cm. Subjects were trained to maintain stable fixation on a central cross (0.4°) throughout stimulus presentation and, just to keep them alert, they had to press a button with the right index finger as soon as they detected a rare and low contrast flicker of the fixation cross (150 ms duration, 2–5 s SOA). Data from these catch-trials was not analyzed. Each run lasted approximately 3 min followed by 30–60 s rest periods, with longer breaks interspersed throughout the runs. A total of 20 runs were conducted to deliver at least 1,000 stimuli to each quadrant for chromatic stimuli and 500 for luminance stimuli (10 runs). Subjects received feedback on both their behavioral performance and their ability to maintain fixation, as monitored by electrooculogram (EOG).
2.2.3 Electrophysiological recording and data analyses
The EEG was acquired using two BrainAmpTM amplifiers and with the Recorder 1.21 software, then it was analyzed using the Analyzer 2.1 software (BrainProducts, GmbH, Munich, Germany). The 64 electrodes were placed according to the 10-10 standard montage system. All scalp channels were initially referenced to the left mastoid (M1). Horizontal eye movements were monitored with bipolar recordings from electrodes at the left and right outer canthi. EOG produced by blinks and vertical eye movements were recorded with an electrode below the left eye, which was referenced to site Fp1. The EEG was digitized at 250 Hz with an amplifier and band-pass filtered between 0.1 and 100 Hz, including a 50 Hz notch filter, and data were stored for off-line averaging. Artifact rejection was performed prior to signal averaging to discard epochs in which deviations in eye position, blinks or amplifier blocking occurred. On average, 6.5% of trials were rejected for violating artifact criteria. Time-locked VEPs were averaged separately per stimulus position (upper-left, upper-right, lower-left, and lower-right). The EEG was segmented into 1,100 ms epochs that began 100 ms prior the stimulus onset (voltage baseline). To reduce high-frequency noise, the averaged VEPs were low-pass filtered at 35 Hz. Data were re-referenced to averaged mastoids. VEP latency and amplitude components were measured as peak voltage deflections within specified time intervals (see results); these measures were taken at the electrode sites where the components were maximal in amplitude.
Two-ways ANOVAs were used to evaluate the effects of chromaticity and stimulus position on each VEP component, comparing chromatic and luminance conditions in the four quadrants for peak latency and amplitude measures (2 × 4 design). The confidence level, α, was set to 0.05 after Greenhouse–Geisser correction.
2.2.4 Modeling of VEP sources
Topographical mapping of scalp voltage and estimation of the dipolar sources of the VEP components in the grand-average waveforms were carried out using Brain Electrical Source Analysis (BESA 2000 v.5.1.8; Megis Software GmbH, Gräfelfing, Germany). The algorithm implemented in BESA was used to estimates the time-course of multiple equivalent dipolar sources activated in the fMRI. Interactive changes in dipole sources orientation lead to a minimization of the residual variance (RV) between the source model and the observed spatiotemporal VEP distribution. In the current study, this analysis used a realistic approximation of the head, with the radius obtained from the average of the studied group (80 mm). This realistic head model uses finite elements derived from an average of 24 individual MRIs and consists of three compartments: brain (including the cerebral spinal fluid), skull and scalp. A spatial digitizer recorded the three-dimensional coordinates of each electrode and three fiducial landmarks (the left and right preauricular points and the nasion). A computer algorithm was used to calculate the best-fit sphere that encompassed the array of electrode sites and determine their spherical coordinates. The mean spherical coordinates for each site averaged across all subjects were used for the topographic mapping and source localization procedures. In addition, individual spherical coordinates were related to the corresponding digitized fiducial landmarks and to landmarks identified on the standard finite element model of BESA 2000. The possibility of interacting dipoles (cross-task) was reduced by seeding dipoles not in adjacent positions and by selecting solutions with relatively low dipole moments with the aid of an “energy” constraint (weighted 20% in the compound cost function as opposed to 80% for the RV). The optimal set of parameters was found iteratively by searching for a minimum in the compound cost function. In addition to the RV, the model quality was estimated by applying residual orthogonality tests (ROTs; e.g., Bocker, Cornelis, Brunia, & Van den Berg-Lenss, 1994). For more details on the BESA method, see Di Russo et al. (2016).
In few words, a fMRI seeded strategy was used to model the dipolar sources of VEPs. Regional sources were constrained (seeded) to all the major fMRI activations and fitted in orientation only. BESA algorithm just optimized the dipole orientation in seeded (fixed) fMRI-based locations. The intervals for fitting (see “Results” section) were chosen based on the explained VEP topography. This method, combined with the retinotopic mapping, enables us to localize the EEG data in relation to visual areas and to anatomical regions with a known functional profile, as already done by our research group in the past (Di Russo & Pitzalis, 2014; Di Russo et al., 2002, 2003, 2005, 2007, 2012, 2016; Pitzalis, Fattori, et al., 2012; Pitzalis, Bozzacchi, et al., 2013; Sulpizio et al., 2017). Using this fMRI-guided VEP localization we produced three source models, one main model using both fMRI and VEP data for chromatic stimulation and two secondary models comparing the sources time-course of chromatic and luminance stimulations. Source modeling was conducted on group average data using group average fMRI seeds.
2.3 fMRI experiments
In the fMRI experiments, we used two-condition stimuli and a block-sequence paradigm (eight 16 s ON, 16 s OFF epochs) for both the main color experiment and to map the motion region MT+. Then, we used periodic stimuli and a phase-encoded paradigm to map the retinotopy of visual cortical areas. Subjects performed three different fMRI protocols as described below:
2.3.1 Color experiment
In the main fMRI experiment, the chromatic stimulation and task were identical to those used in the VEP experiment, except for the number of runs and duration, which differed because of block-sequence paradigm requirements. Further, each quadrant was presented in separate sessions in random order among subjects. In this procedure, 16 s of stimulation (chromatic stimulus for 200 ms every 650 and 1,000 ms) were alternated with 16 s of control condition were only the fixation-cross task was present. This sequence was repeated two times for each quadrant. Hence, the fMRI experiment consisted of eight runs of 4 min each. Prior to scanning, each subject was trained on the task (to maintain stable fixation and detect the flicker of the fixation cross) outside the scanner until the participant was completely familiar with the task. As in the VEP experiment, subjects were also briefly trained inside the scanner with a preliminary warm-up session.
2.3.2 MT+ mapping
Two additional scans were acquired to localize the motion-sensitive region, MT+. Stimuli produced by an X11/OpenGL program (original GL code by A. Dale, ported and extended by M. Sereno) consisted of concentric, thin, light gray rings (0.2 cycles/deg, 0.2 duty cycle) on a slightly darker-gray background, either moving (7°/s ON period) or stationary (OFF period). During the ON block, the concentric rings periodically contracted and expanded (1 s, 1 s) to avoid generating motion aftereffects during the OFF block. The average luminance of the stimulus was 61 cd/m2. The stimulus luminance contrast was low (∼1.5%) to better isolate MT+. It is now generally acknowledged that the relatively large motion-sensitive region found using this localizer and originally labeled V5 (or MT) in humans (Tootell et al. 1995) is probably a complex of several areas (e.g., Kolster, Peeters, & Orban, 2010; Pitzalis et al., 2010). For this reason, here we referred to it as the “MT complex” or “MT+.”
2.3.3 Retinotopic mapping
We mapped polar angle (measured from the contralateral horizontal meridian around the center of gaze) and eccentricity (distance from the center of gaze) using phase-encoded stimuli, as described elsewhere (e.g., Pitzalis et al., 2006, 2010; Pitzalis, Fattori, et al., 2012; Pitzalis, Bozzacchi, et al., 2013; Pitzalis, Sereno, et al., 2013; Pitzalis, Sdoia, et al., 2013; Sereno et al., 1995; Strappini et al., 2015, 2016; Strappini, Galati, Martelli, Di Pace, & Pitzalis, 2017). High-contrast light and dark colored checks counterphase flickered in either a ray- or a ring-shaped configuration (polar angle and eccentricity, respectively). Stimuli moved slowly and continuously, and checks reversed between bright and dark at a rate of 8 Hz. The average luminance of the stimuli was 105 cd/m2. Each subject was presented with periodic stimuli (64 s/cycle, 8 cycles/scan), varying in eccentricity or polar angle, in at least two pairs of scans. The use of a wide-field retinotopic stimulation (see “Experimental Set-up” section) allowed us to define not only the traditional visual areas (V1d, V1v, V2d, V2v, V3, V3A, V7, VP, V4v and V8/VO) but also two dorsal visual areas, V6 and V6Av, recently mapped by our group (Pitzalis et al., 2006; Pitzalis, Sereno, et al., 2013). While V6 responds to the entire contralateral hemifield (Pitzalis et al., 2006), area V6Av is a lower-only retinotopic region (Pitzalis, Sereno, et al., 2013).
2.3.4 Experimental set-up
Stimuli were generated by a control computer (a standard PC equipped with a standard 3D graphics card) located outside the MR room, running an in-house software (Galati et al., 2008) implemented in MATLAB (The MathWorks Inc., Natick, MA) using Cogent 2000 (developed at FIL and ICN, UCL, London, UK) and Cogent Graphics (developed by J. Romaya at the LON, Wellcome Department of Imaging Neuroscience, UCL, London, UK).
Visual stimuli were projected using an LCD video projector (100 Hz refresh rate with anti-aliasing system) with customized lens to a back-projection screen mounted behind the MR tube and visible through a mirror placed inside the head coil. For the color experiment, stimulus luminance and chromaticity were calibrated to match those of the CRT monitor used in the VEP experiment. While for the color experiment we used a standard set-up (23 × 12 deg field of view), for both retinotopic and motion mapping we used a wide-field stimulation (up to 82 deg in total visual extent) like that described by Pitzalis et al. (2006). In all experiments, fixation distance and head alignment were held constant by a chin rest mounted inside the head coil. Subjects’ heads were stabilized with foam padding to minimize movement during the scans. In the color experiment, in which the subject's response was required, manual responses were collected using a magnet-compatible response pad connected to the control computer via optic fibers. Retinotopic and MT+ mapping experiment used passive viewing and continuous central fixation throughout the scan acquisition.
2.3.5 Image acquisition
The MR examinations were conducted at the Santa Lucia Foundation (Rome, Italy) on a 3T Siemens Allegra MR system (Siemens Medical Systems, Erlangen, Germany) equipped for echo-planar imaging. Single-shot echo-planar imaging (EPI) images were collected using blood-oxygenation-level-dependent imaging (Kwong et al., 1992) by a standard transmit-receive birdcage head coil. About 30 coronal slices were 2.5 mm thick (with a 0-mm gap, interleaved excitation order), with an in-plane resolution of 3 × 3 mm, oriented approximately perpendicular to the Calcarine fissure. This voxel size strikes a compromise between sufficient signal-to-noise and the ability to assign activations to correct sides of sulci and gyri. Each participant underwent eight scans for the color experiment (two runs for each quadrant), two scans for the MT+ localizer and six scans for the retinotopic mapping (three runs for each stimulus type-eccentricity and polar angle). Each scan took either 256 s (main experiment and localizer scans) or 512 s (retinotopy), with 128 or 256 single-shot EPI images per slice, respectively (TR = 2,000 ms, TE = 30 ms, TA = 66.6 ms, flip angle = 70°, 64 × 64 matrix, bandwidth = 2,298 Hz/pixel; FOV = 192). In each scan, the first 8 s of each acquisition were discarded from data analysis to achieve a steady state, and the experimental tasks started at the beginning of the fifth volume. A total of 192 scans were carried out on 12 subjects (96 scans for the color experiment, 24 scans to map MT+, 72 scans for retinotopy).
The cortical surface of each subject was reconstructed from a structural scan (T1-weighted MPRAGE, 176 contiguous sagittal slices, 1 × 1 × 1 mm; TR = 2,000 ms, TE = 4.38 ms, flip angle = 8°, matrix 256 × 256, bandwidth = 1,130 Hz/pixel) taken in a separate session. The last scan of each functional session was an alignment scan (also MPRAGE, 1 × 1 × 1 mm) acquired in the plane of the functional scans. The alignment scan was used to establish an initial registration of the functional data with the surface.
2.3.6 Data analyses
The anatomical MRI data were used for cortical surface reconstruction using FreeSurfer v5.0.0 (Dale, Fischl, & Sereno, 1999; Fischl, Sereno, & Dale, 1999; http://surfer.nmr.mgh.harvard.edu) based on standard procedures described in details elsewhere (Hagler, Saygin, & Sereno, 2006; Pitzalis et al., 2006, 2010; Pitzalis, Bozzacchi, et al., 2013; Pitzalis, Sereno, et al., 2013; Pitzalis, Sdoia, et al., 2013; Sereno et al., 1995; Strappini et al., 2015; Tootell et al., 1997). Briefly, high-resolution structural images obtained from each subject were manually registered and averaged. The skull was stripped off by expanding a stiff deformable template out to the dura, the gray/white matter boundary was estimated with a region-growing method, and the result was tessellated to generate a surface that was refined against the MRI data with a deformable template algorithm. By choosing a surface near the gray/white matter border (rather than near pial surface, where macrovascular artifacts are maximal), we could assign activations more accurately to the correct sulcus bank. The surface was then unfolded by reducing curvature while minimizing distortion in all other local metric properties. After reconstruction, each hemisphere was then completely flattened using five relaxation cuts: one cut along the Calcarine fissure, three equally spaced radial cuts on the medial surface, and one sagittal cut around the temporal lobe.
Functional individual data from both two-condition experiment and phase-encoded retinotopy were analyzed using a customized version (retinotopy capable) of FreeSurfer (csurf, M. Sereno, www.cogsci.ucsd.edu/~sereno/.tmp/dist/csurf) based on standard procedures described in many previous studies (e.g., Sereno et al., 1995; Sereno, Pitzalis, & Martínez, 2001; Hagler, Riecke, & Sereno, 2007; Pitzalis et al., 2006, 2010; Pitzalis, Strappini, et al., 2012; Pitzalis, Bozzacchi, et al., 2013; Pitzalis, Sdoia, et al., 2013; Strappini et al., 2015, 2016, 2017). Briefly, pre-processing included motion correction, averaging across the two scans for each stimulus type to increase the signal to noise ratio, and aligning to the structural images. Additional affine transformations that included a small amount of shear were then applied to the functional scans for each subject using blink comparison with the structural images to achieve an exact overlay of the functional data onto each cortical surface. We applied to all data a surface-based smoothing (3.2 mm FWHM) after Fourier analysis by averaging the values of adjacent vertices, with an iterative algorithm implemented in FreeSurfer (Hagler et al., 2006). The p values were estimated on a voxel-by-voxel basis by constructing an F ratio between “signal” (response amplitude at stimulus frequency) and “noise” (amplitude at other frequencies excluding second and third harmonics) with degrees of freedom equal to the number of time points. The phase of the signal at the stimulus frequency was used to map retinotopic coordinates (polar angle or eccentricity). In the standard block-design analysis, pseudo-color scales are usually used to represent the amplitude of the response (after masking the data with a significance threshold). We modulated the saturation of the color as a function of the signal amplitude using a sigmoid function. The sigmoid function was arranged so that visibly saturated phase colors begin to emerge from the gray background at a threshold of p < 10−2. The data at most activated cortical surface points have much higher significance values (p < 10−5 to 10−10). This analysis is used both in mapping studies (i.e., phase-encoded retinotopic data) and to distinguish between positive and negative going MR fluctuations in the case of 2-condition stimulus comparisons (e.g., MT+ mapping). The boundaries of retinotopic cortical areas (V1, V2, V3, V3A, V7, VP, V4v, V8/VO, V6, and V6Av) were defined on the cortical surface for each subject based on phase-encoded retinotopy (e.g., Sereno et al., 1995; Pitzalis et al., 2006) and subsequent calculation of visual field sign, which provides an objective means of drawing borders between areas based on the angle between the gradients in the polar angle and eccentricity with respect to cortical position (Sereno et al., 1995). According to a previous study (Tootell et al., 1995) and also as routinely done in our laboratory (e.g., Pitzalis et al., 2010; Pitzalis, Fattori, et al., 2012; Pitzalis, Strappini, et al., 2012; Pitzalis, Bozzacchi, et al., 2013; Pitzalis, Sereno, et al., 2013; Pitzalis, Sdoia, et al., 2013), MT+ was functionally defined on the cortical surface of each participant as the set of all contiguous voxels within the cortical portion around the middle temporal sulcus (MTs), responding stronger to moving than to static low-contrast concentric rings. For each subject, each thresholded map was projected on the participant's cortical surface and intersected with the borders of the retinotopic regions. This masking procedure defined which regions were significantly activated by the task.
Group data from the color experiment were analyzed by SPM5. Functional images from each participant were co-aligned with the high-resolution anatomical scan (MPRAGE) taken during the same session. Images were motion-corrected, resampled into 3 × 3 × 3 mm3 space, transformed into MNI space using a nonlinear stereotaxic normalization procedure (Friston, Frith, Turner, & Frackowiak, 1995), and smoothed with a three-dimensional Gaussian filter (6-mm full-width-half-maximum). A standard group analysis was performed according to a general linear model (GLM), modeling “ON” blocks as box-car functions convolved with a canonical hemodynamic response function. We used the GLM only for the group analysis. Significance was judged by cluster size at the voxel level. Correction for multiple comparisons was performed using distribution approximations from Gaussian random field (GRF) method implemented in SPM5 using the estimated intrinsic smoothness based on residual images, at the cluster level (p ≤ .001, FWR corrected) after forming clusters of adjacent voxels with an uncorrected threshold of p ≤ .001.
Localization and visualization of individual activations by SPM were achieved using BrainShow (code by G. Galati), an in-house generated software for visualization of fMRI data. This software is implemented in Matlab (The MathWorks Inc.) and allows superimposition of SPM group maps (in MNI space, see above) on the reconstruction of the cortical surface of the average brain provided in the Population-Average, Landmark-, and Surface-based (PALS) atlas and generated using SureFit and Caret software (Van Essen, 2005). BrainShow has been used in previous studies from our and other groups (e.g., Castriota-Scanderbeg et al., 2005; Galati et al., 2008, 2011; Ionta et al., 2011; Pitzalis et al., 2010). Stereotaxic coordinates were calculated through an automatic nonlinear stereotaxic normalization procedure (Friston et al., 1995), performed using the SPM8 software platform (Wellcome Department of Cognitive Neurology, London, UK), implemented in MATLAB (The MathWorks Inc., Natick, MA). The template image was based on average data provided by the Montreal Neurological Institute (MNI).
3 RESULTS
3.1 VEP waveforms and topography
Chromatic-VEP waveforms at relevant electrode sites elicited by stimuli in each of the four quadrants are shown in Figure 2. Topographical features of the major components in the scalp distribution maps are shown in Figure 3. The earliest component (labeled C1) was quite small and had an onset of approximately 70 ms and a peak latency of approximately 100 ms. For upper-field stimulations, the C1 component was negative and most prominent at occipital sites ipsilateral to the midline, while for lower-fields this component reversed in polarity and was largest at occipital sites contralateral to the stimulation. After the C1, a positive deflection (P1) was elicited over contralateral ventral occipito-temporal sites with a peak latency of 140 ms. Then, with a peak latency of 175 ms, a component having a topography that resembled the C1 was also present. This component, labeled C2, shows inverted polarity respect the C1 being positive for upper fields and negative for lower fields. After the C2, a negative deflection (N1) was elicited over contralateral parieto-occipital sites with a peak latency of 220 ms. At about 270 ms a third medial activity was termed as C3, because of its inverting polarity. This component is difficult to distinguish being almost concomitant with the N1 at 250 ms for upper fields and with the P2 at 300 ms for the lower fields. Finally, at 300 ms peak latency, the large positive P2 dominated the waveforms. This component peaked on slightly contralateral occipital sites. The P1, the N1, and the P2 did not invert in polarity for upper versus lower field stimuli.
Grand averaged waveforms of the chromatic VEP in each of the four quadrants at medial occipital (O1/2) and lateral parieto-occipital electrodes (PO7/8) [Color figure can be viewed at wileyonlinelibrary.com]
Spline-interpolated 3D voltage maps of chromatic VEP components found in the grand averaged waveforms in the four quadrants [Color figure can be viewed at wileyonlinelibrary.com]
Luminance-VEP waveforms elicited at relevant electrode sites are shown in Figure 4, and topographical features of the found components in the scalp distribution maps are shown in Figure 5. In Figure 4 left and right visual fields are collapsed together and only upper and lower hemifields are shown at medial and contralateral sites. For luminance stimuli, the C1 had an onset of approximately 60 ms and a peak latency of approximately 85 ms. The C1 was earlier and larger for luminance, but the polarity and distribution of this component was quite similar to chromatic VEP. The luminance P1 was elicited over contralateral parieto-occipital sites with a peak latency of 110 ms. The C2 and the C3 components peaked at 160 and 280 ms, respectively, with a topography similar to chromatic VEP. The N1 was elicited over contralateral occipito-parietal sites with a peak latency of 190 ms. The P2 peaked at 265 ms on contralateral occipital sites. The P1, the N1, and the P2 did not invert in polarity for upper versus lower field stimuli. Statistical analysis showed a general main effect of condition on response latency with all chromatic VEP components delayed compared with luminance VEP (F1,29 > 9.14, p < .0052) independently by the stimulus position which was not significant (F3,87 < 3.56, ns). In addition, the P2 amplitude was lower for chromatic than luminance VEP (F1,29 > 12.52, p < .0020) for all stimulus positions. Other effects and interaction were not significant.
Comparison between chromatic and luminance VEP in upper and lower fields at medial occipital and contralateral parieto-occipital scalp sites [Color figure can be viewed at wileyonlinelibrary.com]
Spline-interpolated 3D voltage maps of luminance VEP components found in the grand averaged waveforms in the four quadrants [Color figure can be viewed at wileyonlinelibrary.com]
3.2 Group fMRI activations
Figure 6a shows contralateral activations produced by chromatic stimuli in each quadrant superimposed on the right and left flattened cortical surfaces of the anatomical template (PALS). This figure also shows the group-averaged location of the motion-sensitive region (MT+; white outline boxes) that was functionally mapped by additional scans as described in the Methods.
VEP-fMRI combination of responses to chromatic stimuli for the four quadrants. (a) Group-averaged contralateral fMRI activations superimposed on the flattened hemisphere (occipital lobe) of the PALS template. The color scale indicates the statistical significance of the fMRI activations. Major sulci (dark gray) and regions are labeled as follows: Parieto-Occipital sulcus (POs), IntraParietal sulcus (IPs), Superior Temporal sulcus (STs), Collateral sulcus (COs), Fusiform gyrus (Fus.gy), Calcarine fissure (Calcarine), Lateral Occipital Region (LOR) and Middle Temporal motion region (MT+). The dashed outline surrounding region MT+ represents the group-averaged location of the motion-sensitive cortex based on separate localizer scans. (b) Source waveforms of dipoles seeded to the found fMRI activations for upper and lower fields. (c) Group-averaged ipsilateral fMRI activations superimposed on the flattened hemisphere (occipital lobe) of the PALS template. Other labels and logos are as in Figure 6a [Color figure can be viewed at wileyonlinelibrary.com]
In the group-averaged data, fMRI activations for chromatic stimuli were observed mainly in a set of posterior visual cortical areas of the contralateral hemisphere. These included regions of the Calcarine fissure, the inferior occipital cortex (fusiform gyrus and collateral sulcus), the lateral occipital region (LOR) and the temporal cortex (well corresponding to MT+). While in MT+, Fusiform and LOR regions the fMRI activation for upper and lower substantially overlaps, in the Calcarine fissure and in other main extrastriate visual areas it is visible a certain degree of spatial segregation. When the upper visual fields were stimulated, the activations were more prominent in the foveal representation of the ventral cortical areas (V1v in the lower bank of the Calcarine fissure, V2v and VP), whereas for lower visual field stimulation activations were produced mainly in the foveal representation of dorsal cortical areas (V1d in the upper bank of the Calcarine fissure, V2d and V3). Table 1 gives the Talairach coordinates (converted from MNI) of the centroids of these chromatic-related activations. Figure 6c shows the group-averaged ipsilateral fMRI activations. This ipsilateral activity was much weaker and inconsistent than the contralateral one. It was not present in V1 and only partially in MT+ (1/4 Hemispheres, Hs), LOR (2/4 Hs), and Fusiform gyrus (3/4 Hs). These results are coherent with previous fMRI evidence (e.g., Tootell et al., 1997; Smith & Muckli, 2010) showing that ipsilateral responses can be seen in higher-order areas (as V3A and VP) while they are nearly absent in early visual areas (as V1 and V2). Although interesting, the fMRI ipsilateral activity was not used to constrain the source localization. Indeed, all our results are based on the contralateral fMRI and EEG responses.
| Upper right | X | Y | Z | Upper left | X | Y | Z |
|---|---|---|---|---|---|---|---|
| Calcarine (V1) | −9 | −97 | −1 | Calcarine (V1) | 7 | −96 | −3 |
| MT+ | −45 | −67 | 6 | MT+ | 48 | −61 | 6 |
| LOR | −27 | −88 | 2 | LOR | 29 | −88 | 1 |
| Fusiform | −42 | −73 | −4 | Fusiform | 39 | −69 | −9 |
| Lower right | X | Y | Z | Lower left | X | Y | Z |
|---|---|---|---|---|---|---|---|
| Calcarine (V1) | −8 | −99 | 5 | Calcarine (V1) | 11 | −96 | 10 |
| MT+ | −45 | −70 | 7 | MT+ | 45 | −61 | 7 |
| LOR | −30 | −90 | 5 | LOR | 30 | −90 | 10 |
| Fusiform | −42 | −76 | 6 | Fusiform | 37 | −66 | 4 |
- Coordinates are given for contralateral activations in response to stimuli in each of the four conditions (values are in mm).
3.3 Single subject fMRI activations
Stimulus-evoked fMRI activations were localized for each subject with respect to the retinotopically organized visual areas, defined based on their calculated field sign, and to motion region MT+ as defined by individual functional localizer (see “Methods” section). The borders of retinotopically organized visual areas (V1d, V2d, V3, V3A, V7, V6, V6Av, V1v, V2v, VP, V4v, and V8/VO) and the border of MT+ were identified for each participant. Using this fine mapping method, the activations in striate and extrastriate visual areas could be distinguished despite their proximity and individual differences in cortical anatomy. Figure 7a shows the color-related fMRI activations in response to contralateral stimuli in each quadrant projected onto the flattened left and right hemispheres of an individual participant.
Individual contralateral (a) and ipsilateral (b) fMRI activations projected onto the flattened left and right hemispheres of a representative subject. Activations in response to contralateral and ipsilateral stimuli in each quadrant are shown in relation to the boundaries of both visual areas, defined in the same subject by retinotopic mapping, and motion region MT+ individually mapped by functional localizer (and indicated by a white outline). As indicated in the semicircular logos, dashed and solid lines correspond to vertical and horizontal meridians, respectively. Major sulci (dark gray) are labeled as follows: horizontal intraparietal sulcus (hIPs), Central sulcus (Cs), Insula (Ins.), Middle Temporal sulcus (MTs), inferior temporal sulcus (ITs). Other labels and logos are the same as those described in Figure 6 [Color figure can be viewed at wileyonlinelibrary.com]
Individual results showed that the most systematically activated cortical visual areas are the shared foveal band of dorsal and ventral V1 (83% of subjects) and the foveal representation of the color center V8/VO (89% of subjects). Area V8 was originally defined as a ventral hemifield within the collateral sulcus and extending onto the fusiform gyrus, with a highly color-selective fovea (Hadjikhani et al., 1998). This ventral visual area is just anterior to the horizontal meridian representation that marks the anterior border of area V4v (Sereno et al., 1995). The location of the fusiform color map found here is compatible with the original description of V8 (Hadjikhani et al., 1998) but also with the more recently defined VO-1, and possibly part of VO-2 (Brewer et al., 2005). Thus, in this article, we call the “V8/VO-1/VO-2” region V8/VO to acknowledge both models.
Like dorsal V3A, this ventral area represents the entire contralateral hemifield, with superior and inferior portions (on the flat map) representing the lower (−) and upper (+) visual fields, respectively (Hadjikhani et al., 1998; see also Di Russo et al., 2002, 2003, 2005). Activation of a region corresponding to this ventral area was also evident in group data shown in Figure 6. In V1 and V8/VO we found a substantially overlapping fMRI activation for upper and lower quadrants with only weak signs of spatial segregation between different portions of the visual field. Similarly, we observed overlapping fMRI activation between upper and lower quadrants also in other regions as the motion-sensitive region MT+ (81%), the horizontal segment of the IPS (hIPS; 67%), the LOR (78%) as well as a region in the superior temporal sulcus (61%). These regions do not show signs of spatial segregation between upper and lower since they are always equally activated.
Finally, perception of chromatic stimulation activated also other dorsal and ventral visual areas in accordance with their retinotopic representation of lower and upper portions of the visual field. When chromatic stimuli were in upper quadrants, functional responses were found in ventral area V2 as well as in areas VP, V4v, V3A, and V7. This pattern of activations was consistently observed in most subjects: V1v (89%), V2v (100%), VP (72%), V4v (78%), V3A (83%), and V7 (94%). When chromatic stimuli were in the lower quadrants, activations were consistently produced in dorsal areas V2d (83%), V3 (83%) V3A (78%), and V7 (56%). Note that V3A and V7 representing the entire contralateral hemifield are consistently activated for both upper and lower stimuli. Although less consistently, activation was also found in the retinotopic motion areas V6 (11%) and V6Av (28%), in accordance with their retinotopic representation of the lower visual field which occupies the more lateral part of area V6 (Fattori, Pitzalis, & Galletti, 2009; Pitzalis et al., 2006) and the entire area V6Av (Pitzalis, Sereno, et al., 2013).
Figure 7b shows the ipsilateral fMRI individual activations. Like for the group data, this ipsilateral activity was much weaker and inconsistent than the contralateral one. It was not present in V1 but mainly in the shared foveal band and in the STs (4/4 Hs) and only partially in MT+ (3/4 Hs), V7 (3/4 Hs), LOR (2/4 Hs), and Fusiform gyrus (2/4 Hs). Although individual maps do not have the same robustness as the average maps, it is worth noting that the ipsilateral response profile observed in the group maps (Figure 6c) also survives at the individual level.
3.4 VEP-fMRI combination
Regions of interest were selected by clustering fMRI spots from the average fMRI maps (Figure 6), and resulting coordinates (Table 1) were used to create fMRI based (seeded) models with the source orientations optimized to those locations. Figure 6b shows the VEP-based time-courses (dipole moment) of the fMRI seeded locations (Figure 6a) for upper and lower quadrant stimuli (left and right responses were collapsed together to increase VEP signal-to-noise ratio). Operatively, first, four regional dipoles that included three orthogonal orientation were seeded (fixed in location) at the fMRI coordinates showed in Table 1. Then, the orientation of each seeded source was released and fitted for each of the found VEP components (C1, P1, C2, N1, C3, and P2) using time windows in which each source showed the maximal global field power. Finally, components were assigned to one or more sources that allowed the best fit. Looking at source time-course, for both upper and lower field, the best four-sources model was built associating the C1, C2, and C3 to V1, the P1 to LOR, the early N1 to Fusiform (V8/VO) and the late N1 to MT+, and the P2 to V8/VO. Among all possibilities, this VEP-fMRI association yielded the lowest RV. The time-course of the V1 source (initially fitted between 80 and 120 ms) well accounted for the C1, peaking at 100 ms and inverting in polarity between the upper and lower hemifields. Furthermore, this same source also accounted for the C2 and the C3 which inverted in polarity like the C1 (at 175 and 300 ms). The time course of the LOR source (initially fitted between 120 and 160 ms) well accounted for the P1 component peaking at 140 ms; a second peak of opposed polarity was present at 200 ms. The time course of the fusiform (V8/VO) source (initially fitted between 180 and 220 ms) well accounted for the early part of the N1 component peaking at 210 ms; a second peak of opposed polarity was present at 280 ms explaining the P2. The time-course of the MT+ source (initially fitted between 220 and 260 ms) accounted for the late part of the N1 component showing a peak at 250 ms; a second small peak of opposed polarity was visible at 350 ms. Note that the association of the late N1 with the MT+ dipole was made after that the fusiform dipole explained the N1 and the P2, but the MT+ dipole still showed a peak activity in the N1 range. We did not force any association between brain areas and a specific VEP component, but we rather studied the time courses of the seeded sourced and suggested functional links between them. Moreover, as final step, to optimize and to test the model stability, all four-source model orientations were simultaneously fitted in the 70–350 ms time range, the model for the upper and lower fields explained 97.3 and 98.0% of the variance, respectively.
Although source localization is based on the group average fMRI maps, the single-subject retinotopic mappings enable us to define the position of each fMRI spots in the individual maps respect to that of known visual areas. From this, we can infer, with a certain probability, the position of the average fMRI spots as already done by our group in the past (Di Russo & Pitzalis, 2014; Di Russo et al., 2002, 2003, 2005, 2007, 2012; Pitzalis, Fattori, et al., 2012; Pitzalis, Strappini, et al., 2012; Pitzalis, Bozzacchi, et al., 2013; Pitzalis, Sereno, et al., 2013; Pitzalis, Sdoia, et al., 2013). When a retinotopic mapping is available, this method is more accurate respect to see how the results hold against a probabilistic atlas. Thus, based on single-subject mappings like those shown in Figure 7, the group-averaged VEP-fMRI activations may be likely assigned here to known visual areas, even though it should be recognized that the labeling of the average cluster is probabilistic, and some imprecision and ambiguity can arise. For example, activity in cortical areas not included in the model are likely explained (absorbed) by the four involved sources. Activations in the Calcarine region (estimated source of the C1, C2, and C3) appear to originate primarily from the foveal representation of area V1, although contributions from V2 and V3 areas cannot be excluded because their proximity and the presence of similar (but not identical) anatomo-functional features such as source orientations variations across patches of cortex activated by relatively large, foveal stimuli as those used in the present study (Ales, Yates, & Norcia, 2010, 2013). However, studies on the C1 topography as those by Kelly, Vanegas, Schroeder, and Lalor (2013) reinforced the hypothesis that area V1 is the dominant generator of the C1 (for a review see also Di Russo & Pitzalis, 2014) (Figure 8).
Comparison between luminance and chromatic source-waveforms. VEP responses to upper and lower quadrants in the four areas included in the model of Figure 6 (V1, LOR, Fus.g, MT+) and in area V3A [Color figure can be viewed at wileyonlinelibrary.com]
The mid-temporal activations may be assigned to the motion-sensitive region MT+ (estimated source of the late part of N1). The group activations in the fusiform gyrus and collateral sulcus appear to correspond to the color-sensitive region V8/VO (estimated source of the early part of N1 and P2).
Considering that a previous study reported the contribution of a dorsal visual area (including V3A) in chromatic human perception (Castaldi, Frijia, Montanaro, Tosetti, & Morrone, 2013), we calculated two secondary models to see the time courses of each visual region (i.e., V1, V8/VO, MT+, LOR, V3A) for both luminance and chromatic stimuli to verify the presence of V3A activity in both conditions. We added V3A in both models because this area was already included in previous studies modeling luminance contrast ERP sources (e.g., Di Russo & Pitzalis, 2014) and because it is sufficiently far from the other modeled areas. We did not include V2, V3, and V7 because there were not specific hypotheses on these areas, but also because they were too close to the other mode did not force led areas risking cross-talk interference. Figure 9 shows the results indicating that area V3A (initially fitted from 100 to 200 ms for luminance and from 120 to 220 ms for color) did contribute to chromatic VEP with an activity initiating at 170 ms and peaking at 200 ms. In the primary model (without V3A, Figure 6), this activity was explained by the later part of the second peak within the LOR (initially fitted in the same range of V3A). In general, chromatic stimuli elicited 15–20 ms delayed responses and larger activity in V1 (C2 stage), LOR (at 130 ms), and Fusiform gyrus (210 ms). On the other hand, V3A and MT+ were larger for luminance stimuli during the whole studied period. In the 70–350 ms time range, after a general orientation fit of all sources, the model for the upper and lower fields explained 97.5 and 98.3% of the variance, respectively for chromatic stimulation and 96.2 and 97.1% of the variance, respectively, for luminance stimuli.
Exemplificative model of the spatiotemporal pattern of the six chromatic VEP components arising from occipital regions V1, V8/VO, MT+, and LOR. Time in ms indicates the peak latency of the found component and blue numbers indicate their temporal order. Black and red lines indicate feed-forward and reentrant feedback signals, respectively [Color figure can be viewed at wileyonlinelibrary.com]
4 DISCUSSION
This study was aimed at localizing the main sources of VEPs produced by pure chromatic stimuli. The present co-registration study was obtained by combining temporal high-resolution ERP measures with the spatial high-resolution fMRI data, integrated with retinotopic and motion mapping data at the individual level. This mapping procedure enabled us to functionally link, with a certain probability, the chromatic VEP components within known visual areas and anatomical regions with a known functional profile, as previously demonstrated by our group for other VEP paradigms (Di Russo & Pitzalis, 2014; Di Russo et al., 2002, 2003, 2005, 2007, 2012, 2016; Pitzalis, Fattori, et al., 2012; Pitzalis, Strappini, et al., 2012; Pitzalis, Bozzacchi, et al., 2013; Sulpizio et al., 2017).
4.1 Feed-forward and reentrant feedback signals in the chromatic VEPs: A complex spatiotemporal profile
We found a sequence of occipital activities more complex than those typically reported before (Celesia, 2005), including feed-forward and reentrant feed-back signals as summarized in the schematic model shown in Figure 9. The processing in the cortical network started in V1 approximately 75 ms after the stimulus onset (C1 component, peak activity at 100 ms), then the activity was detected in the LOR at 140 ms (P1), again in V1 at 175 ms (C2), and then in V8/VO at 210 ms (early N1), in MT+ at 250 ms (late N1), and in V8/VO again at 280 ms (P2). Finally, in V1 for the third time at 300 ms (C3). As any brain model, there are some limitations as the fact that more areas than those considered may be present even if in lower extent.
Recording the C1 component, our results confirm that this component is mainly localized in area V1 as evident from the extensive VEP literature on C1 (see Introduction). The C1 has been found by our group, using luminance stimuli in the four quadrants, in many visual stimulation paradigms, such as pattern-onset, pattern-reversal, motion-onset, and steady-state (Di Russo et al., 2002, 2005, 2007, 2012). Therefore, the C1 seems to be a ubiquitous phenomenon related to the activity of the V1 area and likely reflects the cortical volley from the lateral geniculate nucleus. However, it cannot be excluded, given the spatial contiguity between the areas, that the C1 component reflects also some activity from V2 and V3 as response to the chromatic stimuli. In this regard, it is possible to speculate that the peak latency of C1, reflecting also the contribution of V2 and V3, is later than the peak latency of a “pure” V1 dipole.
After a first feed-forward activation, V1 is again activated, at least two more times (at 175 and 270 ms). These latter activations presumably represent re-entrant feedback signals from higher-level areas (Di Russo et al., 2007, 2012), which were also much stronger than the first one (the C1). So far, evidence that the chromatic VEP reflects V1 activity was mainly speculative. In fact, for several studies the presence of VEPs activity over the occipital cortex along the midline, was sufficient to indicate that the signals come from V1 (Foxe et al., 2008; Kulikowski et al., 2002; Nakamura et al., 2000; Onofrj et al., 1995; Xing et al., 2015). Highsmith and Crognale (2010) proposed that early chromatic VEP should arise from V1 because of the lack of attention effect on the C1 component (e.g., Baumgartner et al., 2017; Di Russo et al., 2003, 2012; for a review see Ding, Martinez, Qu, & Hillyard, 2014). Some studies challenged this view indicating the presence of attention effects on the C1 component (Kelly, Gomez-Ramirez, & Foxe, 2008; Rauss et al., 2009), although this result was never further confirmed. More importantly, a recent study that carefully replicated the same experiment and analysis strategies as in Kelly et al. (2008) found no evidence for an attention-based modulation of the C1 (Baumgartner et al., 2017). Furthermore, normal and reliable chromatic VEPs have been recorded in cases of cerebral achromatopsia. In these patients although color appearance was lost and lesions were observed in ventromedial extrastriate cortex, V1 responses to color were unaffected (Crognale, Duncan, Shoenhard, Peterson, & Berryhill, 2013; Victor, Maiese, Shapley, Sidtis, & Gazzaniga, 1989). The combined evidence from source localization, lack of attentional effects, and cerebral achromatopsia supported at speculative level that early VEP activity should be an index of V1 responses to color. Here, with a combined VEP-fMRI retinotopic study, we show evidence of the role of V1 area as one of the major chromatic VEP stations.
While the C1 component is mainly associated to V1, the localization of the P1 component strictly depends on the experimental conditions. For luminance stimuli, this component has been localized in area V3A here as well as in our previous studies (Di Russo et al., 2002, 2005, 2007, 2012). In contrast, for chromatic stimuli as those used here, the P1 was localized in the LOR.
The early part of the N1 was produced by the V8/VO with a peak activity at 210 ms. So far there were no chromatic VEP studies that localized one of their components in the fusiform gyrus with the support of fMRI and high-resolution EEG recording. Previous work has suggested that chromatic VEP must originate early in the cortical pathways (e.g., Highsmith & Crognale, 2010) but the few electrodes used and the lack of information regarding the source not enabled reliable VEP localization.
The later part of the N1 was associated with the motion-sensitive region MT+. Even though this region has been previously found for motion-onset VEP (Di Russo et al., 2012), its latency was much shorter than here, ranging from 70 to 100 for motion stimuli and 250 ms for chromatic stimuli. This delayed intervention of MT+ may be explained by the activity of the color system that is notoriously slower than the magnocellular pathway. This delay is particularly enhanced comparing motion to color stimuli (more than 150 ms) but is also present for all the other components (and relative brain areas) if chromatic and luminance pattern-onset VEP are compared. The earlier V1 activation (C1) is minimally delayed for color stimuli (about 15 ms), while the other components showed about 30 ms of delay for color stimuli.
Overall, comparing present VEP data with literature is evident that chromatic VEP is more complex than previously thought. Instead of the two components (the N1 and the P2) reported so far (e.g., Celesia, 2005), we showed the presence of at the least six components arising from the occipital cortex. The N1 and the P2 are the largest components, but earlier than them the C1, the P1, and the C2 are also present. This result indicates a similar complexity as for luminance stimulation but with a stronger involvement of V1 (C1–C3) and V8/VO (N1 and P2), which is not present for pattern-onset luminance stimulation. On the other hand, parietal activity associated to the early N1 found for luminance was not detectable here (Di Russo et al., 2002). The direct comparison made between chromatic and luminance stimulation (Figure 9) confirmed the literature showing larger and late V1, early LOR, and V8/VO activity for chromatic stimulation. Luminance stimulation produced large activity in late LOR, V3A, and MT+. The presence of weak V3A activity for chromatic stimulation has been confirmed also at VEP level. Compared with red-green chromatic stimulation, the yellow-black luminance stimuli elicited smaller but faster V1 response. This result indicates a larger magnocellular contribution, thus confirming that the stimulus is suitable as a luminance condition. However, since the presence of color signals (yellow), the luminance stimulus likely elicits some activity also in neurons that respond to colors. For this reason, an important caveat in interpreting the differences between chromatic and luminance stimuli is that they are ideal to set apart parvo- and magnocellular systems (which are able to detect chromatic and luminance contrast, respectively) rather than color from non-color brain cells.
It should be cautioned that the use of hemodynamic imaging to substantiate the estimated locations of ERP sources, as done also in other studies although without retinotopic mapping (e.g., Bonmassar et al., 2001; Heinze et al., 1994; Mangun et al., 2001; Martinez et al., 1999, 2001; Snyder, Abdullaev, Posner, & Raichle, 1995; Vanni et al., 2004), is subject to certain caveats (Dale & Halgren, 2001; Rosa, Daunizeau, & Friston, 2010). First and foremost, there is the assumption that the hemodynamic response obtained with fMRI is driven by the same neural activity that gives rise to the ERP. Regarding the visual-evoked activity, such a correspondence appears to be optimal for human medial occipital cortex (including the Calcarine fissure) and is less definite for extrastriate visual areas (Gratton, Goodman-Wood, & Fabiani, 2001). Moreover, it stands to reason that more accurate source models can be achieved for the initial than for the subsequent VEP activity, even though both early and late components receive contributions from multiple, spatially and temporally overlapping cortical generators (Ales et al., 2010; Hagler et al., 2006). As suggested by Hagler (2014) using magnetoencephalography (MEG) and MRI/fMRI methods, in the first 100 ms post-stimulus, MEG receives contributions mainly from V1, but in less extend also from V2 and V3. Another limitation is the interpretation of those components defined as re-entrant activity (the C2 at 160 ms, the C3 at 270 ms and the P2 at 300 ms), that could represent feedforward activity evoked by stimulus off-set (off response). However, in this study, the 400 ms stimulus duration excludes this interpretation. Finally, a further potential limitation in using close dipole locations is the “leakage” or “cross-talk” of the estimates among brain areas (e.g., Hauk & Stenroos, 2014) that constrain the proximity and dipoles number contained in a model.
4.2 Cortical areas involved in color perception
Although colors perception is a fundamental component of primate vision, physiological and neural bases of color perception is still matter of debate. Indeed, during the last years, it has been under question whether we have a unique color center in the brain or whether other visual areas, as the primary visual area V1, concur to the color perception. As well as it is still unclear the sensitivity to colors of the dorsal motion regions as MT+. Present results add new evidence on both these issues.
4.3 Is there a unique color center in the brain?
Early single-unit studies on monkeys (e.g., Lueck et al., 1989; Zeki, 1983a,1983b,1983c,1983d) and neuroimaging studies on normal subjects (e.g., Castaldi et al., 2013; Hadjikhani et al., 1998) suggested that one ventromedial cortical area located in the collateral sulcus (V8 plays a specific role in color perception. Some of these authors pointed to consider this cortical region as the only color center in the brain where the color awareness is accomplished. Support to this notion came also from clinical studies on achromatopsic patients with selective loss of color vision due to damage in the lingual and caudal fusiform gyri where the so-called “color center” is located, sparing V1 (Bartels & Zeki, 2000; Beauchamp, Haxby, Rosen, & DeYoe, 2000; Damasio, Yamada, Damasio, Corbett, & McKee, 1980; Merigan, Freeman, & Meyers, 1997; Zeki, 1990). In a landmark statement, Crick and Koch (1995) argued against primary visual cortex being the site for color consciousness, citing the fact that V1 neurons did not display the properties necessary for color constancy, a cornerstone of our conscious experience. The general view was that neurons having such properties do not occur until area V4 or beyond, where the receptive fields are sufficiently large (Schein & Desimone, 1990).
This view was then challenged by numerous experimental human and macaque evidence speaking in favor of V1 role in color vision (e.g., Gegenfurtner & Kiper, 2003; Hanazawa, Komatsu, & Murakami, 2000; Johnson, Hawken, & Shapley, 2001). For example, macaque V1 neurons are influenced by light outside their receptive fields (MacEvoy & Paradiso, 2001; Wachtler, Sejnowski, & Albright, 2003), and perform important steps in computing colors also within their receptive fields (Conway, 2001; Johnson et al., 2001). The idea of high color-selectivity in foveal V1 is consistent with several pieces of evidence such as data from the distribution of cones in the retina (Curcio, Sloan, Packer, Hendrickson, & Kalina, 1987) and reports confirmed the existence of double-opponent cells (thought to provide the basis for color constancy) in the primary visual cortex (Tso & Gilbert, 1988) and not only in V4 (as previously suggested, see Zeki, 1983a,1983b,1983c,1983d).
Here we showed for the first time that the neural generators of the main components of the color VEP massively involve both V1 and V8/VO, with feed-forward and multiple reentrant feedback signals in these two visual regions which processes color information in three and two temporal windows, respectively (Figure 9). The presence of multiple peak activities in V1 shown here reaffirms the importance of the primary visual areas in the color perception not simply as the first visual station playing a nonspecific role but as one the most important neural correlate of color vision. Our results are consistent with fMRI human studies showing a larger V1 response to a red-green chromatic stimulus than to a luminance stimulus (Engel, Glover, & Wandell, 1997; Hadjikhani & Tootell, 2000; Kleinschmidt, Lee, Requardt, & Frahm, 1996). Note that a color-selective activation in foveal V1 was always present also in those studies showing an involvement of the fusiform gyrus (Hadjikhani et al., 1998). The role of V1 in color perception has been further confirmed in fMRI studies on humans on the chromatic adaptation (Engel & Furmanski, 2001; Engel et al., 1997; Mullen, Chang, & Hess, 2015), on chromatic spatial features (Castaldi et al., 2013) and on the generation of color after-images (Livingstone & Hubel, 1984; Michael, 1978; but see Hadjikhani et al., 1998 for color after-images in V8).
Overall, evidence from monkey and human studies reviewed above together with present combined VEP-fMRI results suggests that V8/VO is not the only color brain center and that also V1 and other ventral occipitotemporal areas (including V2) contribute in human perception for color.
4.4 Color signals in dorsal motion regions
Present results showed that neural generators of main chromatic VEP components involve not only the ventral stream (foveal V1 and V8/VO) but also some motion-sensitive regions of the dorsal stream, as observed in some previous fMRI experiments (Castaldi et al., 2013; D'Souza, Auer, Strasburger, Frahm, & Lee, 2011; Liu & Wandell, 2005; Mullen, Thompson, & Hess, 2010).
A first activity was detected in the LOR, a region overlapping retinotopic dorsal areas, such as V3B (Smith, Greenlee, Singh, Kraemer, & Hennig, 1998) and V4d (Tootell & Hadjikhani, 2001). LOR is known mainly for its motion sensitivity (e.g., Tootell and Hadjikhani, 2001; Pitzalis et al., 2010; ; Pitzalis, Fattori, et al., 2012; Pitzalis, Strappini, et al., 2012) and is also part of the kinetic occipital motion-sensitive region described by Orban's group (Dupont et al., 1997; Orban et al., 1995; Van Oostende, Sunaert, Van Hecke, Marchal, & Orban, 1997). While in the conspicuous literature about visual motion there are a lot of references to LOR, in the research field about color vision we can only find very few studies about the involvement of this region. For instance, Zeki, Perry, and Bartels (2003) found that the cortex between V3d and V5 (where LOC is located; Morrone et al., 2000; Smith et al., 1998; Tootell & Hadjikhani, 2001; Van Oostende et al. 1997) responded with equal selectivity to shapes defined by color contrast or motion contrast. Later, Self and Zeki (2005) demonstrated, using a fMRI adaptation technique, that color and motion cues are integrated into the LOC, and this increases the ability of the observer to recognize objects defined by more than one visual cue. In addition, Castaldi et al. (2013) measured BOLD selectivity to patterns, modulated in luminance and chromatic contrast, with different phases but matched amplitude spectra. Interestingly, they found a widespread activation in several regions including the dorsal LO which showed a preference for congruent-phase stimuli and for odd-symmetry stimuli, both for luminance and color.
A later activity was detected in the lateral motion sensitive complex MT+. In the fMRI experiment, BOLD activation in MT+ was found very consistently across subjects (81%). Early reports suggested that MT+, perhaps the most important dorsal area for the analysis of visual motion, lacks color selectivity (Zeki, 1983c). In accordance with this idea, responses of neurons within MT+ are lower to moving equiluminant stimuli than to moving stimuli with luminance contrast (Riecansky, Thiele, Distler, & Hoffmann, 2005; Seidemann, Poirson, Wandell, & Newsome, 1999; Thiele, Dobkins, & Albright, 2001). These and other observations (e.g., Mullen et al., 2015) have led to the notion that MT+ is colorblind. But MT+ is not entirely unresponsive to chromatic signals. Albright and collaborators, using a combination of physiological and psychophysical techniques, later showed that color information does contribute to the responses of MT+ neurons (Croner & Albright, 1999; Dobkins & Albright, 1994; Thiele, Dobkins, & Albright, 1999). Moreover, Gegenfurtner et al. (1994) showed that most MT+ neurons do respond to chromatic variations, although much less vigorously than to luminance modulation. Thus, it seems to be clear that “color signals” penetrate MT+, but it is unlikely that such signals encode hue or contribute to color perception. It seems more likely that MT+ cells are sensitive but not selective to color (e.g., Wandell et al., 1999; see for review Conway et al., 2014; Gegenfurtner and Kiper, 2003). Note that the distinction between sensitivity to color and selectivity for color is crucial: while color sensitivity is evident in many neurons across many brain regions, color selectivity seems relatively rare.
In the individual fMRI data, we observed color-related activations not only in MT+ and LOR but also in the motion-sensitive regions V6 and V6Av in the parieto-occipital sulcus. While the activations observed in MT+ and LOR were reliable and very consistent across subjects (81 and 78%, respectively), the activations in areas V6 and V6Av were observed only in a minority of subjects (11 and 22%, respectively) and were indeed not visible on the average maps (and for this reason they do not have a VEP counterpart). These two regions are motion sensitive in both macaque (e.g., Galletti, Fattori, Battaglini, Shipp, & Zeki, 1996) and human data (Pitzalis et al., 2010, 2015; Pitzalis, Sereno, et al., 2013; Pitzalis, Sdoia, et al., 2013) and there are so far no studies in both species speaking about the involvement of these areas in the color vision. Here we showed that these two motion areas are weakly (if any) activated by color visual attribute of the stimulus. Monkey (Galletti et al., 2001) and human data (Pitzalis, Bozzacchi, et al., 2013) support the presence of direct connections between the medial motion area V6 and the lateral motion region MT+, and it is possible that the weak color-related signals we observed in area V6 might be inherited from MT+.
4.5 Conclusive remarks
Here we combined multiple neuroimaging and neurophysiological methods to describe the complex spatiotemporal pattern of cortical response induced by purely chromatic stimulation. We found a complex interaction between striate and extrastriate visual areas with feedforward and feedback processing. Chromatic processing strongly involved V1 and V8/VO, but also included dorsal regions. In summary, we confirm that there is not a unique color center in the brain and that color vision produces a multifaceted neural interaction between many different (dorsal and ventral) areas.
