Volume 26, Issue 7 pp. 2074-2084
Full Access

Spatial and temporal analysis of fMRI data on word and sentence reading

Sven Haller

Sven Haller

Institute of Radiology, Department of Neuroradiology, and

Department of Diagnostic Radiology, University Hospital Basel, Petersgraben 4, CH 4031 Basel, Switzerland

Search for more papers by this author
Markus Klarhoefer

Markus Klarhoefer

Institute of Radiology, Department of MR Physics, Basel, Switzerland

Search for more papers by this author
Jens Schwarzbach

Jens Schwarzbach

Center for Mind/Brain Sciences, University of Trento, Rovereto (TN) 38068 Italy

Search for more papers by this author
Ernst W. Radue

Ernst W. Radue

Institute of Radiology, Department of Neuroradiology, and

Search for more papers by this author
Peter Indefrey

Peter Indefrey

FC Donders Centre for Cognitive Neuroimaging, Nijmegen, The Netherlands

Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands

Search for more papers by this author
First published: 25 September 2007
Citations: 18
Dr S. Haller, as above.
E-mail: [email protected]

Abstract

Written language comprehension at the word and the sentence level was analysed by the combination of spatial and temporal analysis of functional magnetic resonance imaging (fMRI). Spatial analysis was performed via general linear modelling (GLM). Concerning the temporal analysis, local differences in neurovascular coupling may confound a direct comparison of blood oxygenation level-dependent (BOLD) response estimates between regions. To avoid this problem, we parametrically varied linguistic task demands and compared only task-induced within-region BOLD response differences across areas. We reasoned that, in a hierarchical processing system, increasing task demands at lower processing levels induce delayed onset of higher-level processes in corresponding areas. The flow of activation is thus reflected in the size of task-induced delay increases. We estimated BOLD response delay and duration for each voxel and each participant by fitting a model function to the event-related average BOLD response. The GLM showed increasing activations with increasing linguistic demands dominantly in the left inferior frontal gyrus (IFG) and the left superior temporal gyrus (STG). The combination of spatial and temporal analysis allowed a functional differentiation of IFG subregions involved in written language comprehension. Ventral IFG region (BA 47) and STG subserve earlier processing stages than two dorsal IFG regions (BA 44 and 45). This is in accordance with the assumed early lexical semantic and late syntactic processing of these regions and illustrates the complementary information provided by spatial and temporal fMRI data analysis of the same data set.

Introduction

Written language comprehension involves a number of processing levels that are at least in part hierarchically organized. Following Perfetti (1999), we distinguish a visual processing level, an orthographic/phonological processing level, a lexical processing level, and sentence-level syntactic and semantic processing. We apply a combined spatial and temporal analysis of functional magnetic resonance imaging (fMRI) to characterize these processing steps.

The majority of fMRI experiments analyse only the spatial distribution of activation clusters, usually in the framework of general linear model (GLM) analyses (Friston et al., 1995). This powerful technique has the advantage of high sensitivity but lacks the ability to discriminate directly without a priori knowledge between stimulus-related early low-level and task-related late high-level activations.

Temporal analyses can discriminate such activations yet have been a domain of electroencephalography (EEG) and magnetencephalography (MEG) owing to their excellent temporal resolution (order of milliseconds). The temporal resolution of fMRI is much lower but according to some authors may be of the order of a few hundred milliseconds or even less (Hernandez et al., 2002; Bellgowan et al., 2003; Formisano & Goebel, 2003). A more recent study even suggests a ‘functional’ resolution in the range of 40–100 ms (Eichele et al., 2005). Assuming this temporal resolution, it seems justified to expect that a sequence of cognitive processing steps may be reflected in timing differences of spatially localized blood oxygenation level-dependent (BOLD) responses (Menon et al., 1998; Menon & Kim, 1999), allowing the resolution of the flow of information in cortical networks (Hernandez et al., 2002). As the haemodynamic response might be confounded by regional differences in neurovascular coupling (Aguirre et al., 1998; Huettel & McCarthy, 2001; Saad et al., 2001), direct comparisons of the estimated BOLD delays and durations between brain regions cannot be used reliably to infer differences in onset and duration of neuronal responses. We avoid this limitation of between-regions variation based on the assumption of a constant local neurovascular coupling within regions (Bellgowan et al., 2003). We parametrically varied linguistic processing demands and compare task-induced within-region BOLD delay differences to obtain indirect evidence about the temporal ordering of neuronal activation between regions. We reasoned that in a hierarchical cognitive processing system, increasing task demands at lower processing levels would induce delayed onset of higher-level processes. The flow of activations can thus be inferred by sorting for task-induced delay differences.

We manipulated the complexity of written stimuli at all processing levels to induce longer processing times at all stages. We expect increasing activations of the spatial GLM data analysis in language-related areas. At the same time, we expect delayed onset of processing at higher-level areas. More specifically, given that MEG and electrophysiological data suggest that processing of written stimuli up to the word level is fast and automatic (< 200 ms; for an overview see Salmelin et al., 2000), we expected our experimental manipulation not to affect the BOLD delay in left superior temporal gyrus (STG) assumed to be involved in lexical phonological processing (Indefrey & Levelt, 2000, 2004) but to cause an increase in BOLD delay in regions involved in processes beyond the word level. In particular, we predicted the strongest increase in BOLD delay with increasing task demands in a response-related motor area. We also expected a dissociation with respect to BOLD delay increase between frontal regions that have been described as being involved in semantic and phonological processing and frontal regions involved in syntactic processing (Bookheimer, 2002).

Methods

Participants

Sixteen native German speakers (eight males, aged 21–39 years, mean 27.7, SD 5.2 years) participated in the experiment. Participants had no history of medical, neurological or psychiatric disorders. They gave written informed consent prior to participating in this study. All participants were right-handed [mean score of 17.2, ranging from 13 to 20 in the online version of the Edinburgh Handedness Inventory (Oldfield, 1971)]. They were familiarized with the tasks using a training programme with different stimuli than in the actual fMRI experiment. The study was approved by the local ethics committee of Basel, Switzerland, in accordance to the declaration of Helsinki.

Stimuli and tasks

The experiment involved three language comprehension conditions designed to yield reaction time differences due to increasing linguistic demands (see Table 1). In each condition, two stimuli were presented simultaneously on a computer screen in whole-sentence format on two lines. In the COMPLEX condition, participants were presented with two syntactically different sentences that had either a similar or a different meaning. In sentence pairs with similar meanings (50% of the stimuli), the sentences differed in voice (‘The dog chases the cat in the garden’–‘The cat is being chased by the dog in the garden’) or fronting of an adjunct denoting location (‘The dog chases the cat in the garden’–‘In the garden, the cat chases the dog’) but shared the assignment of thematic roles. In sentence pairs with different meanings, the assignment of thematic roles was reversed (e.g. ‘The dog chases the cat in the garden’–‘The dog is being chased by the cat in the garden’). The participants indicated whether the meaning of the two sentences was reversed. In the MEDIUM condition, the stimuli were sentence pairs, half of which had antonym meanings (e.g. ‘The room is bright’–‘The room is dark’) and the other half had a neutral semantic relationship (‘The room is bright’–‘The room is large’). The participants indicated whether the sentence meanings were antonym or not. In the SIMPLE condition, all words except for the adjectives of every stimulus were replaced by strings of ‘x’s. The participants indicated whether the adjectives were antonyms (e.g. ‘bright’–‘dark’) or not (‘bright’–‘large’). In all three conditions, the participants responded by pressing with the left thumb one of two keys for the answers ‘Yes’ or ‘No’. Participants were instructed to respond as correctly and quickly as possible. We used 84 stimuli for SIMPLE and MEDIUM and 60 stimuli for COMPLEX, which were divided into four different experimental sets. Each set was applied to four subjects. Stimuli requiring ‘Yes’ and ‘No’ answers were counter-balanced.

Table 1. Overview of experimental conditions
Condition and stimulus Graphemic/ Phonological Lexical Semantic Syntactic
SIMPLE
 xxx xxxx bright xx + +
 xx xxx dark xxxxxx xx
MEDIUM
 The room is bright ++ ++ +
 The room is green
COMPLEX
 The dog chases the cat in the garden +++ +++ ++
 In the garden, the dog chases the cat
  • Examples of visual stimuli and tasks are given in columns 2 and 3. The assumed recruitment of linguistic processing levels is given in the last three columns. +, low demands; ++, medium demands, +++, high demands.

Three further conditions involved covert language production tasks and will be reported in a separate publication.

As shown in Table 1, the three comprehension conditions differed both with respect to the processing levels involved and with respect to the demands at every processing level. While all conditions involved graphemic/phonological processing and retrieval of lexical meaning, the amount of linguistic material and hence the load on graphemic/phonological processing and retrieval of lexical meaning increased from SIMPLE to COMPLEX. Compared with SIMPLE, stimuli in the MEDIUM condition furthermore had syntactic structure. This was also the case in the COMPLEX condition; however, in this condition parsing of the syntactic structures was more demanding due to the use of sentences with non-canonical word orders. In the COMPLEX condition, the semantic decision could not be based on lexical meaning alone but required a sentence-level meaning representation based on the syntactic structure of the sentences.

Experimental set-up and procedure

The stimuli were projected onto a translucent screen mounted to the table of the scanner. The participants saw the stimuli via a mirror on the head coil. The visual angle was approximately 30°. The scanner room was darkened during the experiment. Stimuli were presented in a mixed design with sparse stimulus events grouped (blocked) per condition to avoid anticipatory activation related to syntactic processing in the SIMPLE condition. Each time series (run) consisted of an initial fixation cross presentation (12 s) followed by six stimulus presentation blocks corresponding to the three comprehension conditions of the present study plus the three language production conditions reported elsewhere. In each comprehension block the judgement to be performed was indicated in capital red letters for 3 s (e.g. ‘SIND ES GEGENTEILE?’‘Are they opposites?’) followed by a central fixation cross for 3 s. Thereafter, stimuli were presented in lower-case black letters against a white background for 6, 8 or 10 s (SIMPLE, MEDIUM, COMPLEX, respectively). Each stimulus was followed by a central fixation cross for 8.2 s. This jittering of stimuli presentation relative to the MR data acquisition [repetition time (TR) of 3 s] created no significant differences between conditions with respect to the stimulus onset relative to data acquisition (repeated-measures pair-wise t-tests between conditions, no correction for multiple comparisons). Blocks consisted of seven SIMPLE stimuli, seven MEDIUM stimuli or five COMPLEX stimuli. The order of blocks within runs was randomized. The total duration of each run was 11 min. The experiment consisted of three runs. Participants' responses were registered and analysed for accuracy and reaction time relative to stimulus onset.

Functional imaging

Imaging was performed on a 1.5-T scanner (SIEMENS Symphony, SIEMENS Erlangen, Germany) and functional T2*-weighted images covering the whole brain were obtained with a single-shot echoplanar pulse sequence (EPI) using the following parameters: matrix size 64 × 64, field of view (FOV) 192 × 192 mm, 30 slices, 4-mm slice thickness, no interslice gap, flip angle 90°, TR 3 s, echo time (TE) 59 ms. The first three of 330 volumes were discarded from further analysis to avoid non-steady-state effects caused by T1 saturation. After functional scanning, three-dimensional (3D) T1-weighted MPRAGE images with 1 mm3 isotropic voxel size (Matrix 256 × 256 × 176) were acquired for cortex normalization and cortex surface reconstruction.

Data analysis

Anatomical and functional images were analysed using BrainVoyager (Brain Innovation, Maastricht, The Netherlands). Anatomical scans were segmented for identification of the white–grey matter boundary, which was then used for cortical surface reconstruction and flattening. Pre-processing of functional time series consisted of 3D motion correction, interscan slice timing correction using sinc interpolation (Hajnal et al., 1995), Gaussian spatial filtering (FWHM 8 mm), high-pass temporal filtering (three cycles in the time course) and transformation into Talairach space (Talairach & Tournoux, 1988).

GLM analysis

Data were analysed with a random/mixed-effects GLM: single-subject fixed-effects analyses with the three runs as separated measurements at the first level, random-effects analysis at the second level. One predictor was created for each experimental condition by convolving the stimulus time course with the standard haemodynamic reference function (HRF) of BrainVoyager QX version 1.2 (simple gamma function, delta 2.5, tau 1.25; Boynton et al., 1996). Activations reported were corrected for multiple comparisons by using the false discovery rate (FDR) approach (Genovese et al., 2002) with a false-positive probability of q(FDR) < 0.0001 (main effects of conditions) or q(FDR) < 0.01 (contrasts between conditions). The extent threshold was 250 mm3. The core of the presented investigation is the volume of interest (VOI) analysis as described below. We analysed task-induced BOLD signal change because this simple and robust parameter is largely free of a priori assumptions for the GLM analysis. Concerning the GLM analysis, our focus was the between-condition difference analysis. We chose correction for multiple comparisons based on FDR to show all differences between conditions. The q < 0.01 implies that 1% of reported activations are false positives. A more conservative correction for multiple comparisons, e.g. familywise error, would reduce the ratio of false positives, but at the cost of increasing false negatives. This means that some true activation differences would be missed.

Time-resolved analysis

The temporal analysis is based on studies and in principle approximates a model function to the event-related average BOLD response to estimate temporal properties (Buckner et al., 1998; Menon et al., 1998; Carpenter et al., 1999; Richter et al., 2000; Liao et al., 2002; Bellgowan et al., 2003). Compared with previous investigations, we modified the model function. We used a boxcar function with the parameters delay (delta) and duration (tau) convolved with a standard canonical HRF consisting of two gamma functions (Friston et al., 1998) taken from the SPM2 package (http://www.fil.ion.ucl.ac.uk/spm/). The resulting model function was fitted to the event-related average (ERA) BOLD response relative to stimulus onset (see Fig. 1). By varying delta and tau of the boxcar function, delay (delta) and duration (tau) of the BOLD response were estimated separately for each voxel and each subject. The analysis of the preprocessed functional data was performed in MATLAB (http://www.mathworks.com). Because intraindividual variations of the BOLD response are smaller than intersubject variations (Kruggel & von Cramon, 1999; Handwerker et al., 2004), we separately calculated individual delta and tau parameter maps, which were then averaged for the group analysis. The ERA relative to the stimulus presentation was estimated for each voxel of each subject and each experimental condition by re-sampling with 200-ms resolution using a cubic spline interpolation. The amplitude of the ERA was linearly normalized to a range of 0–1. The model function was fitted to the ERAs using a multidimensional non-linear minimization function (Nelder-Mead simplex search) implemented in MATLAB.

Details are in the caption following the image

The principle of the presented time-resolved analysis. A variable boxcar function (A) convolved with a fixed haemodynamic reference function (B) is fitted to the event-related average BOLD response (C). The procedure estimates the delay (delta) and duration (tau) of the BOLD response for each voxel of each subject.

We considered the fit of the model function to the ERA as successful when the following criteria were met: (1) the estimated delta and tau were within limits corresponding to the observed reaction time differences between conditions (−2 to 4 s and 2 to 10 s, respectively); (2) the amplitude of the ERA prior to normalization was at least 0.35% of the BOLD signal change; and (3) the sum of the squared difference between ERA and fitted model function was less than 0.8.

Delay maps

In accordance with the GLM analysis, we calculated delay maps for the condition SIMPLE and delay difference maps for the comparisons MEDIUM–SIMPLE and COMPLEX–MEDIUM. For every comparison, we included all voxels where a successful fit according to the criteria described above was possible in at least half of the subjects. The resulting maps were re-imported and visualized in BrainVoyager.

Volume of interest analysis

We defined six cubic (11 × 11 × 11 mm3) VOIs. Three language-related VOIs were centred at coordinates located in areas of the left inferior frontal gyrus (IFG) according to coordinates published by Bookheimer (2002) that are involved in semantic (ventral inferior frontal gyrus, pars orbitalis, IFG VENTRAL, x = −43, y = 26, z = 0), syntactic (mid inferior frontal gyrus, pars triangularis, IFG MID, x = −45, y = 15, z = 12) and phonological (dorsal inferior frontal gyrus, pars opercularis, IFG DORSAL, x = −47, y = 8, z = 27) processing. Additionally, we defined a left superior temporal VOI (STG) centred at x = −60, y = −39, z = 10 in an area that has been related to lexical phonological processing in reading (Indefrey & Levelt, 2000, 2004). A stimulus-related VISUAL VOI was centred at the Talairach coordinates (TAL) x = −15, y = −78, z = 4 in the occipital cortex. A response-related MOTOR VOI was centred at TAL x = 40, y = −28, z = 60 in the right motor cortex (response button press with left thumb). VOIs did not overlap. For the statistical comparisons between VOIs we calculated individual means of delta and tau. In all conditions, delta in VISUAL was considered as the baseline and subtracted from the delta of the other VOIs. Delta, tau, response time and response accuracy were analysed using repeated-measures anova followed by post-hoc Newman–Keuls-corrected paired t-tests.

Results

Behavioural data

The average response latencies for the conditions SIMPLE, MEDIUM and COMPLEX were 2.61 ± 0.54, 3.55 ± 0.63 and 5.39 ± 0.59 s (mean ± SD). A repeated-measures anova showed a main effect of condition (F2,15 = 152.4, P < 0.0001). Post-hoc Newman–Keuls-corrected pairwise comparisons showed that the average response latencies differed significantly between all conditions (P < 0.001). The percentages of correct responses (accuracy) for the conditions SIMPLE, MEDIUM and COMPLEX were 94.5 ± 7.4, 92.8 ± 7.1 and 76.3 ± 16.9% (mean ± SD). There was a main effect of condition (F2,15 = 16.48, P < 0.0001). Pairwise analyses showed significant differences in accuracy between SIMPLE and COMPLEX and between MEDIUM and COMPLEX (P < 0.001 for both comparisons). No significant difference was found between SIMPLE and MEDIUM.

GLM analysis

A random/mixed-effects GLM analysis of the three experimental conditions revealed significant activations in the SIMPLE condition (relative to low-level fixation baseline) and activation increases between SIMPLE and MEDIUM and MEDIUM and COMPLEX as shown in Fig. 2 and Table 2. Note that activation increases in the COMPLEX condition may have remained undetected in some areas because of reduced sensitivity due to the smaller number of stimuli in this condition. Bilateral activation of visual areas and right sensorimotor activation were present in all conditions. Activations in the left posterior inferior frontal gyrus and the left posterior superior temporal gyrus were also present in all conditions and increased from SIMPLE to COMPLEX. There was a significant increase in the maximum BOLD amplitude between SIMPLE and MEDIUM in STG and IFG VENTRAL, between MEDIUM and COMPLEX in all VOIs except MOTOR and IFG VENTRAL, and between SIMPLE and COMPLEX in all VOIs except MOTOR. Figure 3A and Table 3 depict the amplitude of the BOLD response of the three conditions in the VOIs in which we subsequently conducted the time-resolved analysis.

Details are in the caption following the image

(A) Results of the GLM analysis for the condition SIMPLE (relative to low level fixation baseline) and the contrasts MEDIUM–SIMPLE and COMPLEX–MEDIUM. Colour-coded are t-values of significant activations superimposed on the MNI template provided by BrainVoyager QX in lateral views. Significant activation clusters are listed in Table 2. (B) Estimated delays of the time-resolved analysis for the conditions SIMPLE, and in analogy with the GLM analysis the differences MEDIUM–SIMPLE and COMPLEX–MEDIUM. The colour code represents delay (delta) in seconds relative to activation onset of VISUAL VOI. The delay is slightly negative in some visual areas in SIMPLE because the assumed BOLD delay of the HRF model function of approximately 6 s exceeded the estimated BOLD delay in the data.

Table 2. Activation clusters of the GLM analysis for the condition SIMPLE and the contrasts MEDIUM–SIMPLE and COMPLEX–MEDIUM
Size (mm3) Maximum t x y z Side Anatomical region BA
SIMPLE (FDR < 0.0001)
 300 062 24.4 1 ± 33 −30 ± 35 26 ± 24 Left/right Medial frontal gyrus 6
Left/right Middle frontal gyrus 6, 8, 9
Right Precentral gyrus 4
Right Postcentral gyrus 1, 2, 3
Left/right Superior parietal lobule 7
Left/right Inferior parietal lobule 40
Left/right Anterior cingulate 24, 32
Left/right Precentral gyrus 6
Left/right Inferior frontal gyrus 44, 45, 46, 47
Left/right Cuneus 17, 18, 30
Left/right Insula 13
Left/right Transverse temporal gyrus 41
Left/right Middle occipital gyrus 18, 19, 37
Left/right Inferior occipital gyrus 18
Left Middle temporal gyrus 21
Left/right Lingual gyrus 17
Left/right Fusiform gyrus 19, 37
SIMPLE (FDR < 0.0001) local maxima*
21.9  25  −81 −7 Right Lingual gyrus
Inferior occipital gyrus
17
18
21.9  25  −81 −7 Right Lingual gyrus
Inferior occipital gyrus
17
18
17.0  −38  −78 −5 Left Lingual gyrus
Inferior occipital gyrus
17
18
13.3  40  −28  60 Right Precentral gyrus
Postcentral gyrus
4
3
12.5  −50   7  42 Left Middle frontal gyrus 6, 8
10.5   0  5  49 Left/right Medial frontal gyrus 6
9.6  30  −52  47 Right Superior parietal lobule 7
9.3  33  22  10 Right Anterior insula 13
8.9 − 57  −15  28 Left Precentral gyrus 3
8.9  52  −13  30 Right Precentral gyrus 3
8.5  −24  −57  47 Left Superior parietal lobule 7
7.9  −30  28  11 Left Anterior insula 13
7.6  −46  15  12 Left Inferior frontal gyrus 44
MEDIUM − SIMPLE (FDR < 0.01)
 17 028 11.7 −36 ± 14 −63 ± 22 −2.7 ± 6.6 Left Middle occipital gyrus 18, 19
Left Lingual gyrus 17
Left Superior temporal gyrus 22, 42
Left Middle temporal gyrus 21, 37
Left Fusiform gyrus 19, 37
Left Transverse temporal gyrus 41
 16 290 9.2 −42 ± 8.2 96 ± 13 30 ± 1.7 Left Middle frontal gyrus 6 9
Left Inferior frontal gyrus 44, 45, 46, 47
Left Insula 13
  8790 9.0 45 ± 6.2 1.4 ± 10 45 ± 11 Right Middle frontal gyrus 6, 9
  7524 7.4 2.8 ± 5 −2.3 ± 9.1 60 ± 6.2 Left/right Medial frontal gyrus 6
  4071 7.4 24 ± 4.3 −85 ± 4 1.6 ± 6.4 Right Middle occipital gyrus 18
Right Lingual gyrus 17
  1588 6.0 −29 ± 3 −48 ± 6 45 ± 2.8 Left Superior parietal lobule 7
  1421 6.9 28 ± 2.7 5.9 ± 6.4 13 ± 3 Right Insula 13
  597 6.2 37 ± 3.2 −46 ± 3.9 −12 ± 4.6 Right Fusiform gyrus 37
  518 5.9 47 ± 2 20 ± 4.3 −0.71 ± 1.8 Right Inferior frontal gyrus 47
  328 5.6 45 ± 1.9 −23 ± 4.4 11 ± 3.5 Right Superior temporal gyrus 22
Right Transverse temporal gyrus 41
COMPLEX –MEDIUM (FDR < 0.01)
 73 556 15.8 −3 ± 21 −65 ± 19 6.5 ± 17 Left/right Precuneus 7, 19, 31
Left/right Cuneus 17, 18, 30
Left/right Lingual gyrus 18
Left/right Middle occipital gyrus 18, 19
Left/right Inferior occipital gyrus 18
Left/right Fusiform gyrus 19, 37
 15 141 10.6 −40 ± 8.6 3.8 ± 12 43 ± 12 Left Middle frontal gyrus 6, 9
Left Inferior frontal gyrus 44, 45, 46
  6751 11.2 −1.7 ± 5.3 3.4 ± 6.8 58 ± 6.2 Left/right Medial frontal gyrus 6
  3453 8.2 −52 ± 4.7 −39 ± 4.8 11 ± 4.8 Left Superior temporal gyrus 42, 22
Left Middle temporal gyrus 21
  2675 8.5 31 ± 4.6 −6.2 ± 3.4 57 ± 4.6 Right Middle frontal gyrus 6
  1226 5.5 42 ± 3.8 15 ± 7.9 36 ± 2.4 Right Middle frontal gyrus 6, 9
  1141 6.4 −31 ± 3 25 ± 3 14 ± 2.9 Left Insula 13
  353 5.2 −55 ± 2.7 −3.6 ± 2.2 −1.9 ± 1.4 Left Superior temporal gyrus 22
  431 5.5 32 ± 2.6 27 ± 2.2 14 ± 2 Right Insula 13
  333 5.3 30 ± 1.9 −51 ± 2.2 39 ± 2.3 Right Superior parietal lobule 7
  • *For the condition main effect of SIMPLE, local activation maxima are additionally provided. Size of activation clusters in mm3, maximum t-value, centre of gravity in Talairach space (x, y, z), side, anatomical region and Brodmann area (BA).
Details are in the caption following the image

(A) Maximum BOLD response in the volumes of interest VISUAL, STG, IFG VENTRAL, IFG MID, IFG DORSAL and MOTOR. Error bars indicate standard deviation. (B) Estimated delay (delta) of the time-resolved analysis in the same VOIs as above. Delay in VISUAL is normalized to zero in the analysis. (C) Estimated duration (tau) of the time-resolved analysis. Error bars indicate 95% confidence intervals.

Table 3. Overview of significant BOLD amplitude differences between conditions within VOIs
BOLD maximum
VISUAL STG IFG VENTRAL IFG MID IFG DORSAL MOTOR
anova F = 60.82*** F = 16.02*** F = 7.772** F = 4.793* F = 5.165* NS
Post-hoc pair-wise Newman-Keuls corrected t-test
 SIMPLE–MEDIUM NS * * NS NS n.a.
 MEDIUM–COMPLEX *** ** NS * *
 SIMPLE–COMPLEX *** *** ** * *
  • Top row, repeated-measures anova. Other rows, post-hoc pairwise comparisons of conditions with Newman–Keuls-corrected repeated-measures t-tests. n.a., not applicable; NS, not significant; *P < 0.05, **P < 0.01, ***P < 0.001.

Time-resolved analysis of haemodynamic data

The estimated delta (delay) for the three experimental conditions is shown in Fig. 2B for the whole brain. We compared the estimated delta (delay) and tau (duration) for different conditions within VOIs (see Fig. 3). Main effects of condition on delay were found in IFG DORSAL, IFG MID and MOTOR. Pairwise comparisons showed significant delay increases between SIMPLE and COMPLEX and between MEDIUM and COMPLEX in all three VOIs (see Table 4). Delay increases between SIMPLE and COMPLEX and MEDIUM and COMPLEX were significantly (P < 0.05) stronger in IFG MID than in IFG VENTRAL and STG, but did not differ significantly in any other direct pairwise comparison of VOIs. Main effects of condition on duration were found in all VOIs. Pairwise comparisons showed significant duration increases between SIMPLE and COMPLEX in all VOIs. Significant duration increases between SIMPLE and MEDIUM were found in all VOIs except MOTOR. Significant duration increases between MEDIUM and COMPLEX were only found in IFG DORSAL and MOTOR. Duration increases between conditions did not differ significantly in direct comparisons between VOIs.

Table 4. Overview of significant delay (delta) and duration (tau) differences between conditions within VOIs
DELAY (DELTA) DURATION (TAU)
STG IFG VENTRAL IFG MID IFG DORSAL MOTOR VISUAL STG IFG VENTRAL IFG MID IFG DORSAL MOTOR
anova NS NS F = 14.3
(****)
F = 8.4
(**)
F = 6.4
(**)
F = 7.9
(**)
F = 9.6
(***)
F = 7.2
(**)
F = 36.4
(****)
F = 53.7
(****)
F = 10.0
(***)
Post-hoc pair-wise Newman-Keuls corrected t-test
 SIMPLE–MEDIUM n.a. NS NS NS ** * ** *** *** NS
 MEDIUM–COMPLEX *** ** * NS NS NS NS *** **
 SIMPLE–COMPLEX *** ** ** ** *** ** *** *** ***
  • Top row, repeated-measures anova. Other rows, post-hoc pairwise comparisons of conditions with Newman–Keuls-corrected repeated-measures t-tests. n.a., not applicable; NS, not significant; *P < 0.05, **P < 0.01, ***P < 0.001, ****P < 0.0001.

The size of reaction time (RT) differences between conditions corresponded to the sum of delay and duration differences in the MOTOR VOI (see Fig. 4). Duration increases in the other VOIs were larger than RT increases and larger than delay increases in the MOTOR VOI.

Details are in the caption following the image

Delay (delta), duration (tau) and reaction time (RT) differences (seconds) between conditions for all volumes of interest. (A) MEDIUM–SIMPLE, (B) COMPLEX–MEDIUM.

Discussion

In reading, a cascade of cognitive processing steps must be performed, which involves at least visual, graphemic, lexical, and – for sentence-level stimuli – syntactic processes. We combined spatial and temporal analysis of fMRI to better characterize neuronal activation evoked by written language comprehension at the word and sentence level and to resolve the flow of processing. Participants performed semantic decision tasks on written words (SIMPLE condition), simple sentences (MEDIUM condition) and complex sentences (COMPLEX condition). Spatial analysis was performed in standard GLM. Concerning the temporal analysis, direct delay comparison between regions might be confounded by local differences in the neurovascular coupling (Aguirre et al., 1998; Huettel & McCarthy, 2001; Saad et al., 2001). To avoid this problem, we parametrically increased linguistic task demands and compare only within-region task-induced BOLD delay between areas.

Left IFG and STG

The spatial fMRI data analysis with a random/mixed-effects GLM revealed a network of areas that were significantly activated in all conditions, including the visual cortex, the IFG, the STG and prefrontal areas. The activation patterns correspond to those previously observed for word reading (for overviews see Pugh et al., 1996; Fiez & Petersen, 1998; Indefrey & Levelt, 2000, 2004; Turkeltaub et al., 2002; Price & Mechelli, 2005) and sentence reading (for overviews see Kaan & Swaab, 2002; Indefrey, 2004). The most relevant increase in activation in the spatial analysis was present in left IFG and left STG.

To analyse the left IFG in more detail, we defined three VOIs according to the literature that are involved in different linguistic processing aspects: semantic IFG VENTRAL, phonological IFG DORSAL and syntactic IFG MID (Bookheimer, 2002). Additionally, we defined a VOI in the left posterior STG assumed to be involved in lexical processing (Indefrey & Levelt, 2000, 2004). For comparison, we furthermore defined VOIs subserving both the earliest (visual processing – VISUAL) and the latest processing stages (preparation of motor response – MOTOR). The spatial analysis showed increasing BOLD responses in all IFG VOIs and STG between SIMPLE and COMPLEX, increasing BOLD from SIMPLE to MEDIUM in IFG VENTRAL and STG, and finally increasing BOLD from MEDIUM to COMPLEX in IFG MID, IFG DORSAL and STG. This suggests that changing from word-level (SIMPLE) to sentence-level (MEDIUM) modified IFG VENTRAL and STG but not IFG MID or IFG DORSAL. On the other hand, changing from simple sentences (MEDIUM) to complex sentences (COMPLEX) modified IFG MID, IFG DORSAL and STG but not IFG VENTRAL. The temporal analysis revealed delay increases between conditions in the two dorsal VOIs of the left IFG (IFG DORSAL and IFG MID) and in the MOTOR VOI, but not in the ventral left inferior frontal VOI (IFG VENTRAL) or left STG. According to the logic of our experiment, this implies a lower-level or early-stage processing in IFG VENTRAL and STG but higher-level or late-stage processing in IFG DORSAL and IFG MID. These findings are in agreement with recent time-resolved fMRI data on auditory sentence processing (Dehaene-Lambertz et al., 2006), which identified a later onset of the haemodynamic response in IFG compared with STG. Because that study used direct between-region delay comparisons, which may be confounded by region-specific neurovascular coupling, our data provide important complementary evidence in this respect.

Associating IFG MID with a later processing level is in line with the a priori hypothesis of a role of this VOI in syntactic processing. According to this hypothesis, the observed delay in the COMPLEX condition might be due to an increase in processing load at presyntactic processing levels. The substantial size of the delay increase of the order of 1 s indicates that the initiation of syntactic processing was delayed until several words of the stimulus input had been visually, graphemically/phonologically and lexically processed. Such a long delay of syntactic processing is at variance with current models of syntactic parsing, which assume parsing to proceed in an incremental manner starting from the first word (Kempen, 1998; Friederici et al., 2000; Vosse & Kempen, 2000). Compared with normal sentence processing, the onset of syntactic processing may have been delayed due to the limited variability and hence a certain degree of predictability of the syntactic structures in our task.

Although the observed delay in the COMPLEX condition suggests an involvement of IFG MID, located in Broca's area, in syntactic processing or even a subsequent processing stage, this does not mean that this VOI cannot be involved in earlier processing stages as well. If the later response was the dominant response with a higher BOLD amplitude, the temporal analysis would detect this delay although an earlier and weaker response might be present. Note that IFG MID was also active in the SIMPLE condition, which involved only single words and hence did not require any sentence-level syntactic processing. Broca's area is considered to be involved in syntactic processing based on many studies showing a regional cerebral blood flow increase for syntactic compared with non-syntactic stimuli (for overviews see Kaan & Swaab, 2002; Indefrey, 2004). However, haemodynamic studies reporting stronger activation for sentences as compared with words do not exclude a word-level syntactic function (retrieval if lexical syntactic features) of Broca's area. In our view this is exactly the point where our analysis has provided additional information. If IFG MID supported only a word-level syntactic function, we should not have been able to observe a delay increase. The fact that we did observe it supports the hypothesis that this region subserves a later and thus not a word-level processing stage. Moreover, Broca's area also subserves non-syntactic processes. There is, for example, good evidence for pre- and post-lexical phonological processing functions in reading (Pugh et al., 1996; Indefrey & Levelt, 2000, 2004; Bookheimer, 2002). The fact that the other dorsal inferior frontal VOI, IFG DORSAL, showed the same response pattern as IFG MID across conditions does not support different functional roles as suggested by Bookheimer (2002).

Our findings suggest that the most ventral VOI of the posterior inferior frontal lobe (IFG VENTRAL), despite its proximity to the other frontal VOIs, subserves an earlier processing component. This result confirms Bookheimer's (2002) conclusion of a different functional role of this part of the IFG compared with more dorsal areas. The result is, furthermore, compatible with the proposed role of the ventral IFG in semantic processing but suggests that this is rather lexical (early) than sentence-level (late) semantic processing.

The comparison of IFG MID and IFG VENTRAL demonstrates the relevance of the complementary information provided by spatial and temporal analyses. The two regions showed very similar BOLD response increases with increasing linguistic demands. In contrast, the temporal analysis discriminated these two regions. This allowed not only the conclusion that they are involved in different processes but also that IFG VENTRAL must be involved in an earlier process than IFG MID.

Partial parallel processing

According to our expectations, the data did not imply strictly serial processing architecture. In a system with such architecture, a later processing stage awaits the output of an earlier processing stage before it becomes active. This was clearly not the case in our data, where delay increases between conditions showed a temporal ordering of processing stages but were considerably smaller than the summed delay and duration increases in VOIs subserving earlier processing stages. In other words, the activation in VOIs subserving later processing stages (IFG MID, MOTOR) started before the task-induced additional processing at earlier (visual, lexical) stages was completed. This temporal pattern suggests a cascaded processing architecture in which earlier processing levels feed partial output to subsequent levels, such that different processing levels work to some degree in parallel.

With respect to the MOTOR VOI it is certainly plausible to relate its haemodynamic activation to a late processing stage, namely the preparation and execution of the motor response. However, the delay in MOTOR did not equal the sum of the duration increases at earlier processing stages, suggesting that in the more complex conditions motor responses were initiated later but before all relevant information from earlier processing stages was available. Given that the complexity of the motor response as such did not differ between conditions, the increase in MOTOR activation duration between conditions might thus not be due to motor processing but rather reflect a kind of ‘hold’ state during which further response-relevant information was processed.

Limitations of temporal analysis

The temporal analysis presented is based on previous investigations that estimated temporal parameters by approximating model functions to the event-related average BOLD responses (Buckner et al., 1998; Menon et al., 1998; Carpenter et al., 1999; Richter et al., 2000; Liao et al., 2002; Bellgowan et al., 2003). BOLD delay and duration are estimated simultaneously and independently. In analogy to most standard fMRI softare packages, we modified model function into a boxcar function convolved with an HRF function (Friston et al., 1998). Given the similarities between spatial GLM and temporal analyses, fundamental assumptions apply to both methods. In particular, we assume that within a region, the neurovascular coupling/HRF remains constant during the short period of an fMRI experiment. This assumption is considered appropriate in general (Menz et al., 2006), although it may be over-simplified in special circumstances (Duann et al., 2002). In accordance with most GLM analyses, we assumed a boxcar function for the neuronal activation although the exact neuronal response function is unknown. A general concern are inter-individual and to a lesser degree regional intra-individual differences in the exact shape of the HRF (Aguirre et al., 1998; Saad et al., 2001). Like most GLM analyses and a recent time-resolved fMRI study using a different approach (Dehaene-Lambertz et al., 2006) we assumed a uniform HRF. This simplifying assumption is supported by the observation that voxels in regions that are regarded to be task-relevant show a far smaller variability than voxels outside these regions of interest (Neumann et al., 2003). Because the shape of BOLD responses is much less variable within subjects than across subjects (Aguirre et al., 1998) we performed individual time-resolved analyses in the first step and performed the group average only in the second step.

A specific concern for our temporal analysis resides in the observation that the BOLD response amplitudes differed between regions and, more importantly, also showed a differential increase between conditions. BOLD response amplitude differences can in principle influence the estimated onset of neuronal activation. To minimize such a putative confounding effect, we normalized the BOLD amplitude prior to the delay estimation. Furthermore, the observed pattern of amplitude and delay changes across regions makes such a putative confounding effect unlikely for the present data. Of the regions that showed a BOLD amplitude increase, two (STG and IFG VENTRAL) did not show an increase of the estimated delay and two others did (IFG DORSAL and IFG MID). Conversely, there was no significant difference in BOLD amplitude in MOTOR yet a clear delay increase.

Given these concerns, our design included two internal validations. First, concerning duration, we increased stimulus duration and expected corresponding BOLD duration increases in stimulus-related visual cortex. The estimated duration in the VISUAL VOI was identical to the stimulus presentation duration in the SIMPLE condition (6 s). As expected, it increased from SIMPLE to COMPLEX. However, this increase was smaller than the stimulus presentation increase of 4 s, suggesting that the duration parameter underestimated duration increases between conditions also in other areas. Second, concerning delay, we expected the increased response times to parallel longer delays in response-related motor cortex, which should stop after the response. RT differences between conditions should therefore be reflected in corresponding time differences of the end of the estimated activation in the MOTOR VOI given by the sum of estimated delay and duration increases. This was indeed the case, as the sum of estimated delay and duration increases between conditions in this VOI corresponded quite exactly to the RT increases. Given this evidence for a valid estimation of the sum of delay and duration increases and the possible underestimation of duration increases discussed above, delay increases might have been to some degree overestimated overall. Note that such a global misestimation would not affect the finding of differential delay increases across regions, as duration increases between conditions were uniform in all areas.

Alternative temporal analysis approaches represent temporal characteristics of the BOLD response in statistical weights by introducing one or several temporal derivatives of stimulus time-course-based basis regressor(s) (Friston et al., 1998, 2000, 2003) or by shifting stimulus time-course-based basis regressor(s) in time (Formisano et al., 2002). We did not choose such an approach because estimates beyond the range of 1 s become less accurate (Handwerker et al., 2004).

Limitations of the experimental design

The written stimuli were chosen such that (1) demands at all processing levels were increased between conditions and (2) the corresponding increases in processing times would be sufficiently strong to be detected with fMRI, i.e. in the order of seconds. The behavioural data we obtained during fMRI measurements showed the expected increase in processing time between conditions. The response latencies increased by 0.9 s between the SIMPLE and the MEDIUM and by 1.8 s between the MEDIUM and the COMPLEX conditions. The increasing demands at all linguistic processing levels, however, complicates the interpretation when considering only the spatial GLM analysis. Additionally the duration uniformly increased with increasing task demands in all VOIs, in contrast to the delay. This suggests that, as expected, the processing load increased at all processing levels.

The condition duration increased from SIMPLE to COMPLEX. We chose this approach because it allowed validation of the estimated BOLD duration in the visual area, as discussed above. In principle, such duration increases might induce systematic confounds between conditions if there was a differential increase in BOLD duration in different regions. Such a differential duration increase in different regions is, however, also possible if the condition duration is constant. Because we observed uniform duration increase in all regions, putative systematic confounds related to increased task duration appear unlikely.

The response accuracy decreased from MEDIUM to COMPLEX, but not from SIMPLE to MEDIUM. We reason that increasing task demands in general induce increasing response latency, decreasing accuracy or a combination of both processes. The response latency increased from SIMPLE to MEDIUM (and from MEDIUM to COMPLEX). We reason that the most plausible explanation for this pattern, despite unchanged response accuracy, is increasing task demands from SIMPLE to MEDIUM to COMPLEX, in accordance with our experimental task design.

Implications for other experiments

It was shown that time-to-onset misestimates as small as 1 s influence model parameter estimation and therefore random-effects analyses across subjects (Handwerker et al., 2004). The results of the present study suggest therefore that areas subserving later processing components are systematically underestimated when using a GLM analysis with only stimulus time-course-based regressors, in particular for short events and complex tasks. In part, local and task-induced delay differences in the BOLD response may be compensated for by introducing temporal derivatives of regressor (Friston et al., 1998, 2000, 2003). Note, however, that even when the model includes temporal derivatives, the comparison of the magnitude of two responses based on a GLM analysis may be biased, if there is a difference in the delay of the responses (Liao et al., 2002).

Improvements in MR technology including higher field strength and parallel imaging will improve the temporal resolution of fMRI in the future. This will allow us to apply the proposed within-region task-induced temporal analysis of fMRI in a wide spectrum of cognitive experiments even with significantly smaller modifications in reaction times.

Conclusion

The combination of spatial and temporal analysis allowed a functional differentiation of IFG subregions involved in written language comprehension that was not possible with the spatial GLM analysis alone. Ventral IFG region (BA 47) and STG subserve earlier processing stages than two dorsal IFG regions (BA 44 and 45). This is in accordance with the assumed early lexical semantic and late syntactic processing of these regions.

Acknowledgement

This work was supported, in part, by a grant from the Swiss National Science Foundation (grant 205321-104292/1).

Abbreviations

  • BA
  • Brodmann area
  • BOLD
  • blood oxygenation level-dependent
  • ERA
  • event-related average
  • ER-fMRI
  • event-related functional magnetic resonance imaging
  • FDR
  • false discovery rate
  • FWE
  • familywise error
  • GLM
  • general linear model
  • HRF
  • haemodynamic response function
  • IFG
  • inferior frontal gyrus
  • RT
  • reaction time
  • STG
  • superior temporal gyrus
  • TAL
  • Talairach space
  • VOI
  • volume of interest.
    • The full text of this article hosted at iucr.org is unavailable due to technical difficulties.