The situation or the person? Individual and task-evoked differences in BOLD activity

2.1 Task descriptions

Surface-based activation maps were collected from 416 unrelated, healthy subjects provided by the HCP (Van Essen et al., 2013) as they performed seven tasks: Emotion, gambling, language, motor, relational, social, and working memory. A brief description of each HCP task is presented here, but for more details see Barch et al. (2013). The working-memory task was an N-back task with eight condition types of 0-back and 2-back conditions with faces, places, tools, and body parts presented as stimuli. The gambling task was a card guessing game, which involved guessing the number on a mystery card to win or lose money in two different conditions: “mostly win” or “mostly lose” outcome conditions. In the motor task, there were six condition types where subjects were directed to perform particular motor movements from the fingers, toes, or tongue in response to visual cues. The language task was a story-comprehension task with interleaved blocks of two condition types: Brief auditory stories and simple arithmetic problems (e.g., “fourteen plus twelve.”). The social task was a “theory of mind” task where participants are presented with short video clips of two condition types: Interacting or randomly moving shapes. In the relational task, subjects are presented with pairs of objects and told to distinguish them on either subject-specified dimensions in one condition or experimenter-specified dimensions in the other condition. In the emotion task, participants were told to match either emotionally expressive faces (e.g., angry or fearful) in one condition or shapes in the other condition. In total, each subject completed 24 task conditions across the seven tasks.

2.2 Task-fMRI preprocessing and analysis

A detailed description of the HCP structural and functional minimal-preprocessing pipeline is provided in Glasser et al. (2013). We briefly describe the surface-based functional preprocessing and analysis pipeline relevant to the question of inter-subject variability: (a) Subject's volume scans are mapped to CIFTI gray-ordinates standard space (32k Conte69 mesh) using an advanced “areal-feature-based” multi-modal registration algorithm (MSMAll; features used included myelin maps, resting-state networks, and resting state topographic maps), (b) 2 mm FWHM surface-based smoothing using a geodesic Gaussian algorithm, (c) 2 mm subcortical parcel-constrained smoothing (only the cortical surface vertices were used in subsequent analyses), and (d) conversion to CIFTI dense time series format for further processing. Subsequent preprocessing included an extra surface-based smoothing using a geodesic Gaussian algorithm with 4 mm FWHM (no further smoothing was conducted). Computation of whole-brain activation maps for each subject was computed using a standard general linear model (GLM) analysis using FSL's FMRIB's Improved Linear Model (FILM) with autocorrelation correction (Woolrich, Ripley, Brady, & Smith, 2001). Task condition regressors were constructed by convolution with a canonical hemodynamic response function (HRF; Glover, 1999). Temporal derivatives of each convolved regressor were included in the GLM to account for timing differences, but estimates for these terms were not used for the analysis. The standardized beta estimates (z-statistics) were used as BOLD “activation” estimates for each vertex and used in further analyses.

2.3 Construction of whole-brain activation map distance matrices

Whole-brain activation map distance matrices were constructed at the vertex-level, and across a range of resolution sizes at a ROI level (N = 100, 200, 400, 600, and 800; higher number of ROIs corresponds to a more fine-grained parcellation) using the whole-cortex Schaefer parcellation (Schaefer et al., 2017). Activation estimates for each ROI were calculated as the mean standard beta estimate for all vertices contained within the ROI. As whole-brain cortical activation patterns were of interest, noncortical vertices were excluded from the computation of the distance matrix. The “distance” between two whole-brain activation maps represents the dissimilarity in activation estimates (estimated from the GLM described above) of all vertices or ROIs between each map. Note, the activation estimates for all vertices or ROIs were not thresholded before the distance computation, meaning the activation estimates of all vertices or ROIs contributed to the distance computation. The whole-brain activation map distance matrix was constructed by computing both the City block (L1-norm) and Euclidean (L2-norm) distance metrics between all pairs of whole-brain activation maps across all subjects. Vertex- and ROI-values of each activation maps were z-score standardized before computation of distance to remove arbitrary mean and scaling differences. Pearson correlation and cosine distance, both equivalent (up to a constant factor) to squared Euclidean distance on z-scored data (Berthold & Höppner, 2016), were attempted but resulted in a singular distance matrix with negative sums of squares residual estimates in the MDMR analysis (R² > 1). However, the relative difference between the task condition versus subject estimates was similar for the correlation/cosine distance and Euclidean distances. The whole-brain activation map distance matrix is constructed by computing the distance (City block or Euclidean) between all subject's task-condition activation maps (N = 24 task-conditions). The resultant 9,984 (416 subjects × 24 task-conditions) by 9,984 task-activation map distance matrix was then input to the MDMR analysis. An illustration of this process on a hypothetical task-fMRI dataset of two subjects and two tasks is presented in Figure 1.

2.4 Multivariate distance matrix regression

2.5 Clustering analysis

To further explore the organization between whole-brain activation maps, we applied the Louvain modularity maximization graph-clustering algorithm (Blondel, Guillaume, Lambiotte, & Lefebvre, 2008) to a whole-brain activation map similarity matrix. The whole-brain activation similarity matrix was computed from the Pearson correlation similarity between each whole-brain activation map using the N = 100 resolution Schaefer parcellation, as this was found to be most responsive to task differences (see Section 3). However, we also demonstrate that the partition from this ROI-parcellation resolution was minimally different from the partitions using higher-resolutions or vertex-level data. Roughly, the objective of the graph-clustering algorithm (implemented in the Brain Connectivity Toolbox; Rubinov & Sporns, 2010) is to partition the whole-brain activation maps into communities that maximize the within-cluster similarity of whole-brain activation patterns. Negative weights (i.e., negative correlation coefficients) were set to contribute asymmetrically to the modularity objective function (Rubinov & Sporns, 2011). In addition, an iterative fine-tuning algorithm was used that recursively applied the Louvain algorithm until no further increases in the modularity objective function were achieved (Geerligs, Rubinov, Cam-CAN, & Henson, 2015). Because modularity-maximization algorithms suffer from a well-known resolution limit (Fortunato & Barthélemy, 2007), the modularity objective function is often modified with a resolution parameter (Betzel & Bassett, 2016; Geerligs et al., 2015) to control the resulting “resolution” of the clusters. To choose a value for this parameter, we searched across a range of resolution parameter values, and chose the values that produced the most consistent partitions across 50 runs of the algorithm, measured as the average adjusted Rand index (Traud, Kelsic, Mucha, & Porter, 2011) using code provided in the Network Community toolbox (http://commdetect.weebly.com/). All values for the parameter from 0.75 to 1.04 yielded perfect across-run consistency, with a drop-off in consistency thereafter (Supporting Information Figure S3). There is a “rebound” after this drop-off in the consistency parameter at the value of 1.13. Thus, we present two clustering solutions at the following values of the resolution parameter, 1.04 and 1.13, that present a “coarse” and “fine-grained” view of the relationships among activation maps, respectively.

The resulting clusters provide insight into what types of task conditions drive the greatest differences among whole-brain activation patterns, and how consistent these differences are across subjects. To interpret each cluster, we inspected the number of activation maps from each task condition in each cluster. The number of task condition activation maps in each cluster were visualized in a word cloud created using the wordcloud function in MATLAB. To visualize the activation patterns associated with each cluster, we computed the average (centroid) activation pattern by computing the mean for all vertices across cluster members.

3.1 MDMR results

The results of the MDMR analysis revealed that task condition differences explained a greater amount of variance (100 ROI/City block: R² _{task-paradigm} = 0.3404, R² _{task-condition} = 0.467) in whole-brain activation patterns than between-person differences (100 ROI/City block: R² _subject = 0.143) for all resolutions of the Schaefer parcellation (Figure 2). Of note, all comparisons of R² values were statistically significant, as confirmed by zero overlap in their bootstrapped R² distributions (visually apparent from Figure 2). This was true for both the City block distance and Euclidean distance. However, this finding was reversed at the vertex-level with the City block distance: Between-person differences explained a greater amount of variance in whole-brain activation patterns than did task paradigm. One contributing factor for this reversal is the “resolution” at which distances are computed: As the resolution increases from the 100 ROI Schaefer parcellation to the vertex-level, the amount of variance explained by task condition decreases, while between-person differences increases. Another contributing factor to this reversal is the differential emphasis on large distances by the City block and Euclidean distance metrics. The Euclidean distance metric places greater weight on large distances given that the difference between each vertex or ROI is squared, which is not the case with the City block metric. Thus, large activation differences in ROIs or vertices between task paradigms are given greater weight in the total distance calculation with the Euclidean distance metric. Together, these factors may explain the reversal observed between the vertex and ROI-level resolutions for the City block distance metric.

3.2 Task-specific post hoc MDMR results

To further analyze between-person versus task condition variability of whole-brain activation patterns, we applied the MDMR analysis separately for each task, modeling both conditions within each task and between-person differences. The results suggest that within a task paradigm, between-person differences explain a substantially greater amount of variance than task condition differences of that task paradigm (Figure 3). The size of this difference was dependent on task paradigm. For example, in the Gambling task, the difference between the “punish” and “reward” conditions explained a minimal amount of variance in activation map similarity (City block/vertex task R² = 0.0024) compared with persistent individual differences (City block/vertex subject R² = 0.8649). In contrast, the difference between the “math” and “reward” condition in the Language task explained a greater amount of variance (City block/vertex task R² = 0.0632), with a comparable percentage of explained variance due to between-person differences (City block/vertex subject R² = 0.8632). Consistent with the total MDMR results, there was an increase in explained variance due to between-person differences and decrease due to task conditions when moving from lower to higher ROI resolutions. An additional observation of note is that areas of high between-person standard deviation (SD) had a consistent overlap with areas of high magnitude activation estimates across all task conditions (Figure 3). Thus, brain areas highly responsive to task conditions on average, across subjects, exhibit the greatest between-person variability in those responses compared with low-response areas. In addition, activation pattern similarity was greater across high SD vertices (top 20%) compared with low SD vertices (bottom 20%; Supporting Information Figure S2). As these high SD vertices overlapped with high average activation vertices, these findings indicate that activation pattern similarity was greater in task-response vertices, compared with nontask-responsive vertices.

3.3 Clustering results

To further examine the type of experimental manipulations across task conditions that produce disparate or similar activation patterns across subjects, we used a graph-based clustering approach. We first computed a whole-brain activation pattern similarity matrix across all subjects' activation maps. The similarity matrix was constructed by computing the Euclidean similarity (inverse of distance) across all ROIs of the Schaefer 100 parcellation between all subjects' activation maps. We chose the Euclidean metric and Schaefer 100 ROI parcellation because this resolution and metric were most responsive to task-conditions (Figure 2). The objective function of the graph-clustering algorithm contains a resolution parameter (“gamma”) that controls whether the resulting clusters are large (“coarse”) or small (“fine-grained”), what we refer to as the “dimensionality” of the clustering. To choose a value for this parameter, we searched across values of the parameter that maximized the across-run clustering consistency (see Section 2; Supporting Information Figure S3). We chose two cluster solutions: A low-dimensional (coarse) and high-dimensional (fine-grained) cluster solution.

The low-dimensional cluster solution yielded three large clusters (containing 9,306 out of 9,984 activation maps), corresponding to a language/motor (C5), social/emotion (C2), and working-memory/gambling/emotion/relational task clusters (C3) (Figure 4). The largest cluster (C3; N = 4,835) contained a wide variety of task conditions. One prominent commonality among these task conditions was that they required active attention to rapidly presented visual cues. This was not the case for the activation maps that belonged to the other two large clusters, C2 and C5. C2 primarily contained activation maps from the task conditions of the social task, which required passive viewing of visual stimuli. C5 primarily contained activation maps from the task conditions of the language and motor tasks, which involved responding to auditory stimuli and movements of various body parts to visual cues, respectively. Two smaller clusters, C7 and C8, were also present in the clustering solution. C7 (N = 205) primarily contained face and body stimuli conditions from the working memory task (both from the 0-back and 2-back conditions). C8 (N = 411) primarily contained the visual cue task condition activation maps from the motor task. While most activation maps belonging to a single task-condition were classified into the same cluster, this was not the case for every task condition. For example, the activation maps for the “shapes” condition of the Emotion task was roughly split between C2 (N = 229) and C3 (N = 131). Notably, the social task activation maps that primarily constitute C2 were elicited by passive viewing of visual shapes. Thus, these observations suggest that some subject's activation maps look more similar to other task conditions (e.g., social task) than other subject's activation maps of the same task condition.

The high-dimensional cluster solution yielded a more “fine-grained” view of the relationships among the activation maps (Figure 5). Half of the clusters in this solution primarily contained activation maps from either a single task condition or task paradigm. C6 primarily contained activation maps from the Emotion task, C39 with the task conditions of the relational task, C43 with the visual cue condition of the motor task, C44 with the task conditions of the social task, and C49 with the task conditions of the gambling task. The task condition activation maps of the working-memory task were split between four clusters: C3 with primarily “0-back” conditions and the “face” condition of the Emotion task, C14 with “0-back” body and face stimuli conditions, C28 with “2-back” body and face stimuli conditions, and C32 with primarily “2-back” conditions. Interestingly, the language/motor cluster from the low-dimensional cluster solution remained mostly intact in C12 of the high-dimensional cluster solution.

We also tested the degree to which the low- and high-dimensional clustering of whole-brain activation maps is driven by between-person differences, regardless of task conditions. We calculated the normalized mutual information (NMI; Meilă, 2007) between the cluster membership and subject membership of each activation map (varies from 0 to 1; 1 representing maximum similarity). The NMI between the cluster membership and subject membership indices was 0.0231 for the low-dimensional solution, and 0.0901 for the high-dimensional solution. In other words, the contribution of between-person differences to the low- and high-dimensional cluster solutions is minimal. However, the NMI values are nonzero, indicating that between-person differences did contribute to the cluster solution for some subjects.

4.1 Between-person and task-condition differences in BOLD activity

The primary goal of this study was to estimate the variability in BOLD activation maps explained by between-person and task-condition differences using a novel MDMR approach. In addition, we sought to determine the moderating influences on these estimates. Such influences include parcellation resolution, choice of distance metric, and task-paradigm coding. In agreement with Gratton et al. (2018), we find that task conditions explain greater variance between whole-brain activation pattern differences compared with persistent individual differences. This was the case for all parcellation “resolutions” considered. However, at the vertex-level (N = 59,412), we find that between-person differences explained a greater amount of variance than task conditions, for at least the City block distance metric, in agreement with Miller et al. (2009). More generally, we find that as the resolution at which whole-brain activation maps are compared increases, the variance explained by between-person differences increases more than that explained by task-condition differences decreases. This finding should not be surprising, given that there is much more variability in the fine-scale structure of the human cortex, as opposed to its large-scale arrangement. While comparing whole-brain activation patterns at lower ROI resolution reduces the dimensionality of the comparison, it also acts as a spatial smoothing operation, reducing potential small differences in the anatomical location of functional areas (Bijsterbosch et al., 2018; Mikl et al., 2008). Thus, increasing the parcellation resolution (e.g., 800 ROI parcellation or vertex-level) would reduce the extent of this implicit smoothing, thereby potentially increasing inter-subject variability of whole-brain activation patterns. Of note, spatial smoothing in the pre-processing of task-activation maps was minimal, 4 mm FWHM geodesic Gaussian smoothing. On the other hand, task condition differences in whole-brain activation patterns are most evident at the coarse ROI level (e.g., 100 ROI parcellation), suggesting that differences between task conditions are most likely to be reflected in large-scale spatial changes in task-evoked activity.

4.2 Differences across levels of resolution

A corollary of this finding is that stable between-person differences in whole-brain activation patterns evoked by a task are largely due to small differences in the spatial extent of task-evoked activity that are more resolved at the vertex- or “fine-grained” ROI-level, as opposed to a “coarser” ROI parcellation. These small individual differences in the spatial extent in BOLD activation are not guaranteed to represent the spatial extent of task-responsive neural populations between subjects. At the resolution of a vertex or voxel, individual variability may be due to differences in vascular innervation or cross-subject alignment of neuro-anatomical features, such as cortical folding patterns. In terms of vascular innervation, the measured spatial specificity of blood flow changes induced by neurovascular coupling is limited in gradient-echo (GE) BOLD fMRI due to venous drainage from the site of a task-responsive neural population. In fact, changes in venous blood flow can be detected as activation as far as 4 mm away from a task-activated neural population (Parkes et al., 2005; Turner, 2002), well above the 2 mm voxel resolution now commonly used in GE-BOLD fMRI (and the surface vertex-level used here). In addition, cross-subject alignment of anatomical features could still be an issue at a “high” resolution. Though a sophisticated multi-modal surface normalization procedure (MSM-All) was used by the HCP for this dataset (Glasser et al., 2013), cross-subject alignment issues, however minimal, are still likely to be present (Guntupalli, Feilong, & Haxby, 2018). These factors make for a complicated interpretation of whole-brain activation pattern comparisons at a “fine-grained” resolution near to the vertex or voxel-level, which would not be present a coarser ROI parcellation level.

One central matter is what level of resolution is of greatest interest to task-fMRI researchers. It is a common practice to only interpret clusters of significantly active voxels/vertices with cluster-correction based thresholding techniques. In this case, larger brain regions are of interest, rather than individual vertices/voxels (Smith & Nichols, 2009). Furthermore, some researchers may restrict their analysis to a priori regions of interest from a data-driven parcellation (Gordon et al., 2017; Power et al., 2011) or previously reported brain coordinates. However, in some analyses of task fMRI data, interpretation of BOLD activity at the voxel/vertex level is a common practice. For example, multivariate pattern analysis (MVPA) commonly analyzes patterns of BOLD activity estimated at the voxel/vertex level. In analyses of the hemodynamic response using task-fMRI data, it is also quite common to analyze at the voxel/vertex level to distinguish different vascular components of brain tissue (e.g., large veins vs. capillaries). Thus, we reiterate the sentiment of Schaefer et al. (2017) and others that there is likely no optimal resolution for all task-fMRI analyses. The level of resolution, and thus the degree of between-person heterogeneity, will be dependent on the research question.

4.3 Mathematical and empirical differences in distance metrics

This study also highlights the fact that similarity/distance metrics can perform differently in different contexts. For example, we find that whole-brain activation patterns are less different across task-conditions when the City block distance metric was used, as opposed to the Euclidean distance. This was reversed for between-person differences: Whole-brain activation patterns are less different across subjects when the Euclidean distance was used, as opposed to the City block distance. The differences between the Euclidean and City block distances used here, as well as the Pearson correlation and cosine distance, can be explained in terms of their weighting of large distances. Properly normalized (i.e., z-scored beforehand to eliminate mean and scaling differences), City block, Euclidean, and Pearson Correlation/Cosine distance differ solely in terms of the weight given to the difference between two points when comparing two vectors. City block distance is the least sensitive to large differences, simply summing the absolute differences between each vertex/ROI between whole-brain activation patterns. The Euclidean distance sums the square of the difference between each vertex/ROI, followed by taking the square root of this sum. This squaring operation gives large differences between the activation estimates of a vertex/ROI differentially more weight compared to smaller differences. The correlation and cosine distance (equivalent if the mean is subtracted from each vertex/ROI) are equivalent (up to a constant factor) to the z-score normalized squared Euclidean distance: Simply summing the squared differences without taking the square root of this sum, which places even greater weight on larger differences. As illustrated above, these mathematical properties produce concrete empirical differences in the MDMR analyses above (Figure 2). For a concrete example, task-condition explains 46.68% of the variability in whole-brain activation pattern differences for the City block distance, 49.81% of the variability for the Euclidean distance, and 73.89% of the variability for the Pearson correlation/cosine distance. However, the differences between the distance metrics are likely more pronounced in our sample of task-activation maps. As can be observed from both the task-condition mean activation maps (Figure 3) and clustering results (Figure 5), many of the activation maps in this sample exhibit the same canonical task-positive/task-negative activation pattern. Differences between activation maps are mostly restricted to focal regions of this dominant task-positive/task-negative activation pattern. Thus, the City block distance would be more sensitive to the overall dominant task-positive/task-negative activation pattern and place less weight on these “outlier” regions. In contrast, the Euclidean and correlation/cosine distance would place greater emphasis on these “outlier” regions.

4.4 Differences between task-conditions

A further issue in comparisons of between-person and task condition differences in BOLD activation patterns is the choice of task conditions. Task conditions with many different stimulus and response features would be expected to be more dissimilar in their evoked BOLD activation patterns than task conditions sharing largely the same features. In the case of a comparison between persistent individual differences and task conditions containing wildly different stimulus and response features, we would expect task conditions to explain a much greater amount of variance between whole-brain activation patterns than individual differences. This goes for differences between task conditions across task paradigms, as well as differences between task conditions within a task paradigm. For example, we observed very minimal differences between the task conditions of the gambling task. In fact, the difference in task conditions of the gambling task explained only 0.24% of the differences between whole-brain activation patterns at the vertex-level. This suggests that the different features between the conditions of the gambling task, mainly differing probability of gains versus losses, produce minimal observable changes in whole-brain activation patterns. In contrast, different features between the task conditions of the language task, auditory arithmetic calculations versus passive auditory listening to stories, explained 6.32% of the differences between whole-brain activation patterns at the vertex-level. These findings emphasize that the neural response estimated using BOLD fMRI is differentially responsive to experimental manipulations of the task-fMRI environment in ways that may not be expected by current cognitive theory. This further motivates a cognitive ontology project that attempts to identify those features of the task-fMRI environment that explain the greatest differences in whole-brain BOLD activation patterns (Anderson, 2015; Bolt et al., 2017; Lenartowicz et al., 2010; Poldrack & Yarkoni, 2016).

4.5 Other factors contributing to differences in BOLD activation patterns

Several factors that could contribute to differential estimates of between-person and task condition variability were not explored in the main results of this manuscript:

4.6 Unsupervised clustering of activation maps

To identify those features in our sample of task conditions that produce disparate or similar activation patterns we used a graph-based clustering approach. Observations of both a low- and high-dimensional cluster solution yielded interesting findings. The cluster solutions revealed similarities in whole-brain activation patterns across task conditions and task paradigms. In the low-dimensional clustering solution, three large clusters were observed that contained primarily social/emotion (C2), language/motor (C5), and working-memory/gambling/relational/emotion (C3) task-condition activation maps. The task condition activation maps that belonged to the three clusters had quite distinct stimulus and response features. C2 activation maps primarily involved passive reception of visual stimuli. Of note, many of the “shape” condition activation maps of the emotion task also belonged to this cluster, which involved identifying whether simple line-drawn circles and ovals are the same or different. The ease of the task demands in this condition may be similar to passive reception of visual stimuli. C5 activation maps almost exclusively contained activation maps from the language and motor tasks, involving responses to presented arithmetic or story stimuli, and motor movements of hands, feet, or tongue regulated by a flashing visual fixation cross. The clustering results also revealed interesting observations regarding between-person differences. In both the low- and high-dimensional cluster solutions, some subject's BOLD activation maps were more similar to other task-conditions, than other subject's activation maps in the same task condition. For example, the activation maps for the “shapes” condition of the Emotion task was roughly split between two clusters in the low-dimensional cluster solution.

One notable observation from both cluster solutions is the high overall similarity in whole-brain centroid activation patterns between all clusters. This pattern is the classic “task-positive/task-negative” activation pattern that is consistently observed across a large variety of task paradigms (Bolt et al., 2017; Fedorenko, Duncan, & Kanwisher, 2013; Hugdahl, Raichle, Mitra, & Specht, 2015; Raichle et al., 2001). Minor differences in this overall pattern are what distinguish most centroid activation patterns of each cluster from each other. For example, C2 and C3 from the low-dimensional cluster solution are largely differentiated in terms of the differential higher activation in the lateral posterior cortex (Figure 5). These findings are consistent with a previous description of the nested structure of whole-brain BOLD activation patterns (Bolt et al., 2017), in which task-states are differentiated terms of permutations of a canonical task-positive/task-negative activation pattern.