1 INTRODUCTION

In the last years, the indirect estimation of structural disconnection has become an increasingly popular alternative to directly measured structural disconnectomes. The latter are available only from specific diffusion weighted imaging sequences such as Diffusion Tensor Imaging (DTI), which are only sparsely available for most clinical cases. Software packages such as the Lesion Quantification Toolkit (LQT; Griffis et al., 2021), the Brain Connectivity and Behavior Toolkit (BCBtoolkit; Foulon et al., 2018), and the Network Modification (NeMo) tool (Kuceyeski et al., 2013) use lesion maps from neurological patients as seeds to estimate which fiber tracts intersect with the lesion, based on a tractography atlas of healthy individuals. In combination with a gray matter atlas, they further allow the creation of pairwise (also referred to as connection-wise, parcel-wise, region-wise, region-to-region, or ROI-to-ROI) disconnection measures between each gray matter region-pair.

Disconnection measures created this way have a decisive theoretical advantage over voxel-wise approaches in univariate statistics concerning the so-called “partial injury problem” (Kinkingnéhun et al., 2007; Rorden et al., 2009; Sperber et al., 2019). The partial injury problem argues that (in voxel-wise designs) true lesion-deficit associations are deluded when lesions of different patients fall into different parts of the ground truth region (i.e., the entire area truly eliciting the behavioral deficit). In contrast, pairwise approaches are based on the summation of voxels according to predefined atlas parcels, so that a direct overlap of lesioned voxels is less essential for the successful discovery of an association. The creation of region-pairs even allows considering the mutual involvement of spatially distributed lesions due to a damaged connection between them, which further increases the statistical power in univariate analyses for pairwise disconnection measures compared to single voxels (see Sperber et al., 2022).

The mechanisms underlying the partial injury problem also apply to the pooled analysis of lesions in the left and right hemispheres. Even if structures relevant to a function of interest are represented in both hemispheres as is the case for non-lateralized behaviors, lesion-symptom mapping that analyzes unilateral left and unilateral right lesions together causes voxels of either hemisphere to be statistically underpowered. In a patient with the behavioral deficit in question and a lesion in the left hemisphere, the neural correlates of that deficit in the opposing right hemisphere are represented as fully intact, whereas these voxels are damaged in a patient with a lesion to the right hemisphere, and vice versa. That, in turn, weakens the association between each hemisphere and the deficit. Voxels of one hemisphere thus act as opponents for voxels of the other one. Therefore, individual analyses for each hemisphere have been established as the standard in voxel-wise lesion-symptom mapping (Frenkel-Toledo et al., 2019, 2021). However, so far it is unclear whether or not this problem applies also to pairwise disconnection approaches.

Recent literature addressing target behaviors in patients suffering from brain injury primarily analyzed structural disconnections in a joint analysis, that is, a single analysis based on patients with unilateral left and with unilateral right hemispheric lesions (e.g., Errante et al., 2022; Kuceyeski et al., 2016; Moore et al., 2022; Pan et al., 2022; Smaczny et al., 2023; Urbanski et al., 2016), and only a minority in individual analyses separated by hemisphere (e.g., Rosenzopf et al., 2023). The question thus arises whether joint or separate analyses, or both equally, are valid to study pairwise structural disconnections in humans. Theoretically, the uneven distribution of inter-hemispheric disconnections (possible in the whole sample) and intra-hemispheric disconnections (possible only in patients with lesions in the respective hemisphere) might introduce a similar bias as known from lesions in opposing hemispheres (i.e., the partial injury problem). Joint analyses could underpower intra-hemispheric disconnections and decrease their chances to be uncovered (despite a true association to the deficit). Simulations offer an excellent approach to investigate these questions. We thus examined the effects of joint analyses versus analyses separated by hemisphere to answer the question whether it is more valid to analyze left and right hemispheric lesions jointly or separately in pairwise disconnection studies.

2 METHODS

All analyses were performed in MATLAB R2019a (The MathWorks, Inc., Natick, USA) via custom scripting. We conducted two simulation paradigms, for which we defined three pairwise ground truth connections: one inter-hemispheric connection and one intra-hemispheric connection in each hemisphere (see Figure 1). Region-pairs were based on the Brainnetome Atlas (BNA; Fan et al., 2016). The inter-hemispheric connection extended between ventral area 9/46 of the middle frontal gyrus (MFG) of the left and right hemispheres; the intra-hemispheric connection extended between medial area 38 of the superior temporal gyrus (STG) and ventral agranular insular area of the insular gyrus of either hemisphere. To mimic a typical clinical cohort, we created samples that consist to one third of critical cases (i.e., patients with damage to a ground truth connection and the associated target behavior) and two thirds that represent “control” cases.

2.2 Simulation 2: Lesion-based simulation

In the second simulation, we pursued a more realistic approach by generating 3D lesion maps and indirectly estimating their corresponding pairwise disconnections. We generated 50 binary lesion maps per hemisphere as ellipsoids with random radiuses in standard MNI space. For each hemisphere, we created randomly distributed lesions, of which some were localized along the two targeted pairwise connections. After lesion map generation, maps were masked by ventricles, cerebellum, extracerebral space, and the opposite hemisphere at the midsagittal plane. The overlap of the simulated lesions is shown in Figure 2. Mean lesion volume was 13.6 mL (SD: 10.0 mL) in total, 13.4 mL (SD: 10.8 mL) for left-sided lesions, and 13.7 mL (SD: 9.3 mL) for right-sided lesions.

We then indirectly estimated the structural disconnectome: the generated binary lesion maps served as seeds to track all fibers passing through each lesion. The LQT with the default options was selected for this purpose. We used the default tractography atlas based on 842 healthy individuals by the Human Connectome Project (Yeh et al., 2018) and the BNA as gray matter parcellation atlas. For each simulated lesion, the obtained pairwise disconnection matrix contains the relative damage for each region-pair that has a direct anatomical connection according to the tractography atlas. In accordance with our first simulation, lesions with a pairwise disconnection of at least 30% for a target connection were defined as critical lesion leading to the behavior of interest; for each hemisphere, we generated 8 critical lesions with damage to the inter-hemispheric target connection and 8 critical lesions with damage to the intra-hemispheric target connection (total critical lesions: 16 affecting the inter-hemispheric, 8 the left-sided intra-hemispheric, and 8 the right-sided intra-hemispheric ground truth connection; see Table 1 for details).

For the simulation of the target behavior, we again used the maximum pairwise disconnection score among the three ground truth connections (and set the range to 0 to 1). Contrary to Simulation 1, the disconnection scores differed to some degree between hemisphere and type of connection in the less controlled Simulation 2 (cf. Table 1). To simulate different natural scenarios of statistical power, we inserted a specified proportion of noise to the maximum disconnection score; we tested noise levels of 0%, 10%, 25%, 45%, 50%, 55%, and 75% of the total behavioral score. The noise value was pseudo-randomly defined and equal for inter- and intra-hemispheric critical subsamples to ensure that none of the target connections accidentally received overall more noise than the other that would lead to an unequal blurring of the connection-behavior association. The pseudo-random noise generation had the additional advantage that all simulation analyses (left and right hemispheric samples as well as different noise levels) were directly comparable. We then implemented mass-univariate analyses with permutation-based correction for multiple testing as explained above for the first simulation.

In addition, we looked at the effect of a greater sample size by extending the sample to 120 lesions per hemisphere (N_control = 80; N_{critical-inter} = 20; N_{critical-intra} = 20; for an overlay image, see Figure S1 in the Supplement). The following procedure was similar to the one described above, except that we kept pairwise connections damaged in at least 20 cases and that we tested noise levels of 70% and 80% instead of 45% and 55% due to the increased statistical power.

3 RESULTS

In both the first and second simulations, the joint analysis and the individual analyses separated by hemisphere revealed very different pairwise disconnection patterns. In general, analyses separated by hemisphere showed a more realistic image with respect to the simulated connection-behavior association of the respective inter- or intra-hemispheric ground truth connections.

3.1 Simulation 1: Pairwise disconnection-based simulation

When analyzing left and right hemispheric pairwise disconnections together, only the inter-hemispheric disconnection became significantly associated to the simulated behavior (t = 8.37; p < .0001; Figure 3). This indicates that varying degrees of relevance were assigned to the association between inter- and intra-hemispheric disconnections to the target behavior, whereby the intra-hemispheric disconnections were not detected to be significantly associated to the target behavior (t = 4.83; p > .05). In contrast, when separating the analyses according to hemisphere, damage to both ground truth connections of each hemisphere, that is, the inter-hemispheric as well as the intra-hemispheric connection, was significantly linked to the simulated behavior (t = 5.86; p < .05; Figure 3). Interestingly, the obtained t-statistics for inter- and intra-hemispheric disconnections were identical, indicating that none of them were subject to a methodology-induced bias and in fact reflect reality, as the ground truth connections were indeed simulated as being equally relevant (cf. Table 1). Besides the ground truth connections, no further disconnections became significant.

3.2 Simulation 2: Lesion-based simulation

The obtained t-statistics for each ground truth connection and for the different tested noise levels are visualized in Figure 4a. For the intra-hemispheric ground truth connections, the t-statistics are reduced in the joint analysis compared to the hemisphere-specific analysis, whereas for the inter-hemispheric ground truth connection, the t-statistic is enhanced in the joint analysis compared to the separate analyses. In more detail, the inter-hemispheric connection was remarkably more significant than both intra-hemispheric connections in the joint analysis, while in contrast, the intra-hemispheric connections were (slightly) more significant than the inter-hemispheric one in both individual analyses (see Figure 4a). Hence, we observed a reversed order of the inter-hemispheric disconnection being superior to the intra-hemispheric disconnection for the joint analysis and inferior for the separate analyses. With increasing noise, the overall power of the ground truth connections decreased, which led to intra-hemispheric connections no longer surpassing the significance threshold in the joint analysis, as it is the case for 55% noise, for instance (see Figure 4a). Looking at the numbers of significant inter- and intra-hemispheric disconnections (Figure 4b,c), we found a large amount of significant inter-hemispheric disconnections compared to a (much) smaller amount of intra-hemispheric disconnections surpassing the significance threshold in the joint analysis across noise levels. The results obtained by the joint analysis approach suggest that mainly inter-hemispheric disconnections are associated with the behavior of interest. On the other hand, when investigating left and right hemispheric lesions and their corresponding disconnectomes in separate analyses, we found fewer significant inter-hemispheric disconnections than intra-hemispheric ones for left and right-sided lesions (Figure 4b,c).

When analyzing a much larger sample of 240 lesions in total, we found similar effects as the ones described above for the two investigated analysis approaches (see Figure S2 in the Supplement). Greater statistical power due to a larger sample size does therefore not prevent from the statistical bias induced by the joint analysis. However, the statistical power did benefit from an increased sample size so that certain ground truth connections could be detected in the separate analyses for noise levels, for which those connections could not be detected by the separate analysis approach using the smaller sample (e.g., 50% noise level).

4 DISCUSSION

In the present simulation study, we tested whether a single analysis of both unilateral left and unilateral right hemispheric lesions together or analyses separated by hemisphere would result in comparable pairwise, region-to-region structural disconnection results, or whether one or the other analysis approach should be preferred. We associated behavior with pairwise structural disconnections, which were either (i) directly simulated or (ii) estimated indirectly from simulated unilateral left and unilateral right hemispheric lesions. We observed that joint and separate analyses resulted in very different patterns of disconnection. Analyses separated by hemisphere showed a more realistic image with respect to the simulated connection-behavior association of the respective inter- or intra-hemispheric ground truth connections.

In the joint analysis of left and right hemispheric lesions, intra-hemispheric disconnections were statistically underpowered and thus surpassed the significance threshold with more difficulty compared to inter-hemispheric disconnections. In contrast, analyses separated by hemisphere did not neglect one type of disconnection. Our first, pairwise disconnection-based simulation, which represents a scenario of perfect circumstances, clearly preferred the approach of analyzing both hemispheres separately, because only this approach could detect all ground truth connections. This finding indicates that the procedure of separate analyses is methodologically seen the undisputedly best approach. Our second simulation, which represents the noisier but more natural scenario, showed similar effects. Different from our first simulation, with increasing noise, the separate analyses as well as the joint analysis could not detect all ground truth connections. However, this issue is more of a general problem resulting from lower effect sizes due to insufficient sample sizes. Our simulation revealed that—at the same noise level—an increased sample size led to the detection of all underlying ground truth connections in the separate analysis approach, whereas the statistical bias was still preserved in the joint analysis. Our data thus suggest that—depending on different effect sizes, sample sizes, and background noise of the respective data set—the sole application of analyses separated by hemisphere can already be sufficient, while in other cases, separate analyses might not catch all ground truth disconnections because of the statistical threshold used and the insufficiently large effect size of the missed disconnection. However, increasing the sample size might eliminate this issue as shown by the present simulation. Also, reporting the uncorrected results might be an option to deal with small sample sizes.

In a recent article of our group, we conducted both a joint analysis of unilateral left and right hemispheric patients as well as individual analyses separated by hemisphere on real data of stroke patients suffering from “pusher syndrome” (Rosenzopf et al., 2023). We observed that the results obtained from the respective analyses were in accordance with the here described pattern. Indeed, we obtained a smaller amount of significant results when analyzing both hemispheres together; all additional disconnections that appeared in the separate analyses were ipsilesional intra-hemispheric disconnections (Rosenzopf et al., 2023). Additional research is needed to further develop the understanding of the implications that the choice of analysis approach has on clinical data.

Besides the statistical bias that results to the disadvantage of intra-hemispheric disconnections, there are further aspects arguing against the application of joint analyses. First, even if all ground truth connections would survive the significance threshold due to sufficiently large effect sizes, this analysis approach still impacts the level of significance of disconnections. Intra-hemispheric disconnections would still be misinterpreted in a joint analysis as less relevant than inter-hemispheric ones. In fact, they might be ignored since most researchers tend to discuss only the most significant findings, especially when there are dozens of significant items. Second, another argument against joint analyses is that findings are less interpretable with respect to intra-hemispheric disconnections, since a significant intra-hemispheric disconnection does not allow any meaningful conclusion about the whole sample (i.e., the behavior/deficit of interest) because such a disconnection could only be caused by part of the sample. Third, even if one merges homologous left-sided and right-sided intra-hemispheric connections in the connectivity matrix together to solve the issue of statistical imbalance in the joint analysis, results would not be reliable as the assumption of homologous structural connections sharing also homologous functions cannot be considered as universally valid in studies dealing with cognitive functions in the human brain.

Sperber et al. (2022) concluded that the partial injury problem is less of an issue in disconnection studies since pairwise measures can sum up damage that affects the same fiber tract or pairwise connection regardless of distance between them. While this is true for the ground truth correlate of a single hemisphere, our present results demonstrated that this conclusion is not valid for ground truth correlates located within both hemispheres. We have shown that mechanisms similar to those that cause power limitations in voxel-wise mapping of lesion-symptom-associations also operate in pairwise disconnection studies if the sample consists of lesions in both hemispheres. Joint analyses decrease the probability of intra-hemispheric disconnections truly associated with a behavior of interest to be uncovered. Inter-hemispheric connections can be damaged in both left and right-brain damaged patients, while, on the other hand, (direct) intra-hemispheric connections can only be disturbed within the lesioned hemisphere. In short, the necessary statistical power might not be achieved for intra-hemispheric disconnections when analyzing left- and right-sided lesions together, while inter-hemispheric disconnections profit from the increased sample size. An exclusively performed joint analysis can therefore solely make specific inferences on the potential involvement of inter-hemispheric disconnections while systematically underestimating possibly existing intra-hemispheric disconnections. Analyses separated by hemisphere, on the other hand, seem to reflect the true associations more validly. The statistical power in this approach might be overall lower than in the joint analysis due to the smaller sample size and potentially smaller effect size of a ground truth connection. However, neither the power of inter-hemispheric nor intra-hemispheric disconnections is systematically biased in analyses separated by hemisphere. The latter approach therefore does not favor the results for one connection type. The obtained t-statistics of the ground truth connections of either hemisphere were closer together in the individual analyses compared to the joint analysis, reflecting the simulated data in a more realistic manner.

Since the systematic bias induced by the joint analysis is a consequence of the sampling rather than the applied statistical approach, it would also persist in multivariate approaches and Bayesian frameworks, which are known to alleviate some issues associated with univariate approaches and frequentist statistics, respectively. In the joint analysis, inter-hemispheric disconnections would receive larger Bayes factors than intra-hemispheric disconnections, reflecting the same mechanisms described in the present simulation using t-statistics. Moreover, the present conclusions retrieved from pairwise structural disconnection-deficit-associations by unilateral brain lesions, such as strokes or tumors, are very likely to also apply to tract-wise disconnection measures, which define the damage not based on disconnected region-pairs but on the disconnected white matter fiber tracts. Tracts, like region pai, can be characterized as inter-hemispheric (when the tract passes through both hemispheres) or intra-hemispheric (when the tract remains within one hemisphere) and are therefore likely susceptible to the same statistical bias we found here for pairwise measures. However, future studies should verify this assumption.

5 CONCLUSION

The results derived from the present simulation study on pairwise structural disconnection analyses indicate that jointly analyzing unilateral left and unilateral right hemispheric lesions leads to a methodological bias, whereas the approach of individual analyses separated by hemisphere represents the accurate way to deal with such data sets. Joint analyses systematically underestimate possibly existing intra-hemispheric disconnections, which might therefore be considered less relevant or even completely missed. In contrast, the approach of hemispheric separation avoids such biased results. Hence, we propose that analyses separated by hemisphere should be considered as standard for pairwise (and tract-wise) structural disconnection studies.

CONFLICT OF INTEREST STATEMENT

The authors report no competing interests.