Reliability of the MRI-based Brain Atrophy and Lesion Index in the evaluation of whole-brain structural health

1 INTRODUCTION

Aging is a process characterized by deficit accumulation over time.1 As is the situation with the body,2 various degenerative changes can also accumulate in the brain,3 and they interact to overwhelm repair processes, causing high-level failure in brain functions, and can lead to cognitive decline and dementia.4 To collectively evaluate the additive effects of several common structural deficits on brain function, a semiquantitative rating scale, the Brain Atrophy and Lesion Index (BALI), has been validated with the application of multiple research datasets.5, 6 The BALI assesses global atrophy (GA) and lesions in both the supratentorial and infratentorial compartments. Categorized changes are assessed as lesions in the gray matter (eg, cortical infarcts) and dilated perivascular spaces in the subcortical white matter as well as lesions in the periventricular regions, deep white matter, basal ganglia, and the surrounding regions.4-6 By integrating multiple deficits in the aging brain into one scale, the BALI has been used as a proxy measure of brain health status and a way to model the dynamic brain changes in the process of aging.3, 4, 7

To date, the BALI has demonstrated sensitivity in the relationship with age and cognition, differentiating people with different cognitive diagnoses and those with high risks for cognitive decline, and predicting those converted to dementia.3-8 Moderate-to-high reliability has been reported for BALI based on different sequences.4, 9, 10 Nevertheless, previous studies have chiefly used high-field MRI (eg, 3.0 tesla) with a relatively high signal-to-noise ratio and assessed the brain of older subjects with many age-associated changes.4, 9, 10 It is yet to be determined (a) whether the BALI method can also be reliably applied in evaluating people of younger ages and (b) whether the data acquired from clinically conventional MRI may also be used to score BALI with acceptable reliability.

To test the generalizability of the BALI, here, we examined the measurement property of the BALI using the data from the adults of a wide age range and at both high and conventional MRI field strengths.

2 METHODS

2.3 Evaluation of the Brain Atrophy and Lesion Index

As described elsewhere,4-10 the BALI is a semiquantitative summary rating scale, adapted from several well-established scales that assess localized structural changes.11, 12 Changes that are integrated into the BALI evaluation include gray matter lesions (eg, cortical infarcts) and subcortical dilated perivascular spaces (GM-SV), deep white matter lesions (DWM), periventricular white matter lesions (PV), lesions in the basal ganglia and surrounding areas (BG), lesions in the infratentorial compartment (IT), and GA.

Applying the BALI rating schema, a value between 0 and 3 was assigned to assess a change in each category, with a higher score indicating greater severity. In the categories DWM and GA, the values of 0-5 were used, allowing the capture of more severe changes and thereby avoiding ceiling effects. The “other findings” category was included to record other possible changes such as neoplasm, trauma, idiopathic normal-pressure hydrocephalus, focal asymmetry, and deformity, each of which is sometimes seen in older adults. The BALI total score was calculated as the sum of subscores of all the seven categories.5

Figure 1 shows examples from the sample of the BALI categories evaluated as having different subscores (Figure 1).

3 RESULTS

As detailed in Table 1, the randomly selected subsamples for reliability testing well represented the samples under different experimental conditions, regarding age and other demographic features (Table 1). The BALI total score differed by age (4.20 ± 2.27 for the younger group vs 11.37 ± 2.79 for the older group), which was consistent for both raters and with no age vs field strength interactions (F_age = 77.02/64.73, F_{field strength} = 2.51/5.22, F_interaction = 0.34/0.22).

Considering the reliability of the BALI total score (Table 2), the ICC for the interrater agreement was 0.94 (95% CI = 0.90-0.97) for the overall sample. The interrater ICC values were also good for each of the different experimental conditions: 0.96 for younger, 0.92 for older, 0.92 for 1.5T, and 0.94 for 3.0T. There was no statistical difference in the ICC between age groups (Z = 1.43, P = 0.153) or field strengths (Z = 0.60, P = 0.549).

The ICC for intrarater agreement of the BALI total score was consistently high for each rater (ICC ≥ 0.90 for the total sample). There was no difference in the intrarater ICC between younger (≥0.88) and older (≥0.91) age groups (Z ≤ 1.82, P ≥ 0.069), or between 1.5T (≥0.88) and 3.0T (≥0.91; Z ≤ 1.61, P ≥ 0.107; Table 2).

The reliability for the BALI category subscores was also high for the overall sample tested; the Cohen K coefficients ranged between moderate (0.61) for DWM and perfect (0.85) for GM-SV (Table 3). The Cohen K values were more significant in the younger than in the older subjects for GM-SV (0.83 vs 0.46) and BG (0.79 and 0.52), but not for other subscores; in contrast, the Cohen K values were consistently higher (except for DWM) at 3.0T (0.68 for IT to 0.85 for GA) than at 1.5T (0.50 for BG to 0.65 for GM-SV), demonstrating robust identification of these brain changes due to higher field strength (Table 3).

Under any given condition, the intrarater agreement rates for any BALI category subscore were always better than moderate and mostly comparable between the raters (Table 3).

4 DISCUSSION

In the present study, we investigated the reliability of the BALI evaluation using clinical routinely acquired 2D T2-weighted MRI at 1.5T and 3.0T in a convenience sample of younger and older adult participants. Reliability tests were conducted on the BALI total score and each of the category subscores. Inter- and intrarater agreement rates were compared between age groups (younger vs older) and field strengths (1.5T vs 3.0T). Our data suggested consistently high inter- and intrarater agreement rates with the BALI total score under different experimental conditions, which did not differ between the different age groups and field strengths. The data also suggested moderate-to-perfect inter- and intrarater agreement rates with each BALI category subscore regardless of age group and field strength, even though the reliability for a few categories was higher at 3.0T than at 1.5T.

Previous research has repeatedly shown that the BALI can be used to capture and summarize several common structural changes in the aging brain, thereby providing a way to study the impact of global structural changes on brain function in older adults.5-10 It has been known that various structural brain changes can coexist with brain aging, reflecting heterogeneous profiles in the process of age.15 Importantly, brain changes, including cerebral volume loss, can start early in the adult life span.16 Many of these changes can be clinically less significant or meaningful when considered individually, but when combined, they produce additive effects on function.4 Indeed, our previous work has shown that BALI total score differed significantly by age and in older people with different cognitive conditions.5-8 Here, the high reliability of the BALI total score and subscores in both younger and older subjects suggests that the BALI method may also be used in studying younger adults. Thus, the evidence yielded from the present study is in support of extending the BALI method beyond older ages, allowing enhanced generalizability of the BALI for the global assessment of brain health changes in broader study settings. This contribution is of significance, due to the critical opportunity to investigate the accumulation of deficits in the brain prior to older adulthood.

The BALI focuses on morphologic changes from the widely available routine clinical sequences.10 Here, we have further tested the reliability of the BALI evaluation using MR imaging acquired at both 1.5T and 3.0T. Consistent with a previous report on older adults,9 the data from the present study involving both older and younger adults have confirmed that the method is easy to master and the evaluation time is manageable.5 The interrater agreement rate of the BALI total scores is satisfactory at both field strengths and by both raters, indicating the robustness of the BALI total score as a global measure of structural brain health. Meanwhile, the increased reliability for several category subscores at 3.0T demonstrates advantages of high-field strength: The higher signal-to-noise ratio can benefit BALI evaluation, allowing the raters to more sensitively identify subtle changes and robustly grade the BALI categories.

Our data must be interpreted with caution. In this study, both raters who evaluated the images are experts in diagnostic neuroimaging and have well mastered the BALI method. Whether nonneuroradiologist raters may require additional training and practice to achieve the same level of reliability for each experimental condition remains to be determined. However, previous studies have shown highly reliable rating scores for older participants using 3.0-T T1-weighted MRI by nonneuroradiologist raters trained with the BALI method (using the rating schema descriptions, atlas, examples, and case discussions).5, 17, 18 Furthermore, our study used a subsample of the subjects from one study site for the reliability testing. Although the statistical analyses suggested satisfactory reliability, to what extent that the findings can be generalized to the general population deserves further research with increased sample sizes.

In conclusion, our study suggests that multiple structural brain changes can be collectively evaluated with the use of the BALI total score in adults of a wide age range. The BALI score based on both 1.5-T and 3.0-T T2WI is highly reliable in capturing global brain changes that can accumulate across the adult life course. High-field MRI can further improve the robustness in detecting subtle changes.

CONFLICT OF INTEREST

Tao Gu receives a fellowship award from Beijing Hospital to conduct postdoctoral research in Canada. Hui Guo receives a fellowship award from the China Scholarship Council to conduct postdoctoral research in Canada.