2.1 The 13 meta-dimensions (13-MD) classification system

The 13-MD is a measure of GHRQoL consisting of 33 items that fall into 13 meta-dimensions. A meta-dimension is a set of related components (items) designed to gauge a particular facet of health. The 13-MD measures different aspects of health, including five meta-dimensions for physical health, four meta-dimensions for mental health, three meta-dimensions for social health, and one meta-dimension for sexuality and intimacy. These meta-dimensions include “Body Functioning,” “Cognitive abilities, senses, and language,” “Sleep and energy,” “Self-esteem and self-acceptance,” “Physical pain and discomfort,” “Mobility and physical disability,” “Daily activities and work,” “Social and leisure activities,” “Social and interpersonal relationships,” “Citizenship, inclusion, and autonomy,” “Depression, anxiety, and anger,” “Well-being,” and “Sexuality and intimacy” [12]. For example, the meta-dimension “Body functioning” is composed of items like “Breathing,” “Eating,” and “Relieving oneself (urine and fecal matter).” This meta-dimension is one of the five meta-dimensions that aim to measure the respondent's physical health. Each item offers response options ranging from 5 to 7 levels, allowing for a vast number of potential health states (i.e., 1.42 × 10²⁶ possible health states). The measure was originally developed in French through a Delphi procedure, with an initial version containing 15 meta-dimensions. This version underwent psychometric validation through a survey targeting the general population of Quebec [12]. The final version with 13 meta-dimensions was established after observing misfitting items. An English version of the 13-MD is also available [13].

2.2 Study population

The final version of the 13-MD was submitted to a survey and data from the general French-speaking population of Quebec were collected through an online design survey in 2023. Participants were recruited by Dynata Inc., a company specialized in online surveys. To ensure representativity across gender, age, and education, a quota sampling method was used. To be eligible, participants must reside in Quebec (Canada) and be over 18 years old. Participants rated items on a scale from 1 (best level) to 5–7 (worst level) [12] in the final version of the 13-MD [13]. To date, no utility score can be derived from the instrument and a value set is planned to be developed to make it usable in QALY calculation for CUA. Only participants who provided complete responses were included in the analysis, and any incomplete observations were removed. Considering the Quebec population, a minimum of 1067 participants was required to ensure representativity with a confidence level of 95% and a risk of error below 3%.

2.3 Psychometric tests

To ensure better comparativeness, the same aspects of the previous validation process were considered [12]. The degree to which items are connected and measure the same construct, known as internal reliability or consistency, was evaluated. Cronbach's α was computed to measure internal reliability (or consistency) which informs the extent to which the different items of an instrument measure a same construct or are interconnected [14]. The Cronbach's α is generally between 0 and 1 and a value between 0.70 and 0.95 was targeted (i.e., a higher coefficient means that items are correlated and measure the same concept) [15, 16]. Additionally, items that if eliminated would increase the Cronbach's α score by 2% were identified [15]. We also computed the item-test correlation, which measures the correlation between each item and the overall test correlation. The item-rest correlation was calculated to measure the correlation between one item and the other items. Furthermore, we examined the interitem correlation to assess the level of correlation between the items. We expected the interitem coefficients to range from 0.2 to 0.4 [17], item-test coefficients to be high and approximately constant across items, and item-rest coefficients to be greater than 0.2 [18].

All items were examined for floor and ceiling effects, which indicate the proportion of respondents who always selected the worst and best possible answers, respectively (i.e., the lowest and best levels describing health). This was done to ensure that the instrument's reliability was maintained and that the response levels allowed us to record variations between users [19]. To rule out the absence of ceiling or floor effects, less than 15% of responses should always indicate the best/worst achievable score [19].

A principal component analysis (PCA) was implemented to conduct an exploratory factor analysis (EFA) which helps in identifying the underlining factor structure or model for a set of variables [20, 21]. Using directions called principal components, the PCA helps in minimizing the dimensionality of the data and preserving most of the variation within it [22]. This exploratory analysis was conducted to determine if the same distribution pattern of the different items across the various components would be observed when using the new collected data as in previous analyses [12]. If this pattern holds true, we can then confirm the stability of its structure even when using different data sets.

The Bartlett test of sphericity and the Kaiser Meyer Olkin (KMO) test were also performed to assess the data's suitability for factor analysis. If results were satisfying, the PCA was performed. In our case, the varimax rotation, which is the most popular rotation method was applied [23]. It consists in searching for a rotation of the initial components to maximize the loadings' variance [23]. The PCA helped analyze the loading patterns and confirm items grouped together in the same meta-dimension (i.e., groups consisting of related items that assess a specific health characteristic) [24]. Items that contributed less than 20% to the components were considered problematic.

The EFA conducted through the PCA helps to identify and confirm the factor structure previously obtained including the number of factors and pattern of factors loadings. In addition to this exploratory analysis, a confirmatory factor analysis (CFA) was also performed. Unlike EFA which is more used as a descriptive technique, CFA, conducted through a structural equation model (SEM), allows to specify the structure in advance and assess how well the previously obtained structure replicates the sample covariance matrix [21, 25]. The CFA also provides evidence in supporting the convergence of the 13-MD's structure [25]. Various statistics, including the root mean square error of approximation (RMSEA), standardized root mean square residual (SRMR), comparative fit index (CFI), Tucker and Lewis Index (TLI), Akaike's information criterion (AIC), and Bayesian information criterion (BIC) were compiled and compared to the results of our previous analyses. We expected the RMSEA to be lower than 0.08 (the closer to 0, the better), SRMR to fall between 0.05 and 0.1, CFI and TLI to exceed 0.9, and AIC and BIC to be low [24, 26-29].

The item response theory (IRT) model was used to assess the questionnaire's ability to distinguish between individuals in different regions on the latent continuum. IRT aims to understand or forecast the relationship between unobservable qualities or attributes (latent traits) and their observable manifestations (results or reactions) [30-32]. The IRT calculates the likelihood that individuals with varying levels of the latent trait will respond to the items in different ways. Among its many benefits, IRT enables the identification of how the level of precision may vary across different items or levels of the instrument, unlike classical test theory, which uses a single estimate to describe instrument's reliability [31]. The discriminant coefficient was of interest as a coefficient of 1.0 indicates good discrimination by the item; individuals with low ability (health) are much less likely to achieve a higher score compared to those with average or high ability. Therefore, a higher discrimination coefficient indicates better ability of the item to distinguish and provide accurate information. It is important to avoid a negative discrimination coefficient as it would imply that the probability of achieving a higher score decreases as the respondent's ability (health) increases [33].

Instrument development frequently employs the Rasch analysis method which is part of the IRT models and is probabilistic. The Rasch procedure is an analytical approach that aids in the development of new instruments and in assessing the quality of existing questionnaires [34]. The Rasch analysis allows us to analyze the fit of the questionnaire (unidimensionality) using various measures such as person/item separation reliability, person/item separation index, standard error of measurement (SEm), infit and outfit statistics, and others [6, 34, 35]. The person/item separation index and person/item separation reliability were expected to be greater than 0.2 and 0.7, respectively [36]. As for the SE, the smaller the value the better it is [36]. Ideally, infit and outfit statistics should fall between 0 and 1.5, indicating that the item functions as intended within this range [37]. Additionally, the point-measure correlation (PTMEASUR-AL) was calculated to determine the correlation between the item and the entire set of items for all observations. Generally, there is a close and positive relationship between the model's predicted expected value (EXP) and correlation value (CORR) [38]. Furthermore, the Rasch analysis helped identify any potential differential item functioning (DIF), which is used to assess the fairness of the items. DIF techniques evaluate whether different respondent subgroups have varying probabilities of responding to items in different ways [19, 39]. The Rasch-Welch, Mantel, and cumulative log-odd in logits (CUMLOR) statistics are commonly used to evaluate DIF. According to the guidelines provided on the Winsteps website, an item with a CUMLOR absolute value of less than 0.43 is considered to have a negligible difference in functioning. A value between 0.43 and 0.64 indicates a slight to moderate difference in functioning. A value above 0.64 indicates the presence of a moderate to large difference in functioning.

2.4 Meta-dimensions and items' ranking

The meta-dimensions and items of the 13-MD were ranked to determine their relative importance. The same participants who completed the 13-MD also performed the ranking tasks within the same survey. They were given 100 points and asked to distribute them among the 13 meta-dimensions. After that, they were asked to distribute another 100 points among the items of each meta-dimension. There was no intention to compare items not belonging to the same meta-dimension, as ranking 33 items would have been too cognitively demanding. If a respondent used all their points for a meta-dimension or item, leaving some with missing values, those missing values were replaced with zeros, assuming empty scores did not carry any weight for the respondent. Standardized weights were calculated to account for respondents who gave values exceeding the initial 100 points. For the ranking of meta-dimensions, each meta-dimension's score was divided by the sum of all meta-dimensions scores and then multiplied by 100. The same process was applied to the ranking of items, where the items' scores were divided by the total score of the corresponding meta-dimension. This ensured that all weights fell within a range of 0–100.

Participants with incomplete or unusable responses were excluded. Unusable responses referred to respondents who did not seem to understand the task and instead provided comments rather than the required scores.

For all analyses, Winsteps Rasch 5.1.5.2 and Stata BE 17 (Stata Corp) were utilized.

3.1 Psychometric tests

The confirmatory analysis of the 13-MD was conducted using data collected through an online survey involving the general population of Quebec. 1337 French-speaking participants were invited to participate in this survey. During the screening process, 238 observations were excluded, due to incompleteness. The remaining 1099 observations were included for psychometric tests. Descriptive statistics collected from the respondents can be found in Table 1.

Cronbach's α was 0.9524, which falls within the desired range. When we eliminated any of the items, none of them resulted in an increase in Cronbach α's value by 2% or more. Additionally, all items in the item-test, item-rest and interitem correlations displayed good and desired coefficients (see Table 2). Approximately 2.91% of participants always selected the best option available, while only 0.09% always chose the worst option for each response choice. Based on these results, we can conclude that the 13-MD does not suffer from floor or ceiling effects, as the percentages are significantly below the previously mentioned cut-off point of 15%.

PCA was conducted to validate the pattern described by the correlation analysis. Before conducting PCA, the KMO test and Barlett test of sphericity were performed. The KMO test indicated a sampling adequacy of 0.95 (i.e., the higher the value, the better it is). The Barlett test revealed a significant χ² value of 22257.455 (p < 0.001) indicating sufficient intercorrelations in our data to proceed with factor analysis. To determine the components retained in the varimax rotation, an unrotated PCA was carried out. Six components were retained and accounted for 64.27% of the total variance. The contribution of each item to the explained variance of each component was ascertained using PCA with varimax rotation. All items appeared to have coefficients greater than 0.2. It was observed that almost all items belonging to the same meta-dimension were in the same component (see Table 3).

Regarding the CFA, the estimated SEM showed good statistics, very close to those observed in our previous results using a different data set [12]. The statistics obtained in Model 2 (compared to those of our previous analysis named Model 1) are presented in Table 4. In both models, the goodness of fit of the 13-MD is confirmed. All variables in our model also appeared to be statistically significant (p < 0.05).

Using the IRT model with graded responses and Rasch analysis, the model fit was tested. When evaluating the discriminant capacity of the items, all discriminant coefficients showed signs that were of high value (>1) and significant. Rasch analysis yielded similar results (see Table 5). The item separation reliability (0.99), item separation index (12.46), person separation reliability (0.87), and person separation index (2.57) were all satisfactory. Additionally, the SE was extremely low at 0.2.

One item, specifically “Anxious or stressed,” appeared to be associated with moderate to large DIF. However, its coefficient did not exhibit any significant issues in the end (see Table 6).

The excellent results obtained confirmed the goodness of fit of our questionnaire. No item revealed any unwanted score during the validation process. It is thus confirmed that the 13-MD is a valid tool designed to measure what it is intended to.

3.2 Meta-dimensions and items' ranking

In the psychometric validation, 1099 respondents provided complete responses, with 15 respondents giving unusable responses. Therefore, the ranking procedure was based on the responses of the remaining 1084 participants. The ranking of meta-dimensions and the ranking of items within each meta-dimension are presented in Table 7.

Ranking scores were found to be evenly distributed among meta-dimensions indicating that each meta-dimensions holds significance. However, the meta-dimension “Well-being” received the highest score of 11.45 suggesting it is the most crucial aspect of health for the respondents. This was followed by “Sleep and energy” with a score of 8.86, and “Body functioning” with a score of 8.65. The items “Fulfilled,” “Sleep,” and “Breathing” were the top-ranked within their respective meta-dimensions. On the other hand, the meta-dimensions “Sexuality and intimacy,” “Citizenship, inclusion, and autonomy,” and “Depression, anxiety, and anger” were deemed as the least important, scoring at 6.50, 6.03, and 5.07, respectively.