Methodological issues in a meta-analysis^†

In a recent article published in HEPATOLOGY, Awad et al.1 present a meta-analysis comparing peginterferon alfa-2a and peginterferon alfa-2b for hepatitis C treatments. Using data from eight trials, the authors concluded that the proportion of patients achieving sustained virological response (SVR) with peginterferon alfa-2a was significantly higher than the proportion achieving SVR with peginterferon alfa-2b. Of the 3070 patients in the Individualized Dosing Efficacy Versus Flat Dosing to Assess Optimal Pegylated Interferon Therapy (IDEAL) trial, 1016 were included in the meta-analysis, even though they received a dosage of 1.0 μg/kg/week, which is lower than the approved starting dosage of peginterferon alfa-2b (1.5 μg/kg/week). This letter, however, focuses on concerns about methodological issues in the meta-analysis.

The validity of the random effects model used in the article is dependent on a large number of individual studies, each with sufficiently large samples.2-4 In Awad et al.'s study,1 only eight trials were used, five of which had fewer than 50 observations per arm. The largest trial5 alone represented nearly 70% of observations. Based on that trial, past research has shown no difference between peginterferon alfa-2a and peginterferon alfa-2b with respect to SVR. This underlines the importance of sensitivity analyses for examining the robustness of Awad et al.'s conclusions. Because the authors showed that there was no evidence of heterogeneity (I² = 0) among the eight studies that they included, a fixed effects model approach is appropriate. I applied a fixed effects exact inference procedure (proposed by Tian et al.6) as a sensitivity analysis to the data shown in Fig. 2 of Awad et al.'s article. This method is unbiased even for small samples or when only a small number of studies are included in the meta-analysis. This robust method yielded a 95% confidence interval for the risk ratio of 0.988-1.214, which included the null value of 1. A similar analysis limited to the approved starting dose of 1.5 μg/kg/week (which excluded 1016 of the 3070 patients in the IDEAL trial) also showed a lack of a statistically significant difference with an exact 95% confidence interval of 0.967-1.214.

The discrepancy between the findings based on large sample methods and those based on exact methods emphasizes the need for thorough sensitivity analyses using a variety of appropriate statistical methods whenever possible. Here, the use of an exact method, which is likely more appropriate because of the limited number of studies, shows no statistically significant difference between the two drugs; this result directly contradicts the findings of Awad et al.1

Aside from the biases resulting from the reliance on large sample properties when statistical analyses of small samples are being performed, current meta-analyses often reduce a complicated, multitrial meta-analysis to a single parameter from which absolute conclusions are drawn. A number of recent articles, including ones by Wang et al.7 and Cai et al.,8, 9 present a method that, applied to the issue of peginterferon alfa-2a versus peginterferon alfa-2b, would provide clinicians with clearer guidance about which product is most likely to be an appropriate treatment for any given patient.