The impact of individual and methodological factors in the variability of response to methylphenidate in ADHD pharmacogenetic studies from four different continents^†‡

INTRODUCTION

Pharmacogenetics addresses the association between genes and clinical response to pharmacological interventions [Goldstein et al., 2003]. Targeting genes related to the pharmacokinetics or pharmacodynamics of a medication, pharmacogenetics aims to understand the variability among individuals in the rates of adverse reactions or clinical improvement associated with medication use. Besides its potential in tailoring interventions to the genetic background of individuals, pharmacogenetics can illuminate neural pathways of specific disorders [Polanczyk et al., 2007b].

Several research groups have evaluated the association of individual polymorphisms and the response to methylphenidate (MPH) in subjects with attention-deficit/hyperactivity disorder (ADHD) [Polanczyk et al., 2005; Stein and McGough, 2008]. These studies focused mainly on genes believed to be associated with the disease [Faraone et al., 2005], predominantly from the dopaminergic system [Roman et al., 2002; Stein et al., 2005; Mick et al., 2006; Cheon et al., 2007; Kooij et al., 2008], but also from the noradrenergic [Polanczyk et al., 2007b] and serotoninergic systems [Tharoor et al., 2007; Zeni et al., 2007]. To date, there are few replication studies for each polymorphism of interest and the existing replications have provided inconsistent results [Roman et al., 2002; Stein et al., 2005]. Methodological aspects of the studies, such as variability in design, diagnostic and outcome measures used, as well as limitations, such as not controlling for confounding factors, have been hypothesized to be related to these differing results [Polanczyk et al., 2005]. Moreover, considering that the effect of a single gene on the response to medication is expected to be small, most samples are probably underpowered to detect existing effects [Goldstein et al., 2003].

Collaborative studies aggregating samples from diverse international centers have the potential to assess the effect of polymorphisms on methylphenidate's response with adequate power [Goldstein et al., 2003]. Nevertheless, this strategy has been criticized because it may introduce sampling variability due to different ethnicity and methodological strategies between studies. In other words, if methodological aspects of the studies or ethnicity were related to heterogeneity in the response to MPH between investigations, the potential gain in power would be offset by the introduction of noise into analyses. In a meta-analysis of methylphenidate for treating adult ADHD, Faraone et al. [2004] found a significant association between type of rater, dose and variability in efficacy of methylphenidate in adults with ADHD. In a subsequent meta-analysis of medications used to treat ADHD in children, Faraone et al. [2006] reported that the design of the trial (parallel vs. crossover) and the conceptualization of outcome (end score vs. change of score) were significantly associated with the reported effect size of methylphenidate. Such issues would likely also add variability to pharmacogenetic studies of ADHD.

Thus, we decided to conduct a two stage analysis: (a) we evaluated the role of methodological aspects in the variability of ADHD symptom improvement between studies using meta-regression analyses in the available data sets on pharmacogenetics from the ADHD Molecular Genetics Network; and (b) we assessed the role of individual characteristics of the subjects in the variability of ADHD symptoms improvement using multivariate regression analysis in the same data sets. It is important to stress that we have not included genetic data in our analyses, although all studies included here were designed to evaluate the role of specific polymorphisms on the response to methylphenidate. Conceptually, prior to aggregating studies to evaluate the effect of specific polymorphisms on response to methylphenidate, it is essential to assess how methodologic features that differ among studies contribute to estimates of treatment response, because such factors will contribute noise to cross-site pharmacogenetic analyses if not addressed.

MATERIALS AND METHODS

This study has been conducted in the context of the Pharmacogenomics Working Group of the ADHD Molecular Genetics Network, which brings together collaborative research groups for studying the molecular genetics of ADHD [Faraone, 2003]. Participants of the Working Group were invited to collaborate providing data on pharmacogenetic studies of MPH in subjects with ADHD (published or not) that have been conducted at participating centers. Data on each individual were requested, and these included demographic and clinical variables, as well as variables related to the treatment. Furthermore, the following study-level data were requested: continent, design, symptom-rating scale used, rater who completed the scale, and sample size.

We computed the effect size of MPH for each study through the standardized mean difference, using change-from-baseline measure. We calculated the change in total ADHD symptoms from pre- to post-treatment within individuals and used the pooled standard deviation to generate the effect size. This strategy corrects for the correlation between measures within a patient for open label studies [Curtin et al., 2002; Elbourne et al., 2002].

Analyses were conducted separately for variables at the study and individual levels. Random-effect meta-regression analysis evaluated the effect of study level variables on the heterogeneity of results across different studies. This strategy has been previously applied to assess the effect of methodological and demographic variables in ADHD prevalence estimates [Polanczyk et al., 2007a] and in methylphenidate's effect size [Faraone et al., 2004, 2006]. The study level variables assessed were: continent where the study was conducted, design of the study (open-label vs. randomized controlled trial [RCT]), sample size, symptom-rating scale used, and rater who completed the scale. Variables associated with the outcome at a P value <0.2 in univariate analyses were included in the multivariate model. Given the results obtained, we conducted an additional analysis, which consisted of a multivariate model comprising all variables independently of the univariate results.

The association between individual level variables and percentage of change from pre- to post-treatment in combined inattention and hyperactivity-impulsive symptoms (negative results indicate decrease of symptoms) was analyzed with linear regression analysis. Independent factors assessed were: time from pre- to post-treatment evaluations, baseline severity of ADHD symptoms, age, gender, ethnicity, ADHD subtype, presence of conduct disorder (CD), oppositional defiant disorder (ODD), any mood or anxiety disorder, any other disorder (eating, substance use/abuse, tic, Tourette syndrome, enuresis/encopresis), IQ, previous use of MPH, MPH formulation administered (immediate release [IR], long-acting [LA], osmotic release [OROS]), MPH dose (per kilograms per day), number of times MPH was administered a day if immediate release, and concomitant use of another medication. Initially, the association between independent factors and baseline severity of ADHD symptoms was investigated with simple linear regression analysis to check for colinearity. Subsequently, the association between each independent factor and the outcome was evaluated, with baseline ADHD score included as a covariate when appropriate. Finally, a multiple linear regression model was constructed including all explanatory variables associated with the outcome in univariate analysis at a P value <0.2. A backward elimination procedure was applied for the construction of the final model. At this stage, a P value <0.05 was considered as statistically significant. All analyses were conducted with STATA version 9.2.

RESULTS

Nine samples [Stein et al., 2005; van der Meulen et al., 2005; Mick et al., 2006; Cheon et al., 2007; Reinhardt et al., 2007; Polanczyk et al., 2007b; Kooij et al., 2008; Purper-Ouakil et al., 2008] from seven centers were included in the study. Characteristics of the samples are described in Table I. The pooled random effect size of methylphenidate for the treatment of combined inattentive and hyperactive-impulsive symptoms was estimated to be 1.32 (CI 95% 0.88–1.76). Findings indicated the presence of heterogeneity among samples in the change of ADHD symptoms after treatment with methylphenidate (Q = 108.34; df = 8, P < 0.001, τ² = 0.42). Univariate meta-regression analyses revealed that study design (P = 0.002) was significantly associated with heterogeneity of results, while continent, symptom-rating scale, rater, and sample size were not. All variables were included in the multivariate model, and again only design of the study was associated to heterogeneity of results (P = 0.001).

At a second stage, the samples were aggregated and the effect of individual level factors for the change of ADHD symptoms from pre- to post-treatment of the 782 individuals was assessed (Table II). The following independent factors were associated to the outcome at a P value <0.2 in univariate analyses: age, ADHD subtype, ODD, any mood disorders, any other disorders, IQ, number of times MPH was administered, previous use of medication, baseline severity of ADHD symptoms, and time of follow-up. Since information on the number of times MPH was administered was available for only 200 individuals, it was excluded from analyses. In the final multivariate model, age (β = −0.45, SE = 0.08, P < 0.001), presence of ODD (β = 9.58, SE = 2.38, P < 0.001) and baseline severity of ADHD symptoms (β = −0.41, SE = 0.06, P < 0.001) were associated with the percentage of change from pre- to post-treatment in combined ADHD symptoms. This model accounted for 8% in the variance in the change of scores from pre- to post-treatment. To further understand the effect of age on response to methylphenidate, we calculated the effect size of methylphenidate for preschoolers, school-age children, adolescents and adults. Results indicated an effect size above 1.2 for all strata, with minor differences between age-ranges related to the variability of data (data available upon request).

DISCUSSION

This study evaluated the association between individual and study level factors and response to MPH aggregating pharmacogenetic studies from diverse cultural backgrounds. Study design (open-label vs. RCTs) was significantly associated with heterogeneity of results. Furthermore, in the analyses of individual level characteristics, age, comorbid ODD, and baseline severity of ADHD symptoms were associated with change in ADHD symptoms during treatment with MPH.

This is the third study conducted to date that evaluated the association of methodological characteristics and response to methylphenidate and the first one to assess exclusively pharmacogenetic studies. Faraone et al. [2004] assessed six RCTs of adults with ADHD treated with methylphenidate and detected significant effects for type of rater (self- vs. physician rating) and dose but not for design of the RCTs (crossover vs. parallel). In a subsequent study, Faraone et al. [2006] evaluated the association between methodological aspects of 29 RCTs of ADHD youth and improvement of ADHD symptoms with stimulants and non-stimulants medications. The authors detected the association between effect sizes of medications and both design (crossover vs. parallel RCT designs) and type of score (outcome vs. change scores). Other characteristics, such as type of raters, the score categories used to assess efficacy, the use of fixed-dose versus titration for best dose designs, whether or not subjects with a history of non-response were excluded, exclusion of nonresponders, or use of a placebo lead-in were not significantly associated with medications' effect size.

In our study, we compared exclusively open-label studies, without a control group, versus RCTs. Since only three RCTs were included, we were not able to test differential effects of parallel versus cross-over RCTs in the heterogeneity of results. Our analyses demonstrated that open studies yielded higher effect sizes of methylphenidate in comparison to RCTs. This result was expected, since effect size calculation for the former kind of studies does not take into account the placebo effect, over-estimating the effect of the intervention. Although a relationship between any polymorphism and a placebo effect is yet to be demonstrated in the ADHD pharmacogenetic arena, the inclusion of a control group for analyses is always the best strategy. This may be accomplished with randomized controlled trials assuring that treatment and control groups “are equal with respect to all features,” except the treatment assignment, or with observational, non-randomized trials. The latter design reflects patients from “real world” and, once accounting for the differences between patients from groups with appropriate statistical techniques (e.g., propensity scores), can yield results of significant clinical interest. In regard to the type of outcome measure, we decided to use only change scores. This method takes into account the baseline symptom severity, which is important since patients with higher initial severity have more “room” to present reduction of symptoms that those mildly symptomatic. Moreover, since the studies have followed individuals for a relatively short period of time, it is desirable to take into account the trajectory of response and not solely the endpoint. Thus, we could not assess the effect of outcome measure on heterogeneity of results, as previously evaluated [Faraone et al., 2006]. Change-from-baseline outcome measures are advantageous in open label studies without a control group and in parallel RCTs [Elbourne et al., 2002]. However, cross-over designs present unique characteristics that must be considered when aggregated in meta-analysis (e.g., possibility of carryover, order effect, binary data) [Elbourne et al., 2002]. Furthermore, due to the lack of data from the majority of the studies included, we were not able to evaluate the effect of study or individual level characteristics on the occurrence of adverse events, which is a clinically significant outcome that deserves further attention from future pharmacogenetic studies.

This study aggregated data from nine samples from four different continents (Europe, North America, South America, and Asia), and demonstrated that the origin of the sample was not associated with differential pattern of response to methylphenidate in the meta-regression analysis. The resulting sample size (n = 782) allowed the evaluation of ethnicity as an individual level variable, which was not significantly associated with the outcome. This is the first large-scale international study to evaluate the response to methylphenidate aggregating studies from different continents. These findings are in the same direction of those from the National Institute of Mental Health Multimodal Treatment Study of ADHD (MTA), which detected a substantial and clinically similar response to MPH between African Americans, Latinos, and Caucasians [Arnold et al., 2003]. These are also in the same direction as findings of a previous study where the site of the study was not related to heterogeneity in ADHD prevalence estimates [Polanczyk et al., 2007a]. However, these two studies differ in the sense that the current one is not a systematic review and our results are related to the set of pharmacogenetic studies included, which is not necessarily the same for all methylphenidate trials for ADHD. As we have previously stressed, we have not assessed the effect of specific polymorphisms on the outcome. In this way, we are not hypothesizing that specific polymorphisms implicated in the clinical response to methylphenidate are equally distributed across different ethnic backgrounds. This is a matter for future studies.

Age was significantly associated with improvement in ADHD symptoms in the pooled analysis in the same direction of previous evidence [Taylor et al., 1987; Buitelaar et al., 1995]. Although MPH is equally effective in treating children and adolescents with ADHD [Findling et al., 2001] and presents similar effect sizes for different age groups [Faraone et al., 2004, 2006; Brown, 2006], our results indicate the importance of taking in consideration the age range of subjects in the interpretation of pharmacogenetic results. Samples should be preferentially composed of individuals with narrow age ranges or this variable must be equally distributed across genotypes. Comparisons across studies conducted with different age groups must take this potential confounder into account. Furthermore, we detected an association between the presence of ODD and a reduced response to methylphenidate. Goez et al. [2007] evaluated the response to MPH in 1,122 children with ADHD. The authors detected a large subgroup of patients with comorbid ODD and anxiety disorders that presented significantly lower response. On the other hand, in 165 preschoolers treated with MPH, the presence of three or more comorbid disorders was related to the lack of response to treatment, while one (primarily ODD) or two comorbidities were not related to a decreased response to treatment [Ghuman et al., 2007]. In the MTA Study, which evaluated 579 children with ADHD, those with comorbid ODD and/or CD were similar to those without comorbidities in terms of response to treatment. However, the comorbid subgroup presented persistent differences at the end-point, indicating a less favorable prognosis [Jensen et al., 2001]. It is important to note that the evaluation of the role of comorbid conditions in the treatment of ADHD is usually conducted with statistically underpowered samples due to the prevalence of these conditions. In this regard, pooled samples allow a more adequate evaluation of these factors.

The possibility to identify study level characteristics of studies related to variability of results is recent, with three studies conducted in the ADHD field to date [Faraone et al., 2004, 2006; Polanczyk et al., 2007a], although meta-analysis is a frequently employed statistical technique. Even if overall tests of heterogeneity are non-significant, which can occur due to decreased statistical power, it is of interest to study the influence of methodological characteristics on variability of results [Thompson and Higgins, 2002].

The aggregation of ADHD pharmacogenetic studies imposes an additional methodological difficulty in comparison to the aggregation of genetic association studies, since the measurement of the effect of intervention can be influenced by a number of variables, including differences between subjects, design, delivery of treatment, and measurement. This is not a major problem within studies, since all patients are evaluated in the same manner, irrespectively of their genotype. However, this can be a source of variability when comparing results among studies. Nonetheless, our findings indicate joint analysis of pharmacogenetic studies is possible, since no fixed variables, such as the site where the study was conducted or ethnic background of individuals, was significantly related to outcome. Nevertheless, we recommend that analyses should be preferentially conducted in groups stratified by the design of the study. In addition, based on experts' advice [Higgins et al., 2002; Thompson and Higgins, 2002], future meta-analyses should evaluate the role of study level characteristics on variability of results, independent of results of heterogeneity tests. This will add to the understanding of what methodological characteristics are related to variability of results, and for this reason must be kept fixed in replication studies. This may facilitate the planning and execution of future multi-site, international studies and, ultimately, appropriate and generalizable data comparison.