Accounting for Repeat Enrollments During an Emergency Clinical Trial: The Rapid Anticonvulsant Medications Prior to Arrival Trial (RAMPART)
Abstract
Objectives
The objectives were to describe the frequency of repeat enrollment within a specific exception from informed consent trial testing benzodiazepine treatment of prehospital status epilepticus and to estimate the effect of repeat enrollments on the analysis of the primary outcome.
Methods
This was a secondary analysis of data collected as part of the Rapid Anticonvulsant Medication Prior to Arrival Trial (RAMPART), a study comparing intramuscular midazolam to intravenous lorazepam given by paramedics to patients with prehospital status epilepticus. Subjects in RAMPART achieved a successful primary outcome if they had cessation of seizures by the time of emergency department arrival. Data were collected on all subjects, but only the first enrollment for each individual was used in the primary analysis. The patterns of repeat enrollment are described, along with the demographics of these subjects. In addition, an intraclass correlation coefficient (ICC) was estimated to assess the amount of within-subject correlation and its effect on the estimated treatment effect when all enrollments are included in the analysis.
Results
A total of 1,023 enrollments occurred in RAMPART among 893 unique individuals (range of repeat enrollment observed = two to 14). The ICC for seizure cessation within individual was low at 0.119; when excluding subjects with benzodiazepine crossover, the ICC was 0.094.
Conclusions
In clinical trials of emergency conditions with interval complete resolution, accounting for repeat enrollments is feasible. The RAMPART experience demonstrated that in this setting the within-subject correlation is low and can be accounted for at relatively low statistical cost.
In most clinical trials, patients participate once. In trials of emergency conditions such as status epilepticus with early enrollment, patients may inadvertently or intentionally be enrolled multiple times. Many important, high-morbidity emergency conditions recur in at-risk individuals. Specific examples include status asthmaticus, head trauma, and status epilepticus.1 In the prehospital setting or early in the emergency department (ED) visit, evaluations of treatments for such conditions by clinical trials frequently employ exception from informed consent (EFIC), which makes it more difficult to prospectively identify potential reenrollers.2 Within a clinical trial, repeated measurements of outcome data from the same individual may be positively correlated. Ignoring this correlation may bias the variance estimate and lead to spurious results.3
Independence of observations is a key assumption of many statistical techniques.4 In brief, independence means that knowledge of a given observation does not affect the probability distribution of other observations. In a typical clinical trial, by measuring an outcome in one patient, the outcome probability distribution of the other is unaffected. However, independence is commonly violated with clustered data. In the case of an EFIC trial with repeat enrollments, if some patients are more or less likely to respond to treatment, then additional enrollments by those patients may not provide truly independent information. In this case, the extent to which some patients have either highly refractory or easily treated status epilepticus can lead to misestimation of the confidence intervals of the treatment effect of interest.
The planning of clinical trials requires careful consideration regarding the preexisting state of knowledge within the disease, along with the scientific and logistical priorities of the actual experiment to be conducted. The field of emergency care research is unique for many reasons; of particular interest within this setting is the potential recurrent nature of serious conditions. As such, one way of addressing the correlation within subjects who may present with the disease in question multiple times is to only enroll a patient once in a clinical trial. On the other hand, practical concerns may make it challenging to determine that a patient has been enrolled previously when an intervention is being delivered under EFIC. A common approach is to limit to the first occurrence of the disease within individual, as this will eliminate the possibility of within-subject correlation biasing the results. Under an EFIC trial, however, the patient may have repeated exposure to a research intervention without the prospect of contributing to scientific knowledge on the primary efficacy outcome. Such repeated enrollments may contribute information regarding safety, however. As such, when planning emergency care research it is important to consider the scientific, ethical, and logistical effects of the approach that will be used to account for repeated presentations with the same condition. Our objectives were to describe the frequency of repeat enrollment within a specific EFIC trial of benzodiazepine treatment for prehospital status epilepticus and the effect of repeat enrollments on the analysis of the primary outcome.
Methods
Study Design
The Rapid Anticonvulsant Medication Prior to Arrival Trial (RAMPART) was a phase 3 randomized, double-blind, noninferiority clinical trial comparing intravenous (IV) lorazepam to intramuscular midazolam via autoinjector for prehospital patients with status epilepticus. The design and results of the primary analysis of RAMPART have been previously published.5 The current investigation is a secondary analysis of RAMPART focused on patients with multiple enrollments. The primary analysis of RAMPART data only included the first enrollment; however, data on response and follow-up were collected for subjects who were enrolled multiple times. Subjects in RAMPART achieved a successful primary outcome if they had cessation of seizures by the time of ED arrival. More detailed methods are in Data Supplement S1 (available as supporting information in the online version of this paper). The trial was conducted under EFIC for emergency research. Institutional review boards for all entities engaged in the research approved the protocol.
Study Setting and Population
RAMPART was conducted primarily in urban and suburban centers comprising the NETT spoke and hub complexes. Study interventions occurred in ambulances, and 33 emergency medical services (EMS) agencies were involved. Patients were transported to 79 receiving hospitals.
Adults and children with estimated body weights of at least 13 kg who required treatment with benzodiazepines for status epilepticus were eligible for inclusion. Based on the previous experience of the Prehospital Treatment of Status Epilepticus trial, we anticipated that a reenrollment rate of up to 15% would be unavoidable.6
Study Protocol
A study box was used to distribute the study treatments and record clinical events in the ambulance. Demographic data, vital signs, medical history, seizure recurrence, ED disposition, and hospital discharge data were collected using predefined procedures. Subjects with repeat enrollments were followed for adverse events as per the protocol and outcome data were collected, but only the first enrollment for each patient was included in the primary outcome analysis.
Data Analysis
The primary analysis of the RAMPART trial used a one-sided test for binomial proportions to test the noninferiority hypothesis.7 A total of 1,024 patients were planned for enrollment, which included an inflation factor to account for a 15% repeat enrollment rate.
In the current investigation, we provide descriptive statistics regarding the RAMPART subjects who had repeat enrollments and assess the effect of accounting for these reenrollments in the analysis. In addition, we summarize each event regarding treatment assignment and treatment success graphically for these subjects.
For this secondary analysis, generalized linear mixed models with a logit link function were employed to estimate the treatment effect accounting for the correlated observations due to repeat enrollments.8 We fit three models: one with all RAMPART events included and ignoring within-subject correlation (simple logistic regression model); a second that accounts for clustering within individuals using a generalized linear mixed model; and a third mixed model, which excludes subjects who had treatment assignment crossover in subsequent enrollments. The intraclass correlation coefficient (ICC) is defined as the between subject variance divided by the total variance or as the proportion of the total variance attributed to clustering, in this case within individuals.
Results
In the RAMPART trial, there were 1023 enrollments among 893 unique individuals. From these 893 individuals there were 85 individuals accounting for 130 reenrollments (64 with two enrollments, 13 with three enrollments, and eight ranged from four to 14). The demographic and clinical characteristics of the reenrollers are provided in Table 1.
| Characteristic | Repeat Enrollers (n = 85) | Single Enrollers (n = 808) |
|---|---|---|
| Age (yr), mean (±SD) | 43.36 (±17.27) | 43.43 (±22.36) |
| Female | 32 (37.65) | 373 (46.16) |
| Ethnicity | ||
| Hispanic or Latino | 13 (15.29) | 93 (11.51) |
| Not Hispanic or Latino | 52 (61.18) | 548 (67.82) |
| Unknown | 20 (23.53) | 167 (20.67) |
| Race | ||
| American Indian or Alaska Native | 0 (0.00) | 8 (0.99) |
| Asian | 1 (1.18) | 21 (2.60) |
| Black or African American | 51 (60.00) | 402 (49.75) |
| Native Hawaiian or other Pacific Islander | 1 (1.18) | 2 (0.25) |
| White | 29 (34.12) | 319 (39.48) |
| More than one race | 0 (0.00) | 14 (1.73) |
| Other | 1 (1.18) | 8 (0.99) |
| Unknown | 2 (2.35) | 34 (4.21) |
| Received high dose (estimated weight > 13 kg) | 78 (91.76) | 694 (85.89) |
| Prior history of epilepsy | ||
| No | 9 (10.59) | 205 (25.37) |
| Yes | 68 (80.00) | 520 (64.36) |
| Not answered (not SE; one SE missing response) | 8 (9.41) | 83 (10.27) |
| Precipitant of episode | ||
| Anticonvulsant withdrawal or noncompliance | 33 (38.82) | 245 (30.32) |
| Idiopathic or refractory breakthrough | 26 (30.59) | 219 (27.10) |
| Acute threshold lowering comorbidity | 8 (9.41) | 52 (6.44) |
| Other or none identifieda | 18 (21.18) | 292 (36.14) |
| Final diagnosis | ||
| Seizure/SE | 77 (90.59) | 726 (89.85) |
| Unable to determine | 4 (4.71) | 23 (2.85) |
| Nonepileptic spell | 4 (4.71) | 59 (7.30) |
| Intubation within 30 minutes of ED arrival | 16 (18.82) | 115 (14.23) |
| Seizures terminated without rescue therapy | 57 (67.06) | 554 (68.56) |
| Recurrent seizure | 27 (31.76) | 98 (12.13) |
| Hospitalized | ||
| No | 37 (43.53) | 299 (37.00) |
| Yes | 48 (56.47) | 502 (62.13) |
| Unknown | 0 (0.00) | 7 (0.87) |
- Data are n (%) unless otherwise noted.
- SE = status epilepticus.
- a Other precipitants include status without prior history, nonepileptic spell, and unable to determine.
Figure 1 shows the number of enrollments, treatment assignments, and responses to treatment among the re-enrollers. There was treatment crossover in 44 of the 85 reenrollers (52%). The treatment effect on the logit scale, the standard error, test statistics, p-value, and ICC for each model are provided in Table 2. Model 1 ignores within subject correlation and when compared to the other methods, has the lowest standard error, and as such is overstating the precision of the estimated treatment effect by failing to account for repeated observations within individuals. Models 2 and 3 use different methods for accounting for within subject correlation; however, there is very little change in the estimated treatment effect and a similarly small decrease in the precision. Both the reduced sample size and the accounting for within-subject correlation contribute to the observed trends.
| Scenario | Treatment Effect | Standard Error | Test Statistic | p-value | ICC |
|---|---|---|---|---|---|
| Model 1: treatment effect ignoring ICC | 0.5493 | 0.1358 | 16.36 | <0.0001 | NA |
| Model 2: accounting for repeat enrollments but ignoring treatment assignment crossover | 0.5590 | 0.1388 | 15.76 | 0.0001 | 0.119 |
| Model 3: accounting for repeat enrollments but deleting the 44 subjects that had treatment assignment crossover (895 enrollments; 849 unique subjects) | 0.5560 | 0.1477 | 14.16 | 0.0005 | 0.094 |
- Treatment effect is parameter estimate and standard error on logit scale. For example, parameter estimate of 0.55 corresponds to OR of 1.7.
- ICC = intraclass correlation coefficient.
Discussion
Repeat enrollments within the RAMPART trial were relatively common, yet still below the projected proportion of 15%. The RAMPART trial had a large treatment effect and we estimated the ICC for Models 2 and 3 to be quite low. As such, the conclusions of the trial would have been unchanged if repeat enrollments were included, with or without accounting for the correlation. However, the interpretation of low versus high ICC values will vary based on the clinical setting and the design setting. For RAMPART, we may have expected a higher ICC value because the cluster is composed of repeated measurements within a subject. This value may have been diluted due to the majority of the subjects having one outcome measure or one enrollment. Despite this, the precision of the estimate of the treatment effect would have falsely increased if the correlation was ignored, as the standard errors would be biased toward smaller values.
While outside the scope of this design, it is well known from animal and other data that status epilepticus becomes harder to discontinue as it lasts longer and becomes more established.9 Given variable recognition, EMS activation, and EMS response times, it is plausible that time to treatment may be a more important factor than simply individual identity and disease phenotype.
The ethics of repeatedly enrolling a patient in a clinical trial deserve some mention. Because this trial was conducted in an emergency situation under EFIC, each seizing patient needed to be treated quickly. The primary method by which a subject could opt out of the trial was via a bracelet. While not part of the protocol or manual of procedures, the study teams involved with the care of patients with large numbers of enrollments took a great deal of time attempting to arrange follow-up for the frequent reenrollers in many cases. Opt-out bracelets were repeatedly given in some cases. Because RAMPART compared two therapies already in clinical use, all subjects, including those enrolled more than once, always received active treatment consistent with existing standard care. Due to the standard trial procedure of only using the first event per subject, these subjects were exposed to research procedures repeatedly, while not contributing additional primary outcome data. These subjects did provide additional safety data, along with the data used by the current report to estimate the effect of repeat enrollment.
Our findings also have implications for clinical care and further research in status epilepticus. Although the intent of this analysis was merely to explore the statistical effect of how reenrollers were handled in RAMPART, we are also able to generate hypotheses related to the associations seen. Second, there was a small but potentially important subset of reenrollers who appear to be consistently refractory to initial benzodiazepines. Improving the initial care of this subset may involve earlier addition of a nonbenzodiazepine anticonvulsant medication, rather than simply optimizing first-line benzodiazepine administration. Future trials investigating this possibility are warranted.
Limitations
The data used for this analysis were collected from a single clinical trial in status epilepticus comparing two benzodiazepines. The degree to which the results may apply to other conditions and the testing of other treatments is unknown. Second, the majority of patients enrolled in RAMPART came from urban areas, with a significant plurality from metro Detroit and greater Cincinnati. The degree to which the epidemiology of epilepsy may vary across the United States is outside the scope of this work, but could have potentially influenced the results if subjects from areas oversampled in RAMPART are not generally representative.
Conclusions
In general, excluding repeat enrollments at the analysis phase is a conservative strategy, particularly in clinical trials of emergency conditions with interval complete resolution. Inflation of the required sample size during the planning stages is necessary in order to account for these postrandomization exclusions. The RAMPART experience demonstrates that exclusion may not always be necessary, but also allows exploration of alternative measures that can be taken at relatively low statistical cost when independence of observations cannot be assumed. If the expected intraclass correlation coefficient was known prior to the planning of RAMPART, the sample size could have been inflated by this factor to account for the correlation at the study planning stages. In summary, we recommend that steps be taken to account for repeat enrollments in emergency trials and encourage other investigators to evaluate and publish the intraclass correlation coefficient values to allow for the more quantitative planning of future trials in such conditions.
