Validation of patient-reported vaso-occlusive crisis day as an endpoint in sickle cell disease studies
Abstract
Individuals with sickle cell disease (SCD) experience vaso-occlusive crises (VOC). Historically, VOC episodes have been assessed through medical utilization, thereby excluding events managed at home. In order to validate a daily patient-reported outcome for patients with SCD to accurately report their VOC status and experience of a pain crisis, a SCD Diary was included in Evaluation of Longitudinal Pain Study in Sickle Cell Disease (ELIPSIS), a longitudinal, six-month, non-interventional study. The daily patient-completed diary included a description of SCD pain crisis, followed by questions on: pain crisis in the past 24 h (VOC Day question; respective response yes or no), worst pain, tiredness, and functioning. Thirty-five patients with SCD participated in ELIPSIS. Analyses were performed to validate the patient-reported VOC Day. Mean symptoms and functioning scores on the first or last VOC Day of a VOC Event were compared using t-tests with the mean of the three non-VOC Days before and after the event. Mean severity of symptoms and functioning scores on all VOC Days compared to all non-VOC Days were higher, with statistically significant mean differences between first/last VOC Days and respective three non-VOC Days (p's < .01). A subset of patients (n = 15) and caregivers (n = 9) were interviewed to evaluate their understanding of the SCD Diary questions. Nearly all confirmed that the pain crisis description accurately described the VOC experience, and participants expressed confidence differentiating SCD crisis pain from everyday pain. These results demonstrate patients can reliably report their experiences with VOC-related pain crises using the SCD Diary.
Novelty statements
What is the new aspect of your work?
In SCD clinical trials, VOC episodes have historically required medical utilization for an episode to be recorded and counted toward assessments of efficacy which is a considerable underestimate of the patient burden of VOC. The SCD Diary described herein is a novel PRO for patients to accurately report the totality of their VOC-related pain crises.
What is the central finding of your work?
Our results indicate that the SCD Diary and specifically the VOC Day question are valid and responsive diary questions that SCD patients can use to reliably report their VOC pain crises without the additional requirement of seeking medical utilization.
What is (or could be) the specific clinical relevance of your work?
Including the SCD Diary, and specifically the VOC Day question, in clinical trials will improve the current paradigm that requires medical utilization to define a VOC episode which will allow future studies to more accurately evaluate the impact of new therapies.
1 INTRODUCTION
Sickle cell disease (SCD) affects approximately one in 2500 births in the United States (US), and incidence among African Americans is approximately one in 300.1, 2 Individuals with SCD frequently experience vaso-occlusive pain episodes, commonly referred to as a vaso-occlusive crisis (VOC), which are caused by the obstruction of blood vessel(s) leading to ischemic tissue injury and severe pain.3 In older patients with SCD, these VOC pain episodes are correlated with increased morbidity and mortality due to the long-term consequences of repeated VOCs causing organ damage.4 VOCs are the most common cause of hospital and emergency department visits among patients with SCD,5 and these episodes result in missed school, work, and decreased health-related quality of life (HRQoL).6, 7 Negative HRQoL impacts cover a range of domains for those living with SCD including physical, emotional, and social functioning8; studies have shown that these impacts are worse for SCD patients as compared to the general population and even to some chronic diseases.8-11 For patients living with SCD, impairments in HRQoL have been found to be particularly profound in the domains of pain, fatigue, and physical functioning,12, 13 with patients experiencing VOCs reporting an even greater HRQoL impact.8
It is difficult to predict the timing of VOC pain episodes, and reliable methods for tracking pain crises over long periods of time are needed. Currently, in clinical practice, physicians rely on patient recall and medical record documentation to provide a history of incidence and severity of VOCs. In clinical trials, VOC episodes historically have required medical utilization for a VOC episode to be recorded and counted. However, the majority of painful VOC episodes are often managed by patients at home14, 15; therefore, the medical utilization definition of a VOC episode likely captures only a subset of the VOCs that patients experience. Indeed, the Pain in Sickle Cell Epidemiology Study (PiSCES) results demonstrated that the frequency of SCD patient-reported crisis pain was higher than what was captured by their healthcare providers, and crisis pain was largely managed at home.16
Requiring medical utilization for a VOC episode to be recorded underestimates patient burden of VOC and limits the ability to fully evaluate the benefit of new treatments.17-19 Thus, there is a need for a patient-reported outcome (PRO) measure to accurately capture and record VOCs.
The development of a PRO for VOCs aligns with the recommendations from the American Society of Hematology (ASH) and the US Food and Drug Administration (FDA) panel who concluded that because the most common SCD symptoms are subjective and best reported by the patient, PRO measures must be included to assess clinical endpoints in therapeutic trials. It was also recommended that PROs assessing the SCD experience should focus on pain, affect, and functioning.20 The panel acknowledged that there was little evidence for measures used to assess pain managed at home and suggested an electronic daily diary as a potential option. Daily monitoring using an electronic patient-reported outcome (ePRO) device may allow for more accurate and regular capture of the patient's pain experience.21, 22 Concurrent with these recommendations, a PRO was developed and implemented as an electronic daily diary (hereafter referred to as the SCD Diary) in an observational, phase 0 trial among a sample of patients with SCD (ELIPSIS).15 One goal of the ELIPSIS study was to evaluate the utility of the SCD Diary, including the ability of the VOC Day question to accurately capture the patient experience with VOC crises. While the feasibility of monitoring out-of-hospital pain and using the patient-reported VOC Day as an endpoint in clinical trials has been published,15 this manuscript describes the methods and analyses used to evaluate and document the content and construct validity of the SCD Diary, and specifically, the patient-reported VOC Day endpoint.
2 METHODS
2.1 ELIPSIS study
ELIPSIS was a non-interventional, longitudinal study that assessed novel biomarkers to identify and document the natural history of VOCs occurring in patients with SCD with hemoglobin SS (HbSS) or S-β thalassemia0 (HbS-β0) genotype. Details of the study design were published previously.15 Patients were at least 12 years old and had a confirmed diagnosis of stable SCD (defined as no significant complications for at least one week prior to study entry). Institutional review board approval was obtained for the study and procedures followed Good Clinical Practice guidelines.
2.2 Patient-reported data
Patients completed the daily SCD Diary, responding to questions regarding their experiences with VOC, pain, tiredness, functioning, and medical utilization. The SCD Diary was developed based on qualitative interviews with patients with SCD and their caregivers.14, 16, 23 Patients completed the SCD Diary daily on a secure, electronic handheld device that complied with FDA regulations for electronic records (21 CFR, Part 11). A description of the handheld device was previously published.15 A description of pain crisis was included in the SCD Diary: “People like yourself describe a pain crisis day as a day when your pain is more than usual, you cannot do what you would normally do, you may be more tired than usual, and often use extra pain medication to get by. Sometimes you may need to speak to or see a doctor or go to the emergency room or hospital, and other times you may treat yourself at home.” This description was followed by the question: “Did you have a pain crisis in the past 24 h?” A VOC Day was recorded if a patient responded “yes” to the diary's VOC Day question. A VOC Event was counted as a sequence of VOC Days and could include intervening, single, non-VOC Days with no pain crisis reported by patients. A VOC Event was considered to be resolved when there were no recorded VOC Days for two or more consecutive days. Patients used the SCD Diary to rate their worst pain severity, tiredness, and functioning within the previous 24-h period using an 11-point numeric rating scale (NRS). For the worst pain and tiredness items, a “0” indicated the absence of the symptom and the anchor for a “10” was the worst imaginable severity of the symptom. Functioning was assessed through four questions about usual physical activities, social activities, daily activities, and self-care. For the functioning items, the anchor for a “0” was “cannot do at all” and a “10” represented “can do completely as usual.” The functioning items were subsequently reverse coded such that higher scores indicated greater impact on function. If a VOC Day was reported, patients were also asked to record their choice of medical utilization.
2.3 Quantitative analyses
Descriptive statistics were used to summarize patient demographics and the SCD Diary scores (i.e., worst pain, tiredness, and functioning) on VOC and non-VOC Days, as well as to summarize the acuity of VOC Days. The completion rate was calculated as a proportion of days that the SCD Diary was completed throughout the study; completers were defined as patients with >80% diary completion.24
To assess the construct validity of the SCD Diary, repeated measures correlations (repeated measures longitudinal mixed models utilizing all available observations and considering the correlation introduced by having multiple observations for each patient) were used to examine the relationship between individual items of the SCD Diary for: (1) the first day of VOC, (2) VOC Days, and (3) non-VOC Days. A correlational value of: <0.3 was considered small, 0.3 to 0.5 was considered moderate, and >0.5 was viewed as strong.25 Known groups validity demonstrating the ability of the SCD Diary items to distinguish between VOC and non-VOC Days was examined by comparing the individual worst pain, tiredness, and functioning items for non-VOC Days and VOC Days. Paired t-tests were conducted to assess the responsiveness of the PRO scores. Responsiveness was examined by comparing the mean SCD Diary item scores of the three days prior to the first day of a VOC Event with the scores on the first day of the VOC Event, and between the last VOC Day of a VOC Event with the three non-VOC Days following the end of an event.
2.4 Qualitative validation
A subset of patients (n = 15) and caregivers (n = 9) in the ELIPSIS trial participated in a single qualitative semi-structured, 60-min interview in-person or via telephone with a trained interviewer. The primary objective was to obtain additional qualitative data regarding: how patients defined sickle cell VOC, whether caregivers know when the person for whom they cared was experiencing a crisis, information on meaningful change regarding a reduction in VOC Days, barriers to diary completion, and improvements to the diary monitoring process.
3 RESULTS
3.1 Quantitative results
3.1.1 Baseline demographic and clinical characteristics
Thirty-seven patients enrolled in ELIPSIS, of which 35 completed the study. The average age (± standard deviation [SD]) of patients was 24.7 ± 9.5 years and just over half were female (51.4%) (Table 1). Only 10 patients (28.6%) completed their SCD Diary more than 80% of the time. The average diary completion rate for completers was 88.6%, while the average completion rate for non-completers was 57.4%. Mean age was slightly lower in the non-completer group, and the ages ranged from 24 to 46 for dairy completers and 13 to 48 for diary non-completers. There were 12 adolescents in the study (ages 13 to 17) who were all in the non-completer group.
| Total sample (N = 35) | Diary completers >80% (N = 10) | Diary non-completers ≤80% (N = 25) | |
|---|---|---|---|
| Mean age, years | |||
| Mean (SD) | 24.7 (9.5) | 29.8 (6.7) | 22.7 (9.8) |
| Median (range) | 24.0 (13.0–48.0) | 28.0 (24.0–46.0) | 18.0 (13.0–48.0) |
| Sex, n (% male) | 17 (48.6%) | 6 (60.0%) | 11 (44.0%) |
| SCD diagnosis, n (%) | |||
| HbSS | 30 (85.7%) | 10 (100.0%) | 20 (80.0%) |
| HbS-β0 Thal | 5 (14.3%) | 0 (0.0%) | 5 (20.0%) |
| Height, cm | |||
| Mean (SD) | 168.2 (9.6) | 169.6 (12.6) | 167.6 (8.3) |
| Median (range) | 167.6 (151.6–186.0) | 166.3 (156.2–186.0) | 167.6 (151.6–185.4) |
| Weight, kg | |||
| Mean (SD) | 66.6 (17.4) | 67.0 (17.5) | 66.4 (17.7) |
| Median (range) | 64.0 (44.9–119.2) | 70.4 (44.9–95.7) | 63.5 (45.4–119.2) |
| BMI | |||
| Mean (SD) | 23.5 (6.0) | 23.0 (4.4) | 23.7 (6.7) |
| Median (range) | 22.1 (14.8–43.3) | 22.4 (18.4–32.8) | 21.5 (14.8–43.3) |
| Baseline Hb | |||
| Mean (SD) | 8.5 (1.5) | 8.6 (1.6) | 8.5 (1.6) |
| Median (range) | 8.6 (4.6–11.7) | 8.3 (6.8–11.7) | 8.6 (4.6–10.9) |
| Prior hydroxyurea use, yes, n (%) | 20 (57.1%) | 6 (60.0%) | 14 (56.0%) |
- Abbreviations: BMI, body mass index; Hb, hemoglobin; SCD, sickle cell disease; SD, standard deviation.
3.1.2 Repeated measures correlations between SCD diary scores
SCD Diary items on the first day of a VOC Event were strongly correlated with one another (worst pain range: r = .574 to .584; tiredness range: r = .574 to .693; all p < .0001) (Table S1). Similarly, the SCD diary items for VOC Days were strongly correlated (worst pain range: r = .648 to .658; tiredness: r = .648 to .758; all p < .0001) (Table S2). Non-VOC Day correlations did not converge due to the large number of “0” responses (i.e., no impact) reported for the functioning items.
3.1.3 VOC Days and Events in ELIPSIS
Over the six-month observation period in ELIPSIS, the mean number of VOC Days per patient for the total sample was 9.9 ± 18.5 days (Table 2). Among participants who were considered completers, the mean number of VOC Days was 19.9 ± 30.7 days and 5.9 ± 8.5 for non-completers. This large discrepancy is driven in part by one outlier in the completer group who reported 91 VOC Days. With this participant removed, the mean number of VOC Days for diary completers was 12.0 ± 18.9. Thirty-one patients (88.6%) reported at least one VOC Day during the study; the mean number of VOC Days per patient for this subgroup was 11.2 ± 19.3 days and the mean duration of VOC Events was 2.7 ± 3.3 days. The mean severity of worst pain was higher on VOC Days compared to non-VOC Days (7.6 vs. 2.6, respectively). Similarly, the mean severity of tiredness was higher on VOC Days compared to non-VOC Days (6.3 vs. 4.1, respectively) (Table 3). For the functioning items, mean impact scores were higher on VOC Days (usual physical activities = 3.6; social activities = 2.6; daily activities = 3.5 self-care = 2.9) compared to non-VOC Days (usual physical activities = 2.6; social activities = 2.1; daily activities = 2.5; self-care = 1.8). The magnitude of difference between VOC Days and non-VOC Days for the functioning scores was smaller than the differences observed for worst pain and tiredness scores between VOC and non-VOC Days.
| (N = 35) | Diary completers >80% (N = 10) | Diary non-completers ≤80% (N = 25) | |
|---|---|---|---|
| Incidence of VOC Eventsb | |||
| N | 35 | 10 | 25 |
| Mean number of VOC Events per person (SD) | 3.3 (3.9) | 4.9 (6.5) | 2.6 (2.0) |
| Median (range) | 3.0 (0.0–22.0) | 3.0 (0.0–22.0) | 2.0 (0.0–7.0) |
| Number of VOC Daysa per patient | |||
| N | 35 | 10 | 25 |
| Mean number of VOC Days per person (SD) | 9.9 (18.5) | 19.9 (30.7) | 5.9 (8.5) |
| Median (range) | 4.0 (0.0–91.0) | 5.5 (0.0–91.0) | 4.0 (0.0–41.0) |
| Number of VOC Days for patients reporting at least 1 VOC Day | |||
| N | 31 | 9 | 22 |
| Mean number of VOC Days per person (SD) | 11.2 (19.3) | 22.1 (31.7) | 6.7 (8.8) |
| Median (range) | 4.0 (1.0–91.0) | 7.0 (1.0–91.0) | 4.0 (1.0–41.0) |
| Duration of VOC Events for patients with VOC | |||
| N | 31 | 9 | 22 |
| Mean Days (SD) | 2.7 (3.3) | 3.7 (4.0) | 2.3 (2.9) |
| Median (range) | 1.4 (1.0–14.7) | 3.0 (1.0–13.9) | 1.3(1.0–14.7) |
| Rate of diary completion | |||
| N | 35 | 10 | 25 |
| Mean (SD) | 66.3% (19.8%) | 88.6% (4.1%) | 57.4% (16.1%) |
| Median (range) | 68.4% (15.1%–95.4%) | 89.2% (81.9%–95.4%) | 63.4% (15.1%–77.6%) |
- a The VOC Day is a self-report measure of VOC status during the previous 24 h. It is a response to the dichotomous (yes/no) item “Did you have a pain crisis in the past 24 h?” A response of “yes” to this item was counted as one VOC Day.
- b A VOC Event is a sequence of VOC Days that may also include intervening, single, non-VOC Days with no pain crisis. A VOC Event is considered to have resolved when there are no reported VOC Days for two consecutive study days.
| VOC Daya (N = 345) | Worst PRO score of a VOC Eventb (N = 113) | Mean severity of VOC Eventc (N = 113) | Non-VOC Day (N = 3895) | |
|---|---|---|---|---|
| Worst pain | ||||
| Mean (SD) | 7.6 (1.9) | 7.6 (2.5) | 7.1 (2.4) | 2.6 (3.1) |
| Median (range) | 8.0 (0.0–10.0) | 8.0 (0.0–10.0) | 7.3 (0.0–10.0) | 1.0 (0.0–10.0) |
| Tiredness | ||||
| Mean (SD) | 6.3 (2.4) | 6.4 (2.8) | 5.9 (2.7) | 4.1 (3.2) |
| Median (range) | 6.0 (0.0–10.0) | 7.0 (0.0–10.0) | 5.8 (0.0–10.0) | 4.0 (0.0–10.0) |
| Usual physical activities | ||||
| Mean (SD) | 3.6 (2.6) | 4.1 (3.3) | 3.5 (3.0) | 2.6 (3.2) |
| Median (range) | 3.0 (0.0–10.0) | 4.0 (0.0–10.0) | 3.0 (0.0–10.0) | 2.0 (0.0–10.0) |
| Social activities | ||||
| Mean (SD) | 2.6 (3.0) | 3.5 (3.3) | 2.9 (3.1) | 2.1 (3.0) |
| Median (range) | 2.0 (0.0–10.0) | 3.0 (0.0–10.0) | 2.0 (0.0–10.0) | 0.0 (0.0–10.0) |
| Daily activities | ||||
| Mean (SD) | 3.5 (2.7) | 3.8 (3.4) | 3.2 (3.2) | 2.5 (3.3) |
| Median (range) | 3.0 (0.0–10.0) | 3.0 (0.0–10.0) | 2.0 (0.0–10.0) | 1.0 (0.0–10.0) |
| Self-care activities | ||||
| Mean (SD) | 2.9 (2.6) | 3.3 (3.2) | 2.8 (3.0) | 1.8 (2.8) |
| Median (range) | 3.0 (0.0–10.0) | 3.0 (0.0–10.0) | 1.5 (0.0–10.0) | 0.0 (0.0–10.0) |
- Abbreviations: PRO, patient-reported outcome; SD, standard deviation; VOC, vaso-occlusive crisis.
- a The VOC Day is a self-report measure of VOC status during the previous 24 h. It is a response to the dichotomous (yes/no) item “Did you have a pain crisis in the past 24 h?” A response of “yes” to this item was counted as one VOC Day.
- b Worst PRO score of a VOC Event based on worst individual score of worst pain, tiredness, and functioning items.
- c If more than one day of VOC Event, PRO ratings were averaged across all VOC Days.
3.1.4 Responsiveness of PRO scores
For worst pain, the mean patient score on the first VOC Day of a VOC Event was 7.1 ± 2.3, which was significantly higher (p < .01) compared to the mean score on the three non-VOC Days prior to the event (3.3 ± 3.1) (Table 4). Similarly, the mean worst pain score on the last VOC Day was significantly higher (p < .01) compared to the mean score on the three non-VOC Days following a VOC Event (7.0 ± 2.3 vs. 3.7 ± 3.0, respectively). Differences in the tiredness scores between VOC Days and non-VOC Days were also statistically significant (p < .01) as the mean tiredness score on the first VOC Day of the event was 5.7 ± 2.7 compared to 4.9 ± 2.6 the three non-VOC Days prior to the event. The mean tiredness score on the last VOC Day of an event was 5.7 ± 2.6 compared to a mean score of 4.6 ± 2.5 on the three non-VOC Days following the event (p < .01).
| SCD diary item | Na | Non-VOC Days (3 days prior to event) | First VOC Day of VOC Event | Absolute difference | t-test | Last VOC Day of VOC Event | Non-VOC Days (3 days post event) | Absolute difference | t-test |
|---|---|---|---|---|---|---|---|---|---|
| Mean (SD) | Mean (SD) | Mean (SD) | t-value | Mean (SD) | Mean (SD) | Mean (SD) | t-value | ||
| p-valueb | p-valueb | ||||||||
| Worst pain | 95 | 3.3 (3.1) | 7.1 (2.3) | 3.8 (2.9) | −12.8 <.01 |
7.0 (2.3) | 3.7 (3.0) | 3.3 (2.5) | −12.8 <.01 |
| Tiredness | 95 | 4.9 (2.6) | 5.7 (2.7) | 0.8 (2.1) | −3.80 <.01 |
5.7 (2.6) | 4.6 (2.5) | 1.1 (2.6) | −4.02 <.01 |
| Usual physical activities | 95 | 2.1 (2.6) | 3.5 (3.2) | 1.4 (2.7) | −4.95 <.01 |
3.3 (2.8) | 2.4 (2.5) | 0.9 (2.1) | −4.20 <.01 |
| Social activities | 95 | 1.6 (2.5) | 3.0 (3.2) | 1.4 (2.8) | −4.86 <.01 |
2.6 (2.9) | 1.7 (2.4) | 1.0 (2.4) | −3.81 <.01 |
| Daily activities | 95 | 1.8 (2.7) | 3.2 (3.3) | 1.4 (2.8) | −4.93 <.01 |
2.9 (3.0) | 2.0 (2.6) | 0.9 (2.4) | −3.54 <.01 |
| Self-care activities | 95 | 1.5 (2.4) | 2.7 (3.1) | 1.3 (2.5) | −4.96 <.01 |
2.5 (2.8) | 1.5 (2.2) | 1.0 (2.3) | −421 <.01 |
- Note: Although there were a total of 114 events across 31 patients in ELIPSIS, there were 18 VOC Events without three days of PRO data prior to the event and one VOC Event without PRO data the day of the event. Therefore, only 95 VOC Events across 30 patients had the PRO data that are reflected in this table.
- Abbreviations: SD, standard deviation; VOC, vaso-occlusive crisis.
- a VOC event is a sequence of VOC days that may also include intervening, single, non-VOC days with no pain crisis. A VOC event is considered to have resolved when there are no reported VOC days for two consecutive study days.
- b Paired t-test comparing SCD diary scores by pre-VOC days (mean of scores for the three days before first day of VOC event) and paired t-test comparing SCD diary scores by pre-VOC Days (mean of scores for the three days after resolution of VOC Event).
Similar results were demonstrated for the usual physical activities, social activities, daily activities, and self-care scores, with statistically significant differences (p < .01) between VOC Day and non-VOC Days for the pre- and post-VOC Events (Table 4). For example, for patient-rated ability to do usual physical activities, the mean score on the first VOC Day of an event was 3.5 ± 3.2 compared to the mean score of 2.1 ± 2.6 for the three non-VOC Days prior to the VOC Event. The mean score for the usual physical activities item on the last VOC Day was 3.3 ± 2.8, while the mean score for the three non-VOC Days following the VOC Event was 2.4 ± 2.5.
3.1.5 Severity by highest acuity of VOC Day
Very few patients reported having a doctor's visit (n = 3) or telephone call with a doctor/nurse (n = 4) as their medical utilization choice on VOC Days. In general, the mean worst pain score reported by patients increased as higher medical acuity interactions were sought (Table 5). The self-treated at home category had the lowest mean worst pain score (n = 271; mean = 7.3 ± 1.7) during a VOC Event; however, this mean score was still substantially higher than the mean score on non-VOC Days (2.6 ± 3.1). The mean level of tiredness was somewhat higher when patients indicated a hospital stay (n = 45; mean = 7.3 ± 2.5) compared to when self-treating at home (n = 271; mean = 6.1 ± 2.3). This mean level of tiredness for individuals who self-treated their VOC at home was still higher than the mean level of tiredness indicated by individuals on non-VOC Days (4.1 ± 3.2). The same trend was seen for all four functioning questions (usual physical activities, social activities, daily activities, and self-care activities), with higher levels of functioning impact observed for those patients who sought interaction with a medical professional (Table 5).
| Highest acuity of day | |||||
|---|---|---|---|---|---|
| Treated at home (N = 271) | Emergency room (N = 23) | Hospital stay (N = 45) | Telephone with doctor/nurse (N = 4) | Doctor's office/clinic visit (N = 3) | |
| Worst pain | |||||
| Mean (SD) | 7.3 (1.7) | 7.6 (2.5) | 8.7 (2.1) | 8.8 (1.9) | 9.7 (0.6) |
| Median (range) | 7.0 (0.0–10.0) | 8.0 (0.0–10.0) | 10.0 (0.0–10.0) | 9.5 (6.0–10.0) | 10.0 (9.0–10.0) |
| Tiredness | |||||
| Mean (SD) | 6.1 (2.3) | 6.4 (3.1) | 7.3 (2.5) | 6.0 (4.9) | 8.7 (2.3) |
| Median (range) | 5.0 (0.0–10.0) | 7.0 (0.0–10.0) | 7.5 (2.0–10.0) | 7.0 (0.0–10.0) | 10.0 (6.0–10.0) |
| Usual physical activities | |||||
| Mean (SD) | 3.2 (2.1) | 4.0 (3.2) | 5.6 (3.4) | 8.8 (2.5) | 5.3 (2.5) |
| Median (range) | 3.0 (0.0–10.0) | 4.0 (0.0–10.0) | 6.0 (0.0–10.0) | 10.0 (5.0–10.0) | 5.0 (3.0–8.0) |
| Social activities | |||||
| Mean (SD) | 2.1 (2.5) | 3.3 (3.3) | 5.1 (3.4) | 8.5 (3.0) | 4.7 (3.5) |
| Median (range) | 1.0 (0.0–10.0) | 2.0 (0.0–10.0) | 5.0 (0.0–10.0) | 10.0 (4.0–10.0) | 5.0 (1.0–8.0) |
| Daily activities | |||||
| Mean (SD) | 3.0 (2.2) | 3.7 (3.7) | 5.6 (3.6) | 6.3 (4.8) | 5.3 (2.5) |
| Median (range) | 3.0 (0.0–10.0) | 3.0 (0.0–10.0) | 6.0 (0.0–10.0) | 7.5 (0.0–10.0) | 5.0 (3.0–8.0) |
| Self-care activities | |||||
| Mean (SD) | 2.4 (2.1) | 3.3 (3.3) | 4.8 (3.3) | 8.8 (2.5) | 5.3 (2.5) |
| Median (range) | 3.0 (0.0–10.0) | 3.0 (0.0–10.0) | 5.0 (0.0–10.0) | 10.0 (5.0–10.0) | 5.0 (3.0–8.0) |
- Abbreviations: SD, standard deviation; VOC, vaso-occlusive crisis.
- a The VOC Day was self-reported by the subject of a VOC during the previous 24 h. It is a response to the dichotomous (yes/no) item “Did you have a pain crisis in the past 24 h?” A response of “yes” to this item were counted as one VOC Day.
3.2 Qualitative validation
Fifteen patients with SCD were interviewed, including 11 adults (mean age: 31 years) and four adolescents (mean age: 16 years) (Table 6). The majority were male (64% adults; 75% adolescents) and Black (100% adults; 75% adolescents). Nine caregivers were interviewed (mean age: 39 years). The majority of caregivers were female (89%), and all were Black (100%).
| Baseline characteristic | Total sample (N = 24) | Adolescent patients (12–17 years old) (n = 4) | Adult patients (n = 11) | Caregivers (n = 9) |
|---|---|---|---|---|
| Mean age (years) (SD) | 31.6 (12.8) | 15.8 (1.5) | 31.1 (11.7) | 39.2 (10.4) |
| Median (range) | 28 (14–54) | 16.0 (14–17) | 27.0 (18–49) | 39.0 (21–54) |
| Mean age of the sickle cell patient (years) (SD) | 22.2 (10.8) | |||
| Median (range) | 18 (14–49) | |||
| Gender, n (% male) | 11 (46%) | 3 (75%) | 7 (64%) | 1 (11%) |
| Racial background, n (%) | ||||
| American Indian or Alaska Native | 1 (4%) | 1 (25%) | 0 | 0 |
| Black or African American | 23 (96%) | 3 (75%) | 11 (100%) | 9 (100%) |
| Ethnicity, n (%) | ||||
| Hispanic or Latino | 2 (8%) | 0 | 1 (9%) | 1 (11%) |
| Not Hispanic or Latino | 21 (88%) | 4 (100%) | 9 (82%) | 8 (89%) |
| Missing | 1 (4%) | 0 | 1 (4%) | 0 |
| Current grade, N (%) | ||||
| 8th | 1 (25%) | |||
| 10th | 1 (25%) | |||
| 11th | 1 (25%) | |||
| 12th | 1 (25%) | |||
| Highest level of education, n (%) | ||||
| Secondary/high school | 3 (27%) | 1 (11%) | ||
| Some college | 7 (64%) | 4 (44%) | ||
| College degree | 0 | 3 (33%) | ||
| Othera | 1 (9%) | 1 (11%) | ||
| Employment status, n (%) | ||||
| Employed, full-time | 1 (9%) | 5 (56%) | ||
| Employed, part-time | 2 (18%) | 1 (11%) | ||
| Homemaker | 0 | 1 (11%) | ||
| Student | 3 (27%) | 1 (11%) | ||
| Unemployed | 1 (9%) | 0 | ||
| Retired | 0 | 1 (11%) | ||
| Disabled | 3 (27%) | 0 | ||
| Otherb | 1 (9%) | 0 | ||
| Marital status, n (%) | ||||
| Never married | 15 (75%) | 9 (82%) | 6 (67%) | |
| Married | 4 (20%) | 1 (9%) | 3 (33%) | |
| Not applicable | 1 (5%) | 1 (9%) | 0 | |
| Living/domestic situation, n (%) | – | – | ||
| Live with both parents in the same home | 3 (20%) | 3 (75%) | 0 | |
| Live with mother only | 1 (7%) | 1 (25%) | 0 | |
| Living alone | 3 (20%) | 0 | 3 (27%) | |
| Living with a partner, spouse, family, or friends | 8 (53%) | 0 | 8 (73%) | |
| Do you currently live with the sickle cell disease patient? N (%) | ||||
| Yes | 7 (78%) | |||
| No | 2 (22%) | |||
| Relationship to the sickle cell disease patient, n (%) | ||||
| Parent | 6 (67%) | |||
| Spouse | 0 | |||
| Otherc | 3 (33%) | |||
| How long have you been the caregiver for this patient with sickle cell disease? (Years, months) | ||||
| Mean (SD) | 15.8 (5.1) | |||
| Median (Range) | 17 (6–22.2) | |||
| Do you feel that you can tell when the sickle cell disease patient you care for is having a pain crisis? N, % | ||||
| Yes | 9 (100%) | |||
- Abbreviation: SD, standard deviation.
- a “Other” includes: “barber college” (patient) and “technical certification” (caregiver).
- b “Other” includes: “entrepreneur”.
- c “Other” includes: Sister (n = 2) and Caregiver (n = 1).
Nearly all participants (93% patients; 89% caregivers) confirmed that the description of pain crisis included in the SCD Diary accurately conveyed the experience of a SCD pain crisis; however, many participants (62% patients; 75% caregivers) expressed that the description could be improved by replacing the phrase “when your pain is more than usual” with “when your pain is more severe than usual.” Patients with SCD most commonly defined a sickle cell crisis using words and phrases such as “extreme pain,” “uncontrollable pain,” and “pain that requires hospitalization.”
Patients and caregivers expressed confidence in differentiating SCD crisis pain from everyday pain. One patient explained “if the pain pretty much keeps me from doing anything, basically crippling me, uh, because of the intensity of the pain, then I know that this is a sickle cell crisis.” Patients stated that they can tell the difference between everyday pain and crisis pain in a variety of ways. Some SCD patients commented that pain severity or intensity increased during a crisis (n = 7/15, 47%); others noted that the location of pain was different in a crisis (n = 3/15, 20%), and several patients explained that they simply understood the difference between everyday pain and crisis pain based on their experience with crises pain (n = 3/15, 20%). Several SCD patients described how crisis pain could be a “stiff” pain specific to one body location (n = 2/15, 13%) and how crisis pain could inhibit their body movement (n = 2/15, 13%). Others commented that pain medication typically would not relieve crisis pain (n = 2/15, 13%) and that the duration of pain is generally longer during a pain crisis (n = 1, 7%). More than half of the SCD patients (n = 8/15, 53%) explained that they considered the severity of the pain (as opposed to the duration of pain) when determining if they were experiencing a pain crisis; 27% of patients considered both pain severity and duration. All caregivers reported that they could tell when the person they cared for was experiencing a pain crisis. Caregivers said that indicators of a pain crisis included increased quietness in their patient, inactivity, verbalization of pain, not wanting to eat or drink, and crying.
Patients and caregivers were asked what would constitute a meaningful change in terms of a reduction in VOC pain crises. All patients indicated that having fewer sickle cell pain crises per year would be meaningful with the majority of patients and caregivers endorsing a 50% decrease in the number of pain crises and the duration of crises as meaningful. Using the 11-point NRS for pain, most patients reported that either a two- (n = 5/11, 45%) or one-point change (n = 4/11, 36%) would be meaningful, whereas most caregivers thought that either a two- or three-point change (n = 3/8, 38%) or a five-point reduction (n = 3/8, 38%) would be needed to be most meaningful.
Patients reported a variety of barriers to completing the SCD Diary daily during ELIPSIS, including: having a pain crisis (40%), being forgetful (40%), did not have access to the SCD Diary at the time of completion (30%), and forgetting the SCD Diary at home when hospitalized for a crisis (30%). All of the patients and caregivers agreed that the idea of downloading an application to their personal device and receiving texts and/or email reminders would increase their ability to complete the SCD Diary daily.
4 DISCUSSION
Participants in the ELIPSIS study were able to use the SCD Diary to reliably report their VOC-related pain crises. Moreover, results from these analyses demonstrate that the SCD Diary is valid and can be implemented into clinical trials to capture the full patient experience of VOC-related pain crises. This finding is critical as it signals the ability to shift from a medical utilization definition of a VOC to a patient-reported definition. Previously reported results from ELIPSIS15 showed that the majority of VOC Events (62.3%) were treated at home, 18.4% culminated in a hospitalization, 17.5% resulted in direct healthcare utilization, while the remaining (1.8%) had indirect healthcare utilization. These results, in conjunction with results from the PiSCES14 study that found VOC pain was largely managed at home, indicate that relying on the medical utilization operational VOC definitions results in a substantial number of missed VOC episodes. Having a reliable, validated PRO measure that includes an item to capture patient-reported VOC pain crises (that may or may not culminate in a hospitalization or healthcare utilization) is critical in providing a more complete picture of the SCD patient experience.
In ELIPSIS, patients reported higher mean worst pain scores on VOC Days compared to non-VOC Days (all p < .01), which is consistent with the expectation that pain is greater on VOC Days than non-VOC Days. In addition, patient-reported scores for tiredness and functioning (physical activities, social activities, daily activities, self-care activities) were higher (i.e., worse) on VOC Days compared to non-VOC Days. This demonstrates that patients can distinguish between their VOC-related experiences from typical or everyday experiences. Further, higher diary scores (i.e., more severe pain, tiredness and impact on functioning) were observed on higher acuity days, which suggests a link between worsening self-reported symptoms and impacts and the need to seek medical intervention. Together, these results support the construct validity of the SCD Diary and the ability of patients to use the SCD Diary to accurately and reliably report on their VOC-related experiences.
Results from these analyses demonstrated that SCD Diary scores were responsive to changes in VOC status as statistically significant differences in scores between the VOC Day and non-VOC Days, for the 3-day intervals pre- and post-VOC Events, were seen for worst pain, tiredness and functioning. This indicates that: the SCD Diary items detect a true change in status when patients report a VOC Day, and the VOC Day is a strong indicator of decline in patient HRQoL status. This aligns with previous findings in adolescents with SCD showing that the impact of crisis pain on physical functioning decreases as crisis pain improves.26
Patients and caregivers who participated in a qualitative interviews were able to clearly differentiate between everyday pain and SCD crisis pain, and they endorsed and understood the SCD pain crisis description included in the SCD Diary. The majority of patients indicated that the severity of pain, as opposed to the duration of pain, was the primary factor driving their response to the VOC Day question. Patients and caregivers also provided insight into what they would consider to be a meaningful change in pain crises, with the majority endorsing a 50% decrease in the number and the duration of crises as meaningful. These insights may help to inform future analyses of meaningful change in pain crises when using the SCD Diary.
Several study limitations should be considered when reviewing the data. Adherence to diary completion in ELIPSIS was limited, with only 10 of the 35 participants being considered completers (>80% of diary completion). Completion of the diary declined from the first month of the study (84%), with an overall completion rate of 67% over the full six month observation period.15 However, based on additional analyses (data on file), there appear to be no significant associations between the SCD Diary completion rate and number of VOC Days, VOC Events, participant age, sex, or use of hydroxyurea. In the qualitative interviews, participants indicated that pain crisis, forgetfulness, not having the SCD Diary with them at the time to complete it, and forgetting the diary at home when hospitalized were the primary reasons for not completing the diary. Of the 15 participants interviewed, only 3 participants were considered completers (>80% of diary completion). Therefore, the insight from the non-completer participants was captured and provides an understanding of the challenges of completing the daily diary. Both patients and caregivers provided feedback indicating that having the SCD Diary available as a downloadable phone application and receiving reminder texts or emails may help raise the rates of adherence. These suggestions will be considered for future trials using the SCD Diary, and indeed, it is critical for researchers designing trials to consider that when patients experience a pain crisis, they may be less likely to want to complete the diary, or to remember to do so. To reduce missing data, researchers should aim to minimize hurdles to completion by utilizing brief, easy to use measures, and considering alternate data collection methods (e.g., caregiver completion).
Additionally, ELIPSIS had a limited sample with patients recruited from a single site; therefore, the generalizability of these results to a more geographically diverse population is not known. To further support the validity of the SCD Diary, the measure should be evaluated in future trials with larger and more diverse SCD populations. Finally, the observational nature of the ELIPSIS study design was a limiting factor. Future analyses should be conducted to assess the SCD Diary in an interventional trial. This is particularly important for examining the responsiveness of the SCD Diary to detect changes in the patient in the context of therapeutic interventions.
Several minor modifications have been made to the SCD Diary to enhance content's clarity. Based on patient and caregiver feedback, the SCD Diary description of a sickle cell pain crisis was revised from “… your pain is more than usual” to “… your pain is more severe than usual.” In addition, minor revisions were also made to the anchor description wording for the tiredness item's response scale. Additionally, only one of the four functioning items—the usual physical activities item—was retained in the SCD Diary to minimize respondent burden. The usual physical activities question was selected because it had the greatest response to patient VOC status. While the other three functioning items were responsive to VOC status, they did not substantially enhance the understanding of a VOC Day/Event. Moreover, by shortening the functioning component of the diary to a single item, the daily completion burden on patients is reduced, which may improve compliance rates. Results from recent qualitative interviews (n = 21) confirmed that the revised SCD Diary incorporating these minor modifications was well understood by patients and caregivers (data on file).
A major strength of this study was the combination of qualitative and quantitative methods within the same patient sample to demonstrate the validity of the VOC Day question. Results from these analyses indicate that the SCD Diary, and specifically the VOC Day question, is a valid instrument that SCD patients can use to reliably report their experiences with VOC-related pain crises. The inclusion of this measure in clinical trials will improve the current paradigm requiring medical utilization to define a VOC episode, and bring the patient voice into the documentation of VOC episodes, thus allowing future studies to more accurately evaluate the impact of new therapies and their efficacy in reducing VOC-related pain crises.
AUTHOR CONTRIBUTIONS
Contribution: All authors collaborated on the study design, analysis plan, interpretation of data, and/or outline of the manuscript. Brooke M. Currie and Karin S. Coyne drafted and revised the manuscript text, and Kathleen W. Wyrwich, Christine L. Baker, Sheryl Pease, Steven Arkin, Debra D. Pittman, and Michael Callaghan reviewed, provided input, and provided final approval of the manuscript.
ACKNOWLEDGMENTS
The authors acknowledge all patients and individuals who contributed to the study. The authors thank Mary Kate Ladd, MA for her writing support, Dawn Ri'chard for her editorial contributions, Denis Rybin for reviewing the manuscript, and David Beidler for his contribution.
FUNDING INFORMATION
This study was sponsored by Pfizer Inc.
CONFLICTS OF INTEREST
Christine L. Baker, Steven Arkin, and Debra D. Pittman are current employees and stockholders of Pfizer Inc. Sheryl Pease was an employee and stockholder of Pfizer Inc. at the time of manuscript development. Kathleen Wyrwich is a former Pfizer Inc. employee, and holds Pfizer stock and stock options, and Eli Lilly and Company stock. She is currently employed at Bristol Myers Squibb Company. Karin S. Coyne is a current employee of Evidera, and Brooke M. Currie was an employee of Evidera at the time of manuscript development. Dr. Coyne and Ms. Currie were paid consultants to Pfizer for this work and in connection with the development of this manuscript. Michael Callaghan is an employee of Central Michigan University and Children's Hospital of Michigan, and he is the medical director at Agios Pharmaceuticals.
PATIENT CONSENT STATEMENT
All participants provided informed consent or assent.
Open Research
DATA AVAILABILITY STATEMENT
The authors confirm that the full data supporting the findings of this study are available within the article or in the online supplement.
