Portable electronic vision enhancement systems in comparison with optical magnifiers for near vision activities: an economic evaluation alongside a randomized crossover trial
Abstract
Purpose
To determine the incremental cost-effectiveness of portable electronic vision enhancement system (p-EVES) devices compared with optical low vision aids (LVAs), for improving near vision visual function, quality of life and well-being of people with a visual impairment.
Methods
An AB/BA randomized crossover trial design was used. Eighty-two participants completed the study. Participants were current users of optical LVAs who had not tried a p-EVES device before and had a stable visual impairment. The trial intervention was the addition of a p-EVES device to the participant's existing optical LVA(s) for 2 months, and the control intervention was optical LVA use only, for 2 months. Cost-effectiveness and cost-utility analyses were conducted from a societal perspective.
Results
The mean cost of the p-EVES intervention was £448. Carer costs were £30 (4.46 hr) less for the p-EVES intervention compared with the LVA only control. The mean difference in total costs was £417. Bootstrapping gave an incremental cost-effectiveness ratio (ICER) of £736 (95% CI £481 to £1525) for a 7% improvement in near vision visual function. Cost per quality-adjusted life year (QALY) ranged from £56 991 (lower 95% CI = £19 801) to £66 490 (lower 95% CI = £23 055). Sensitivity analysis varying the commercial price of the p-EVES device reduced ICERs by up to 75%, with cost per QALYs falling below £30 000.
Conclusion
Portable electronic vision enhancement system (p-EVES) devices are likely to be a cost-effective use of healthcare resources for improving near vision visual function, but this does not translate into cost-effective improvements in quality of life, capability or well-being.
Introduction
One of the major issues faced by people with visual impairment (VI) is the inability to perform simple tasks such as reading and writing. In the United Kingdom (UK), National Health Service (NHS) low vision clinics assess the needs of people with VI and provide low vision aids (LVAs), such as optical magnifiers, as a means to relieve some of the difficulties in performing everyday tasks. Due to the simple nature of these devices, patients often require a variety of aids for different tasks. There is evidence that some optical LVAs are never used due to the limitations of these devices (McIlwaine et al. 1991).
In recent times, electronic alternatives to optical magnifiers have become more readily available as prices of electronic equipment have fallen. Portable electronic vision enhancement systems (p-EVES) are becoming a relatively inexpensive way for people with VI to enhance their ability to complete simple tasks without additional assistance (see Fig. 1 for an example of a p-EVES device). The additional benefits of p-EVES systems over traditional optical magnifiers include binocular viewing, habitual working distance, variable magnification, adjustable contrast settings and freeze frame facility (Taylor et al. 2014). In the Welsh NHS low vision service, p-EVES devices are now provided to eligible patients and anecdotal evidence suggests that they are both popular and successful (Charlton et al. 2011). However, at present p-EVES devices are not routinely provided by NHS low vision services throughout the rest of the UK.
The hypothesis is that in comparison with optical LVAs, the p-EVES devices are less tiring to use and thus can be used for a longer duration and that they are more versatile so can be used for a wider range of tasks. If this is the case, the user will experience greater independence and is likely to rate their health-related quality of life (HRQoL) and well-being more highly. Previous research has shown only limited evidence that non-portable EVES devices are more effective than optical LVAs at improving performance of everyday tasks (Peterson et al. 2003; Culham et al. 2004), and to date, there has been no published evidence regarding the effectiveness and cost-effectiveness of p-EVES devices.
Visual impairment (VI) is known to be one of the leading causes of depression in older people; 13% of people with VI have significant depressive symptoms (Evans et al. 2007), 75% of whom are not currently receiving treatment (Nollett et al. 2016). Services and adaptations to help people with VI could potentially be an effective and cost-effective means of improving the independence, mental health and quality of life of people with untreatable VI.
Our aim was to firstly determine the incremental cost-effectiveness of use of a p-EVES device plus optical LVA compared with optical LVA use alone (using ‘near vision’ visual function as the measure of effect). Secondly, we aimed to estimate the cost per quality-adjusted life year (QALY) gained from use of a p-EVES device plus optical LVA compared with optical LVA use alone.
Patients and Methods
Design
The p-EVES study was a single-centre two-arm randomized crossover study designed to determine the acceptability, effectiveness and cost-effectiveness of p-EVES devices as compared to optical LVAs for ‘near vision’ tasks in adults with moderate to severe visual impairment. In this paper, we report on cost-effectiveness; full effectiveness results will be reported in a separate paper which is currently under review (Taylor J, Bambrick R, Brand A, Bray N, Dutton M, Harper RA, Hoare Z, Ryan B, Edwards RT, Waterman H, Dickinson CM; unpublished data), and a qualitative paper reporting acceptability is currently being prepared. The p-EVES study was conducted at Manchester Royal Eye Hospital, UK. A published study protocol describes the study design and methodology in detail (Taylor et al. 2014). The p-EVES study was registered with clinical trials.gov (Identifier: NCT01701700), received favourable National Research Ethics Service (NRES) ethical approval and conformed to the tenets of the Declaration of Helsinki. Written informed consent was obtained from all participants.
Study population
The trial was carried out between 2013 and 2015 in Manchester, UK. Participants were recruited through optometrist led low vision clinics at the Manchester Royal Eye Hospital. Patients were considered eligible to participate if they were aged 18 or over, had a visual impairment secondary to a stable ocular pathology, had binocular distance visual acuity of 0.7 or worse and/or log contrast sensitivity of 1.20 or worse and were currently in possession of at least one near vision optical LVA. A total of 780 patients from low vision clinics at Manchester Royal Eye Hospital were assessed, 680 were excluded and 100 randomized. Of the 680 excluded, 533 did not meet the inclusion criteria, 117 declined, 27 could not be contacted, and in three cases, the recruitment forms completed by referring clinicians were illegible.
Intervention
An AB/BA crossover trial design was used. Participants were randomly allocated to one of two possible study arms: Group 1 or Group 2. Group 1 received the two interventions A and B in the order AB, while Group 2 received the interventions in the order BA. Intervention A was a 2-month period using optical LVAs and a p-EVES device. Intervention B was a 2-month control period using optical LVAs only. During intervention A, optical LVAs were retained: they could be used for tasks for which the p-EVES was not suitable, or instead of the p-EVES if preferred. Additional information is provided in the previously published protocol (Taylor et al. 2014).
Participants selected one of four possible p-EVES devices depending on their needs: the Optelec Compact 4HD (£545), the Optelec Compact+ (£249), the Schweizer eMag 43 (£399) and the Eschenbach Mobilux Digital (£399). As per current NHS practice, participants were not given extended formal training on how to use the p-EVES devices beyond basic operational demonstration.
Intervention adherence
Adherence was considered in two ways: firstly, did the participants engage with the purpose of the study and try to use the p-EVES to carry out a range of tasks, and, secondly, did participants find the p-EVES useful enough to persist with it. Regarding the first point, participants in each arm of the study were phoned after 1 week to ask about any difficulties in using the devices, and to arrange re-instruction if required. For each participant, the Manchester Low Vision Questionnaire (MLVQ) was used to record the frequency of use of aids; the type/number of tasks completed using each aid; the longest duration of use of each aid; and the type/number of tasks that could not be completed using each aid (or which required additional assistance).
Measurement of effectiveness
Two primary measures of effectiveness were used: near vision visual function, as measured using the NV-VFQ-15 (Stelmack & Massof 2007), for the cost-effectiveness analysis; and vision-related quality of life, as measured using the VisQoL (Misajon et al. 2005), for cost-utility analysis. The EQ-5D-5L (Herdman et al. 2011), ICECAP-A (Al-Janabi et al. 2012) and WHO-5 (Bonsignore et al. 2001) were used as part of sensitivity analyses to examine the effect of different measures on results.
Outcome measures
The NV-VFQ-15 is a 15-item ‘near vision’ visual function outcome measure which was developed for this study from the ‘near vision’ items on the NV-VFQ-48 (Stelmack & Massof 2007). A validated algorithm for scoring vision function questionnaires was used to score the NV-VFQ-15 (Massof 2007). This scoring approximates a Rasch analysis. Using this scoring system, the lowest possible ‘near vision’ visual function score on the NV-VFQ-15 is −3.36 logits, and the highest possible score is 5.07 logits, therefore resulting in a total score scale out of 8.43 logits.
The VisQoL is specifically designed to measure vision-related quality of life (Misajon et al. 2005) and is one of the first outcome measures to allow utility values and QALYs to be calculated for the purpose of economic evaluation in vision-related interventions (Peacock et al. 2008). It is scored from 0 (state of death) to 1 (perfect health).
The EQ-5D is a generic, validated HRQoL measure (EuroQoL Group 1990). We used the five-level (5L) version of the EQ-5D as this is considered to be more sensitive than the original three-level (3L) version (Janssen et al. 2013). At the time of analysis, a validated UK value set was not available to score the EQ-5D-5L, and therefore, a crosswalk value set was used to assign weights for domain levels based on the 3L scoring system (van Hout et al. 2012). The EQ-5D-5L was analysed to produce an index score between 0 (state of death) and 1 (perfect health).
The ICECAP-A is a validated capability measure focussing on well-being beyond health and is scored from 0 (no capability) to 1 (full capability) (Al-Janabi et al. 2012). The WHO-5 is a validated emotional well-being index, which is highly sensitive for screening depressive symptoms (Lowe et al. 2004). A total score out of 100 is calculated from the raw score.
Measurement of costs
Costing the intervention
Costs for the p-EVES devices ranged from £249 to £545 per device. Additional staff time in low vision clinics for demonstration of the devices was collected and costed. Salary costs were estimated using published NHS banding data for 2014/2015 (Royal College of Nursing 2014). With an applied overhead rate of 60.7% (Curtis 2011), per hour p-EVES device demonstration costs equated to £24.52.
Costing carer time
See Table 1 for a breakdown of carer tasks, time codes and costs. Costs were calculated for change in carer time elicited by the availability of the p-EVES aid as compared to access to optical LVAs only. Due to the crossover design, this was measured as change from baseline (Visit 1) to Visit 2 for Group 1, and change from Visit 2 to Visit 3 for Group 2, as Group 2 essentially had a staggered baseline due to the ‘BA’ order of interventions. The results are reported as carer time freed up through use of p-EVES. Carer time was costed at an hourly rate of £6.80 (Curtis 2014).
| MLVQ near vision task | Estimated time (minutes) | Estimated carer cost (£)a | Estimated frequency if not stated |
|---|---|---|---|
| Reading letters/cards/other correspondence | 15 | £1.70 | Daily |
| Reading instructions (packets, tins, bottles, medicines, etc.) | 15 | £1.70 | Daily |
| Reading ‘ordinary’ print books/newspapers/magazines | 30 | £3.40 | Daily |
| Reading telephone directory to check numbers | 5 | £0.57 | Weekly |
| Reading markings on dial (cooker, radio/hi-fi, washer, etc.) | 5 | £0.57 | Daily |
| Reading shop prices/labels | 60 | £6.80 | Twice weekly |
| Read the time on your watch | 5 | £0.57 | Daily |
| Reading large print books/newspapers | 30 | £3.40 | Daily |
| Identifying money | 5 | £0.57 | Daily |
| Writing own letters, cards, etc. | 30 | £3.40 | Weekly |
| Signing your own name | 5 | £0.57 | Twice weekly |
| Reading own writing | 15 | £1.70 | Weekly |
| Filling in cheques, forms, etc. | 30 | £3.40 | Weekly |
| Special hobby (e.g. stamps, models, painting, music) | 30 | £3.40 | Weekly |
| DIY/repair/fixing task | 30 | £3.40 | Weekly |
| Sewing/knitting/needlework/mending | 30 | £3.40 | Weekly |
| Watching TV | 15 | £1.70 | Daily |
| Reading street signs/bus numbers/directions etc. | 5 | £0.57 | Twice weekly |
| Watching an event/trip/theatre | 180b | £20.40 | Monthly |
| Mobile phone | 15 | £1.70 | Daily |
| Looking at photographs | 15 | £1.70 | Daily |
| Other | NA | NA | NA |
- a Based on gross hourly salary of £6.80 for public and independent sector care worker (Curtis 2014).
- b Includes travel and activity time for carer, as carer would need to be present throughout.
- MLVQ = Manchester Low Vision Questionnaire, DIY = do it yourself.
Need for task assistance was measured using the MLVQ, which asks respondents to estimate how often they require assistance from another person on 21 different tasks ranging from reading small print to writing letters. Participants were also asked to estimate daily/weekly frequency and length of time taken for the MLVQ tasks, but participants reported issues in making these estimates. In order to maintain uniformity in the analysis, we consulted with a person with over 40 years’ experience of living with a VI to aid in assigning time codes and estimating daily/weekly frequencies for each task on the MLVQ.
Analysis of effects and costs
Twelve participants with no follow-up data and six participants with only 2-month follow-up data were excluded from the analyses. Therefore, a total of 82 of 100 randomized participants were included in the economic analyses. A total of 38 participants were randomized to Group 1 and 44 to Group 2.
CROS t-tests were used to account for participant, period and treatment factors. The results indicated that there was not a statistically significant effect of moving from intervention A to intervention B (or vice versa) on any of the economic evaluation measures, and therefore, the data for the two interventions could be grouped by intervention (A or B) across the two study groups, allowing us to expand the sample size to include participants as their own comparators.
Carer time costs were collated for both groups and compared between the intervention A and intervention B time-points. Mean differences in costs between the two interventions were calculated using nonparametric bootstrapping, and run on 5000 iterations, to produce 95% confidence intervals around these differences (Briggs et al. 1997). Discounting was not undertaken as the length of interventions did not exceed 12 months.
Primary cost-effectiveness analysis
All economic analyses were conducted from a societal perspective to take account of the financial burden of carer time. All costs are in £ sterling for 2014. The primary cost-effectiveness analysis compared change in ‘near vision’ visual function, as measured using the NV-VFQ-15, with carer and intervention costs for the two interventions. Due to the crossover design, effectiveness of the p-EVES devices was measured as change from baseline (Visit 1) to Visit 2 for Group 1, and change from Visit 2 to Visit 3 for Group 2. Effectiveness of the optical LVA intervention was measured as change from Visit 1 to Visit 3 for Group 1, and change from Visit 1 to Visit 2 for Group 2 to account for the different order of interventions in the two study groups.
An incremental cost-effectiveness ratio (ICER) was estimated using the NV-VFQ-15 as the measure of effect. Bootstrapping, based on 5000 iterations, was used to generate 95% confidence intervals around the ICER estimates. An ICER plane was generated to show the distribution of costs and effects, and a cost-effectiveness acceptability curve (CEAC) was generated to show the probability of cost-effectiveness at a specified threshold of £1000 per point improvement on the NV-VFQ-15.
Primary cost-utility analysis
The QALY is a well-established outcome measure in health economics which is used to aggregate the quantity and quality of life experienced in a given health state. In the UK, the National Institute for Health and Care Excellence (NICE) specifically endorses the QALY as a primary outcome measure in economic evaluations (NICE 2013). Previous research has indicated that generic measures such as the EQ-5D are not sensitive to quality of life changes related to vision and visual function (Tosh et al. 2012). The VisQoL was therefore used as the primary outcome measure to generate a cost per QALY, ICER plane and CEAC for comparison with the NICE ceiling of £20 000 to £30 000 per QALY in the UK (NICE 2013).
Area under the curve (AUC) calculations were used to estimate the overall utility experienced during the different intervention periods, and therefore ICER calculations were based on difference in utility experienced during each 2-month intervention period.
In order to calculate AUC, a baseline score is required; however, due to the crossover design, participants in Group 1 received the p-EVES intervention before the optical LVA only control. This meant that calculating AUC for the optical LVA intervention was not possible for Group 1 without introducing bias due to prior experience with a p-EVES device. Therefore, optical LVA AUC data was only used from Group 2. This resulted in a sample size of N = 82 in the p-EVES plus optical LVA intervention arm and N = 44 in the optical LVA only arm.
Sensitivity analysis: outcome measures
Using the bootstrapping methods detailed earlier, probabilistic sensitivity analysis was conducted to generate ICER planes illustrating variation in cost-effectiveness from varying basic assumptions about effectiveness and costs.
Deterministic sensitivity analysis was conducted by altering the effectiveness results and costs of the intervention; secondary cost-effectiveness, cost-utility and cost-capability analyses were carried out to examine the effect of different outcome measures on the economic evaluation results. In the cost-effectiveness sensitivity analysis, the WHO-5 was used as an alternative measure of effectiveness due to the high incidence of depression in this population (Evans et al. 2007). In the cost-utility sensitivity analysis, the EQ-5D-5L was used as an alternative measure of utility. Additionally, the ICECAP-A was used to calculate an estimated cost per year of full capability (YFC); the YFC approach is an alternative to the QALY framework which focuses on a broader measurement of well-being beyond health and physical functioning (Mitchell et al. 2015).
Sensitivity analysis: intervention costs
Three adjusted intervention cost sensitivity analyses were undertaken. In the first sensitivity analysis, the highest cost device (£545) was applied to all participants alongside a standardized hour of training (£24.52); in the second, the lowest cost device was applied to all participants (£249) with no training costs; and in the third, a real-world Welsh NHS low vision service p-EVES cost (£150) was applied to all participants (at the time of this trial the Welsh service used the Compact+, not the £150 low-cost device).
Results
Demographics
Demographic characteristics of the sample are presented in Table 2.
| Group 1 (N = 49) | Group 2 (N = 51) | |
|---|---|---|
| Age (years) mean (SD) | 69.79 (19.97) | 72.94 (16.63) |
| Gender Male n (%) | 20 (40.8) | 18 (35.3) |
| Ethnicity n (%) | ||
| White British | 45 (91.8) | 48 (94.1) |
| Other | 4 (8.2) | 3 (5.9) |
| Registration (CVI) status n (%) | ||
| SSI | 22 (44.9) | 21 (41.2) |
| SI | 18 (36.7) | 25 (49.0) |
| Not registered | 9 (18.4) | 5 (9.8) |
| Residential status n (%) | ||
| Alone | 19 (38.8) | 24 (47.1) |
| With spouse/partner | 11 (22.4) | 10 (19.6) |
| With family/friends | 19 (38.8) | 17 (33.3) |
| Employment status n (%) | ||
| Employed | 4 (8.2) | 3 (5.9) |
| Unemployed | 6 (12.2) | 7 (13.7) |
| Retired | 38 (77.6) | 41 (80.4) |
| FTE | 1 (2.0) | 0 (0) |
| Binocular distance VA (logMAR) mean (SD) | 0.95 (0.30) | 0.96 (0.25) |
| Near VA (M units) at 25 cm mean (SD) | 2.66 (1.67) | 2.53 (1.32) |
| Central visual field status (CCVFT grade) mean (SD) | 2.45 (1.42) | 3.10 (1.54) |
| (Log) Contrast sensitivity (Pelli Robson) mean (SD) | 0.78 (0.37) | 0.75 (0.31) |
- CCVFT = California Central Visual Field Test, CVI = Certificate of visual impairment, FTE = full-time education, SD = standard deviation, SI = sight impaired, SSI = severely sight impaired, VA = visual acuity.
Effectiveness
The mean difference in effect between the p-EVES plus optical LVA intervention and optical LVA only intervention equated to a visual ability improvement of 0.567 [95% confidence interval (CI) 0.279 to 0.847] on the NV-VFQ-15; QALY gains of 0.006 (95% CI −0.007 to 0.019) and 0.007 (95% CI −0.007 to 0.021) on the VisQoL and EQ-5D-5L, respectively; well-being improvement of 0.292 (−5.073 to 5.561) on the WHO-5; and YFC gains of 0.007 (95% CI −0.004 to 0.016) on the ICECAP-A. The CROS t-tests revealed a significant treatment effect for the NV-VFQ-15 [p < 0.001, ES = 0.57 (CI 0.33, 0.81)] and VisQoL [p = 0.04, ES = 0.01 (CI −0.02, 0.05)], but not the WHO-5 [p = 0.884, ES = 0.32 (CI −4.08, 4.72)], EQ-5D-5L [p = 0.09, ES = 0.03 (CI −0.01, 0.07)] or ICECAP-A [p = 0.523, ES = 0.01 (CI −0.02, 0.03)].
The statistical findings therefore indicate that the intervention including p-EVES was effective at improving ‘near vision’ visual function and providing additional QALYs (measured using a vision-specific measure) as compared to optical LVAs.
Costs
Device costs
Of the 82 participants included in the economic analyses, 28 received a p-EVES device costing £545, 50 received a p-EVES device costing £399 and four received a p-EVES device costing £249. The average cost of a p-EVES device was £441.54 (SD £81.53). The analysis was focussed on the incremental costs and benefits of p-EVES; therefore, it was not necessary to cost pre-existing optical LVAs which participants already had available.
Optometrist time costs
Additional staff time required to demonstrate how to use the p-EVES devices ranged from five to 30 min, with an average of 15 min (SD 6). On average, this equated to an additional staff time cost of £5.98 (SD £2.40) per participant. Combined with the p-EVES device costs, the average cost of the p-EVES plus optical LVA intervention was £447.52 (SD £82.29).
Carer time costs
During the 2-month p-EVES plus optical LVA intervention period (intervention A), average carer time amounted to 29.28 hr, an opportunity cost of £199.07 (SD £155.44) to the carer(s). During the 2-month optical LVA only control period (intervention B), average carer time amounted to 33.74 hr, an opportunity cost of £229.40 (SD £150.89) to the carer(s). The mean difference in carer costs was therefore −£30.33 (4.46 hr) with a bootstrapped 95% CI of −£76.10 to £15.99, equating to a reduced monthly carer cost of £15.17 per participant.
Total costs
Factoring in the intervention device costs, staff time and carer time costs, a mean difference of £417.19 was found between interventions A and B (total costs of £646.59 and £229.40, respectively), with a bootstrapped 95% CI of £366.04 and £465.89. For the cost-utility analysis, the mean difference in total costs was slightly lower at £398.95 (CI £336.69 to £460.15) due to higher mean total costs for the optical LVA subsample (N = 42; £247.64).
Primary cost-effectiveness analysis
In total, 164 sets of data were included in the cost-effectiveness analyses; 82 participants in both intervention arms, as the crossover design allowed participants to act as their own comparators. See Table 3 for a breakdown of effectiveness and ICER results. The mean difference in total costs between the p-EVES plus optical LVA intervention arm and the optical LVA only arm (£417.19, CI £366.04 to £465.89) was divided by the mean effect (0.567 NV-VFQ-15 points), producing an estimated ICER of £735.77 (CI £481.03 to £1525.18) per unit change in ‘near vision’ visual function.
| Trial outcomes | Effectiveness | Incremental effectiveness | ICER | Probability of cost-effectiveness | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Baseline mean (SD) | p-EVES mean (SD) | LVA mean (SD) | p-EVES mean effect (SD) | LVA mean effect (SD) | Effect mean difference | 95% CI for mean difference | ICER point estimate | 95% CI for ICER | Probability at threshold | ICER plane NW % | ICER plane NE % | |
| NV-VFQ-15 | 1.371 (1.28) | 2.015 (1.323) | 1.448 (1.284) | 0.645b (1.103) | 0.078 (0.714) | 0.567 | 0.279 – 0.847 | £735.77c,f | £481.03 – £1525.18 | 0.840i | 0 (0) | 5000 (100) |
| VisQoL a | 0.555 (0.218) | 0.582 (0.230) | 0.553 (0.228) | 0.095b (0.036) | 0.089 (0.035) | 0.006 | −0.007 – 0.019 | £66,490.00d,g | £23,054.5h | 0.14j | 949 (18.98) | 4051 (81.02) |
| EQ-5D–5L a | 0.629 (0.232) | 0.643 (0.217) | 0.611 (0.245) | 0.105 (0.036) | 0.098 (0.040) | 0.007 | −0.007 – 0.021 | £56,991.43d,g | £19,801.2h | 0.214j | 803 (16.06) | 4197 (83.94) |
| ICECAP-A a | 0.729 (0.183) | 0.755 (0.153) | 0.746 (0.166) | 0.125 (0.025) | 0.118 (0.029) | 0.007 | −0.004 – 0.016 | £56,991.43e,g | £26,447.6h | 0.094j | 589 (11.78) | 4411 (88.22) |
| WHO-5 | 54.683 (24.262) | 55.268 (21.883) | 54.976 (23.075) | 0.585 (19.097) | 0.293 (14.791) | 0.292 | −5.073 – 5.561 | £1,428.70c,f | £89.27h | 0.438i | 2329 (46.58) | 2671 (53.42) |
- a mean effect and ICERs based on 2month area under the curve outcomes
- b significant treatment effect p < 0.05
- c cost per point improvement
- d cost per QALY
- e cost per YFC
- f calculated using mean cost difference of £417.19 (full sample)
- g calculated using mean cost difference of £398.95 (reduced LVA control sample due to AUC calculation)
- h one-tailed bootstrapped 5% confidence limit, upper limit could not be calculated
- i at £1000 threshold
- j at £30,000 threshold
- Abbreviations: AUC = area under the curve, CI = confidence interval, ICER = incremental cost-effectiveness ratio, LVA = low vision aid, NE = North East, NW = North West, p-EVES = portable electronic vision enhancement systems, QALY = quality-adjusted life year, SD = standard deviation, YFC = year of full capability.
Figure 2A shows the ICER plane with 5000 points (100%) in the north-east quadrant, where the intervention is more costly but more effective. Figure 2B presents the corresponding CEAC, which shows the probability of cost-effectiveness at a range of thresholds. At a cost of £481.03, the intervention has a 2.5% probability of cost-effectiveness; at £1525.18, there is a 97.5% probability of cost-effectiveness. The ICER estimate falls between the confidence limits with a cost-effectiveness probability of 48% at a cost of £735.77 per unit change. At a threshold of £1000 per point improvement, the NV-VFQ-15 results demonstrate an 84.4% probability of cost-effectiveness.
Primary cost-utility analysis
A total of 82 participants were included in the p-EVES plus optical LVA arm and 44 in the optical LVA only control arm (due to issues with AUC caused by the crossover design). See Table 3 for a breakdown of effectiveness and ICER results. Effectiveness was expressed as the difference in mean QALYs experienced between the two interventions over a 2-month follow-up period, which equated to 0.006 years or 2.19 days using the VisQoL as the primary measure of utility. Mean difference in total costs (£398.95, CI £336.69 to £460.15) was divided by the mean difference in effect between the two interventions (0.006, CI −0.009 to 0.019), yielding an estimated cost per QALY of £66 490: over double the upper NICE cost threshold of £30 000 per QALY. Using the VisQoL, 19% of the bootstrapped estimates fell in the north-west quadrant, where the intervention is more costly and less effective (see Fig. 2C). Therefore, because the CEAC did not exceed 81% maximum probability of cost-effectiveness, upper confidence limits could not be calculated. A one-sided 95% lower confidence limit was calculated at £23 055. Figure 2D presents the corresponding CEAC for the VisQoL. At a threshold of £30 000 per QALY, there is a 14% probability of cost-effectiveness using the VisQoL as the measure of effect.
Sensitivity analysis: secondary cost-effectiveness analysis
A secondary cost-effectiveness analysis was conducted using well-being (measured using the WHO-5) as the measure of effect. See Table 3 for effectiveness and ICER results. Mean effect on the WHO-5 was calculated at 0.292 (95% CI −5.073 to 5.561), producing an ICER of £1428.70 (one-tailed lower 95% CI 89.27) per unit change in well-being. In total, 2329 (46.6%) of the bootstrapped estimates fell in the north-west quadrant, where the intervention is more costly and less effective. At a threshold of £1000 per point improvement, the WHO-5 demonstrates a 47.8% probability of cost-effectiveness, although it should be noted that this is for a one-point improvement on a well-being scale scored from 0 to 100.
Sensitivity analysis: secondary cost-utility and cost-capability analysis
A secondary cost-utility analysis was conducted using the EQ-5D-5L as the measure of utility, and cost-capability analysis using the ICECAP-A as the measure of capability. See Table 3 for effectiveness and ICER results. Effectiveness was expressed as the difference in mean QALYs (EQ-5D-5L) and YFCs (ICECAP-A) experienced between the interventions over a 2-month follow-up period, which equated to 0.007 years or 2.56 days on both the EQ-5D-5L and ICECAP-A, producing a cost per QALY/YFC of £56 991. One-sided 95% lower confidence limits were calculated at £19 801 for the EQ-5D-5L and £26 448 for ICECAP-A. At a threshold of £30 000 per QALY/YFC gain, probability of effectiveness was 21.4% using the EQ-5D-5L and 9.4% using the ICECAP-A.
Sensitivity analysis: adjusted intervention costs
Table 4 shows the sensitivity analysis results for all adjusted intervention costs. The higher cost sensitivity analysis (£569.52 per p-EVES intervention) increased the mean difference in total costs from £417.19 to £554.19 (£398.95 to £535.95 in cost-utility analysis), increasing ICERs by between 33% and 34%. In the lower cost sensitivity analysis (£249 per p-EVES intervention), the mean difference in total costs decreased from £417.19 to £233.67 (£398.95 to £215.43 in cost-utility analysis), decreasing ICERs by between 46% and 48%. In the ‘real-world’ sensitivity analysis (£150 per p-EVES intervention), the mean difference in total costs decreased from £417.19 to £119.67 (£398.95 to £101.43 in cost-utility analysis), decreasing ICERs by between 71% and 75%.
| Base case ICER | Lower intervention cost ICERa | Higher intervention cost ICERb | Welsh NHS intervention cost ICERc | |
|---|---|---|---|---|
| NV-VFQ-15 | £735.77 | £379.95 | £977.41 | £211.06 |
| VisQoL | £56 991.43 | £35 905.34 | £89 325.34 | £16 904.79 |
| EQ-5D | £66 490.00 | £30 776.01 | £76 564.58 | £14 489.82 |
| ICECAP | £56 991.43 | £30 776.01 | £76 564.58 | £14 489.82 |
| WHO | £1428.70 | £737.78 | £1897.92 | 409.82 |
- a All device costs reduced to £249, no optometrist time costs.
- b All device costs increased to £545, 1 hr optometrist time (£24.52) for all participants.
- c All device costs reduced to £150, no optometrist time costs.
- ICER = incremental cost-effectiveness ratio.
The sensitivity analyses demonstrate that with lower device costs, p-EVES interventions could potentially be cost-effective according to the NICE cost per QALY threshold of £20 000 to £30 000; in the ‘real-world’ sensitivity analysis, cost per QALY estimates fell to between £14 490 and £16 905 (EQ-5D-5L and VisQoL, respectively). It should be noted that the ‘real-world’ p-EVES cost of £150 is for a different device to the ones used in this study, and therefore, the equivalence of outcomes can only be assumed.
Discussion
Summary
The mean incremental cost, including carer costs, of the p-EVES intervention was £417.19. Bootstrapping gave an ICER of £735.83 per unit change in ‘near vision’ visual function, as measured by the NV-VFQ-15. Considering that this scale is scored from −3.36 to 5.07, the ICER point estimate relates to a 6.7% improvement in ‘near vision’ visual function. At a threshold of £1000 per point improvement, the p-EVES intervention had an 84.4% probability of cost-effectiveness using the NV-VFQ-15 as the measure of effect. Cost per point improvement on the WHO-5 was almost 50% over the £1000 threshold, which is particularly high when considering that the scale ranges from 0 to 100.
QALY and YFC results did not show the p-EVES intervention to be cost-effective in comparison with optical LVAs; however, sensitivity analyses demonstrated that lower cost p-EVES devices could yield better cost-effectiveness outcomes. Overall, the results indicate that p-EVES devices are a cost-effective means of improving ‘near vision’ visual function for people with a VI, but this does not translate to equivalent improvements in self-reported quality of life, capability or well-being.
Current p-EVES devices supplied by the NHS in UK cost approximately £150, on top of the cost of any other optical LVAs the patient may need. This is an important consideration, as the results indicate that p-EVES devices could supplement optical LVAs, not replace them. At up to five times the cost of an optical LVA, p-EVES devices are an additional cost to NHS low vision services. Even if p-EVES were to replace all currently supplied optical LVAs, additional funding would be needed to account for the extra cost of p-EVES compared with optical LVAs. The need for additional NHS funding to enable provision of p-EVES devices is therefore apparent.
Strength and limitations
Participants were only given 2 months with a p-EVES device. It would perhaps have been beneficial to extend the time-points to allow more prolonged use of the p-EVES devices. It was difficult to collect exact data regarding carer time and activities using the MLVQ as participants had difficulty accurately recalling the frequency and duration of assistance. In order to counter this, we consulted with a person with VI to develop a set of assumptions regarding activity duration and frequency. Although this is not the most robust approach to analysing need for assistance, it allowed us to utilize a uniform approach to costing the data. Due to a lack of certainty in economic evaluations, it is common place for health economists to apply assumptions where robust information or data is not available (Mogyorosy & Smith 2005). We used probabilistic sensitivity analysis, in the form of bootstrapping, to illustrate the variation in cost-effectiveness as a result of varying basic assumptions about effectiveness and costs. This accounts for some of the uncertainty in the data as a result of the assumptions made in the analysis.
The analysis benefited from the incorporation of both a generic and disease-specific measure of HRQoL to allow comparisons between measures. Another strength was the inclusion of carer time costs. Loss of productivity and the opportunity costs of informal care are often neglected in economic evaluations, but from a societal perspective these are important considerations.
Comparison with other literature
To our knowledge, this paper presents the first robust cost-effectiveness and cost-utility analyses of p-EVES devices. Although economic evidence is lacking in this field of research, a number of previous studies have compared different types of LVAs, including electronic devices. To date, limited evidence has shown that non-portable EVES devices are more effective than optical LVAs at improving ‘near vision’ tasks such as reading performance. Culham et al. (2004, 2009) found that some EVES provide better performance of ‘near vision’ tasks, but likewise optical LVAs were more effective for other tasks. Peterson et al. (2003) found that in general, EVES enabled faster reading speeds, but optical LVAs facilitated faster performance of item location tasks. There is no published evidence about the effectiveness of the most technologically advanced p-EVES devices available today, such as those used in this trial.
Implications for future research
Only the NV-VFQ-15 and VisQoL measures demonstrated a significant treatment effect for the p-EVES intervention. The results from the NV-VFQ-15 suggest that the p-EVES intervention could potentially be a cost-effective means of improving ‘near vision’ visual function. However, the results from the preference-based utility and capability measures (VisQoL, EQ-5D-5L and ICECAP-A) show that the p-EVES intervention could not be proven to be a cost-effective approach to improving health status or well-being, therefore indicating that improvement to ‘near vision’ visual function does not drastically affect overall health status, or that standard measures of health and well-being are not sufficiently sensitive to measure change in this population.
Previous literature shows that the EQ-5D measures can be insensitive to utility change caused by vision and visual function (Bozzani et al. 2012; Longworth et al. 2014). In order to examine the appropriateness of vision-related and generic measures of utility in this setting, the VisQoL and EQ-5D-5L were compared in the cost-utility analyses.
The VisQoL was designed to be sensitive to vision-related quality of life and is an appropriate means of rapidly estimating utility for the purpose of economic evaluation (Peacock et al. 2008). It is therefore interesting that both the VisQoL and EQ-5D-5L exhibited relatively similar results in terms of incremental utility gains between the intervention arms (VisQoL = 0.006; EQ-5D-5L = 0.007). The EQ-5D-5L showed greater overall utility estimates and a lower cost per QALY estimate.
If we consider the VisQoL to be more sensitive in this population, these results would suggest that the EQ-5D-5L was in fact overestimating utility in this population, but both are insensitive in picking up utility gains from ability to perform normal tasks. This is an important consideration for future research within this field, as accurate utility measurement is paramount in the calculation of QALYs. Furthermore, only the VisQoL showed a significant difference between the intervention arms. Considering these points, the VisQoL appears to be the most appropriate measure of utility in this setting.
Using the NV-VFQ-15, a 0.567 or 6.7% improvement in visual function was found after the p-EVES intervention. Equivalently sized changes to utility and capability were not found using any of the measures. It is first important to reiterate that each time-point in the study was only 2 months long, and therefore, there may have been insufficient time for the p-EVES intervention to facilitate change to broader aspects of health and well-being. Furthermore, the NV-VFQ-15 only measures change in terms of ‘near vision’ visual function. Quality of life, well-being and capability are affected by a large array of factors, and therefore, we would not expect to see equivalent increases to utility as visual function.
