Implementation of A New, Mobile Diabetic Retinopathy Screening Model Incorporating Artificial Intelligence in Remote Western Australia
Funding: This activity was supported by the Western Australian Future Health Research and Innovation Fund (grant ID DoH202310626/1), the McCusker Charitable Foundation, and the Fred Hollows Foundation.
ABSTRACT
Objective
Diabetic retinopathy (DR) screening rates are poor in remote Western Australia where communities rely on outdated primary care-based retinal cameras. Deep learning systems (DLS) may improve access to screening, however, require validation in real-world settings. This study describes and evaluates the implementation of a new, mobile DR screening model that incorporates artificial intelligence (AI) into routine care.
Design
Prospective, population-based study.
Setting
The model was co-designed with local Aboriginal communities and implemented in the remote, Pilbara region of Western Australia. A research officer without formal healthcare qualification performed retinal screening aboard a Mercedes Sprinter Van using an automated retinal camera with integrated AI diagnostics. Patients received their diagnosis on-the-spot and completed an evaluation survey. A remote clinician provided supervision and on-the-spot telehealth consultation for referable disease.
Participants
People with diabetes from the Pilbara region.
Main Outcome Measure(s)
Number of people screened, acceptability of AI to patients.
Results
From February to August 2024, DR screening was provided to 9 communities across the Pilbara region. 78 patients provided research consent, of which 56.4% were Aboriginal or Torres Strait Islanders. 10.3% of retinal photos had referable DR and 8.4% of photos were ungradable. 96% of patients were ‘Happy with the use of AI’.
Conclusion
Our new model for AI-assisted DR screening was culturally safe, acceptable to patients and effective, demonstrating an 11-fold increase in screening rates compared to 2023 Pilbara data. In remote Australian settings, AI-assisted DR screening may overcome historical barriers to service provision and improve minimisation of preventable blindness.
Summary
-
What is already known on this subject?
- ○
98% of visual loss from diabetic retinopathy (DR) is preventable through frequent screening.
- ○
Aboriginal Australians and Australians living rurally are under-screened due to barriers such as a lack of access to care.
- ○
Artificial intelligence (AI) deep learning systems are reportedly accurate at diagnosing retinal photographs—however, their real-world utility in screening under-served populations is not well validated.
- ○
-
What do we now know as a result of this study that we did not know before?
- ○
Our prospective study describes and evaluates a new mobile DR screening model incorporating artificial intelligence in a real-world population in remote Western Australia.
- ○
DR screening with AI assistance is acceptable to patients, including Aboriginal patients.
- ○
DR screening with AI assistance has the potential to improve DR screening rates in rural populations who have historically been underserved by previous screening models.
- ○
1 Introduction
Diabetic retinopathy (DR) is a common microvascular complication of diabetes where, over time, the blood vessels at the back of the eye (retina) become damaged by high blood sugar levels. DR is a significant cause of preventable vision loss and blindness globally [1, 2], and is the leading cause of blindness in working-age adults worldwide [3]. Regular eye screening, typically comprising visual acuity and retinal photographs, is crucial to identifying disease early, when treatment can prevent permanent vision loss [4]. The current Australian guidelines recommend annual DR screening for Aboriginal and Torres Strait Islander people [5], yet only an estimated 34% were screened in 2019–2020 [2]. Inadequate DR screening in Aboriginal Australians is a critical gap in service provision as Aboriginal and Torres Strait Islander people have higher diabetes prevalence, are twice as likely to develop DR, and six times more likely to develop ‘referable’ DR, which can progress to blindness if unrecognised and untreated [6, 7]. This disparity in screening service provision is especially pronounced in rural and remote areas of Australia, which have 19 times less ophthalmology workforce than the average Australian availability [2].
For over a decade, Lions Outback Vision (LOV) has endeavoured to minimise disparity in eye care service provision by providing ophthalmology and optometry services throughout rural and remote Western Australia (WA), through outreach services and more recently through the ‘Kimberley Hub’. The large, mobile ‘Vision Vn’ travels to over 20 towns but does not have the capacity to provide routine DR screening and cannot travel on rough roads to reach more remote communities. The Pilbara region, where Lions Outback Vision is the sole public ophthalmology provider, remains particularly under-serviced from a DR screening perspective. To combat low screening rates, primary care-based retinal cameras were installed in several local health services in the Pilbara region during a 2016 government initiative. Yet in all of 2022, only 23 people received DR screening through this model due to high staff turnover, the subsequent lack of staff trained to operate the cameras, and asynchronous diagnosis of photographs, which necessitates multiple episodes of care with poorer compliance to follow-up (unpublished LOV data).
An alternative model for DR screening is needed to improve screening rates and prevent vision loss in remote Western Australia. Advances in technology have resulted in automated retinal cameras operable by staff without formal healthcare qualifications. Furthermore, certain cameras are integrated with synchronous artificial intelligence (AI) diagnostic capability, with multiple AI deep learning systems (DLSs) demonstrating excellent sensitivity and specificity for diagnosing DR [8]. However, few DLSs for DR have been validated on Aboriginal Australian populations and in regional settings. Furthermore, with the high prevalence of Aboriginal people living in rural Australia (18% in the East Pilbara c.f. WA average of 3.3% [9]), ensuring culturally safe use of AI is of paramount importance. This prospective study aims to (1) evaluate the real-world feasibility and effectiveness of AI-assisted DR screening in remote Western Australia and (2) evaluate the acceptability to patients. The accuracy of AI diagnosis compared to human diagnosis will be the focus of a future study.
2 Materials and Methods
2.1 Stakeholder and Community Engagement
Lions Outback Vision underwent extensive stakeholder and community engagement in the Pilbara region of Western Australia, prior to co-designing our new model for DR screening. The aim of this engagement was to ensure our model would be effective, culturally appropriate, and complement, not duplicate, existing services in the region. An ethical framework for evaluation was co-developed with support from several local Aboriginal Community-Controlled Health Organisations (ACCHO) including the Pilbara Aboriginal Health Alliance, Puntukurnu Aboriginal Medical Service and Mawarnkarra Health Service. Ethical approval for research was granted by the Western Australian Aboriginal Health Ethics Committee (HREC1294).
LOV also collaborated with local health services such as ‘Integrated Chronic Disease Care’, ‘Strengthening Aboriginal Health’ and ‘Diabetes Education’ in order to synchronise the implementation of our new DR screening model with existing services where possible, maximising patient engagement. LOV further maximised patient engagement by texting and calling patients with diabetes overdue for screening prior to visiting their communities. These patients were identified on LOV's electronic medical record and from records shared by local ACCHOs and WA Country Health Service. Walk-in appointments were encouraged with support from local Aboriginal elders and social media (Facebook) posts to local groups were made to further spread awareness of LOV's screening activity. Patients were offered free reading glasses to further incentivise engagement in screening. Regular stakeholder and community meetings were conducted throughout LOV's screening activity to identify ongoing issues needing improvement.
2.2 Model of Care
- RO records patient demographic information (name, date of birth, sex, ethnicity) onto LOV's electronic medical record.
- RO measures distance visual acuity using a 3-m Snellen chart.
- RO takes retinal photographs of both eyes using the Topcon Maestro2 camera (automatically uploaded to Topcon Harmony online viewing platform). Note: some patients require pupil dilation with 0.5% tropicamide eye drops (after eye pressure measurement using an iCare tonometer).
- AI DLSs (Thirona RetCAD, EyRIS SELENA+) synchronously generate PDF screening reports, with concrete diagnoses (as per International Clinical Diabetic Retinopathy and Diabetic Macular Oedema Disease Severity Scale [10]).
- RO discusses report with the patient, at the point of care, and formulates a concrete follow-up plan.
- Copy of report is shared with patient's primary care provider.
Apart from the incorporation of AI diagnosis, our model is analogous to other screening models. A remote telehealth clinician was available at all times to support the research officer and supervise the screening process. If there was any significant down-time during screening (e.g., waiting for pupillary dilation), patients were shown ‘Bad Sugar Bad Eyes’, a culturally appropriate educational video about DR developed and validated by LOV, available in both English and Martu Wangka, the two most spoken languages in the Pilbara region [11]. Informed consent to use data for research was sought from all patients prior to screening, although declination did not affect the screening process. Patients were informed of the use of AI for point-of-care diagnosis and the RO was trained by our Aboriginal Eye Health Coordinator on how to explain the use of AI in a culturally appropriate way. Consenting patients were also invited to complete a ‘patient evaluation survey’ after screening. The survey comprised five yes/no questions assessing patient satisfaction, one Likert scale rating the ‘overall experience’ and an open-ended comments field for additional feedback.
2.3 Statistical Analysis
Categorical data are presented as numbers (percentage), and normally distributed continuous data are presented as means ± SD. Bivariate analysis was conducted by chi-squared test for categorical data.
3 Results
Between 14 February 2024 and 20 August 2024, the Sprinter Van provided DR screening to 9 communities across the Pilbara region (Figure 1). Consent for research was obtained from 78 participants (39 (50.0%) male, 44 (56.4%) Aboriginal or Torres Strait Islander, mean ± SD age of 51.6 ± 12.8 years at visit). In total, 155 retinal photographs were included in this study (one patient had a prosthetic eye). As seen in Table 1, 126 (81.3%) photographs had non-referable DR (117 no DR, 9 mild DR) and 16 (10.3%) had referable DR (10 moderate non-proliferative DR, 4 severe non-proliferative DR, 2 proliferative DR). Only 13 (8.4%) photographs were ungradable, of which 10 (10/111 = 9.0%) were taken on the Optomed Aurora IQ camera and 3 (3/44 = 6.8%) on the Topcon Maestro2 camera with no difference in gradability between the two cameras (p = 0.66). Of the 78 consenting participants, 54 had confirmed type 1 or type 2 diabetes mellitus (28 (51.9%) male, 32 (59.3%) Aboriginal or Torres Strait Islander, mean ± SD age of 53.9 ± 12.1 years at visit). However, there were 24 patients unsure of their diabetes status for whom screening was still provided (all 24 had no DR).
| Non-referable DR | Referable DR | Ungradable | Total | |
|---|---|---|---|---|
| Optomed Aurora IQ | 85 (77%) | 16 (14%) | 10 (9%) | 111 |
| Topcon Maestro2 | 41 (93%) | 0 (0%) | 3 (7%) | 44 |
| Total | 126 (81%) | 16 (10%) | 13 (8%) | 155 |
- Abbreviation: DR, diabetic retinopathy.
The evaluation survey was completed by 50 (64.1%) consenting participants. The results are represented in Table 2 and are highly positive. 96% of responders were ‘Happy with the use of AI’ and left additional comments such as ‘Amazed at the technology!’ and ‘So good to have these facilities and practitioners come to remote towns where healthcare is limited!’. When rating the overall experience on a 5-point Likert Scale from ‘Very Good’ to ‘Very Bad’, all participants either chose ‘Good’ (7 participants; 14%) or ‘Very Good’ (43 participants; 86%).
| Yes | No | Unsure | |
|---|---|---|---|
| Did you feel the procedure was explained clearly? | 50 (100%) | 0 | 0 |
| Did you feel comfortable during your eye check? | 49 (98%) | 1 (2%) | 0 |
| Do you understand why you had your eyes checked? | 49 (98%) | 0 | 1 (2%) |
| Were you happy with the use of AI? | 48 (96%) | 0 | 2 (4%) |
| Would you recommend the service to your family and friends? | 50 (100%) | 0 | 0 |
- Abbreviation: AI, artificial intelligence.
4 Discussion
4.1 Translating the Theoretical Promise of AI Into Real-World Impact
To our knowledge, our study is the first to describe the incorporation of AI diagnostic assistance into routine practice for DR screening in a rural Australian setting. Although DLSs for DR have existed for some years, the feasibility of incorporation into routine practice has typically been validated in metropolitan, centralised eye clinics staffed by ophthalmic nurses with minimal technological barriers [12-14], and studies in rural regions have been limited to non-Australian settings, including rural India [15] and rural China [16]. Given the unique challenges of healthcare service provision in rural Australia, including the issues of workforce shortage and maintaining cultural appropriateness, our study addresses an important gap in knowledge by demonstrating that AI can be safely and feasibly used rurally, and this is explored further in Sections 4.1.1 and 4.1.2. The number of patients screened between February and August 2024 is 11 times more than were screened by primary care-based cameras during the same period in 2023 (unpublished LOV data). This is an encouraging validation of the feasibility and acceptability of our model. The rate of 10.3% for referrable DR in our study supports the need to enhance screening in rural Australia to minimise preventable vision loss and disease burden.
4.1.1 Empowering Non-Ophthalmic Workforce to Perform High Fidelity DR Screening
Only 8.4% of retinal photographs taken during the implementation of our model were ungradable, which is comparable to the existing literature [17]. This validates the feasibility of training a research officer without formal health qualifications to perform high-fidelity retinal screening using intuitive, automated cameras, both handheld and table mounted. The idea that a primary care-based workforce could similarly be trained to perform high-fidelity screening using modern cameras is, by extension, supported, and this could be crucial to overcome the lack of ophthalmology and optometry workforce in rural Australia. Uptake of primary care-based retinal screening models has historically been poor due to ‘difficult-to-use’ retinal cameras that relied on manual transfer of photographs for diagnosis, resulting in delayed feedback to patients and providers. The successful real-world validation of modern retinal cameras, which automatically upload and diagnose photographs on-the-spot, demonstrates the potential for AI to deliver seamless screening and promote equitable access to eyecare for disadvantaged populations [18] across rural Australia.
4.1.2 Incorporating AI Into a Culturally Safe and Sustainable Model for DR Screening
The importance of stakeholder and community engagement in co-designing effective, safe, and sustainable health projects for Aboriginal populations is well-established in existing literature [19]. Engagement of the Aboriginal community was critical in shaping our model of care. Community meetings ensured we were aware of important cultural considerations and could co-design acceptable solutions; for example, it is not appropriate for a female patient to be alone inside a van with a male research officer, thus LOV encouraged patients to bring family members with them to attend screening. Community engagement also helped to maximise the impact of our study by allowing us to engage patients known to local ACCHOs in need of screening. Although the acceptability of AI-assisted DR screening has previously been validated [20, 21], including in Australian populations [22], there is a paucity of literature validating the use of AI in Aboriginal patients. One New Zealand study revealed that 64% of Pacific Islanders and 39% of Māori participants would prefer ‘human-led’ screening even if it meant longer wait times, compared to 38% of Asian participants and 28% of participants with European ancestry [23]. This was posited to be due to cultural differences, with Pacific Island and Māori people known to place a high value on personal interaction in their cultural norms. As Aboriginal people are also a demographic minority within Australia, who demonstrate unique cultural values, our study bridges an important gap in understanding by demonstrating that DR screening with AI can be safely implemented for rural Aboriginal patients, and this is supported by the findings from our evaluation survey. The sustainability of DR screening in remote communities relies largely on endorsement from local organisations. As emphasised earlier in Section 4.1.1, our study supports the feasibility of implementing modern, automated retinal cameras for use in rural Australia, and thus, with a growing emphasis on sovereignty and empowerment of ACCHOs to improve outcomes for Aboriginal patients, it may be desirable to further explore the potential for ACCHOs to coordinate their own DR screening programs, with less reliance on outreach from visiting specialists.
4.2 Limitations of a Mobile Service
Although the mobile nature of our Sprinter Van has augmented eyecare service provision and enabled DR screening to be performed in more remote locations than LOV's larger Vision Van, we experienced several difficulties which may impede any mobile service. These included: equipment malfunction, internet failure, ensuring the personal safety of our outreach research officer and cost. On one occasion, our Topcon Maestro2 retinal camera malfunctioned after a visit to Jigalong, most likely due to damage from the rough roads travelled on. The increased risk of equipment malfunction and consequent cost and inconvenience of repair must be considered when implementing a mobile screening service in a remote setting. The absence of reliable internet connection also at times impeded efficient uploading of retinal photographs to our remote viewing platform for human and AI-assisted diagnosis, resulting in a more disjointed patient experience and possibly, selection bias in completion of the evaluation survey. Thirdly, ensuring the wellbeing and safety of our sole research officer was challenging. Due to the paucity of rural accommodation, the research officer needed to sleep in the Sprinter Van on several occasions. The ‘three weeks on, one week off’ screening schedule was also demanding and in instances of illness or unforeseen circumstance, screening activity had to be cancelled, to the detriment of patients and local community. Lastly, cost is a limitation of mobile services as in addition to equipment malfunction, daily allowances and accommodation costs must be covered, as do operating costs of the vehicle. Whilst mobile service provision may continue to have a role in remote settings, it is possible that refinement of primary care-based models will be a more sustainable and scalable solution, as the risk of damage to equipment, unreliable internet and threat to staff wellbeing may be mitigated by utilising ‘permanent’ local infrastructure and workforce, and costs may also be reduced.
4.3 Future Directions
It is important to validate the diagnostic accuracy of DLSs on the population for which they will be used especially for minority populations [24, 25]. Although it is beyond the scope of the present paper to prospectively validate the diagnostic accuracy of Thirona RetCAD and EyRIS SELENA+, Lions Outback Vision are completing a separate study aimed at validating the accuracy of these two DLSs used during our new model for screening. LOV have previously conducted research demonstrating high sensitivity and specificity for Google's ARDA DLS on retinal photographs taken from Aboriginal patients [26]. Thus, we will also include ARDA in a three-way prospective comparison against Thirona RetCAD and EyRIS SELENA+. We expect that validation of DLSs for DR on a real-world rural population with high Aboriginal representation may help to substantiate a more autonomous use of AI diagnostic assistance in future, without constant clinician supervision. This may further overcome the issues of workforce shortage often seen in rural service provision.
Beyond validation of the diagnostic accuracy of Thirona RetCAD, EyRIS SELENA+ and Google ARDA for DR screening on a real-world rural population with a largely Aboriginal demographic, Lions Outback Vision are also planning a separate study in the field of AI ‘oculomics’. Certain DLSs [27] can not only diagnose DR with high accuracy but also diagnose cardiovascular disease elsewhere in the body, from retinal photographs. The potential to accurately diagnose non-ophthalmic disease, including ischaemic heart disease and heart failure, is very promising as cardiovascular disease is collectively the largest cause of mortality worldwide [28] and disproportionately affects Aboriginal Australians [29]. Opportunistically and non-invasively diagnosing systemic disease from retinal imaging alone may promote improved recognition of disease and earlier intervention, especially in underserviced populations unable to undertake less accessible diagnostic testing like echocardiography or invasive angiography.
Health economic considerations are critical to analyse any new intervention's future sustainability in clinical practice. The population impact and cost-effectiveness of AI-based DR screening have already been validated in an Australian model [30] although a health economics analysis plan tailored to Lions Outback Vision's model is being developed by collaborators at Oxford University, with the specific objective of evaluating the cost-effectiveness of a statewide implementation that extends beyond the Pilbara region. Given the limitations of a mobile screening service, the most sustainable option may be to augment screening in existing primary care settings by replacing outdated retinal cameras with modern, automated devices that require minimal training and integrating validated DLSs to seamlessly provide an on-the-spot diagnosis. The potential for cardiovascular assessment via oculomics will further support primary care providers in ensuring holistic care, and the integration of AI-assisted DR screening into existing services will be more empowering for ACCHOs compared to a mobile model that relies solely on visiting outreach services. Patients with referable DR detected with ‘point-of-care’ diagnostic support in primary care settings would still be able to access ‘on-demand’ or ‘on-call’ video telehealth provided by Lions Outback Vision across regional Western Australia to decide if an in-person review is needed on the Vision Van or elsewhere. The importance of on-call telehealth in facilitating patient access to specialist eye care in rural WA has previously been validated [31].
5 Conclusion
Incorporation of AI into DR screening is feasible in remote settings, is acceptable to patients, and has potential to improve screening rates in remote Australian communities and among Aboriginal patients. Whilst we have demonstrated that mobile screening with AI is achievable, it may be more sustainable to augment screening in primary care settings by adopting upgraded, automated retinal cameras and using AI to provide on-the-spot diagnostic assistance to local providers. Further research is needed to validate the accuracy of AI in diagnosing DR and systemic disease, and to evaluate economic feasibility prior to broader integration of AI into routine practice in DR screening.
Author Contributions
Qiang Li: investigation, writing – original draft, writing – review and editing. Jocelyn J. Drinkwater: conceptualization, formal analysis, funding acquisition, investigation, methodology, writing – original draft, writing – review and editing. Kerry Woods: writing – review and editing, methodology. Emma Douglas: funding acquisition, methodology, writing – review and editing. Alex Ramirez: investigation. Angus W. Turner: conceptualization, funding acquisition, methodology, writing – review and editing.
Acknowledgements
We would like to acknowledge the in-kind support of our technology partners: Thirona, RetCAD, EyRIS, Topcon Healthcare, and Optomed. We would equally like to acknowledge the unwavering local support of our WA/Pilbara regional partners: Pilbara Aboriginal Health Alliance, Puntukurnu Aboriginal Medical Service, Mawarnkarra Health Service, Royal Flying Doctor Service, Nintirri Centre, Karratha Central Healthcare, Aboriginal Health Council of Western Australia, Diabetes WA, WA Country Health Service, Integrated Chronic Disease Care Program, IBN Group, Strengthening Aboriginal Health Program, Panaceum Pilbara, Jigalong Community Inc., Ashburton Aboriginal Corporation, Gumala Aboriginal Corporation, and Yinhawangka Aboriginal Corporation. Open access publishing facilitated by The University of Western Australia, as part of the Wiley - The University of Western Australia agreement via the Council of Australian University Librarians.
Ethics Statement
Ethical approval for research was granted by the Western Australian Aboriginal Ethics Committee (HREC1294).
Conflicts of Interest
The authors declare no conflicts of interest.
Open Research
Data Availability Statement
The data that support the findings of this study are available from the corresponding author upon reasonable request.
