Impact of technological and departmental changes on incident rates in radiation oncology over a seventeen-year period
Abstract
Introduction
Advancements in technology and processes are designed to bring improvement. However, this is often achieved in parallel with increases in complexity, simultaneously presenting opportunities for new types of errors. This study aims to contextualise the impact of internal departmental changes upon radiation incidents and near misses recorded.
Methods
A timeline of events and a comprehensive incident categorisation system were applied to all radiation incidents and near misses recorded at the Princess Alexandra Hospital Radiation Oncology department from 2003 to 2019, inclusive. Descriptive statistics were performed to identify the type and number of incidents reported during the time period in relation to potential changes within the department, with a focus on the implementation of an electronic environment.
Results
Over the seventeen-year period, 157 incidents and 76 near misses were reported. The majority of incidents were classified as ‘procedural’ (78%), with ‘treatment’ being both the highest point of error and point of detection (49% and 85%, respectively).
The largest number of incidents and near misses were reported in 2018 (n = 39) which was also a year that experienced the largest number of departmental changes (n = 16), including the move to a completely electronic planning process.
Conclusions
Changes within the department were followed by an increasing number of reported incidents. Proactive measures should be undertaken prior to the implementation of major changes within the department to aid in the minimisation of incident occurrence.
Introduction
Radiation therapy (RT) is an inherently multifaceted field.1 Frewen et al.2 identified 60 process steps with 141 sub-processes in the RT pathway at the Princess Alexandra Hospital (PAH); all of which have a potential for error. RT is a discipline of constant and rapid advancement in technology and increased reliance on automated electronic systems.1, 3 The implementation of new technologies and innovations has been proven to improve the efficiency of quality assurance (QA) processes and reduce occurrence of incidents, such as human errors.4, 5 Research conducted by Fraass et al.6 supports this, stating that the use of computer-controlled delivery systems for routine complex three-dimensional conformal radiation therapy (3DCRT) treatments leads to a decrease in treatment delivery errors. This and other reported benefits are likely due to a reduction in manual data entry.6 However, it has been concluded that such enhancements and the reliance upon technology itself can paradoxically facilitate new sources of error and lead to the development of potential incidents that were previously non-existent.1, 7-10 Furthermore, in 2009, the International Commission on Radiological Protection (ICRP) Report 1128 identified new types of incidents are occurring due to the abundance of computers with increasingly complicated software and complexities of treatment processes. Hence, as the complexity of practices increases, the risk of a potential error also increases.1 Although Record and Verify (R&V) Systems have been reported to reduce radiation errors, they do not eliminate them completely and can even themselves become a source of error.11
It is widely reported that the majority of incidents in RT are relatively minor, resulting in little to no clinical impact.3, 12 However, most process errors in RT are preventable.13 In order to implement countermeasures to mitigate risks to patients, incident reporting and analysis are essential.13, 14 Incident analysis is aimed at improving patient safety and requires a holistic approach, whereby incidents and near misses should not be addressed in isolation from the magnitude of complex environmental factors that co-exist. In 2014, the Queensland Government’s Best Practice Guide to Clinical Incident Management highlighted the importance of considering the context of incidents when analysing contributing factors and recommending actions.15 Hence, for effective and completeness of incident analysis there is a need to consider the interdependence and interconnectivity between the multitude of activities, events, and surroundings that occur simultaneously.15 In addition to this, the ICRP Report 112 places high importance on the need to learn from previous incidents and to take a proactive approach by considering the statistical likelihood of these errors reoccurring and to continuously anticipate potential errors.8
The purpose of this study was to retrospectively review radiation incident and near miss data from the Radiation Oncology PAH Ipswich Road (ROPAIR) campus in the context of a timeline of events and changes within the work environment. Any potential relationships between these changes and radiation incident and near miss occurrences were explored. Particular focus was given to the electronic environment.
Methods
Department
The ROPAIR department houses one of the largest public radiotherapy departments in Queensland, Australia, with the first patient undergoing treatment in 2002. Seventeen years on, the department has rapidly grown and now operates two Computed Tomography (CT) scanners, five linear accelerators (linacs), a Leksell Gamma Knife Perfexion (Elekta AB, Stockholm, Sweden), Mosaiq R&V system (IMPAC Medical Systems, Stockholm, Sweden) and Pinnacle (Koninklijke Philips, Amsterdam, Netherlands), Gamma Plan (Elekta AB, Stockholm, Sweden) and Monaco (Elekta AB, Stockholm, Sweden) treatment planning systems, with 65 full-time equivalent (FTE) radiation therapists.
At ROPAIR, there is an established no-blame culture, therefore, the focus is on incident prevention and investigating system failures to allow appropriate improvements to be made. Incidents and near misses were initially recorded using PRIME (Acclaim Safety Systems Ltd) and then from 2017 using RiskMan (Correct Care Australasia). An ethics waiver was granted by the Metro South Health Human Research Ethics Committee for the conduct of this quality improvement study.
Data collection
A radiation incident at the ROPAIR department is defined as ‘an unintended deviation of an event or series of events that has led to an adverse effect (on a patient)’.4, 16 A near miss is classified as ‘an unwanted or unexpected change from what was intended that did not reach the patient but had the potential to cause an adverse effect if it remained undetected’.16-18
All radiation incidents and near misses that were reported during the period encompassing January 2003 to December 2019 inclusive were recorded in a departmental database after having been logged in the hospital-wide reporting system. For the purposes of this study, any incidents relating to software faults, such as XVI cone-beam CT (CBCT) scan terminations, were excluded, to eliminate a potential skew in the results. Furthermore, to assist in data homogeneity, all data including both patient numbers and incidents were related to linac-based treatments. Any near misses detected by the existing departmental QA processes were not considered as near miss events, as if these errors are being detected by these fixed checkpoints, it is evidence that these well-established systems are effective and working as intended. Furthermore, to include this data would not be beneficial for analysis and makes the reporting system cumbersome, placing increased demand on staff, potentially resulting in decreased reporting compliance.
Taxonomy
As Queensland has no standardised categories for incident reporting, the Incident Review Committee within the department devised a customised error-type taxonomy system that was relevant and specific to the ROPAIR department. This allows for the categorisation of radiation incidents and near misses, therefore, the data collected can be utilised clinically to identify areas of potential improvement. The categories used are detailed in Table 1. The actual and potential severity assessment codes were also determined for each incident and near miss, as a method of grading the risk and likely frequency of an event. All incidents and near misses were discussed at quarterly meetings and categorised based on consensus.
| Category | Subcategory |
|---|---|
| Accessories | Bolus |
| Shielding | |
| Immobilisation | |
| Imaging | Incorrect imaging setting or procedures performed |
| Imaging not reviewed when required | |
| Incorrect match of pre-treatment image | |
| Injury | Patient injury |
| Staff injury | |
| Visitor injury | |
| Procedural | Calculation |
| Communication | |
| Documentation | |
| Setup | |
| Quality Assurance | |
| Technology | Software fault |
| Hardware fault |
Causal factors and departmental changes
There are often multiple contributing factors when an incident or near miss occurs, however, for the purpose of data collection, the most dominant causal factor was determined for each event, using a root cause analysis method. The causal factor categories can be seen in Table 2. Also recorded were different steps in workflow processes, which assisted with determining the prominent causal factor. For example, the point of error refers to the point at which the incident originated, and the point of detection refers to the point in the workflow process that the error or near miss was discovered.
| Causal Category |
Incidents n (%) |
Near Misses n (%) |
Total Incidents n (%) |
|---|---|---|---|
| Insufficient training or education | 23 (14.6%) | 11 (14.5%) | 34 (14.6%) |
| Inadequate supervision | 1 (0.6%) | 1 (1.3%) | 2 (0.9%) |
| Poor communication | 16 (10.2%) | 12 (15.8%) | 28 (12%) |
| Human error | 59 (37.6%) | 22 (28.9%) | 81 (34.8%) |
| Protocol or procedural failure | 36 (22.9%) | 29 (38.2%) | 65 (27.9%) |
| Environmental factors and Internal Systems | 22 (14%) | 1 (1.3%) | 23 (9.9%) |
| Total | 157 | 76 | 233 |
Changes within the department over the seventeen-year period were documented as a timeline of events and categorised as either major or minor by the Incident Review Committee. Each event was categorised in terms of the type of change. A major change was defined as something significant with an expected large impact on multiple areas of the department, such as the implementation of Volumetric Modulated Arc Therapy (VMAT); and minor changes were less significant and non-complex changes within the department, such as the introduction of new stabilisation equipment. A summary of the timeline can be seen in Table 3.
| Year | Level | Category | Event |
|---|---|---|---|
| 2002 | Major | New Equipment and Software | New CT scanner |
| Major | New Equipment and Software | New planning system (Eclipse) | |
| Major | Electronic | All digital imaging implemented | |
| Major | New Equipment and Software | Two new linear accelerators (linacs) installed | |
| 2007 | Major | New Equipment and Software |
Mosaiq Implementation |
| Major | New Equipment and Software | Third linac installed, with cone beam computed tomography (CBCT) capabilities | |
| 2009 | Major | New Practice | Intensity Modulated Radiation Therapy (IMRT) and increased daily imaging implemented |
| 2010 | Major | New Practice | Stereotactic Body Radiation Therapy (SBRT) program commenced |
| 2012 | Major | New Equipment and Software | Fourth linac installed, specifically for SBRT (with HexaPOD and ExacTrac) |
| Major | New Equipment and Software | Fifth linac installed | |
| Major | Staffing | Major increase in staff numbers (of 19%) | |
| Major | New Equipment and Software | Commenced use of in-house quality assurance (QA) software developed to compare TPS (treatment planning system) data to R&V (Record and Verify) system data | |
| Major | New Practice | First Volumetric Modulated Arc Therapy (VMAT) treatment | |
| 2013 | Major | New Equipment and Software | Sixth linac installed |
| Major | New Equipment and Software | Two new planning systems installed (Pinnacle and iPlan) | |
| 2014 | Major | New Practice | Planning teams started |
| Major | Electronic | Electronic prescription commences | |
| 2015 | Major | New Practice | Commence using Deep Inspiration Breath-Hold (DIBH) |
| Minor | Staffing | Increased staff education on incident reporting | |
| 2016 | Major | Electronic | Move to completely paperless treatment process |
| Major | New Practice | First flattening filter free (FFF) patient treated | |
| Minor | Staffing | Incident Review Committee | |
| 2017 | Major | New Practice | Introduction of RiskMan |
| 2018 | Major | Electronic | Move to completely paperless planning process |
| Minor | Imaging | Intrafraction imaging introduced |
Statistical analysis
Descriptive statistics were presented using mean and standard deviation (SD) for continuous data and using frequency and percentage for categorical data. All descriptive analyses were performed using the R statistical software.19
Results
Incident rates
Of all events reported (n = 233), 76 were categorised as near misses. For this study, radiation incidents and near miss rates were calculated per patient attendance or fraction. From 2003 to 2019, the rate of radiation incidents and near misses per 100 patient attendances was 0.05; excluding near misses, this rate fell to 0.04. The least number of incidents were recorded in 2004 and 2011 (n = 3 for both years). With the largest number of incidents reported in 2018 (n = 39), followed by 2016 (n = 27).
The dominant category for events was ‘procedural’, accounting for 78% (n = 181), which is a rate of 0.04 per 100 patient attendances, of all errors and near misses. The sub-categories of ‘setup’ closely followed by ‘quality assurance’ accounted for 54%, which is a rate of 0.03 per 100 patient attendances, when combined (n = 66, 28%; and n = 61, 26%). In 2010, there was an increased frequency of ‘other’ incidents in terms of point of error, compared to the other sixteen years (2 out of 5 incidents in 2010 compared to 0 or 1 incidents in all the other years, range [0%-11%]). Overall, ‘treatment’ was both the highest point of error and detection (n = 115, 49%; and n = 197, 85% respectively), as shown in Figure 1. There was also a peak in incidents that recorded ‘planning’ (n = 12) and ‘treatment’ (n = 14) as the point of error in 2016.
Causal factors
All incidents and near misses were classified according to nine identified causes outlined in Table 2. ‘Human Error’ and ‘Protocol or Procedural Failure’ causal factors made up 63% of all incidents, with ‘Inadequate Supervision’ being the cause of the least number of incidents (n = 2, 1%). The largest number of ‘Human Error’ incidents was recorded in 2018 (n = 14), closely followed by 2019 (n = 10), with no year within the seventeen-year time period reporting zero ‘Human Error’ causal factors. Furthermore, ‘Human Error’ was the only causal factor attributed to all errors (n = 3) reported in 2011.
Departmental changes
As seen in Table 3, there have been a large number of changes within the department. The largest number of departmental changes occurred in 2018 (n = 16) with 2002 recording the largest number of major changes (n = 10), as this was the year the department was established. There was an increase in the number of radiation therapist FTE of 542% (from 12 to 65), over the study period. The largest increase in staff numbers of 19% (n = 9; 47 to 56) was seen in 2012, coinciding with department expansion.
During the time period analysed, there was an increase in linac patient attendances of 140% (from 10,652 to 25,534). The largest increase in patient linac attendances was evident in 2008 and 2012 consecutively. In 2008, there was a large increase in patient attendances of almost 30%.
In 2015, there was a considerable increase in staff education surrounding incident reporting, to clarify what is classed as a near miss and to emphasise the importance of reporting near misses. The aim of this education session was to improve staff knowledge on the reporting process and to reiterate the purpose of incident reporting in the existing non-blame cultural environment.
Departmental changes versus category of error
The majority of internal changes in 2016 were major changes (83%), with four out of five (80%) of the major changes relating to the move towards an electronic treatment workflow. The gradual elimination of paper treatment records and implementation of electronic treatment processes commenced in November 2014 and was completed in December 2016. During the implementation process between 2014 and 2016, there was an increase in ‘Procedural’ errors of 200% (7 to 21). After the completion of the electronic treatment process implementation period in 2017, there was a considerable decrease in ‘Procedural’ errors of 33.3% (21 to 14). Over the course of the electronic treatment environment implementation period, an increase in the following causal factors was identified: inadequate supervision, insufficient training, poor communication and protocol failure. In 2018, there were a large number of electronic changes introduced and also an increase in all types of errors, excluding injury, as shown in Figure 2.
Further in-depth descriptive analysis was performed specifically on isocentre and localisation errors before, during and after the implementation of an electronic treatment workflow. This showed that prior to implementation, from 2003 to 2013, there was an average of 3.8 (+/− 1.8) isocentre and localisation errors reported per year. During implementation, between 2014 and 2016, there was an average of 7 (+/− 3) isocentre and localisation errors documented per year. Of the incidents and near misses in 2015 to 2016, the majority (18/22) were near misses related to documentation errors. Post-electronic workflow implementation in 2018, there was a decrease in recorded isocentre and localisation incidents (n = 4).
Discussion
Overall, the findings from this study have highlighted peak time points where incidents and near misses have increased, including 2016 and 2018. Upon review, it is clear that there are numerous factors, such as staff culture and workplace changes, that influence both the reporting and occurrence of incidents and none of these factors should be considered in isolation.
There are numerous detailed studies on error analysis, failure modes and risk factors in Radiation Oncology.2, 7 Systematic and random errors occurring in conjunction with clinical environmental factors and the implementation of new technology have been well documented.16 A 2010 article by Bissonnette and Medlam,7 analysed trends in radiation incidents from 2001 to 2007 with significant changes in technology. Bissonnette and Medlam7 found that increases in treatment complexity resulted in increases in documentation errors, which aligns with the findings of the current study. Smith et al.20 also reported on a fifteen-year review of incident data in 2020, reporting similar results to the current study with regards to increases in near misses and errors related to the implementation of new technology. Their study found an increase in near miss events associated with the introduction of image-guided RT (IGRT) technology.20 However, overall, there is a paucity of literature reporting a comparative analysis of radiation incidents in relation to the implementation of new technology and electronic processes plus changes within a large academic department over a greater than a ten-year period.
From the seventeen years of incidents analysed at this single institution, it was not possible to perform correlation analyses to identify direct relationships between departmental changes and a change in the number of incidents and near misses reported. This is a result of the many variables and confounding factors involved and a potential delay of varying time periods on incident rates.
The data suggests that for some changes there may be a potential six-month delay in impact on incident rates from changes within the department, such as in the case of staff education, but in other cases, such as an increase in staff numbers, there might be an instant effect on incident rates. It is also likely that incident rates are higher at the initial point of implementation and then decrease with time, as new processes then become the new standard of practice. However, these assumptions would need to be tested and validated on a larger dataset.
Incident rates compared to departmental changes
As the ROPAIR department has developed and changed, new incident patterns emerged. In 2011, ‘human error’ was the only causal factor attributed to all errors (n = 3) reported. Human error causal factors then peaked in 2019 (n = 14), whilst protocol failure errors peaked in 2016 (n = 14). The largest increase in staff numbers was seen in 2012, with the same year recording a 37.5% rise in incidents (3 in 2011 to 15 in 2012). This rise in incidents may have been a result of the surge (of 18.6%) in patient attendances in 2012. In 2015, there was a significant increase in staff education surrounding the reporting of incidents and near misses, which in turn may have contributed to the growth in incident reporting evident in 2016. This growth was not necessarily indicative of a decrease in patient safety, but rather more likely due to an improved safety culture that is focused on learning and prevention. Therefore, demonstrating that as with any voluntary (and subjective) reporting system, only reported incidents are reflected in the data and these may not necessarily demonstrate the actual rate of occurrences. An article by Findlay and Ottrey21 showed similar findings stating that incident data will only reflect what is reported and due to a voluntary reporting system this may not be representative of the actual level of occurrence. Daily CBCT for IGRT became the standard of practice in 2016 which may have also attributed to the increase in detection of near miss events that manifested in 2016 (n = 21) that would have potentially previously gone undetected. 2017 saw the introduction of RiskMan, which for ROPAIR proved to be a more streamline reporting process and therefore may have attributed to the rise in reporting in 2018.
Impact of an electronic environment
Various factors may have contributed to the rise in specific causal factors over the period of electronic treatment implementation. For example, 2014 saw the second-largest expansion in staff numbers and in 2015, there was a significant increase in staff education. It is, therefore, likely that this large rise in reporting is credited to the increase in education, rather than the change to an electronic treatment environment, reiterated by Findlay and Ottrey.21 The current study found that factors such as staff education and departmental culture may have impacted the voluntary reporting of errors. Findlay and Ottrey21 also emphasise this by stating that staff become more likely to report incidents as the organisations reporting culture matures. Arnold12 reinforces this by concluding that errors will reduce through building a focus on prevention and harnessing a learning culture. Smith et al.20 also reported similar results, showing an increase in reporting which coincided with the establishment of a department-wide incident review meeting and the promotion of a positive safety culture.
In both 2016 and 2018, there were increases in both electronic changes and errors on treatment. The largest number of minor errors and largest number of overall errors occurred in 2018 (n = 39), most likely attributed to the largest number of internal major and minor changes occurring in this year (n = 11). 2018 also saw the largest number of ‘Human Error’ incidents. This aligns with the Australian Radiation Protection and Nuclear Safety Agency’s Australian Radiation Incident Register Annual Report in 2018 that indicates human error was the primary cause identified for 65% of incidents.22 One of the major changes at ROPAR in 2018 was the implementation of electronic quality check list processes for planning and QA. Interestingly, the electronic changes in 2018 related mostly to planning, however, there was a decrease in incidents in planning in 2018 and an increase in planning incidents in 2019. This may be suggestive of a delayed effect. However, as the electronic process was a gradual implementation alongside other changes during this time period, it is difficult to ascertain which errors and near miss events were directly related to the electronic environment implementation.
Walker et al.10 indicated that some automated processes may reduce the risk of human errors, especially manually calculated isocentre positions. Klein et al.1 also found that with an increase in automation, there must be a parallel increase in QA checks. The article concluded that no computer system can compensate for a team member’s error in judgement, misunderstanding of physical concepts or technical limitations or unsatisfactory planning and delivery of radiotherapy.1 However, Yeung et al.4 state that rather than increasing the number of QA checks, which could cause an overload and inefficiencies, continued staff education can help decrease incidents. In support of this concept, it was noted that the commencement of using Couch Move Assistance in 2016, resulted in a decrease in reported treatment incidents from 14 in 2016, to 12 in 2017. The total number of incidents recorded also decreased from 2016 (n = 27) to 2017 (n = 21), appearing to have positively impacted incident rates. Further to this, the increase in isocentre and localisation incidents and near misses in 2017, followed by a decrease in 2018 is suggestive of effective implementation of the electronic procedures, as in 2018, the new practice has become the new standard practice.
Furthermore, factors not considered within the scope of this study, such as plan and treat patients, treatment site, time of day and technique may contribute to unidentified trends. This also highlights that incident and near miss analysis is a continual process of reviewing and assessing errors over time; as the types of errors observed will inevitably change with changes in practice and technology. It is challenging to make direct correlations between incident rates and departmental changes over time due to the number of confounding factors involved. However, it is beneficial to conduct a thorough review of the data in the context of changes that have occurred to gain valuable insight that may assist in preparation for future changes. As this is a single-institutional study, it would be beneficial to perform a multi-centre analysis. Predictive data would also be useful, but is difficult to ascertain, due to the large number of confounding factors impacting on incident rates.
Conclusion
These results have demonstrated that incidents and near miss events do not occur in isolation but rather are a result of numerous complex potential contributing factors, including, but not limited to the implementation of new electronic technology and practices along with other environmental changes within the department. This research has provided valuable information on areas within the department requiring focus; and placed an emphasis on the need for a more proactive approach to practice that focuses on preventative measures prior to and in conjunction with the implementation of major departmental changes. This will ultimately lead to improved patient safety and quality.
Acknowledgements
Management and staff of Princess Alexandra Hospital Radiation Oncology for their support of this project.
Conflicts of Interest
The authors declare no conflict of interest.
