Measuring and managing radiologist workload: A method for quantifying radiologist activities and calculating the full-time equivalents required to operate a service
Abstract
Introduction
Accurate and transparent measurement and monitoring of radiologist workload is highly desirable for management of daily workflow in a radiology department, and for informing decisions on department staffing needs. It offers the potential for benchmarking between departments and assessing future national workforce and training requirements. We describe a technique for quantifying, with minimum subjectivity, all the work carried out by radiologists in a tertiary department.
Methods
Six broad categories of clinical activities contributing to radiologist workload were identified: reporting, procedures, trainee supervision, clinical conferences and teaching, informal case discussions, and administration related to referral forms. Time required for reporting was measured using data from the radiology information system. Other activities were measured by observation and timing by observers, and based on these results and extensive consultation, the time requirements and frequency of each activity was agreed on. An activity list was created to record this information and to calculate the total clinical hours required to meet the demand for radiologist services.
Results
Diagnostic reporting accounted for approximately 35% of radiologist clinical time; procedures, 23%; trainee supervision, 15%; conferences and tutorials, 14%; informal case discussions, 10%; and referral-related administration, 3%. The derived data have been proven reliable for workload planning over the past 3 years.
Conclusions
A transparent and robust method of measuring radiologists' workload has been developed, with subjective assessments kept to a minimum. The technique has value for daily workload and longer term planning. It could be adapted for widespread use.
Introduction
It is the mark of an educated mind to rest satisfied with the degree of precision which the subject admits and not to seek exactness where only approximation is possible.
Aristotle, 384–322 BC
In 2007, delays in accessing and reporting diagnostic imaging were having a significant negative impact on the flow of inpatients and outpatients through our tertiary hospital services. Depending on clinical urgency, outpatients were waiting over a year to access ultrasound, CT or MRI. Once imaging was completed, there were then significant delays in reporting, with up to 1500 unreported cases waiting. Emergency and inpatients were typically waiting over 12 h for a report (compared with a target report turnaround time of 4 h), and outpatients over 24 h. A project looking at radiology capacity (jointly undertaken by our radiology service and our institution's Business Development Unit (BDU)) identified limited radiologist hours as the single largest contributor to delays across the service.
It was realised that a robust and transparent method of measuring overall workload and predicting work capacity of radiologists in different settings was required. Production engineers within the BDU believed that an activity list-based method could achieve this. The aim of this study was to identify and quantify activities undertaken by radiologists in a teaching hospital environment in order to create such an activity list. Once developed, this could enable managing daily throughput of the department and determining appropriate staffing levels, and could potentially be used for benchmarking different departments and form the basis of national workforce planning.
Method
Observation, measurement and consensus were used to create an activity list, which details all of the clinical activities undertaken by radiologists, and the associated time requirements.
The steps taken to construct the activity list were as follows.
Identify all rostered and non-rostered activities
- Reporting sessions (we do not have protected reporting areas, so radiologists who are rostered to reporting will also perform many of the non-rostered clinical activities)
- Procedure sessions including interventional radiology, biopsy, fluoroscopy and mammography case workup
- Multidisciplinary meetings (MDM) and clinical case review conferences
- Registrar teaching sessions
- Supervision of activities associated with imaging, including supporting other radiology staff (administrative, nursing, medical radiation technologists and sonographers), patient still on table review of imaging and limited scans to check sonographers' findings
- Biopsies or interventional procedures occurring outside rostered biopsy or interventional sessions
- Procedures or examinations, which require radiologist time in addition to the standard image review report dictation process. These have been grouped under the heading of advanced diagnostic studies in the activity list. Examples include radiologists performing intra-operative ultrasound or ultrasound of fetal hearts, and workstation review of imaging, for example, CT colonograms or CT coronary angiograms
- Referral review (for justification and appropriateness), triage and protocolling, and assisting scheduling of imaging
- Clinical communication related to referrals, scheduling and results
- Ad hoc interruptions by clinical teams
- Review of outside imaging
- Consultation/interactions with patients and families
- Supervision and training of radiology registrars and fellows, including review of reports
- Supervision, teaching, and training of other radiology and non-radiology staff and medical students.
Assemble the activity list
- Reporting, subdivided into modalities
- Procedures
- Advanced diagnostic (as defined above)
- Referral-related administration time (includes time for review, protocol and triage assignment, and assisting scheduling)
- Informal case discussions (encompassing ad hoc interruptions by clinical teams)
- Conference (including clinical case review and MDM)
- Registrar and fellow supervision, and tutorials.
Each different activity was given a separate line in an activity list created using Microsoft Excel (Microsoft Corporation, Redmond, WA, USA) (Table 1).
| Activity | Mean volume | Per | Number per month | Time per activity, minutes | Mean hours per day | Mean hours per month | % of total hours |
|---|---|---|---|---|---|---|---|
| Reporting X-ray | 249 | Day | 5410 | 3.42 | 14.2 | 303 | 11.9 |
| Reporting CT | 54 | Day | 1173 | 15.08 | 13.6 | 295 | 11.4 |
| Reporting US | 80 | Day | 1738 | 4.90 | 6.5 | 142 | 5.5 |
| Reporting MRI | 17 | Day | 369 | 18.33 | 5.2 | 113 | 4.4 |
| Reporting mammography | 30 | Week | 130 | 8.90 | 1.7 | 19 | 0.7 |
| Reporting fluoroscopy | 40 | Week | 174 | 5.00 | 0.7 | 14 | 0.6 |
| Reporting interventional | 10 | Day | 217 | 2.33 | 0.4 | 8 | 0.3 |
| Procedure diagnostic† | 39 | Day | 847 | 15.00 | 9.8 | 212 | 8.2 |
| Advanced diagnostic studies‡ | 10.4 | Day | 226 | 50.00 | 8.7 | 188 | 7.3 |
| Procedure interventional | 5.1 | Day | 110 | 65.00 | 5.5 | 119 | 4.6 |
| Procedure fluoroscopy | 27 | Week | 117 | 30.00 | 2.7 | 59 | 2.3 |
| Procedure mammography | 26 | Week | 113 | 15.00 | 1.8 | 28 | 1.1 |
| Referral-related admin | 191 | Day | 4150 | 1.00 | 3.2 | 69 | 2.7 |
| Informal case discussions | 120 | Week | 521 | 30.00 | 12.0 | 261 | 10.1 |
| Conference preparation | 49 | Week | 213 | 35.00 | 5.1 | 124 | 4.8 |
| Conference run-time | 49 | Week | 213 | 50.00 | 8.1 | 177 | 6.8 |
| Registrar, fellow supervision | 12 | Day | 261 | 90.00 | 18 | 391 | 15.1 |
| Tutorial preparation | 9 | Week | 39 | 35.00 | 1.0 | 23 | 0.9 |
| Tutorial run-time | 9 | Week | 39 | 60.00 | 1.8 | 39 | 1.5 |
- †Procedure diagnostic group: time required to review imaging in progress, for example, checking of plain radiographs on request by a medical radiation technologist, checks of ultrasound, CT and MRI while the patient is still on the table. A radiologist interruption occurs in 25% of all ultrasound scans, 10% of all CT and MRI scans, and 5% of all plain radiographs. Fifteen minutes per interruption (from start to return to previous work state at time of interruption) were allowed based on observation. ‡Advanced diagnostic studies: any procedure or examination which requires active radiologist participation separate from the reporting, but is not interventional, biopsy, problem mammography or fluoroscopy. Examples include ultrasound performed by a radiologist such as fetal heart scanning or intraoperative scanning, or examinations involving a radiologist in workstation time such as CT of coronary arteries or CT colonography. US, ultrasound.
Assign times and frequency of occurrence to clinical activities
Time requirements had to be assigned to each activity, and where appropriate, the frequency with which the activities occurred had to be determined for inclusion in the activity list.
Mean reporting times per examination by modality were calculated from data extracted from the radiology information system.1
BDU staff observed, recorded and timed the non-reporting activities of radiologists over a 2-week period. Radiologists were aware of being observed. The total number of cases contributing to the observational data, including the proportion of the total caseload contributing to the observational data over the 2-week period, is unknown. To build acceptance of the model, observational data were presented to the radiologist group to ensure there was consensus that observed volumes and associated times were reasonable.
Specific times and frequency of occurrence were quantified as follows.
Reporting times
All reporting time data were extracted from the radiology information system (RIS) (Comrad, Software Innovations, Christchurch, New Zealand) used in the Radiology Departments at Christchurch Hospital (approximately 650 beds) and the adjacent Christchurch Women's Hospital (approximately 130 beds). These sites are tertiary teaching hospitals, and accredited training sites for the Royal Australian and New Zealand College of Radiology, servicing a local population base of approximately 500 000, as well as taking referrals from a much wider region for specialist services. The Radiology Service has funding for 23 full-time equivalent (FTE) radiologists and 12 registrars, and produces approximately 140 000 reports per year.
Mean report times for each modality were calculated using the technique described by Cowan1 (Appendix I). This included calculation of mean times for the reports associated with procedural type work including that undertaken in mammography (which is specifically problem mammography at our institution, i.e., the assessment and workup of patients with known lesions), interventional radiology and fluoroscopy.
The time taken for radiologists to perform the associated procedures was measured separately and is included under the corresponding group in the procedure section of the list.
Report volumes were entered in the activity list for each modality based on historical emergency, inpatient and outpatient referral volumes. Double reading of mammography was allowed for. The report volumes for procedural work are higher than the procedure volumes in the section below as a number of procedures (such as peripherally inserted central catheter line insertion and endoscopic retrograde cholangiopancreatography) are performed by non-radiologists, but still require a radiologists report.
Procedural activities
Observation of the duration of procedures over a 2-week period by BDU staff was used to assign mean times to interventional, fluoroscopy and mammography procedures. Although there was large variation in interventional radiology procedure times (the sample size was too small to use a procedure-specific approach), when the mean time calculated was applied to the number of procedures done over a longer period of time in order to estimate the total time taken to complete them, the result was sufficiently similar to the actual time taken (i.e. it approximated the number of interventional radiology sessions worked) to be useful for the purpose of creating the activity list.
The procedure diagnostic group allows for the time required to review imaging in progress, for example, checking of plain radiographs on request by a medical radiation technologist, and checks of ultrasound, CT and MRI while the patient is still on the table.
Based on BDU observation, a time allocation per modality for checking was linked to examination volumes as follows: a radiologist interruption occurred/is allowed for in 25% (i.e., one in four) of all ultrasound scans, 10% of all CT and MRI scans, and 5% of all plain radiographs. Fifteen minutes per interruption (from start to return to previous work state at time of interruption) were allowed based on observation.
The activity list has been set up so that adjusting the report volume for any of ultrasound, CT or MRI results in the data in the procedure diagnostic line being automatically updated to reflect the correct average number of times reviewing imaging in progress is required.
Advanced diagnostic studies
Where significant radiologist time was required to complete a diagnostic examination separate from reporting it, and that time was not captured in report times or procedure times, it was included under advanced diagnostic studies.
Biopsies and other minor procedures performed outside of our interventional radiology suite were included in this group, along with examinations where the radiologist is the primary imager, for example, fetal heart and intra-operative ultrasound, and examinations where radiologists spending time at a workstation was required such as CT colonography and CT coronary angiography.
Based on historical examination volumes, the number of this type of study was set at the equivalent of 10.4 per day, with a mean of 50 min allowed (relatively generous to allow for the extremes that can occur in this heterogeneous group).
Referral-related administration time
Following observation of radiologists reviewing referrals for justification, appropriateness, protocol and triage assignment, and assisting scheduling, radiologist time for these activities was estimated at the equivalent of a mean of 1 min per case and applied to CT, MRI, ultrasound, interventional and mammography cases. This line in the activity list is automatically populated based on the report volume lines for the included modalities.
Informal case discussions
This allows time for clinicians discussing cases on an ad hoc basis with radiologists. The time was estimated at 30 min per session based on observation.
Scheduled conferences including preparation time
This includes multidisciplinary team meetings and clinical case review conferences. While the caseload for meetings can vary, our experience is that many meetings have a relatively stable caseload, preparation and run-time, and are of relatively consistent duration due to follow-on meetings. Typical preparation time and duration were derived from a combination of observation and radiologist consensus on reasonable mean times where a group of radiologists shared leading a particular meeting. Times were recorded in a separate Microsoft Excel conference spreadsheet and then overall preparation and duration times were calculated
Registrar training
A time allocation of 90 min per registrar per day is included based on 2-week observational survey completed by registrars recording how much radiologist time was spent on training per session. This time is in addition to the time it would have taken a radiologist to complete a similar volume of the same activity (such as reporting or procedures). The consensus of our radiologist group was that although this is one of the most difficult aspects of our workload to quantify, the observed time seemed reasonable.
Tutorial time including preparation time
The timing and duration of registrar and medical student teaching or tutorial sessions are fixed, and these sessions have traditionally occurred during clinical time in our institution. Mean preparation time was derived by consensus. These times were also recorded in the separate Excel conference spreadsheet.
Calculating the total clinical hours required to operate the service
A unit of measure was defined for each activity (e.g. a report is the unit for reporting), along with a time per unit (e.g. the mean time for a report).
Volumes were calculated on either a daily (d) or weekly (w) basis depending on the activity. For activities that did not occur daily, for example, fluoroscopy and problem mammography, average weekly volumes were used. From this information, average monthly hours were calculated, along with the percentage of total time for each activity.
The total number of clinical hours required per month and per annum was calculated from totalling the average monthly hours per activity. Time for breaks, absenteeism (2%) and annual leave was added. We also added rostered non-clinical time, which is part of the national award for District Health Board employees and set at 30% or three sessions per 10-session week for reading, research, administration, etc.
Adding all these data, we calculated our department's FTE staffing requirement.
Clinical implementation
Once the activity list was assembled and calculations were agreed, data derived from it (in particular, a realistic estimate of reporting time available each day) were used in combination with other production planning tools to plan a reasonable achievable diagnostic reporting workload for the radiologists.
Results
A simplified version of the activity list constructed is presented in Table 1 and summarised in Table 2.
| Activity group | Hours | % of total clinical hours |
|---|---|---|
| Reporting | 900 | 35 |
| Procedure and advanced diagnostic | 606 | 23 |
| Registrar and fellow supervision | 391 | 15 |
| Conferences and tutorials | 364 | 14 |
| Informal case discussions | 261 | 10 |
| Referral-related administration | 69 | 3 |
| Total clinical hours per month | 2591 | 100 |
The activity list calculations yielded a requirement for 31 091 clinical hours (approximately 149 × 4-h clinical sessions per week) to run our service at the level required to keep up with all emergency, inpatient and outpatient diagnostic imaging demand and the other clinical activities related to operating a tertiary imaging and teaching department. Adding in allowances for leave, breaks and absenteeism (e.g. sick leave) yielded a total full-time equivalent requirement of 24.3 (Table 3). This is based on the current New Zealand industrial award definition of ‘full-time’ including three non-clinical sessions per week.
| Hours | |
|---|---|
| Clinical hours from activity list | 31 091 |
| Allowance for absenteeism (2%) | 622 |
| Annual leave (12%) | 3 731 |
| Breaks (6%) | 1 865 |
| Non-clinical time (30%) | 13 325 |
| Total hours per year | 50 634 |
| Total FTE at 40 hours per week | 24.3 |
- Clinical hours from the activity list are combined with leave and other allowances and entitlements to calculate total hours and full-time equivalents (FTEs).
The time required for reporting accounted for 35% of the total required clinical hours, with the largest proportions being required to report CT and plain films. Procedural/interventional work comprised the next biggest category at 23% of total clinical hours (Table 2).
Workload planning based on numbers derived from the activity list (the percentage of rostered time that will be realised as reporting time, and the mean report times) was implemented and has been in place for approximately 4 years. The radiologists as a group have consistently performed as the model predicted. Controlling reporting workload by reducing outpatient bookings or by outsourcing reporting when we are short of radiologist hours has resulted in a significant and sustained reduction in the number of unreported cases at the end of each working day.
Discussion
It is not possible to create a completely accurate and universally applicable method of measuring radiologist workload and productivity. Variation in radiologist group composition, in individual skills and speed, in casemix and in non-reporting clinical activities, is inevitable.
The challenge in measuring something as heterogeneous as radiologists' workload is to reduce subjectivity to a low level and attain a degree of precision with which we can ‘rest satisfied’. Our goal in presenting these data is to demonstrate a method for estimating overall workload that has a significantly reduced level of subjectivity.
Historically, we estimated radiologist requirements on the basis of best guess and approximation. We now know that this approach significantly underestimated the radiologist resource required to operate our department.
Sessions were the usual unit of measure for most of our activities including procedural activities such as biopsy and interventional work. This is a relatively blunt method that does not reflect the cumulative increase in radiologist time required due to gradually increasing volumes, especially in larger, more complex departments where procedural volumes arise from several different sources including interventional, biopsy, fluoroscopy and problem mammography. A significant advantage of discarding sessions as units is that all the activities that make up overall workload are now measured using the same units of time.
The potential benefits of this more comprehensive objective method of quantifying workload are many.
It builds understanding among non-radiologist staff of the complexity of the role of a clinical radiologist, in particular the many activities radiologists undertake that contribute significantly to their workload in addition to reporting. Prior to the construction of the activity list, the majority of non-radiologist staff in our department assumed that radiologists not rostered to perform procedures or take a conference would be fully available for reporting.
It is a useful tool for informing service planning and can be used for planning reasonable achievable daily and weekly workloads. Information from it can be used to set key performance indicators for monitoring performance, although it is important to note that the activity list and any derived performance indicators reflect and are applicable to the performance of radiologists as a group, rather than to individuals. Different radiologists will have different skills and work at different speeds, and this is difficult to change.2
At a more strategic level, if a shortfall of radiologists exists or is anticipated, the needed radiologist hours can be reliably quantified to inform business continuity decisions. The impact on waiting lists can be predicted if the required clinical hours cannot be filled. If spare radiologist capacity ever became available, the activity list could be used to quantify how much additional work the service could reasonably absorb.
The activity list can be, and has been, adapted for use in other sites, but the activities included and the times associated with them are specific to our department. For example, our department likely differs from many others in our relatively high supervision of registrars to a senior level; the spreadsheet does not allow for radiologists remotely supervising registrars performing procedures because this does not happen largely in our hospital. Any department adopting the activity list for local use should validate both the activities included and the associated time requirements to ensure they build a reliable workload model for their site. If using it to calculate required FTE, then allowances for breaks, sick leave, and other causes of absenteeism and annual leave need to be included, along with any other entitlements routinely taken such as study leave, secondment and sabbatical.
If adopted widely, the activity list as a way of modelling radiologist workload could form the basis of a common framework for benchmarking services and establishing workforce requirements, for example, at regional or national level.
To ensure the list remains valid as a model of radiologist workload over time, it must be updated as demands on our service change, and the measurements and assumptions on which it is based should be revalidated from time to time. We aim to do this annually.
There are limitations to the activity list model. Consensus is subjective, and there is inevitable artefact from the observation process; we do not know of any way to eliminate this. The possibility of ‘gaming’ the observation process in order to skew results and perhaps strengthen the case for recruiting more radiologists exists, but this would require a concerted effort by a large number of the radiologists to be significant. There is still a significant subjective component although we believe it is the minimum that could reasonably be achieved given the complexities of modelling workload in the real world of clinical radiology.
Registrar and fellow supervision and training negatively impact on radiologist productivity,3 however, accurately quantifying to what degree, given all the variables that need to be considered is challenging.
Some low-volume (relative to routine work) and difficult-to-quantify work is not specifically allowed for, for example, review of outside imaging or amending previously dictated reports. Other important activities, such as interacting with patients or relatives (outside of procedural sessions) and ensuring communication of significantly abnormal results, are also not specifically allowed for.
To allow for these and other similarly infrequent or unpredictable components of clinical work, we are dependent on generous time allowances (equating to buffer time) in a few lines of the activity list model. For example, allowing 1 min for the reviewing imaging referrals seems excessive when many only take seconds, and advanced diagnostic time allowance is generous for some of the examinations included. Any ‘spare’ time not consumed by these activities gives the model the flexibility it needs to allow for cases that take significantly longer, and for covering other activities and exceptions not specifically accommodated in the activity list.
In conclusion, we have described a technique with reduced subjective component for understanding the workload of our radiologists. It has potential applications for workload planning and for workforce planning at local, regional and national levels across the health system.
Appendix I
Summary of methodology used to derive mean report times per modality
In our system, we do not allow multiple different modality examinations on a single RIS entry, that is, each modality would receive a unique examination identification number and be reported separately. We do allow more than one body area to be imaged per modality examination, for example, CT chest and abdomen would result in a single report. A large sample of reporting times, recorded automatically by the RIS along with the description of each examination being reported, was placed in a database. Analysis was confined to diagnostic reporting by consultant radiologists. A spreadsheet was produced, listing the total number and the frequency of reporting times of each distinct examination. Outliers with exceptionally long report times (more than 10 min for plain radiography, 30 min for ultrasound or 60 min for CT or MRI with some exceptions) were culled; this removed 9.5% of the total. The mean report times were calculated by modality.
