Radiation Oncology—Review Article
A review of interventions to reduce inter-observer variability in volume delineation in radiation oncology
Shalini K Vinod,
Myo Min,
Michael G Jameson,
Lois C Holloway,
Corresponding Author
Shalini K Vinod
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
South Western Sydney Clinical School, University of NSW, Sydney, New South Wales, Australia
Western Sydney University, Sydney, New South Wales, Australia
Correspondence
A/Prof Shalini Vinod, Cancer Therapy Centre, Liverpool Hospital, Locked Bag 7103, Liverpool BC, NSW 1871, Australia.
Email: [email protected]
Search for more papers by this authorMyo Min
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
South Western Sydney Clinical School, University of NSW, Sydney, New South Wales, Australia
Search for more papers by this authorMichael G Jameson
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
Ingham Institute of Applied Medical Research, Liverpool Hospital, Liverpool, New South Wales, Australia
Centre for Medical Radiation Physics, University of Wollongong, Wollongong, New South Wales, Australia
Search for more papers by this authorLois C Holloway
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
South Western Sydney Clinical School, University of NSW, Sydney, New South Wales, Australia
Western Sydney University, Sydney, New South Wales, Australia
Ingham Institute of Applied Medical Research, Liverpool Hospital, Liverpool, New South Wales, Australia
Search for more papers by this authorShalini K Vinod,
Myo Min,
Michael G Jameson,
Lois C Holloway,
Corresponding Author
Shalini K Vinod
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
South Western Sydney Clinical School, University of NSW, Sydney, New South Wales, Australia
Western Sydney University, Sydney, New South Wales, Australia
Correspondence
A/Prof Shalini Vinod, Cancer Therapy Centre, Liverpool Hospital, Locked Bag 7103, Liverpool BC, NSW 1871, Australia.
Email: [email protected]
Search for more papers by this authorMyo Min
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
South Western Sydney Clinical School, University of NSW, Sydney, New South Wales, Australia
Search for more papers by this authorMichael G Jameson
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
Ingham Institute of Applied Medical Research, Liverpool Hospital, Liverpool, New South Wales, Australia
Centre for Medical Radiation Physics, University of Wollongong, Wollongong, New South Wales, Australia
Search for more papers by this authorLois C Holloway
Cancer Therapy Centre, Liverpool Hospital, Liverpool, New South Wales, Australia
South Western Sydney Clinical School, University of NSW, Sydney, New South Wales, Australia
Western Sydney University, Sydney, New South Wales, Australia
Ingham Institute of Applied Medical Research, Liverpool Hospital, Liverpool, New South Wales, Australia
Search for more papers by this authorSK Vinod MBBS, MD, FRANZCR; M Min MBBS, FRANZCR; MG Jameson B Med Rad Phys; LC Holloway PhD.
Conflict of interest: None.
Abstract
Introduction
Inter-observer variability (IOV) in target volume and organ-at-risk (OAR) delineation is a source of potential error in radiation therapy treatment. The aims of this study were to identify interventions shown to reduce IOV in volume delineation.
Methods
Medline and Pubmed databases were queried for relevant articles using various keywords to identify articles which evaluated IOV in target or OAR delineation for multiple (>2) observers. The search was limited to English language articles and to those published from 1 January 2000 to 31 December 2014. Reference lists of identified articles were scrutinised to identify relevant studies. Studies were included if they reported IOV in contouring before and after an intervention including the use of additional or alternative imaging.
Results
Fifty-six studies were identified. These were grouped into evaluation of guidelines (n = 9), teaching (n = 9), provision of an autocontour (n = 7) and the impact of imaging (n = 31) on IOV. Guidelines significantly reduced IOV in 7/9 studies. Teaching interventions reduced IOV in 8/9 studies, statistically significant in 4. The provision of an autocontour improved consistency of contouring in 6/7 studies, statistically significant in 5. The effect of additional imaging on IOV was variable. Pre-operative CT was useful in reducing IOV in contouring breast and liver cancers, PET scans in lung cancer, rectal cancer and lymphoma and MRI scans in OARs in head and neck cancers.
Conclusion
Inter-observer variability in volume delineation can be reduced with the use of guidelines, provision of autocontours and teaching. The use of multimodality imaging is useful in certain tumour sites.
References
- 1Morarji K, Fowler A, Vinod SK, Ho Shon I, Laurence JM. Impact of FDG-PET on lung cancer delineation for radiotherapy. J Med Imaging Radiat Oncol 2012; 56: 195–203.
- 2Pitkanen MA, Holli KA, Ojala AT, Laippala P. Quality assurance in radiotherapy of breast cancer-variability in planning target volume delineation. Acta Oncol 2001; 40: 50–5.
- 3Jansen EP, Nijkamp J, Gubanski M, Lind PA, Verheij M. Interobserver variation of clinical target volume delineation in gastric cancer. Int J Radiat Oncol Biol Phys 2010; 77: 1166–70.
- 4Tai P, Van DJ, Battista J et al. Improving the consistency in cervical esophageal target volume definition by special training. Int J Radiat Oncol Biol Phys 2002; 53: 766–74.
- 5Seddon B, Bidmead M, Wilson J, Khoo V, Dearnaley D. Target volume definition in conformal radiotherapy for prostate cancer: quality assurance in the MRC RT-01 trial. Radiother Oncol 2000; 56: 73–83.
- 6Jameson MG, Kumar S, Vinod SK, Metcalfe PE, Holloway LC. Correlation of contouring variation with modeled outcome for conformal non-small cell lung cancer radiotherapy. Radiother Oncol 2014; 112: 332–6.
- 7Weber DC, Tomsej M, Melidis C, Hurkmans CW. QA makes a clinical trial stronger: evidence-based medicine in radiation therapy. Radiother Oncol 2012; 105: 4–8.
- 8Peters LJ, O'Sullivan B, Giralt J et al. Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02. J Clin Oncol 2010; 28: 2996–3001.
- 9Ohri N, Shen X, Dicker AP, Doyle LA, Harrison AS, Showalter TN. Radiotherapy protocol deviations and clinical outcomes: a meta-analysis of cooperative group clinical trials. J Natl Cancer Inst 2013; 105: 387–93.
- 10Abrams RA, Winter KA, Regine WF et al. Failure to adhere to protocol specified radiation therapy guidelines was associated with decreased survival in RTOG 9704–a phase III trial of adjuvant chemotherapy and chemoradiotherapy for patients with resected adenocarcinoma of the pancreas. Int J Radiat Oncol Biol Phys 2012; 82: 809–16.
- 11Daisne JF, Duprez T, Weynand B et al. Tumor volume in pharyngolaryngeal squamous cell carcinoma: comparison at CT, MR imaging, and FDG PET and validation with surgical specimen. Radiology 2004; 233: 93–100.
- 12Wanet M, Lee JA, Weynand B et al. Gradient-based delineation of the primary GTV on FDG-PET in non-small cell lung cancer: a comparison with threshold-based approaches, CT and surgical specimens. Radiother Oncol 2011; 98: 117–25.
- 13 Faculty of Radiation Oncology. Quality Guidelines for Volume Delineation in Radiation Oncology. Ranzcr, Sydney, Australia, 2015. Available at: http://www.ranzcr.edu.au/resources/professional-documents/guidelines. Accessed 15th December 2015.
- 14Hanna GG, Hounsell AR, O'Sullivan JM. Geometrical analysis of radiotherapy target volume delineation: a systematic review of reported comparison methods. Clin Oncol (R Coll Radiol) 2010; 22: 515–25.
- 15Fotina I, Lutgendorf-Caucig C, Stock M, Potter R, Georg D. Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. Strahlenther Onkol 2012; 188: 160–7.
- 16Jameson MG, Holloway LC, Vial PJ, Vinod SK, Metcalfe PE. A review of methods of analysis in contouring studies for radiation oncology. J Med Imaging Radiat Oncol 2010; 54: 401–10.
- 17Bowden P, Fisher R, Mac MM et al. Measurement of lung tumor volumes using three-dimensional computer planning software. Int J Radiat Oncol Biol Phys 2002; 53: 566–73.
- 18Wong EK, Truong PT, Kader HA et al. Consistency in seroma contouring for partial breast radiotherapy: impact of guidelines. Int J Radiat Oncol Biol Phys 2006; 66: 372–6.
- 19Chao KS, Bhide S, Chen H et al. Reduce in variation and improve efficiency of target volume delineation by a computer-assisted system using a deformable image registration approach. Int J Radiat Oncol Biol Phys 2007; 68: 1512–21.
- 20van Baardwijk A, Bosmans G, Boersma L et al. PET-CT-based auto-contouring in non-small-cell lung cancer correlates with pathology and reduces interobserver variability in the delineation of the primary tumor and involved nodal volumes. Int J Radiat Oncol Biol Phys 2007; 68: 771–8.
- 21Bekelman JE, Wolden S, Lee N. Head-and-neck target delineation among radiation oncology residents after a teaching intervention: a prospective, blinded pilot study. Int J Radiat Oncol Biol Phys 2009; 73: 416–23.
- 22Mitchell DM, Perry L, Smith S et al. Assessing the effect of a contouring protocol on postprostatectomy radiotherapy clinical target volumes and interphysician variation. Int J Radiat Oncol Biol Phys 2009; 75: 990–3.
- 23Reed VK, Woodward WA, Zhang L et al. Automatic segmentation of whole breast using atlas approach and deformable image registration. Int J Radiat Oncol Biol Phys 2009; 73: 1493–500.
- 24Spoelstra FO, Senan S, Le Pechoux C et al. Variations in target volume definition for postoperative radiotherapy in stage III non-small-cell lung cancer: analysis of an international contouring study. Int J Radiat Oncol Biol Phys 2010; 76: 1106–13.
- 25Szumacher E, Harnett N, Warner S et al. Effectiveness of educational intervention on the congruence of prostate and rectal contouring as compared with a gold standard in three-dimensional radiotherapy for prostate. Int J Radiat Oncol Biol Phys 2010; 76: 379–85.
- 26Dewas S, Bibault JE, Blanchard P et al. Delineation in thoracic oncology: a prospective study of the effect of training on contour variability and dosimetric consequences. Radiat Oncol 2011; 6: 118.
- 27Feng M, Moran JM, Koelling T et al. Development and validation of a heart atlas to study cardiac exposure to radiation following treatment for breast cancer. Int J Radiat Oncol Biol Phys 2011; 79: 10–8.
- 28Fuller CD, Nijkamp J, Duppen JC et al. Prospective randomized double-blind pilot study of site-specific consensus atlas implementation for rectal cancer target volume delineation in the cooperative group setting. Int J Radiat Oncol Biol Phys 2011; 79: 481–9.
- 29Schick K, Sisson T, Frantzis J, Khoo E, Middleton M. An assessment of OAR delineation by the radiation therapist. Radiography 2011; 17: 183–7.
10.1016/j.radi.2011.01.003 Google Scholar
- 30Young AV, Wortham A, Wernick I, Evans A, Ennis RD. Atlas-based segmentation improves consistency and decreases time required for contouring postoperative endometrial cancer nodal volumes. Int J Radiat Oncol Biol Phys 2011; 79: 943–7.
- 31Breunig J, Hernandez S, Lin J et al. A system for continual quality improvement of normal tissue delineation for radiation therapy treatment planning. Int J Radiat Oncol Biol Phys 2012; 83: e703–8.
- 32Khoo EL, Schick K, Plank AW et al. Prostate contouring variation: can it be fixed? Int J Radiat Oncol Biol Phys 2012; 82: 1923–9.
- 33Mukesh M, Benson R, Jena R et al. Interobserver variation in clinical target volume and organs at risk segmentation in post-parotidectomy radiotherapy: can segmentation protocols help? Br J Radiol 2012; 85: e530–6.
- 34Nijkamp J, de Haas-Kock DF, Beukema JC et al. Target volume delineation variation in radiotherapy for early stage rectal cancer in the Netherlands. Radiother Oncol 2012; 102: 14–21.
- 35Awan M, Kalpathy-Cramer J, Gunn GB et al. Prospective assessment of an atlas-based intervention combined with real-time software feedback in contouring lymph node levels and organs-at-risk in the head and neck: quantitative assessment of conformance to expert delineation. Pract Radiat Oncol 2013; 3: 186–93.
- 36Lobefalo F, Bignardi M, Reggiori G et al. Dosimetric impact of inter-observer variability for 3D conformal radiotherapy and volumetric modulated arc therapy: the rectal tumor target definition case. Radiat Oncol 2013; 8: 176.
- 37Lorenzen EL, Taylor CW, Maraldo M et al. Inter-observer variation in delineation of the heart and left anterior descending coronary artery in radiotherapy for breast cancer: a multi-centre study from Denmark and the UK. Radiother Oncol 2013; 108: 254–8.
- 38Walker GV, Awan M, Tao R et al. Prospective randomized double-blind study of atlas-based organ-at-risk autosegmentation-assisted radiation planning in head and neck cancer. Radiother Oncol 2014; 112: 321–5.
- 39Yang J, Woodward WA, Reed VK et al. Statistical modeling approach to quantitative analysis of interobserver variability in breast contouring. Int J Radiat Oncol Biol Phys 2014; 89: 214–21.
- 40Stapleford LJ, Lawson JD, Perkins C et al. Evaluation of automatic atlas-based lymph node segmentation for head-and-neck cancer. Int J Radiat Oncol Biol Phys 2010; 77: 959–66.
- 41Boersma LJ, Janssen T, Elkhuizen PH et al. Reducing interobserver variation of boost-CTV delineation in breast conserving radiation therapy using a pre-operative CT and delineation guidelines. Radiother Oncol 2012; 103: 178–82.
- 42van der Leij F, Elkhuizen PH, Janssen TM et al. Target volume delineation in external beam partial breast irradiation: less inter-observer variation with preoperative- compared to postoperative delineation. Radiother Oncol 2014; 110: 467–70.
- 43Giezen M, Kouwenhoven E, Scholten AN et al. Magnetic resonance imaging- versus computed tomography-based target volume delineation of the glandular breast tissue (clinical target volume breast) in breast-conserving therapy: an exploratory study. Int J Radiat Oncol Biol Phys 2011; 81: 804–11.
- 44Jolicoeur M, Racine ML, Trop I et al. Localization of the surgical bed using supine magnetic resonance and computed tomography scan fusion for planification of breast interstitial brachytherapy. Radiother Oncol 2011; 100: 480–4.
- 45den Hartogh MD, Philippens ME, van Dam IE et al. MRI and CT imaging for preoperative target volume delineation in breast-conserving therapy. Radiat Oncol 2014; 9: 63.
- 46Geets X, Daisne JF, Arcangeli S et al. Inter-observer variability in the delineation of pharyngo-laryngeal tumor, parotid glands and cervical spinal cord: comparison between CT-scan and MRI. Radiother Oncol 2005; 77: 25–31.
- 47Viswanathan AN, Erickson B, Gaffney DK et al. Comparison and consensus guidelines for delineation of clinical target volume for CT- and MR-based brachytherapy in locally advanced cervical cancer. Int J Radiat Oncol Biol Phys 2014; 90: 320–8.
- 48Liu C, Gong G, Zhou T, Wang Y, Yin Y, Li B. The error estimate for contouring the brainstem in radiotherapy of head and neck cancer: a multi-center study from north China. J BUON 2014; 19: 484–9.
- 49Liu C, Kong X, Gong G, Liu T, Li B, Yin Y. Error in the parotid contour delineated using computed tomography images rather than magnetic resonance images during radiotherapy planning for nasopharyngeal carcinoma. Jpn J Radiol 2014; 32: 211–6.
- 50Foroudi F, Haworth A, Pangehel A et al. Inter-observer variability of clinical target volume delineation for bladder cancer using CT and cone beam CT. J Med Imaging Radiat Oncol 2009; 53: 100–6.
- 51Altorjai G, Fotina I, Lutgendorf-Caucig C et al. Cone-beam CT-based delineation of stereotactic lung targets: the influence of image modality and target size on interobserver variability. Int J Radiat Oncol Biol Phys 2012; 82: e265–72.
- 52Choi HJ, Kim YS, Lee SH et al. Inter- and intra-observer variability in contouring of the prostate gland on planning computed tomography and cone beam computed tomography. Acta Oncol 2011; 50: 539–46.
- 53Nyholm T, Jonsson J, Soderstrom K et al. Variability in prostate and seminal vesicle delineations defined on magnetic resonance images, a multi-observer, – centre, – sequence study. Radiat Oncol 2013; 8: 126.
- 54Rischke HC, Nestle U, Fechter T et al. 3 Tesla multiparametric MRI for GTV-definition of Dominant Intraprostatic Lesions in patients with Prostate Cancer–an interobserver variability study. Radiat Oncol 2013; 8: 183.
- 55Jensen NKG, Mulder D, Lock M et al. Dynamic contrast enhanced CT aiding gross tumor volume delineation of liver tumors: an interobserver variability study. Radiother Oncol 2014; 111: 153–7.
- 56Weltens C, Menten J, Feron M et al. Interobserver variations in gross tumor volume delineation of brain tumors on computed tomography and impact of magnetic resonance imaging. Radiother Oncol 2001; 60: 49–59.
- 57Mast M, Coerkamp E, Heijenbrok M et al. Target volume delineation in breast conserving radiotherapy: are co-registered CT and MR images of added value? Radiat Oncol 2014; 9: 65.
- 58Villeirs GM, Van VK, Vakaet L et al. Interobserver delineation variation using CT versus combined CT + MRI in intensity-modulated radiotherapy for prostate cancer. Strahlenther Onkol 2005; 181: 424–30.
- 59Guo L, Shen S, Harris E et al. A tri-modality image fusion method for target delineation of brain tumors in radiotherapy. PLoS ONE 2014; 9: e112187.
- 60Riegel AC, Berson AM, Destian S et al. Variability of gross tumor volume delineation in head-and-neck cancer using CT and PET/CT fusion. Int J Radiat Oncol Biol Phys 2006; 65: 726–32.
- 61Breen SL, Publicover J, De Silva S et al. Intraobserver and interobserver variability in GTV delineation on FDG-PET-CT images of head and neck cancers. Int J Radiat Oncol Biol Phys 2007; 68: 763–70.
- 62Caldwell CB, Mah K, Ung YC et al. Observer variation in contouring gross tumor volume in patients with poorly defined non-small-cell lung tumors on CT: the impact of 18FDG-hybrid PET fusion. Int J Radiat Oncol Biol Phys 2001; 51: 923–31.
- 63Hanna GG, McAleese J, Carson KJ et al. (18)F-FDG PET-CT simulation for non-small-cell lung cancer: effect in patients already staged by PET-CT. Int J Radiat Oncol Biol Phys 2010; 77: 24–30.
- 64Patel DA, Chang ST, Goodman KA et al. Impact of integrated PET/CT on variability of target volume delineation in rectal cancer. Technol Cancer Res Treat 2007; 6: 31–6.
- 65Krengli M, Cannillo B, Turri L et al. Target volume delineation for preoperative radiotherapy of rectal cancer: inter-observer variability and potential impact of FDG-PET/CT imaging. Technol Cancer Res Treat 2010; 9: 393–8.
- 66Buijsen J, van den Bogaard J, van der Weide H et al. FDG-PET-CT reduces the interobserver variability in rectal tumor delineation. Radiother Oncol 2012; 102: 371–6.
- 67Whaley JT, Fernandes AT, Sackmann R et al. Clinical utility of integrated positron emission tomography/computed tomography imaging in the clinical management and radiation treatment planning of locally advanced rectal cancer. Pract Radiat Oncol 2014; 4: 226–32.
- 68Metwally H, Courbon F, David I et al. Coregistration of prechemotherapy PET-CT for planning pediatric Hodgkin's disease radiotherapy significantly diminishes interobserver variability of clinical target volume definition. Int J Radiat Oncol Biol Phys 2011; 80: 793–9.
- 69Cattaneo GM, Reni M, Rizzo G et al. Target delineation in post-operative radiotherapy of brain gliomas: interobserver variability and impact of image registration of MR(pre-operative) images on treatment planning CT scans. Radiother Oncol 2005; 75: 217–23.
- 70Grabarz D, Panzarella T, Bezjak A, McLean M, Elder C, Wong RK. Quantifying interobserver variation in target definition in palliative radiotherapy. Int J Radiat Oncol Biol Phys 2011; 80: 1498–504.
- 71Nelms BE, Tome WA, Robinson G, Wheeler J. Variations in the contouring of organs at risk: test case from a patient with oropharyngeal cancer. Int J Radiat Oncol Biol Phys 2012; 82: 368–78.
- 72Weiss E, Richter S, Krauss T et al. Conformal radiotherapy planning of cervix carcinoma: differences in the delineation of the clinical target volume. A comparison between gynaecologic and radiation oncologists. Radiother Oncol 2003; 67: 87–95.
- 73Giraud P, Elles S, Helfre S et al. Conformal radiotherapy for lung cancer: different delineation of the gross tumor volume (GTV) by radiologists and radiation oncologists. Radiother Oncol 2002; 62: 27–36.
- 74Van de Steene J, Linthout N, de Mey J et al. Definition of gross tumor volume in lung cancer: inter-observer variability. Radiother Oncol 2002; 62: 37–49.
- 75Jeanneret-Sozzi W, Moeckli R, Valley JF et al. The reasons for discrepancies in target volume delineation: a SASRO study on head-and-neck and prostate cancers. Strahlenther Onkol 2006; 182: 450–7.
- 76Ost P, De MG, Vercauteren T et al. Delineation of the postprostatectomy prostate bed using computed tomography: interobserver variability following the EORTC delineation guidelines. Int J Radiat Oncol Biol Phys 2011; 81: e143–9.
- 77Roberge D, Skamene T, Turcotte RE, Powell T, Saran N, Freeman C. Inter- and intra-observer variation in soft-tissue sarcoma target definition. Cancer Radiother 2011; 15: 421–5.
- 78Louie AV, Rodrigues G, Olsthoorn J et al. Inter-observer and intra-observer reliability for lung cancer target volume delineation in the 4D-CT era. Radiother Oncol 2010; 95: 166–71.
- 79Nakamura K, Shioyama Y, Tokumaru S et al. Variation of clinical target volume definition among Japanese radiation oncologists in external beam radiotherapy for prostate cancer. Jpn J Clin Oncol 2008; 38: 275–80.
- 80Stanley J, Dunscombe P, Lau H et al. The effect of contouring variability on dosimetric parameters for brain metastases treated with stereotactic radiosurgery. Int J Radiat Oncol Biol Phys 2013; 87: 924–31.
- 81Coles CE, Hoole AC, Harden SV et al. Quantitative assessment of inter-clinician variability of target volume delineation for medulloblastoma: quality assurance for the SIOP PNET 4 trial protocol. Radiother Oncol 2003; 69: 189–94.
- 82Steenbakkers RJ, Duppen JC, Fitton I et al. Reduction of observer variation using matched CT-PET for lung cancer delineation: a three-dimensional analysis. Int J Radiat Oncol Biol Phys 2006; 64: 435–48.
- 83Konert T, Vogel W, MacManus MP et al. PET/CT imaging for target volume delineation in curative intent radiotherapy of non-small cell lung cancer: IAEA consensus report 2014. Radiother Oncol 2015; 116: 27–34.
- 84Min M, Roos D, Keating E et al. External evaluation of the Radiation Therapy Oncology Group brachial plexus contouring protocol: several issues identified. J Med Imaging Radiat Oncol 2014; 58: 360–8.
- 85Eriksen JG, Salembier C, Rivera S et al. Four years with FALCON – An ESTRO educational project: achievements and perspectives. Radiother Oncol 2014; 112: 145–9.
- 86Walker A, Liney GP, Metcalfe PE, Holloway L. MRI distortion: considerations for MRI based radiotherapy treatment planning. Australas Phys Eng Sci Med 2014; 37: 103–13.
- 87van Mourik AM, Elkhuizen PH, Minkema D, Duppen JC, Group DYBS, van Vliet-Vroegindeweij C. Multiinstitutional study on target volume delineation variation in breast radiotherapy in the presence of guidelines. Radiother Oncol 2010; 94: 286–91.
- 88Moretones C, Leon D, Navarro A et al. Interobserver variability in target volume delineation in postoperative radiochemotherapy for gastric cancer. A pilot prospective study. Clin Transl Oncol 2012; 14: 132–7.
- 89Dimigen M, Vinod SK, Lim K. Incorporating a radiologist in a radiation oncology department: a new model of care? Clin Oncol (R Coll Radiol) 2014; 26: 630–5.
- 90Hollingdale AE, Roques TW, Curtin J, Martin WM, Horan G, Barrett A. Multidisciplinary collaborative gross tumour volume definition for lung cancer radiotherapy: a prospective study. Cancer Imaging 2011; 11: 202–8.
- 91Wu DH, Mayr NA, Karatas Y et al. Interobserver variation in cervical cancer tumor delineation for image-based radiotherapy planning among and within different specialties. J Appl Clin Med Phys 2005; 6: 106–10.
- 92Vorwerk H, Zink K, Schiller R et al. Protection of quality and innovation in radiation oncology: the prospective multicenter trial the German Society of Radiation Oncology (DEGRO-QUIRO study). Evaluation of time, attendance of medical staff, and resources during radiotherapy with IMRT. Strahlenther Onkol 2014; 190: 433–43.
- 93Brundage M, Foxcroft S, McGowan T, Gutierrez E, Sharpe M, Warde P. A survey of radiation treatment planning peer-review activities in a provincial radiation oncology programme: current practice and future directions. BMJ Open 2013; 3: e003241.
- 94Hoopes DJ, Johnstone PA, Chapin PS et al. Practice patterns for peer review in radiation oncology. Pract Radiat Oncol 2015; 5: 32–28.
- 95Lawrence YR, Whiton MA, Symon Z et al. Quality assurance peer review chart rounds in 2011: a survey of academic institutions in the United States. Int J Radiat Oncol Biol Phys 2012; 84: 590–5.
- 96Brundage MD, Dixon PF, Mackillop WJ et al. A real-time audit of radiation therapy in a regional cancer center. Int J Radiat Oncol Biol Phys 1999; 43: 115–24.
- 97Boxer M, Forstner D, Kneebone A et al. Impact of a real-time peer review audit on patient management in a radiation oncology department. J Med Imaging Radiat Oncol 2009; 53: 405–11.
- 98Marks LB, Adams RD, Pawlicki T et al. Enhancing the role of case-oriented peer review to improve quality and safety in radiation oncology: executive summary. Pract Radiat Oncol 2013; 3: 149–56.
Citing Literature
