[Retracted] Coding ATC Incident Data Using HFACS: Intercoder Consensus
Retraction(s) for this article
-
Retracted: Coding ATC Incident Data Using HFACS: Intercoder Consensus
- Volume 2021Issue 1Journal of Quality and Reliability Engineering
- First Published online: January 19, 2021
Corresponding Author
Liang Wang
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorYaohua Wang
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorXiaoqiang Yang
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorKai Cheng
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorHaishan Yang
Military Representative Office of the General Department of Armaments in Zhengzhou Area, No. 23, Eastern Rantun Road, Zhengzhou 450051, China
Search for more papers by this authorBaoguo Zhu
Military Representative Office of the General Department of Armaments in Tianjin Area, No. 7, Yuhong Road, Hebei District, Tianjin 300240, China
Search for more papers by this authorChengfei Fan
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorXinwei Ji
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorCorresponding Author
Liang Wang
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorYaohua Wang
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorXiaoqiang Yang
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorKai Cheng
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorHaishan Yang
Military Representative Office of the General Department of Armaments in Zhengzhou Area, No. 23, Eastern Rantun Road, Zhengzhou 450051, China
Search for more papers by this authorBaoguo Zhu
Military Representative Office of the General Department of Armaments in Tianjin Area, No. 7, Yuhong Road, Hebei District, Tianjin 300240, China
Search for more papers by this authorChengfei Fan
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorXinwei Ji
Engineering Institute of Corps of Engineers, PLA University of Science and Technology, No. 1, Haifuxiang, Nanjing 210007, China plaust.edu.cn
Search for more papers by this authorAbstract
Reliability studies for coding contributing factors of incident reports in high hazard industries are rarely conducted and reported. Although the Human Factors Analysis and Classification System (HFACS) appears to have a larger number of such studies completed than most other systems doubt exists as the accuracy and comparability of results between studies due to aspects of methodology and reporting. This paper reports on a trial conducted on HFACS to determine its reliability in the context of military air traffic control (ATC). Two groups participated in the trial: one group comprised of specialists in the field of human factors, and the other group comprised air traffic controllers. All participants were given standardized training via a self-paced workbook and then read 14 incident reports and coded the associated findings. The results show similarly low consensus for both groups of participants. Several reasons for the results are proposed associated with the HFACS model, the context within which incident reporting occurs in real organizations and the conduct of the studies.
References
- 1 Shorrock S. T. and Kirwan B., Development and application of a human error identification tool for air traffic control, Applied Ergonomics. (2002) 33, no. 4, 319–336, 2-s2.0-0036279002, https://doi.org/10.1016/S0003-6870(02)00010-8.
- 2
Wallace B.,
Ross A.,
Davies J.,
Wright L., and
White M., The creation of a new minor event coding system, Cognition, Technology and Work. (2002) 4, no. 1, 1–8.
10.1007/s101110200000 Google Scholar
- 3 Wiegmann D. and Shappell S., A Human Error Approach to Aviation Accident Analysis: The Human Factors Analysis and Classification System, 2003, Ashgate Publishing, Aldershot, UK.
- 4
Wallace B. and
Ross A., Beyond Human Error: Taxonomies and Safety Science, 2006, CRC Press, Boca Raton, Fla, USA.
10.1201/9780203491171 Google Scholar
- 5 Ross A. J., Wallace B., and Davies J. B., Technical note: measurement issues in taxonomic reliability, Safety Science. (2004) 42, no. 8, 771–778, 2-s2.0-3042522891, https://doi.org/10.1016/j.ssci.2003.10.004.
- 6 ElBardissi A. W., Wiegmann D. A., Dearani J. A., Daly R. C., and Sundt T. M., Application of the human factors analysis and classification system methodology to the cardiovascular surgery operating room, Annals of Thoracic Surgery. (2007) 83, no. 4, 1412–1419, 2-s2.0-33947315166, https://doi.org/10.1016/j.athoracsur.2006.11.002.
- 7 Pape A., Wiegmann D., and Shappell S., Air Traffic Control (ATC) related accidents and incidents: a human factors analysis, Proceedings of the 11th International Symposium on Aviation Psychology, 2001, Columbus, Ohio, The Ohio State University.
- 8 Pounds J., Scarborough A., and Shappell S., A human factors analysis of air traffic control operational errors, Aviation Space and Environmental Medicine. (2000) 71, 329–332.
- 9 Shappell S., Detwiler C., Holcomb K., Hackworth C., Boquet A., and Wiegmann D. A., Human error and commercial aviation accidents: an analysis using the human factors analysis and classification system, Human Factors. (2007) 49, no. 2, 227–242, 2-s2.0-33947687150, https://doi.org/10.1518/001872007X312469.
- 10 Wiegmann D., Faaborg T., Boquet A., Detwiler C., Halcomb K., and Shappell S., Human error and general aviation accidents: a comprehensive, fine-grained analysis using HFACS, Department of Transportation, FAA Report, DOT/FAA/AM-05/24, 2005.
- 11 Shappell S. and Wiegmann D., A human error analysis of general aviation controlled flight into terrain accidents occurring between 1990–1998, Department of Transportation, FAA Report, DOT/FAA/AM-03/4, 2003.
- 12
Davies J.,
Ross A.,
Wallace B., and
Wright L., Safety Management: A Qualitative Systems Approach, 2003, Taylor and Francis, London, UK.
10.4324/9780203403228 Google Scholar
- 13 Cohen J., A coefficient of agreement for nominal scales, Educational and Psychological Measurement. (1960) 20, no. 1, 37–46.
- 14 Grove W. M., Andreasen N. C., McDonald-Scott P., Keller M., and Shapiro R., Reliability studies of psychiatric diagnosis. Theory and practice, Archives of General Psychiatry. (1981) 38, no. 4, 408–413.
- 15 Maxwell A. E., Coefficients of agreement between observers and their interpretation, British Journal of Psychiatry. (1977) 130, no. 1, 79–83, 2-s2.0-0017612723.
- 16 Spitznagel E. L. and Helzer J. E., A proposed solution to the base rate problem in the kappa statistic, Archives of General Psychiatry. (1985) 42, no. 7, 725–728, 2-s2.0-0021845522.
- 17 Stanton N. A. and Steverage S. V., Learning to predict human error: issues of acceptability, reliability and validity, Ergonomics. (1998) 41, no. 11, 1737–1756, 2-s2.0-0032211748, https://doi.org/10.1080/001401398186162.
- 18 Wallén-Warner H. W. and Sandin J., The intercoder agreement when using the driving reliability and error analysis method in road traffic accident investigations, Safety Science. (2010) 48, no. 5, 527–536, 2-s2.0-77950528956, https://doi.org/10.1016/j.ssci.2009.12.022.
- 19 Shappell S. and Wiegmann D., Applying Reason: the human factors analysis and classification system (HFACS), Human Factors and Aerospace Safety. (2001) 1, no. 1, 59–86.
- 20 Li W. and Harris D., HFACS analysis of ROC Air Force aviation accidents: reliability analysis and crosscultural comparison, International Journal of Applied Aviation Studies. (2005) 5, no. 1, 65–72.
- 21 Kozlowski S. W. J. and Hattrup K., A disagreement about within-group agreement: disentangling issues of consistency versus consensus, Journal of Applied Psychology. (1992) 77, no. 2, 161–167, 2-s2.0-0000855257.
- 22
Martin P. and
Bateson P., Measuring Behaviour: An Introductory Guide, 1993, Cambridge University Press, Cambridge, UK.
10.1017/CBO9781139168342 Google Scholar
- 23 O′Connor P., HFACS with an additional layer of granularity: validity and utility in accident analysis, Aviation Space and Environmental Medicine. (2008) 79, no. 6, 599–606, 2-s2.0-44449176713, https://doi.org/10.3357/ASEM.2228.2008.
- 24 Olsen N. S. and Shorrock S. T., Evaluation of the HFACS-ADF safety classification system: inter-coder consensus and intra-coder consistency, Accident Analysis and Prevention. (2010) 42, no. 2, 437–444, 2-s2.0-76049095120, https://doi.org/10.1016/j.aap.2009.09.005.
- 25 Reason J., Human Error, 1990, Cambridge University Press, New York, NY, USA.
- 26 Bird F., Management Guide to Loss Control, 1974, Institute Press, Atlanta, Ga, USA.
- 27 Johnson W., Classifying pilot human error causes in A-10 class A mishaps, Graduate Research Project, Embrey-Riddle Aeronautical University, 1997.
- 28 AlWardi Y., Applying human factors analysis and classification system (HFACS) to aviation accidents in the Royal Air Force of Oman (RAFO): a cross cultural comparison, M.S. thesis, 2006, Cranfield University, Bedfordshire, UK.
- 29 Walker D., Applying the human factors analysis and classification system (HFACS) to incidents in the UK construction industry, Graduate Research Project, Cranfield University, 2007.
- 30 Krulak D. C., Human factors in maintenance: impact on aircraft mishap frequency and severity, Aviation Space and Environmental Medicine. (2004) 75, no. 5, 429–432, 2-s2.0-2142640897.
- 31 Patterson J. M. and Shappell S. A., Operator error and system deficiencies: analysis of 508 mining incidents and accidents from Queensland, Australia using HFACS, Accident Analysis and Prevention. (2010) 42, no. 4, 1379–1385, 2-s2.0-77955310409, https://doi.org/10.1016/j.aap.2010.02.018.
- 32 Reinach S. and Viale A., Application of a human error framework to conduct train accident/incident investigations, Accident Analysis and Prevention. (2006) 38, no. 2, 396–406, 2-s2.0-31044436648, https://doi.org/10.1016/j.aap.2005.10.013.
- 33 Baysari M. T., Caponecchia C., McIntosh A. S., and Wilson J. R., Classification of errors contributing to rail incidents and accidents: a comparison of two human error identification techniques, Safety Science. (2009) 47, no. 7, 948–957, 2-s2.0-67349287441, https://doi.org/10.1016/j.ssci.2008.09.012.
- 34 Celik M. and Cebi S., Analytical HFACS for investigating human errors in shipping accidents, Accident Analysis and Prevention. (2009) 41, no. 1, 66–75, 2-s2.0-57849124041, https://doi.org/10.1016/j.aap.2008.09.004.
- 35 James L. R., Demaree R. G., and Wolf G., Estimating within-group interrater reliability with and without response bias, Journal of Applied Psychology. (1984) 69, no. 1, 85–98, 2-s2.0-58049220690, https://doi.org/10.1037/0021-9010.69.1.85.
- 36 Isaac A., Lyons M., Bove T., and Van Damme D., Validation of the human error in ATM (HERA-JANUS) technique, EUROCONTROL, HRA/HSP-002-REP-04, 2003.
- 37 Ross A. J., Wallace B., and Davies J. B., Technical note: measurement issues in taxonomic reliability, Safety Science. (2004) 42, no. 8, 771–778, 2-s2.0-3042522891, https://doi.org/10.1016/j.ssci.2003.10.004.
- 38 Beaubien J. and Baker D., A review of selected aviation human factors taxonomies, accident/incident reporting systems and data reporting tools, International Journal of Applied Aviation Studies. (2002) 2, no. 2, 11–36.
- 39 Dambier M. and Hinkelbein J., Analysis of 2004 German general aviation aircraft accidents according to the HFACS model, Air Medical Journal. (2006) 25, no. 6, 265–269, 2-s2.0-33750361406, https://doi.org/10.1016/j.amj.2006.03.003.
