Accuracy of the Orlando Head CT Criteria in detecting clinically significant abnormalities in patients without head trauma: A systematic review and meta-analysis
David D. Cassidy MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Florida State University College of Medicine, Tallahassee, Florida, USA
Search for more papers by this authorCorresponding Author
Steven G. Rothrock MD
Florida State University College of Medicine, Tallahassee, Florida, USA
Department of Emergency Medicine, Dr P. Phillips Hospital, Orlando Health, Orlando, Florida, USA
Correspondence
Steven G. Rothrock, Department of Emergency Medicine, Dr P. Phillips Hospital, Orlando Health, Orlando, FL USA.
Email: [email protected]
Search for more papers by this authorGrant Wandling MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorChrisi T. Myers BS
Florida State University College of Medicine, Tallahassee, Florida, USA
Search for more papers by this authorKristina Arwady BA
Department of Emergency Medicine, Dr P. Phillips Hospital, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorStephanie Schiffert MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorMax Trojano MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorRyan Sawyers MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorChristian Iuteri MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorConnor Byrne MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorDavid D. Cassidy MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Florida State University College of Medicine, Tallahassee, Florida, USA
Search for more papers by this authorCorresponding Author
Steven G. Rothrock MD
Florida State University College of Medicine, Tallahassee, Florida, USA
Department of Emergency Medicine, Dr P. Phillips Hospital, Orlando Health, Orlando, Florida, USA
Correspondence
Steven G. Rothrock, Department of Emergency Medicine, Dr P. Phillips Hospital, Orlando Health, Orlando, FL USA.
Email: [email protected]
Search for more papers by this authorGrant Wandling MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorChrisi T. Myers BS
Florida State University College of Medicine, Tallahassee, Florida, USA
Search for more papers by this authorKristina Arwady BA
Department of Emergency Medicine, Dr P. Phillips Hospital, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorStephanie Schiffert MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorMax Trojano MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorRyan Sawyers MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorChristian Iuteri MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorConnor Byrne MD
Department of Emergency Medicine Orlando Regional Medical Center, Orlando Health, Orlando, Florida, USA
Residency in Emergency Medicine, Orlando Health, Orlando, Florida, USA
Search for more papers by this authorPresented at the Society for Academic Emergency Medicine Annual Meeting, Philadelphia, PA, May 2025.
Registration: PROSPERO (CRD42024590542).
Supervising Editor: Benton Robert Hunter
Abstract
Background
This systematic review and meta-analysis evaluated the accuracy of the Orlando Head CT Criteria (focal deficit, altered mental status, age≥60, headache with vomiting) in predicting clinically significant abnormalities (CSAs) in ED patients without trauma.
Methods
This study followed PRISMA-DTA guidelines. Searched databases included PubMed, EMBASE, Web of Science, CINAHL, Google Scholar, and gray literature. Studies were included if they contained ED patients without trauma and analyzed accuracy of the Orlando Head CT Criteria. Summary estimates of sensitivity, specificity, diagnostic odds ratio (DOR), and likelihood ratios (LRs) were calculated using a bivariate mixed model with a random-effects approach. Risk of bias and applicability were assessed via QUADAS-2 and certainty of evidence via the GRADE framework. Heterogeneity was assessed via forest plots, a hierarchical summary receiver operating characteristic curve and subgroup analysis. Sensitivity analysis was performed using the leave-one-out method. Threshold effects were investigated via Spearman's correlation and publication bias using Deeks' funnel plot.
Results
Seven studies (8914 patients) were included: four prospective and three retrospective. Orlando Head CT Criteria were 98.4% (95% confidence interval [CI] 97.1%–99.1%) sensitive, 17.9% (95% CI 12.3%–25.3%, 95% CI) specific with a DOR of 13.4 (95% CI 6–29.6) and negative LR (–LR) of 0.09 (95% CI 0.04–0.19) in predicting CSA CT scans. Implementation of these criteria would have decreased CT use by 16.2% (95% CI 9.7%–23.9%). GRADE certainty of evidence was moderate. No subgroup or single study was a source of heterogeneity. There were no threshold effects (Spearman ρ = −0.26). We found no publication bias (Deeks' funnel plot asymmetry test, p = 0.71).
Conclusion
Orlando Head CT Criteria were sensitive in detecting CSA CT scans with a low –LR. Further studies are required to assess whether these criteria can decrease CT use or augment clinical judgment for patients undergoing nontrauma cranial CT.
CONFLICT OF INTEREST STATEMENT
The authors declare no conflicts of interest.
Open Research
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are openly available in pubmed at https://pubmed.ncbi.nlm.nih.gov/.
Supporting Information
| Filename | Description |
|---|---|
| acem70065-sup-0001-SupplementalAppendix1.docxWord 2007 document , 34.7 KB |
Appendix S1. |
| acem70065-sup-0002-SupplementalAppendix2.docxWord 2007 document , 20.8 KB |
Appendix S2. |
| acem70065-sup-0003-SupplementalAppendix3.docxWord 2007 document , 28.2 KB |
Appendix S3. |
| acem70065-sup-0004-SupplementalApprndix4.docxWord 2007 document , 1 MB |
Appendix S4. |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
REFERENCES
- 1 American College of Emergency Physicians Clinical Policies Subcommittee. Clinical policy: critical issues in the management of adult patients presenting to the emergency department with mild traumatic brain injury. Ann Emerg Med. 2023; 81: e63-e105. doi:10.1016/j.annemergmed.2023.01.014
- 2Mower WR, Gupta M, Rodriquez R, Rodriguez R, Hendey GW. Validation of the sensitivity of the National Emergency X- radiography Utilization Study (NEXUS) head computed tomographic (CT) decision instrument for selective imaging of blunt head injury patients: an observational study. PLoS Med. 2017; 14: e1002313. doi:10.1371/journal.pmed.1002313
- 3Stiell IG, Wells GA, Vandemheen K, et al. The Canadian CT rule for patients with minor head injury. Lancet. 2001; 357: 1391-1396. doi:10.1016/s0140-6736(00)04561-x
- 4Rothrock SG, Buchanan C, Green SM, Bullard T, Falk JL, Langen M. Cranial computed tomography in the emergency evaluation of adult patients without a recent history of head trauma: a prospective analysis. Acad Emerg Med. 1997; 4: 654-661. doi:10.1111/j.1553-2712.1997.tb03756.x
- 5Covino M, Gilardi E, Manno A, et al. A new clinical score for cranial computed tomography in emergency department non-trauma patients: definition and first validation. Am J Emerg Med. 2019; 37: 1279-1284. doi:10.1016/j.ajem.2018.09.032
- 6Harris JE, Draper HL, Rhodes AI, Stevens JM. High yield criteria for emergency cranial computed tomography in adult patients with no history of head trauma. J Accid Emerg Med. 2000; 17: 15-17. doi:10.1136/emj.17.1.15
- 7Bhaumik D, Bhaumik SS, Thaker AA, et al. Ordering characteristics predictive of noncontrast CT head positivity in the emergency department. Acta Radiol. 2023; 30: 492-498. doi:10.1016/j.acra.2022.05.001
10.1016/j.acra.2022.05.001 Google Scholar
- 8Callen AL, Chow YA, Chen HR, et al. Predictive value of noncontrast head CT with negative findings in the emergency department setting. Am J Neuroradiol. 2020; 41: 213-218. doi:10.3174/ajnr.A6408
- 9Kelly AM, Kuan WS, Chu KH, et al. Epidemiology, investigation, management, and outcome of headache in emergency departments (HEAD study) – a multinational observational study. Headache. 2021; 61: 1539-1552. doi:10.1111/head.14230
- 10Lemmens CMC, van der Linden MC, Jellema K. The value of cranial CT imaging in patients with headache at the emergency department. Front Neurol. 2021; 12:663353. doi:10.3389/fneur.2021.663353
- 11Nesselroth D, Klang E, Soffer S, et al. Yield of head CT for acute findings in patients presenting to the emergency department. Clin Imaging. 2018; 73: 1-5. doi:10.1016/j.clinimag.2020.11.025
10.1016/j.clinimag.2020.11.025 Google Scholar
- 12Simwatachela E, Ozoh JO, Mabuza LH, Kalinda C. Clinical predictors of abnormal head computed tomography findings in non-trauma patients presenting to a South African emergency department. Front Radiol. 2021; 1:759731. doi:10.3389/fradi.2021.759731
- 13McInnes MDF, Moher D, Thombs BD, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA. 2018; 319: 388-396. doi:10.1001/jama.2017.19163
- 14McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS peer review of electronic search strategies: 2015 guideline statement. J Clin Epidemiol. 2016; 75: 40-46. doi:10.1016/j.jclinepi.2016.01.021
- 15Whiting PF, Rutjes AW, Westwood ME, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med. 2011; 155: 529-536. doi:10.7326/0003-4819-155-8-201110180-00009
- 16Gopalakrishna G, Mustafa RA, Davenport C, et al. Applying Grading of Recommendations Assessment, Development and Evaluation (GRADE) to diagnostic tests was challenging but doable. J Clin Epidemiol. 2014; 67: 760-768. doi:10.1016/j.jclinepi.2014.01.006
- 17Guyatt GH, Oxman AD, Vist G, et al. GRADE guidelines: 4. Rating the quality of evidence – study limitations (risk of bias). J Clin Epidemiol. 2011; 64: 407-415. doi:10.1016/j.jclinepi.2010.07.017
- 18Guyatt GH, Oxman AD, Sultan S, et al. GRADE guidelines: 9. Rating up the quality of evidence. J Clin Epidemiol. 2011; 64: 1311-1316. doi:10.1016/j.jclinepi.2011.06.004
- 19Deeks JJ, Higgens JPT, Altman DG. Chapter 10: Analyzing data and undertaking meta-analyses. In: JPT Higgins, J Thomas, eds. Cochrane Handbook for Systematic Reviews of Interventions. Version 6.5 (updated August 2024). Cochrane; 2024. Accessed January 14, 2025. https://training.cochrane.org/handbook/current/chapter-10#section-10-11
- 20Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005; 58: 882-893. doi:10.1016/j.jclinepi.2005.01.016
- 21Lim HC, Wong EH. Emergent cranial computed tomography in the evaluation of adult non-trauma patients in the emergency department. Hong Kong J Emerg Med. 2004; 11: 197-204.
10.1177/102490790401100402 Google Scholar
- 22Scott-King J, Tieu S, Chiew AL, Lui J, Kirby KA, Chan BS. Clinical decision rule for non-traumatic computed tomography of the brain. Emerg Med Australas. 2019; 31: 974-981. doi:10.1111/1742-6723.13292
- 23Tan I, Young N, Sindhusake DP, et al. Prospective evaluation of selected clinical criteria for cranial tomography in non-trauma adult patients. Emerg Med Australas. 2009; 21: 43-51. doi:10.1111/j.1742-6723.2008.01151.x
- 24Tung C, Lindgren A, Siemund R, van Westen D. Emergency room decision-making for urgent cranial computed tomography: selection criteria for subsets of non-trauma patients. Acta Radiol. 2014; 55: 847-854. doi:10.1177/0284185113506137
- 25McHugh ML. Interrater reliability: the kappa statistic. Biochem Med. 2012; 22: 276-282.
- 26Espejo T, Riedel HB, Messingschlager S, et al. Predictive value and interrater reliability of mental status and mobility assessment in the emergency department. Clin Med (Lond). 2024; 24(2):100027. doi:10.1016/j.clinme.2024.100027
- 27Hollander JE, Go S, Lowery DW, et al. Interrater reliability of criteria used in assessing blunt head injury patients for intracranial injuries. Acad Emerg Med. 2003; 10: 830-835. doi:10.1111/j.1553-2712.2003.tb00624.x
- 28Matteuci MJ, Moszyk D, Migliore SA. Agreement between resident and faculty emergency physicians in the application of NEXUS criteria for suspected cervical spine injuries. J Emerg Med. 2015; 48: 445-449. doi:10.1016/j.jemermed.2014.11.006
- 29Johnstone AJ, Lohlun HC, Miller JD, et al. A comparison of the Glasgow Coma Scale and the Swedish reaction level scale. Brain Inj. 1993; 7: 501-506. doi:10.3109/02699059309008177
- 30Borenstein M. How to understand and report heterogeneity in a meta-analysis: the difference between I-squared and prediction intervals. Integr Med Res. 2023; 12(4):101014. doi:10.1016/j.imr.2023.101014
- 31Green SM. When do clinical decision rules improve patient care? Ann Emerg Med. 2013; 62: 132-135. doi:10.1016/j.annemergmed.2013.02.006
- 32Bellolio MF, Heien HC, Sangaralingham LR, et al. Increased computed tomography utilization in the emergency department and its association with hospital admission. West J Emerg Med. 2017; 18: 835-845. doi:10.5811/westjem.2017.5.34152
- 33Broder J, Warhauer DM. Increasing utilization of computed tomography in the adult emergency department, 2000-2005. Emerg Radiol. 2006; 13: 25-30. doi:10.1007/s10140-006-0493-9
- 34Hess EP, Haas LR, Shah DN, et al. Trends in computed tomography utilization rates: a longitudinal practice-based study. J Patient Saf. 2014; 10: 52-58. doi:10.1097/PTS.0b013e3182948b1a
- 35Kempter F, Heye T, Vosshenrich J, Ceresa B, Jaschke D. Tredns in CT examination utilization in the emergency department during and after the COVID-19 pandemic. BMC Med Imaging. 2024; 24(1): 283. doi:10.1186/s12880-024-01457-4
- 36Mamer LE, Kocher KE, Cranford JA, Scott PA. Longitudinal changes in the US emergency department use of advanced neuroimaging in the mechanical thrombectomy era. Emerg Radiol. 2024; 31: 695-703. doi:10.1007/s10140-024-02260-y
- 37Prabhakar AM, Gottumukkala RV, Hemingway J, Hughes DR, Patel SS, Duszak R Jr. Increasing utilization of emergency department neuroimaging in Medicare beneficiaries from 1994 to 2015. Am J Emerg Med. 2018; 36: 680-683. doi:10.1016/j.ajem.2017.12.057
- 38Wang X, You JJ. Head CT for nontrauma patients in the emergency department: clinical predictors of abnormal findings. Radiology. 2013; 266: 783-790. doi:10.1148/radiol.12120732
- 39Friedman BW. Headache disorders. In: RM Walls, RS Hockberger, M Gausche-Hill, TB Erickson, SR Wilcox, eds. Rosen's Emergency Medicine: Concepts and Clinical Practice. 10th ed. Elsevier; 2023: 1255-1268.
- 40Go S, Kornegay J. Stroke syndromes. In: ME Tintinalli, O Ma, DM Yealy, et al., eds. Tintinalli's Emergency Medicine: A Comprehensive Study Guide. 9th ed. McGraw-Hill Education; 2020. Accessed January 5, 2025. https://accessmedicine.mhmedical.com/content.aspx?bookid=2353§ionid=220293532
- 41Lei C, Smith C. Depressed consciousness and coma. In: RM Walls, RS Hockberger, M Gausche-Hill, TB Erickson, SR Wilcox, eds. Rosen's Emergency Medicine: Concepts and Clinical Practice. 10th ed. Elsevier; 2023: 117-126.
- 42Mutter M, Huff J. Altered mental status and coma. In: JE Tintinalli, O Ma, DM Yealy, et al., eds. Tintinalli's Emergency Medicine: A Comprehensive Study Guide. 9th ed. McGraw-Hill Education; 2020. Accessed January 5, 2025. https://accessmedicine.mhmedical.com/content.aspx?bookid=2353§ionid=189594077
- 43Papa L, Meurer WJ. Stroke. In: RM Walls, RS Hockberger, M Gausche-Hill, TB Erickson, SR Wilcox, eds. Rosen's Emergency Medicine: Concepts and Clinical Practice. 10th ed. Elsevier; 2023: 1224-1242.
- 44 American College of Emergency Physicians Clinical Policies Subcommittee. Clinical policy: critical issues in the evaluation and management of adult patients presenting to the emergency department with acute headache. Ann Emerg Med. 2019; 74: e41-e74. doi:10.1016/j.annemergmed.2019.07.009
- 45 Appropriateness Criteria. Headache. American College of Radiology. Revised 2022. Accessed January 28, 2025. https://acsearch.acr.org/docs/69482/Narrative/
- 46 ASNR Practice Guidelines and Technical Standards. Practice parameter for the performance of computed tomography (CT) of the head. Accessed January 28, 2025. https://www.asnr.org/practice-guidelines/
- 47Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004; 329(7458): 168-169. doi:10.1136/bmj.329.7458.168
- 48Bent C, Lee PS, Shen PY, Bang H, Bobinski M. Clinical scoring system may improve yield of head CT of non-trauma emergency department patients. Emerg Radiol. 2015; 22: 511-516. doi:10.1007/s10140-015-1305-x
- 49Le Nguyen A, Kirkwood B, Hackett L, Buntine P. Validation of the ECHS non trauma cranial CT rule in Australia: a prospective cohort study. Am J Emerg Med. 2022; 52: 225-231. doi:10.1016/j.ajem.2021.12.004
- 50Heskestad B, Baardsen R, Helseth E, Ingebrigtsen T. Guideline compliance in management of minimal, mild, and moderate head injury: high frequency of noncompliance among individual physicians despite strong guideline support from clinical leaders. J Trauma. 2008; 65: 1309-1313. doi:10.1097/TA.0b013e31815e40cd
- 51Pons E, Foks KA, Dippel DWJ, Hunink MGM. Impact of guidelines for the management of minor head injury on the utilization and diagnostic yield of CT over two decades, using natural language processing in a large dataset. Eur Radiol. 2019; 29: 2632-2640. doi:10.1007/s00330-018-5954-5
- 52Stiell IG, Clement CM, Grimshaw JM, et al. A prospective cluster-randomized trial to implement the Canadian CT head rule in emergency departments. CMAJ. 2010; 182: 1527-1532. doi:10.1503/cmaj.091974
