Volume 94, Issue 3 pp. 342-352

ARTIFICIAL INTELLIGENCE IN SURGERY

Exploring the role of an artificial intelligence chatbot on appendicitis management: an experimental study on ChatGPT

Dylan Gracias BBiomed, MD,

Corresponding Author

Dylan Gracias BBiomed, MD

[email protected]

orcid.org/0000-0002-6873-0144

Department of Surgery, Townsville Hospital, Townsville, Queensland, Australia

Correspondence

Dr Dylan Gracias, Department of Surgery, Townsville Hospital, Townsville, QLD 4814, Australia.

Email: [email protected]

Contribution: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing

Search for more papers by this author

Adrian Siu BPharm (Hons), MD, MS,

Adrian Siu BPharm (Hons), MD, MS

orcid.org/0000-0001-8151-9805

Faculty of Medicine and Health, Central Clinical School, The University of Sydney, Sydney, New South Wales, Australia

Surgical Outcomes Research Centre (SOuRCe), Royal Prince Alfred Hospital, Camperdown, New South Wales, Australia

Concord Institute of Academic Surgery, Concord Hospital, Concord, New South Wales, Australia

Contribution: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing

Search for more papers by this author

Ishith Seth BBiomed (Hons), MD, MS,

Ishith Seth BBiomed (Hons), MD, MS

orcid.org/0000-0001-5444-8925

Department of Surgery, Peninsula Health, Melbourne, Victoria, Australia

Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia

Contribution: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing

Search for more papers by this author

Dilshad Dooreemeah MBBS, FRACS,

Dilshad Dooreemeah MBBS, FRACS

Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia

Contribution: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing - original draft, Writing - review & editing

Search for more papers by this author

Angus Lee MBBS, FRACS,

Angus Lee MBBS, FRACS

orcid.org/0000-0002-2276-9924

Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia

Contribution: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing - original draft, Writing - review & editing

Search for more papers by this author

Dylan Gracias BBiomed, MD,

Corresponding Author

Dylan Gracias BBiomed, MD

[email protected]

orcid.org/0000-0002-6873-0144

Department of Surgery, Townsville Hospital, Townsville, Queensland, Australia

Correspondence

Dr Dylan Gracias, Department of Surgery, Townsville Hospital, Townsville, QLD 4814, Australia.

Email: [email protected]

Search for more papers by this author

Adrian Siu BPharm (Hons), MD, MS,

Adrian Siu BPharm (Hons), MD, MS

orcid.org/0000-0001-8151-9805

Faculty of Medicine and Health, Central Clinical School, The University of Sydney, Sydney, New South Wales, Australia

Surgical Outcomes Research Centre (SOuRCe), Royal Prince Alfred Hospital, Camperdown, New South Wales, Australia

Concord Institute of Academic Surgery, Concord Hospital, Concord, New South Wales, Australia

Contribution: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Validation, Visualization, Writing - original draft, Writing - review & editing

Search for more papers by this author

Ishith Seth BBiomed (Hons), MD, MS,

Ishith Seth BBiomed (Hons), MD, MS

orcid.org/0000-0001-5444-8925

Department of Surgery, Peninsula Health, Melbourne, Victoria, Australia

Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia

Search for more papers by this author

Dilshad Dooreemeah MBBS, FRACS,

Dilshad Dooreemeah MBBS, FRACS

Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia

Contribution: Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Writing - original draft, Writing - review & editing

Search for more papers by this author

Angus Lee MBBS, FRACS,

Angus Lee MBBS, FRACS

orcid.org/0000-0002-2276-9924

Department of Surgery, Bendigo Health, Bendigo, Victoria, Australia

Search for more papers by this author

First published: 19 October 2023

https://doi.org/10.1111/ans.18736

Citations: 3

D. Gracias BBiomed, MD; A. Siu BPharm (Hons), MD, MS; I. Seth BBiomed (Hons), MD, MS; D. Dooreemeah MBBS, FRACS; A. Lee MBBS, FRACS.

Share a link

Email
Wechat
Bluesky

Abstract

Background

Appendicitis is a common surgical condition that requires urgent medical attention. Recent advancements in artificial intelligence and large language processing, such as ChatGPT, have demonstrated potential in supporting healthcare management and scientific research. This study aims to evaluate the accuracy and comprehensiveness of ChatGPT's knowledge on appendicitis management.

Methods

Six questions related to appendicitis management were created by experienced RACS qualified general surgeons to assess ChatGPT's ability to provide accurate information. The criteria of ChatGPT answers' accuracy were compared with current healthcare guidelines for appendicitis and subjective evaluation by two RACS qualified General Surgeons. Additionally, ChatGPT was then asked to provide five high level evidence references to support its responses.

Results

ChatGPT provided clinically relevant information on appendicitis management, however, was inconsistent in doing so and often provided superficial information. Further to this, ChatGPT encountered difficulties in generating relevant references, with some being either non-existent or incorrect.

Conclusion

ChatGPT has the potential to provide timely and comprehensible medical information on appendicitis management to laypersons. However, its issue of inaccuracy in information and production of non-existent or erroneous references presents a challenge for researchers and clinicians who may inadvertently employ such information in their research or healthcare. Therefore, clinicians should exercise caution when using ChatGPT for these purposes.

Conflicts of interest

The authors declare no conflict of interest.

Open Research

Data availability statement

All study data is included in the submission.

Supporting Information

References

References marked with asterisk (*) are ChatGPT references.

1Bhangu A, Søreide K, Di Saverio S, Assarsson JH, Drake FT. Acute appendicitis: modern understanding of pathogenesis, diagnosis, and management. Lancet 2015; 386: 1278–1287.
10.1016/S0140-6736(15)00275-5
PubMed Web of Science® Google Scholar
2Wagner M, Tubre DJ, Asensio JA. Evolution and current trends in the management of acute appendicitis. Surg. Clin. 2018; 98: 1005–1023.
10.1016/j.suc.2018.05.006
PubMed Web of Science® Google Scholar
3da Silva JAT. Is ChatGPT a valid author? Nurse Educ. Pract. 2023; 68: 103600.
10.1016/j.nepr.2023.103600
PubMed Web of Science® Google Scholar
4Seth I, Rodwell A, Bulloch G, Seth N. Exploring the role of open artificial intelligence platform on surgical management of knee osteoarthritis: a case study of ChatGPT. J. Clin. Cases Rep. 2023; 5(1): 112–120.
Google Scholar
5Seth I, Rodwell A, Tso R, Valles J, Bulloch G, Seth N. A conversation with an open artificial intelligence platform on osteoarthritis of the hip and treatment. J. Orthop. Sports Med. 2023; 5: 112–120.
10.26502/josm.511500088
Google Scholar
6Jeblick K, Schachtner B, Dexl J, Mittermeier A, Stüber AT, Topalis J, et al. ChatGPT makes medicine easy to swallow: an exploratory case study on simplified radiology reports. arXiv preprint arXiv:221214882. 2022.
Google Scholar
7Eysenbach G. The role of ChatGPT, generative language models, and artificial intelligence in medical education: a conversation with ChatGPT and a call for papers. JMIR Med. Educ. 2023; 9: e46885.
10.2196/46885
PubMed Google Scholar
8Duque GA, Mohney S. Appendicitis in Pregnancy. [Updated 2022 Aug 8]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2023. Available from: https://www.ncbi.nlm.nih.gov/books/NBK551642/
Google Scholar
9*Vons C, Barry C, Maitre S et al. Amoxicillin plus clavulanic acid versus appendicectomy for treatment of acute uncomplicated appendicitis: an open-label, non-inferiority, randomised controlled trial. Lancet 2011; 377: 1573–1579.
10.1016/S0140-6736(11)60410-8
CAS PubMed Web of Science® Google Scholar
10*Anderson RE. The natural history and traditional management of appendicitis revisited: spontaneous resolution and predominance of prehospital perforations imply that a correct diagnosis is more important than an early diagnosis. World J. Surg. 2007; 31: 86–92.
10.1007/s00268-006-0056-y
PubMed Web of Science® Google Scholar
11*Smink DS, Finkelstein JA, Garcia Peña BM, Shannon MW, Taylor GA, Fishman SJ. Diagnosis of acute appendicitis in children using a clinical practice guideline. J. Pediatr. Surg. 2004; 39: 458–463.
10.1016/j.jpedsurg.2003.11.015
PubMed Web of Science® Google Scholar
12Petroianu A, Barroso TVV. Pathophysiology of Acute Appendicitis. JSM Gastroenterol. Hepatol. 2016; 3: 1062–1066.
Google Scholar
13Guo Y, Ye D, Yang G et al. Cluster of acute appendicitis among high school Tibetan students in Nanchang, China: investigation, control, and prevention. Front. Public Health 2022; 10: 889793.
10.3389/fpubh.2022.889793
PubMed Web of Science® Google Scholar
14*Simillis C, Symeonides P, Shorthouse AJ, Tekkis PP. A meta-analysis comparing conservative treatment versus acute appendectomy for complicated appendicitis (abscess or phlegmon). Surgery 2010; 147: 818–829.
10.1016/j.surg.2009.11.013
PubMed Web of Science® Google Scholar
15*Hansson J, Körner U, Khorram-Manesh A, Solberg A, Lundholm K. Randomized clinical trial of antibiotic therapy versus appendicectomy as primary treatment of acute appendicitis in unselected patients. Br. J. Surg. 2009; 96: 473–481.
10.1002/bjs.6482
CAS PubMed Web of Science® Google Scholar
16*Di Saverio S, Birindelli A, Kelly MD et al. WSES Jerusalem guidelines for diagnosis and treatment of acute appendicitis. World J. Emerg. Surg. 2016; 11: 34.
10.1186/s13017-016-0090-5
PubMed Web of Science® Google Scholar
17*Flum DR, Morris A, Koepsell T, Dellinger EP. Has misdiagnosis of appendicitis decreased over time? A population-based analysis. JAMA 2001; 286: 1748–1753.
10.1001/jama.286.14.1748
CAS PubMed Web of Science® Google Scholar
18*Kharbanda AB, Dudley NC, Bajaj L et al. Validation and refinement of a prediction rule to identify children at low risk for acute appendicitis. Arch. Pediatr. Adolesc. Med. 2012; 166: 738–744.
10.1001/archpediatrics.2012.490
PubMed Web of Science® Google Scholar
19*St Peter SD, Sharp SW, Holcomb GW 3rd, Ostlie DJ. An evidence- based definition for perforated appendicitis derived from a prospective randomized trial. J. Pediatr. Surg. 2008; 43: 2242–2245.
10.1016/j.jpedsurg.2008.08.051
PubMed Web of Science® Google Scholar
20*Salminen P, Paajanen H, Rautio T et al. Antibiotic therapy vs appendectomy for treatment of uncomplicated acute appendicitis: the APPAC randomized clinical trial. JAMA 2015; 313: 2340–2348.
10.1001/jama.2015.6154
CAS PubMed Web of Science® Google Scholar
21*Varadhan KK, Neal KR, Lobo DN. Safety and efficacy of antibiotics compared with appendectomy for treatment of uncomplicated acute appendicitis; meta-analysis of randomized controlled trials. BMJ 2012; 344: e2156.
10.1136/bmj.e2156
CAS PubMed Web of Science® Google Scholar
22*Huang L, Yin Y, Yang L, Wang C, Li Y, Zhou Z. Comparison of antibiotic therapy and appendectomy or acute uncomplicated appendicitis in children. A meta-analysis. JAMA Pediatr. 2017; 171: 426–434.
10.1001/jamapediatrics.2017.0057
PubMed Web of Science® Google Scholar
23*Salminen P, Tuominen R, Paajanen H et al. Five-year follow-up of antibiotic therapy versus appendectomy for acute uncomplicated appendicitis: a randomized clinical trial. JAMA 2018; 320: 1259–1265.
10.1001/jama.2018.13201
CAS PubMed Web of Science® Google Scholar
24*Sallinen V, Akl EA, You JJ et al. Meta-analysis of antibiotics versus appendicectomy for non-perforated acute appendicitis. Br. J. Surg. 2016; 103: 656–667.
10.1002/bjs.10147
CAS PubMed Web of Science® Google Scholar
25Collaborative C. A randomized trial comparing antibiotics with appendectomy for appendicitis. N. Engl. J. Med. 2020; 383: 1907–1919.
10.1056/NEJMoa2014320
PubMed Web of Science® Google Scholar
26Gorter RR, Eker HH, Gorter-Stam MA et al. Diagnosis and management of acute appendicitis. EAES consensus development conference 2015. Surg. Endosc. 2016; 30: 4668–4690.
10.1007/s00464-016-5245-7
PubMed Web of Science® Google Scholar
27*Wilms IM, de Hoog DE, de Visser DC, Janzing HM. Appendectomy versus antibiotic treatment for acute appendicitis. Cochrane Database Syst. Rev. 2011; ( 11): CD008359. https://doi.org/10.1002/14651858.CD008359.pub2. Update in: Cochrane Database Syst. Rev. 2020; 10: CD008359. PMID: 22071846.
10.1002/14651858.CD008359.pub2
PubMed Web of Science® Google Scholar
28Zingone F, Sultan AA, Humes DJ, West J. Risk of acute appendicitis in and around pregnancy: a population-based cohort study from England. Ann. Surg. 2015; 261: 332–337.
10.1097/SLA.0000000000000780
PubMed Web of Science® Google Scholar
29Moltubak E, Landerholm K, Blomberg M, Redéen S, Andersson RE. Major variation in the incidence of appendicitis before, during and after pregnancy: a population-based cohort study. World J. Surg. 2020; 44: 2601–2608.
10.1007/s00268-020-05524-z
CAS PubMed Web of Science® Google Scholar
30Seth I, Lim B, Xie Y, Hunter-Smith DJ, Rozen WM. Exploring the role of artificial intelligence chatbot on the management of scaphoid fractures. J. Hand Surg. (Eur. Vol.). 2023; 48(8): 17531934231169858. https://doi.org/10.1177/17531934231169858.
10.1177/17531934231169858
Web of Science® Google Scholar
31Seth I, Bulloch G, Lee CHA. Redefining academic integrity, authorship, and innovation: the impact of ChatGPT on surgical research. Ann. Surg. Oncol. 2023; 30: 5284–5285.
10.1245/s10434-023-13642-w
PubMed Google Scholar
32*Addiss DG, Shaffer N, Fowler BS, Tauxe RV. The epidemiology of appendicitis and appendectomy in the United States. Am. J. Epidemiol. 1990; 132: 910–925.
10.1093/oxfordjournals.aje.a115734
CAS PubMed Web of Science® Google Scholar
33*Rothrock SG, Pagane J. Acute appendicitis in children: emergency department diagnosis and management. Ann. Emerg. Med. 2000; 36: 39–51.
10.1067/mem.2000.105658
CAS PubMed Web of Science® Google Scholar

Citing Literature

Volume94, Issue3

March 2024

Pages 342-352

Exploring the role of an artificial intelligence chatbot on appendicitis management: an experimental study on ChatGPT

Abstract

Background

Methods

Results

Conclusion

Conflicts of interest

Open Research

Data availability statement

Supporting Information

References

Citing Literature

References

Information

About Wiley Online Library

Help & Support

Opportunities

Connect with Wiley

Exploring the role of an artificial intelligence chatbot on appendicitis management: an experimental study on ChatGPT

Abstract

Background

Methods

Results

Conclusion

Conflicts of interest

Open Research

Data availability statement

Supporting Information

References

Citing Literature

References

Related

Information