EVALUATION OF THE PERFORMANCE OF CHATGPT/ARTIFICIAL INTELLIGENCE IN THE MULTIPLE-CHOICE TEST TO OBTAIN THE TITLE OF SPECIALIST IN ORTHOPEDICS AND TRAUMATOLOGY

EVALUATION OF THE PERFORMANCE OF CHATGPT/ARTIFICIAL INTELLIGENCE IN THE MULTIPLE-CHOICE TEST TO OBTAIN THE TITLE OF SPECIALIST IN ORTHOPEDICS AND TRAUMATOLOGY

ABSTRACT Introduction: ChatGPT, an advanced Artificial Intelligence model specialized in natural language processing, shows remarkable abilities, achieving high scores in certification exams in various specialties. This study aims to evaluate ChatGPT’s performance in multiple-choice tests applied...

Full description

Saved in:
Bibliographic Details
Main Authors: LUCAS PLENS DE BRITTO COSTA, DANILO HENRIQUE PIZZO DE CASTRO, RENATO PINHEIRO CORDEIRO, RÔMULO BALLARIN ALBINO
Format: Article
Language:English
Published: Sociedade Brasileira de Ortopedia e Traumatologia 2025-04-01
Series:Acta Ortopédica Brasileira
Subjects:
Artificial Intelligence
Orthopedics
Medical Education
Online Access:http://www.scielo.br/scielo.php?script=sci_arttext&pid=S1413-78522025001000900&lng=en&tlng=en
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:ABSTRACT Introduction: ChatGPT, an advanced Artificial Intelligence model specialized in natural language processing, shows remarkable abilities, achieving high scores in certification exams in various specialties. This study aims to evaluate ChatGPT’s performance in multiple-choice tests applied to obtain specialist certification in Orthopedics and Traumatology. Methods: We used ChatGPT 4.0 to answer 100 questions from the first phase of the Título de Especialista em Ortopedia e Traumatologia 2022 (TEOT) (Specialist in Orthopedics and Traumatology Test). We excluded non-text-based questions. Each question was entered individually into ChatGPT, with a new session initiated for each question. Performance was evaluated regarding number of words and questions’ taxonomic classification. Results: Of the 95 questions analyzed, ChatGPT answered 61.05% correctly and 38.95% incorrectly. There was no statistically significant difference regarding number of words, and ChatGPT’s performance did not vary according to taxonomic level. Conclusion: ChatGPT demonstrated vast knowledge in Orthopedics, with acceptable performance in the TEOT exam. Results suggest ChatGPT’s an educational and clinical resource in Orthopedics, but needs future progress and human supervision for its effective application. Level of evidence IV, Case series.
ISSN:1413-7852

Similar Items