ChatGPT vs. Orthopedic Residents! Who is the Winner?
Objective: In recent advancements in artificial intelligence, ChatGPT by OpenAI has emerged as a versatile tool capable of performing various tasks; however, its application in medicine is challenged by complexities and limitations in accuracy. This article aims to compare ChatGPT’s performance with...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Galenos Publishing House
2024-04-01
|
| Series: | Gazi Medical Journal |
| Subjects: | |
| Online Access: | https://gazimedj.com/articles/chatgpt-vs-orthopedic-residents-who-is-the-winner/doi/gmj.2024.4067 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1841525519260057600 |
|---|---|
| author | Semih Yaş Asim Ahmadov Alim Can Baymurat Mehmet Ali Tokgöz Secdegül Coşkun Yaş Mustafa Odluyurt Tolga Tolunay |
| author_facet | Semih Yaş Asim Ahmadov Alim Can Baymurat Mehmet Ali Tokgöz Secdegül Coşkun Yaş Mustafa Odluyurt Tolga Tolunay |
| author_sort | Semih Yaş |
| collection | DOAJ |
| description | Objective: In recent advancements in artificial intelligence, ChatGPT by OpenAI has emerged as a versatile tool capable of performing various tasks; however, its application in medicine is challenged by complexities and limitations in accuracy. This article aims to compare ChatGPT’s performance with orthopedic residents at Gazi University in a multiple-choice exam to assess its applicability and reliability in the field of orthopedics.
Methods: In this observational study conducted at Gazi University, 31 orthopedic residents were stratified by experience level and assessed using a 50-question multiple-choice test on various orthopedic topics. The study also evaluated ChatGPT 3.5’s responses to the same questions, focusing on both the correctness and reasoning behind the answers.
Results: Orthopedic residents tested, ranging from 6 months to 5 years in experience, scored between 23 and 40 out of 50 in a multiplechoice exam, with a mean score of 30.81, varying by seniority. ChatGPT provided correct answers for 25 out of 50 questions, showing consistency in different languages and times, but also exhibited limitations by giving incorrect responses or stating that the correct answer was not among the choices for some questions.
Conclusion: While ChatGPT can accurately answer some theoretical questions, its effectiveness is limited in interpretive scenarios and in situations with multiple variables, although its accuracy may improve with updates over time. |
| format | Article |
| id | doaj-art-14343373ea1d4586ba18e41219e1fcc9 |
| institution | Kabale University |
| issn | 2147-2092 |
| language | English |
| publishDate | 2024-04-01 |
| publisher | Galenos Publishing House |
| record_format | Article |
| series | Gazi Medical Journal |
| spelling | doaj-art-14343373ea1d4586ba18e41219e1fcc92025-01-17T10:39:39ZengGalenos Publishing HouseGazi Medical Journal2147-20922024-04-0135218619110.12996/gmj.2024.4067ChatGPT vs. Orthopedic Residents! Who is the Winner?Semih Yaş0https://orcid.org/0000-0001-7823-3400Asim Ahmadov1https://orcid.org/0000-0002-9534-3131Alim Can Baymurat2https://orcid.org/0000-0002-0062-621XMehmet Ali Tokgöz3https://orcid.org/0000-0002-4056-3743Secdegül Coşkun Yaş4https://orcid.org/0000-0002-8936-3988Mustafa Odluyurt5https://orcid.org/0000-0003-1039-8430Tolga Tolunay6https://orcid.org/0000-0003-1998-3695Gazi University Faculty of Medicine, Department of Orthopedics and Traumatology, Ankara, TürkiyeGazi University Faculty of Medicine, Department of Orthopedics and Traumatology, Ankara, TürkiyeGazi University Faculty of Medicine, Department of Orthopedics and Traumatology, Ankara, TürkiyeGazi University Faculty of Medicine, Department of Orthopedics and Traumatology, Ankara, TürkiyeAnkara Training and Research Hospital, Clinic of Emergency Medicine, Ankara, TürkiyeZonguldak Çaycuma State Hospital, Clinic of Orthopedics and Traumatology, Zonguldak, TürkiyeGazi University Faculty of Medicine, Department of Orthopedics and Traumatology, Ankara, TürkiyeObjective: In recent advancements in artificial intelligence, ChatGPT by OpenAI has emerged as a versatile tool capable of performing various tasks; however, its application in medicine is challenged by complexities and limitations in accuracy. This article aims to compare ChatGPT’s performance with orthopedic residents at Gazi University in a multiple-choice exam to assess its applicability and reliability in the field of orthopedics. Methods: In this observational study conducted at Gazi University, 31 orthopedic residents were stratified by experience level and assessed using a 50-question multiple-choice test on various orthopedic topics. The study also evaluated ChatGPT 3.5’s responses to the same questions, focusing on both the correctness and reasoning behind the answers. Results: Orthopedic residents tested, ranging from 6 months to 5 years in experience, scored between 23 and 40 out of 50 in a multiplechoice exam, with a mean score of 30.81, varying by seniority. ChatGPT provided correct answers for 25 out of 50 questions, showing consistency in different languages and times, but also exhibited limitations by giving incorrect responses or stating that the correct answer was not among the choices for some questions. Conclusion: While ChatGPT can accurately answer some theoretical questions, its effectiveness is limited in interpretive scenarios and in situations with multiple variables, although its accuracy may improve with updates over time.https://gazimedj.com/articles/chatgpt-vs-orthopedic-residents-who-is-the-winner/doi/gmj.2024.4067chatgptartificial intelligenceorthopedicstraumatology |
| spellingShingle | Semih Yaş Asim Ahmadov Alim Can Baymurat Mehmet Ali Tokgöz Secdegül Coşkun Yaş Mustafa Odluyurt Tolga Tolunay ChatGPT vs. Orthopedic Residents! Who is the Winner? Gazi Medical Journal chatgpt artificial intelligence orthopedics traumatology |
| title | ChatGPT vs. Orthopedic Residents! Who is the Winner? |
| title_full | ChatGPT vs. Orthopedic Residents! Who is the Winner? |
| title_fullStr | ChatGPT vs. Orthopedic Residents! Who is the Winner? |
| title_full_unstemmed | ChatGPT vs. Orthopedic Residents! Who is the Winner? |
| title_short | ChatGPT vs. Orthopedic Residents! Who is the Winner? |
| title_sort | chatgpt vs orthopedic residents who is the winner |
| topic | chatgpt artificial intelligence orthopedics traumatology |
| url | https://gazimedj.com/articles/chatgpt-vs-orthopedic-residents-who-is-the-winner/doi/gmj.2024.4067 |
| work_keys_str_mv | AT semihyas chatgptvsorthopedicresidentswhoisthewinner AT asimahmadov chatgptvsorthopedicresidentswhoisthewinner AT alimcanbaymurat chatgptvsorthopedicresidentswhoisthewinner AT mehmetalitokgoz chatgptvsorthopedicresidentswhoisthewinner AT secdegulcoskunyas chatgptvsorthopedicresidentswhoisthewinner AT mustafaodluyurt chatgptvsorthopedicresidentswhoisthewinner AT tolgatolunay chatgptvsorthopedicresidentswhoisthewinner |