Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry
Introduction: Health care sector, across its numerous domains, has been employing artificial intelligence (AI), encompassing a wide range of tasks involving the various stages of complexity. In fields of research, AI has been widely employed in the scientific writing. The usage is a matter of much d...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wolters Kluwer Medknow Publications
2025-04-01
|
| Series: | Journal of Nature and Science of Medicine |
| Subjects: | |
| Online Access: | https://journals.lww.com/10.4103/jnsm.jnsm_170_24 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849420733119201280 |
|---|---|
| author | Ameena M Siyad A. S. Akhila Subramaniam Ramanarayanan Shabil Mohamed Mustafa Jesline Merly James Priya Babu |
| author_facet | Ameena M Siyad A. S. Akhila Subramaniam Ramanarayanan Shabil Mohamed Mustafa Jesline Merly James Priya Babu |
| author_sort | Ameena M Siyad |
| collection | DOAJ |
| description | Introduction:
Health care sector, across its numerous domains, has been employing artificial intelligence (AI), encompassing a wide range of tasks involving the various stages of complexity. In fields of research, AI has been widely employed in the scientific writing. The usage is a matter of much debate in the recent years. Two names in the AI world stand out: OpenAI’s ChatGPT and Microsoft Copilot. The study was conducted with the objective of assessing the validity of generative AI technologies for identifying relevant literature in dentistry.
Methods:
The study was conducted as a cross-sectional and observational study, during the month of March 2024. The study used both ChatGPT 3.5 and Microsoft Copilot to search dental scientific literature pertaining to topics related to cone-beam computed tomography and cone-beam volumetric tomography and its subdomains including the accuracy, advantages, limitations, and validity. The six components included the (1) authors, (2) reference titles, (3) journal names, (4) publication years, (5) digital object identifiers, and (6) reference links. The accuracy of the reference citations was verified through searching then Medline, Web of Science, Scopus, and Google databases. The data obtained were coded, tabulated, and analyzed using the Statistical Package for Social Sciences for Windows. The accuracy of references to the six components was summarized and expressed as frequency and percentages. The comparison of accuracy between Chat GPT and Copilot was done using the Chi-square test.
Results:
Fifty-five unique references from ChatGPT and 47 from the Copilot were analyzed for validity and accuracy. Concerning the valid titles, 30.90% of the titles provided by ChatGPT and 89.40% provided by Copilot were valid. Author details were correct in 29.10% of the ChatGPT and 74.50% of the Copilot references. Concerning the name of the journal, the corresponding figures were 30.90% and 87.20% respectively.
Conclusions:
Based on our findings, using ChatGPT and Copilot as the sole resource for identifying references to literature reviews in dentistry is not currently recommended. Among the two AI platforms, the Copilot was more accurate and valid, but further research is warranted. |
| format | Article |
| id | doaj-art-93c76323ac8a42ca85796a3d994f5466 |
| institution | Kabale University |
| issn | 2589-627X 2589-6288 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Wolters Kluwer Medknow Publications |
| record_format | Article |
| series | Journal of Nature and Science of Medicine |
| spelling | doaj-art-93c76323ac8a42ca85796a3d994f54662025-08-20T03:31:40ZengWolters Kluwer Medknow PublicationsJournal of Nature and Science of Medicine2589-627X2589-62882025-04-018213513810.4103/jnsm.jnsm_170_24Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in DentistryAmeena M SiyadA. S. AkhilaSubramaniam RamanarayananShabil Mohamed MustafaJesline Merly JamesPriya BabuIntroduction: Health care sector, across its numerous domains, has been employing artificial intelligence (AI), encompassing a wide range of tasks involving the various stages of complexity. In fields of research, AI has been widely employed in the scientific writing. The usage is a matter of much debate in the recent years. Two names in the AI world stand out: OpenAI’s ChatGPT and Microsoft Copilot. The study was conducted with the objective of assessing the validity of generative AI technologies for identifying relevant literature in dentistry. Methods: The study was conducted as a cross-sectional and observational study, during the month of March 2024. The study used both ChatGPT 3.5 and Microsoft Copilot to search dental scientific literature pertaining to topics related to cone-beam computed tomography and cone-beam volumetric tomography and its subdomains including the accuracy, advantages, limitations, and validity. The six components included the (1) authors, (2) reference titles, (3) journal names, (4) publication years, (5) digital object identifiers, and (6) reference links. The accuracy of the reference citations was verified through searching then Medline, Web of Science, Scopus, and Google databases. The data obtained were coded, tabulated, and analyzed using the Statistical Package for Social Sciences for Windows. The accuracy of references to the six components was summarized and expressed as frequency and percentages. The comparison of accuracy between Chat GPT and Copilot was done using the Chi-square test. Results: Fifty-five unique references from ChatGPT and 47 from the Copilot were analyzed for validity and accuracy. Concerning the valid titles, 30.90% of the titles provided by ChatGPT and 89.40% provided by Copilot were valid. Author details were correct in 29.10% of the ChatGPT and 74.50% of the Copilot references. Concerning the name of the journal, the corresponding figures were 30.90% and 87.20% respectively. Conclusions: Based on our findings, using ChatGPT and Copilot as the sole resource for identifying references to literature reviews in dentistry is not currently recommended. Among the two AI platforms, the Copilot was more accurate and valid, but further research is warranted.https://journals.lww.com/10.4103/jnsm.jnsm_170_24accuracyartificial intelligenceliterature searchscientific writingvalidity |
| spellingShingle | Ameena M Siyad A. S. Akhila Subramaniam Ramanarayanan Shabil Mohamed Mustafa Jesline Merly James Priya Babu Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry Journal of Nature and Science of Medicine accuracy artificial intelligence literature search scientific writing validity |
| title | Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry |
| title_full | Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry |
| title_fullStr | Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry |
| title_full_unstemmed | Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry |
| title_short | Assessing the Validity and Accuracy of Artificial Intelligence Technologies for Identifying Relevant Literature in Dentistry |
| title_sort | assessing the validity and accuracy of artificial intelligence technologies for identifying relevant literature in dentistry |
| topic | accuracy artificial intelligence literature search scientific writing validity |
| url | https://journals.lww.com/10.4103/jnsm.jnsm_170_24 |
| work_keys_str_mv | AT ameenamsiyad assessingthevalidityandaccuracyofartificialintelligencetechnologiesforidentifyingrelevantliteratureindentistry AT asakhila assessingthevalidityandaccuracyofartificialintelligencetechnologiesforidentifyingrelevantliteratureindentistry AT subramaniamramanarayanan assessingthevalidityandaccuracyofartificialintelligencetechnologiesforidentifyingrelevantliteratureindentistry AT shabilmohamedmustafa assessingthevalidityandaccuracyofartificialintelligencetechnologiesforidentifyingrelevantliteratureindentistry AT jeslinemerlyjames assessingthevalidityandaccuracyofartificialintelligencetechnologiesforidentifyingrelevantliteratureindentistry AT priyababu assessingthevalidityandaccuracyofartificialintelligencetechnologiesforidentifyingrelevantliteratureindentistry |