Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector
Abstract Several researchers have investigated the consequences of using ChatGPT in the education industry. Their findings raised doubts regarding the probable effects that ChatGPT may have on the academia. As such, the present study aimed to assess the ability of three methods, namely: (1) academic...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-04-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-95387-y |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850153952128008192 |
|---|---|
| author | Matheel AL-Rawas Omar Abdul Jabbar Abdul Qader Nurul Hanim Othman Noor Huda Ismail Rosnani Mamat Mohamad Syahrizal Halim Johari Yap Abdullah Tahir Yusuf Noorani |
| author_facet | Matheel AL-Rawas Omar Abdul Jabbar Abdul Qader Nurul Hanim Othman Noor Huda Ismail Rosnani Mamat Mohamad Syahrizal Halim Johari Yap Abdullah Tahir Yusuf Noorani |
| author_sort | Matheel AL-Rawas |
| collection | DOAJ |
| description | Abstract Several researchers have investigated the consequences of using ChatGPT in the education industry. Their findings raised doubts regarding the probable effects that ChatGPT may have on the academia. As such, the present study aimed to assess the ability of three methods, namely: (1) academicians (senior and young), (2) three AI detectors (GPT-2 output detector, Writefull GPT detector, and GPTZero) and (3) one plagiarism detector, to differentiate between human- and ChatGPT-written abstracts. A total of 160 abstracts were assessed by those three methods. Two senior and two young academicians used a newly developed rubric to assess the type and quality of 80 human-written and 80 ChatGPT-written abstracts. The results were statistically analysed using crosstabulation and chi-square analysis. Bivariate correlation and accuracy of the methods were assessed. The findings demonstrated that all the three methods made a different variety of incorrect assumptions. The level of the academician experience may play a role in the detection ability with senior academician 1 demonstrating superior accuracy. GPTZero AI and similarity detectors were very good at accurately identifying the abstracts origin. In terms of abstract type, every variable positively correlated, except in the case of similarity detectors (p < 0.05). Human-AI collaborations may significantly benefit the identification of the abstract origins. |
| format | Article |
| id | doaj-art-cfa77845888643feae4e7f360e2a3733 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-cfa77845888643feae4e7f360e2a37332025-08-20T02:25:35ZengNature PortfolioScientific Reports2045-23222025-04-0115111010.1038/s41598-025-95387-yIdentification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detectorMatheel AL-Rawas0Omar Abdul Jabbar Abdul Qader1Nurul Hanim Othman2Noor Huda Ismail3Rosnani Mamat4Mohamad Syahrizal Halim5Johari Yap Abdullah6Tahir Yusuf Noorani7Prosthodontic Unit, School of Dental Sciences, Universiti Sains MalaysiaCollege of Dentistry, Al Mashreq UniversityProsthodontic Unit, School of Dental Sciences, Universiti Sains MalaysiaProsthodontic Unit, School of Dental Sciences, Universiti Sains MalaysiaHospital Pakar Universiti Sains MalaysiaHospital Pakar Universiti Sains MalaysiaCraniofacial Imaging Laboratory, School of Dental Sciences, Universiti Sains MalaysiaHospital Pakar Universiti Sains MalaysiaAbstract Several researchers have investigated the consequences of using ChatGPT in the education industry. Their findings raised doubts regarding the probable effects that ChatGPT may have on the academia. As such, the present study aimed to assess the ability of three methods, namely: (1) academicians (senior and young), (2) three AI detectors (GPT-2 output detector, Writefull GPT detector, and GPTZero) and (3) one plagiarism detector, to differentiate between human- and ChatGPT-written abstracts. A total of 160 abstracts were assessed by those three methods. Two senior and two young academicians used a newly developed rubric to assess the type and quality of 80 human-written and 80 ChatGPT-written abstracts. The results were statistically analysed using crosstabulation and chi-square analysis. Bivariate correlation and accuracy of the methods were assessed. The findings demonstrated that all the three methods made a different variety of incorrect assumptions. The level of the academician experience may play a role in the detection ability with senior academician 1 demonstrating superior accuracy. GPTZero AI and similarity detectors were very good at accurately identifying the abstracts origin. In terms of abstract type, every variable positively correlated, except in the case of similarity detectors (p < 0.05). Human-AI collaborations may significantly benefit the identification of the abstract origins.https://doi.org/10.1038/s41598-025-95387-yChatGPTAI-based language modelGenerative pre-trained transformerAI-written contentEducatorsScientific abstracts |
| spellingShingle | Matheel AL-Rawas Omar Abdul Jabbar Abdul Qader Nurul Hanim Othman Noor Huda Ismail Rosnani Mamat Mohamad Syahrizal Halim Johari Yap Abdullah Tahir Yusuf Noorani Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector Scientific Reports ChatGPT AI-based language model Generative pre-trained transformer AI-written content Educators Scientific abstracts |
| title | Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector |
| title_full | Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector |
| title_fullStr | Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector |
| title_full_unstemmed | Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector |
| title_short | Identification of dental related ChatGPT generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector |
| title_sort | identification of dental related chatgpt generated abstracts by senior and young academicians versus artificial intelligence detectors and a similarity detector |
| topic | ChatGPT AI-based language model Generative pre-trained transformer AI-written content Educators Scientific abstracts |
| url | https://doi.org/10.1038/s41598-025-95387-y |
| work_keys_str_mv | AT matheelalrawas identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT omarabduljabbarabdulqader identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT nurulhanimothman identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT noorhudaismail identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT rosnanimamat identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT mohamadsyahrizalhalim identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT johariyapabdullah identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector AT tahiryusufnoorani identificationofdentalrelatedchatgptgeneratedabstractsbyseniorandyoungacademiciansversusartificialintelligencedetectorsandasimilaritydetector |