Detecting LLM-assisted writing in scientific communication: Are we there yet?
Large Language Models (LLMs), exemplified by ChatGPT, have significantly reshaped text generation, particularly in the realm of writing assistance. While ethical considerations underscore the importance of transparently acknowledging LLM use, especially in scientific communication, genuine acknowled...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Sciendo
2024-07-01
|
| Series: | Journal of Data and Information Science |
| Subjects: | |
| Online Access: | https://doi.org/10.2478/jdis-2024-0020 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849317018586578944 |
|---|---|
| author | Lazebnik Teddy Rosenfeld Ariel |
| author_facet | Lazebnik Teddy Rosenfeld Ariel |
| author_sort | Lazebnik Teddy |
| collection | DOAJ |
| description | Large Language Models (LLMs), exemplified by ChatGPT, have significantly reshaped text generation, particularly in the realm of writing assistance. While ethical considerations underscore the importance of transparently acknowledging LLM use, especially in scientific communication, genuine acknowledgment remains infrequent. A potential avenue to encourage accurate acknowledging of LLM-assisted writing involves employing automated detectors. Our evaluation of four cutting-edge LLM-generated text detectors reveals their suboptimal performance compared to a simple ad-hoc detector designed to identify abrupt writing style changes around the time of LLM proliferation. We contend that the development of specialized detectors exclusively dedicated to LLM-assisted writing detection is necessary. Such detectors could play a crucial role in fostering more authentic recognition of LLM involvement in scientific communication, addressing the current challenges in acknowledgment practices. |
| format | Article |
| id | doaj-art-b7bf73064a1144bcb9b385dd0bed9033 |
| institution | Kabale University |
| issn | 2543-683X |
| language | English |
| publishDate | 2024-07-01 |
| publisher | Sciendo |
| record_format | Article |
| series | Journal of Data and Information Science |
| spelling | doaj-art-b7bf73064a1144bcb9b385dd0bed90332025-08-20T03:51:24ZengSciendoJournal of Data and Information Science2543-683X2024-07-019341310.2478/jdis-2024-0020Detecting LLM-assisted writing in scientific communication: Are we there yet?Lazebnik Teddy0Rosenfeld Ariel11Department of Mathematics, Ariel University, Ariel, Israel3Department of Information Science, Bar Ilan University, Ramat Gan, IsraelLarge Language Models (LLMs), exemplified by ChatGPT, have significantly reshaped text generation, particularly in the realm of writing assistance. While ethical considerations underscore the importance of transparently acknowledging LLM use, especially in scientific communication, genuine acknowledgment remains infrequent. A potential avenue to encourage accurate acknowledging of LLM-assisted writing involves employing automated detectors. Our evaluation of four cutting-edge LLM-generated text detectors reveals their suboptimal performance compared to a simple ad-hoc detector designed to identify abrupt writing style changes around the time of LLM proliferation. We contend that the development of specialized detectors exclusively dedicated to LLM-assisted writing detection is necessary. Such detectors could play a crucial role in fostering more authentic recognition of LLM involvement in scientific communication, addressing the current challenges in acknowledgment practices.https://doi.org/10.2478/jdis-2024-0020llm-assisted writingscientific communicationwriting style |
| spellingShingle | Lazebnik Teddy Rosenfeld Ariel Detecting LLM-assisted writing in scientific communication: Are we there yet? Journal of Data and Information Science llm-assisted writing scientific communication writing style |
| title | Detecting LLM-assisted writing in scientific communication: Are we there yet? |
| title_full | Detecting LLM-assisted writing in scientific communication: Are we there yet? |
| title_fullStr | Detecting LLM-assisted writing in scientific communication: Are we there yet? |
| title_full_unstemmed | Detecting LLM-assisted writing in scientific communication: Are we there yet? |
| title_short | Detecting LLM-assisted writing in scientific communication: Are we there yet? |
| title_sort | detecting llm assisted writing in scientific communication are we there yet |
| topic | llm-assisted writing scientific communication writing style |
| url | https://doi.org/10.2478/jdis-2024-0020 |
| work_keys_str_mv | AT lazebnikteddy detectingllmassistedwritinginscientificcommunicationarewethereyet AT rosenfeldariel detectingllmassistedwritinginscientificcommunicationarewethereyet |