AI detectors are poor western blot classifiers: a study of accuracy and predictive values
The recent rise of generative artificial intelligence (AI) capable of creating scientific images presents a challenge in the fight against academic fraud. This study evaluates the efficacy of three free web-based AI detectors in identifying AI-generated images of western blots, which is a very commo...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2025-02-01
|
| Series: | PeerJ |
| Subjects: | |
| Online Access: | https://peerj.com/articles/18988.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850191335028424704 |
|---|---|
| author | Romain-Daniel Gosselin |
| author_facet | Romain-Daniel Gosselin |
| author_sort | Romain-Daniel Gosselin |
| collection | DOAJ |
| description | The recent rise of generative artificial intelligence (AI) capable of creating scientific images presents a challenge in the fight against academic fraud. This study evaluates the efficacy of three free web-based AI detectors in identifying AI-generated images of western blots, which is a very common technique in biology. We tested these detectors on AI-generated western blot images (n = 48, created using ChatGPT 4) and on authentic western blots (n = 48, from articles published before the rise of generative AI). Each detector returned a very different sensitivity (Is It AI?: 0.9583; Hive Moderation: 0.1875; and Illuminarty: 0.7083) and specificity (Is It AI?: 0.5417; Hive Moderation: 0.8750; and Illuminarty: 0.4167), and the predicted positive predictive value (PPV) for each was low. This suggests significant challenges in confidently determining image authenticity based solely on the current free AI detectors. Reducing the size of western blots reduced the sensitivity, increased the specificity, and did not markedly affect the accuracy of the three detectors, and only slightly improved the PPV of one detector (Is It AI?). These findings highlight the risks of relying on generic, freely available detectors that lack sufficient reliability, and demonstrate the urgent need for more robust detectors that are specifically trained on scientific contents such as western blot images. |
| format | Article |
| id | doaj-art-4f37982e5b7b4334b9d4ec07f26a1f53 |
| institution | OA Journals |
| issn | 2167-8359 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ |
| spelling | doaj-art-4f37982e5b7b4334b9d4ec07f26a1f532025-08-20T02:14:57ZengPeerJ Inc.PeerJ2167-83592025-02-0113e1898810.7717/peerj.18988AI detectors are poor western blot classifiers: a study of accuracy and predictive valuesRomain-Daniel GosselinThe recent rise of generative artificial intelligence (AI) capable of creating scientific images presents a challenge in the fight against academic fraud. This study evaluates the efficacy of three free web-based AI detectors in identifying AI-generated images of western blots, which is a very common technique in biology. We tested these detectors on AI-generated western blot images (n = 48, created using ChatGPT 4) and on authentic western blots (n = 48, from articles published before the rise of generative AI). Each detector returned a very different sensitivity (Is It AI?: 0.9583; Hive Moderation: 0.1875; and Illuminarty: 0.7083) and specificity (Is It AI?: 0.5417; Hive Moderation: 0.8750; and Illuminarty: 0.4167), and the predicted positive predictive value (PPV) for each was low. This suggests significant challenges in confidently determining image authenticity based solely on the current free AI detectors. Reducing the size of western blots reduced the sensitivity, increased the specificity, and did not markedly affect the accuracy of the three detectors, and only slightly improved the PPV of one detector (Is It AI?). These findings highlight the risks of relying on generic, freely available detectors that lack sufficient reliability, and demonstrate the urgent need for more robust detectors that are specifically trained on scientific contents such as western blot images.https://peerj.com/articles/18988.pdfResearch integrityAI detectionPaper millsFraudResearch ethicsAccuracy study |
| spellingShingle | Romain-Daniel Gosselin AI detectors are poor western blot classifiers: a study of accuracy and predictive values PeerJ Research integrity AI detection Paper mills Fraud Research ethics Accuracy study |
| title | AI detectors are poor western blot classifiers: a study of accuracy and predictive values |
| title_full | AI detectors are poor western blot classifiers: a study of accuracy and predictive values |
| title_fullStr | AI detectors are poor western blot classifiers: a study of accuracy and predictive values |
| title_full_unstemmed | AI detectors are poor western blot classifiers: a study of accuracy and predictive values |
| title_short | AI detectors are poor western blot classifiers: a study of accuracy and predictive values |
| title_sort | ai detectors are poor western blot classifiers a study of accuracy and predictive values |
| topic | Research integrity AI detection Paper mills Fraud Research ethics Accuracy study |
| url | https://peerj.com/articles/18988.pdf |
| work_keys_str_mv | AT romaindanielgosselin aidetectorsarepoorwesternblotclassifiersastudyofaccuracyandpredictivevalues |