The use of ChatGPT in assessment
ChatGPT has surged interest to cause people to look for its use in different tasks. However, before allowing it to replace humans, its capabilities should be investigated. As ChatGPT has potential for use in testing and assessment, this study aims to investigate the questions generated by ChatGPT by...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
izzet kara
2024-09-01
|
| Series: | International Journal of Assessment Tools in Education |
| Subjects: | |
| Online Access: | https://dergipark.org.tr/tr/download/article-file/3490005 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850067799120019456 |
|---|---|
| author | Mehmet Kanık |
| author_facet | Mehmet Kanık |
| author_sort | Mehmet Kanık |
| collection | DOAJ |
| description | ChatGPT has surged interest to cause people to look for its use in different tasks. However, before allowing it to replace humans, its capabilities should be investigated. As ChatGPT has potential for use in testing and assessment, this study aims to investigate the questions generated by ChatGPT by comparing them to those written by a course instructor. To investigate this issue, this study involved 36 junior students who took a practice test including 20 multiple-choice items generated by ChatGPT and 20 others by the course instructor, resulting in a 40-item test. Results indicate that there was an acceptable degree of consistency between the ChatGPT and the course instructor. Post-hoc analyses point to consistency between the instructor and the chatbot in item difficulty, yet the chatbot’s results were weaker in item discrimination power and distractor analysis. This indicates that ChatGPT can potentially generate multiple-choice exams similar to those of the course instructor. |
| format | Article |
| id | doaj-art-bbcb16928dd64e69b240eb6f0d5639da |
| institution | DOAJ |
| issn | 2148-7456 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | izzet kara |
| record_format | Article |
| series | International Journal of Assessment Tools in Education |
| spelling | doaj-art-bbcb16928dd64e69b240eb6f0d5639da2025-08-20T02:48:12Zengizzet karaInternational Journal of Assessment Tools in Education2148-74562024-09-0111360862110.21449/ijate.1379647618The use of ChatGPT in assessmentMehmet Kanık0https://orcid.org/0000-0002-1737-7678Uluslararası Final ÜniversitesiChatGPT has surged interest to cause people to look for its use in different tasks. However, before allowing it to replace humans, its capabilities should be investigated. As ChatGPT has potential for use in testing and assessment, this study aims to investigate the questions generated by ChatGPT by comparing them to those written by a course instructor. To investigate this issue, this study involved 36 junior students who took a practice test including 20 multiple-choice items generated by ChatGPT and 20 others by the course instructor, resulting in a 40-item test. Results indicate that there was an acceptable degree of consistency between the ChatGPT and the course instructor. Post-hoc analyses point to consistency between the instructor and the chatbot in item difficulty, yet the chatbot’s results were weaker in item discrimination power and distractor analysis. This indicates that ChatGPT can potentially generate multiple-choice exams similar to those of the course instructor.https://dergipark.org.tr/tr/download/article-file/3490005chatgptaiassessmentitem-generationitem analysischatgptaiassessmentitem-generationitem analysis |
| spellingShingle | Mehmet Kanık The use of ChatGPT in assessment International Journal of Assessment Tools in Education chatgpt ai assessment item-generation item analysis chatgpt ai assessment item-generation item analysis |
| title | The use of ChatGPT in assessment |
| title_full | The use of ChatGPT in assessment |
| title_fullStr | The use of ChatGPT in assessment |
| title_full_unstemmed | The use of ChatGPT in assessment |
| title_short | The use of ChatGPT in assessment |
| title_sort | use of chatgpt in assessment |
| topic | chatgpt ai assessment item-generation item analysis chatgpt ai assessment item-generation item analysis |
| url | https://dergipark.org.tr/tr/download/article-file/3490005 |
| work_keys_str_mv | AT mehmetkanık theuseofchatgptinassessment AT mehmetkanık useofchatgptinassessment |