Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish
Multiple-choice questions (MCQs) provide a widely used means of assessing reading comprehension. The automatic generation of such MCQs is a challenging language-technological problem that also has interesting educational applications. This article presents several methods for automatically producin...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Linköping University Electronic Press
2024-12-01
|
| Series: | Northern European Journal of Language Technology |
| Online Access: | https://nejlt.ep.liu.se/article/view/4886 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832591221520859136 |
|---|---|
| author | Dmytro Kalpakchi Johan Boye |
| author_facet | Dmytro Kalpakchi Johan Boye |
| author_sort | Dmytro Kalpakchi |
| collection | DOAJ |
| description |
Multiple-choice questions (MCQs) provide a widely used means of assessing reading comprehension. The automatic generation of such MCQs is a challenging language-technological problem that also has interesting educational applications. This article presents several methods for automatically producing reading comprehension questions MCQs from Swedish text. Unlike previous approaches, we construct models to generate the whole MCQ in one go, rather than using a pipeline architecture. Furthermore, we propose a two-stage method for evaluating the quality of the generated MCQs, first evaluating on carefully designed single-sentence texts, and then on texts from the SFI national exams. An extensive evaluation of the MCQ-generating capabilities of 12 different models, using this two-stage scheme, reveals that GPT-based models surpass smaller models that have been fine-tuned using small-scale datasets on this specific problem.
|
| format | Article |
| id | doaj-art-3890733c936f4e0cbf8fbf38a4f33c28 |
| institution | Kabale University |
| issn | 2000-1533 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Linköping University Electronic Press |
| record_format | Article |
| series | Northern European Journal of Language Technology |
| spelling | doaj-art-3890733c936f4e0cbf8fbf38a4f33c282025-01-22T15:24:16ZengLinköping University Electronic PressNorthern European Journal of Language Technology2000-15332024-12-0110110.3384/nejlt.2000-1533.2024.4886Generation and Evaluation of Multiple-choice Reading Comprehension Questions for SwedishDmytro Kalpakchi0Johan Boye1KTH Royal Institute of TechnologyKTH Royal Institute of Technology Multiple-choice questions (MCQs) provide a widely used means of assessing reading comprehension. The automatic generation of such MCQs is a challenging language-technological problem that also has interesting educational applications. This article presents several methods for automatically producing reading comprehension questions MCQs from Swedish text. Unlike previous approaches, we construct models to generate the whole MCQ in one go, rather than using a pipeline architecture. Furthermore, we propose a two-stage method for evaluating the quality of the generated MCQs, first evaluating on carefully designed single-sentence texts, and then on texts from the SFI national exams. An extensive evaluation of the MCQ-generating capabilities of 12 different models, using this two-stage scheme, reveals that GPT-based models surpass smaller models that have been fine-tuned using small-scale datasets on this specific problem. https://nejlt.ep.liu.se/article/view/4886 |
| spellingShingle | Dmytro Kalpakchi Johan Boye Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish Northern European Journal of Language Technology |
| title | Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish |
| title_full | Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish |
| title_fullStr | Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish |
| title_full_unstemmed | Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish |
| title_short | Generation and Evaluation of Multiple-choice Reading Comprehension Questions for Swedish |
| title_sort | generation and evaluation of multiple choice reading comprehension questions for swedish |
| url | https://nejlt.ep.liu.se/article/view/4886 |
| work_keys_str_mv | AT dmytrokalpakchi generationandevaluationofmultiplechoicereadingcomprehensionquestionsforswedish AT johanboye generationandevaluationofmultiplechoicereadingcomprehensionquestionsforswedish |