The arrangement of response options in multiple-choice test items: verticality is not always better
Multiple-choice tests are widely used to measure learning outcomes. Consequently, constructing high-quality test items is critical, and many authors have advanced item-writing guidelines. One frequently mentioned guideline is to arrange the response options vertically. However, evidence to support t...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2024-10-01
|
| Series: | Frontiers in Education |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/feduc.2024.1442047/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850185353016639488 |
|---|---|
| author | Consuelo Soto Séverin Lions Séverin Lions Gabriel Ortega Melissa Arjona María Paz Blanco Pablo Dartnell Pablo Dartnell Pablo Dartnell |
| author_facet | Consuelo Soto Séverin Lions Séverin Lions Gabriel Ortega Melissa Arjona María Paz Blanco Pablo Dartnell Pablo Dartnell Pablo Dartnell |
| author_sort | Consuelo Soto |
| collection | DOAJ |
| description | Multiple-choice tests are widely used to measure learning outcomes. Consequently, constructing high-quality test items is critical, and many authors have advanced item-writing guidelines. One frequently mentioned guideline is to arrange the response options vertically. However, evidence to support this recommendation is scarce and has only been obtained for items with text-based options. This study aimed at understanding whether the arrangement of options affects performance at solving items with large-sized options, such as graphs and pictures, using objective and subjective measures. Fifty-seven high-school students completed a multiple-choice science and mathematics test with 24 four-choice items, options being graphs or pictures presented in one of four arrangements: vertical without page break, vertical with page break, Z, and inverted N. Response accuracy, response time, and perceived difficulty were obtained for each item. Subsequently, students participated in a cognitive interview about their experiences, practices, perceptions, and beliefs regarding the arrangement of options. Objective measures show that the arrangement of options hardly affected performance, the only effect being that vertical condition with page break resulted in significantly longer response times. Subjective measures show that most students favored the vertical arrangement they consider more common but negatively perceived vertical condition with page break and considered squared arrangements (Z, inverted N) to facilitate visual exploration and comparison between options, as opposed to the vertical arrangement. Results suggest that the vertical arrangement does not offer clear advantages over squared arrangements for items with large-sized options. |
| format | Article |
| id | doaj-art-523902b621f841fdadcd6df60d717da9 |
| institution | OA Journals |
| issn | 2504-284X |
| language | English |
| publishDate | 2024-10-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Education |
| spelling | doaj-art-523902b621f841fdadcd6df60d717da92025-08-20T02:16:45ZengFrontiers Media S.A.Frontiers in Education2504-284X2024-10-01910.3389/feduc.2024.14420471442047The arrangement of response options in multiple-choice test items: verticality is not always betterConsuelo Soto0Séverin Lions1Séverin Lions2Gabriel Ortega3Melissa Arjona4María Paz Blanco5Pablo Dartnell6Pablo Dartnell7Pablo Dartnell8Center for Advanced Research in Education (FB0003), Institute of Education, Universidad de Chile, Santiago, ChileCenter for Advanced Research in Education (FB0003), Institute of Education, Universidad de Chile, Santiago, ChileDepartamento de Evaluación, Medición y Registro Educacional (DEMRE), Universidad de Chile, Santiago, ChileCenter for Advanced Research in Education (FB0003), Institute of Education, Universidad de Chile, Santiago, ChileCenter for Advanced Research in Education (FB0003), Institute of Education, Universidad de Chile, Santiago, ChileCenter for Advanced Research in Education (FB0003), Institute of Education, Universidad de Chile, Santiago, ChileCenter for Advanced Research in Education (FB0003), Institute of Education, Universidad de Chile, Santiago, ChileCenter for Mathematical Modeling (FB210005), Universidad de Chile, Santiago, ChileDepartment of Mathematical Engineering, Universidad de Chile, Santiago, ChileMultiple-choice tests are widely used to measure learning outcomes. Consequently, constructing high-quality test items is critical, and many authors have advanced item-writing guidelines. One frequently mentioned guideline is to arrange the response options vertically. However, evidence to support this recommendation is scarce and has only been obtained for items with text-based options. This study aimed at understanding whether the arrangement of options affects performance at solving items with large-sized options, such as graphs and pictures, using objective and subjective measures. Fifty-seven high-school students completed a multiple-choice science and mathematics test with 24 four-choice items, options being graphs or pictures presented in one of four arrangements: vertical without page break, vertical with page break, Z, and inverted N. Response accuracy, response time, and perceived difficulty were obtained for each item. Subsequently, students participated in a cognitive interview about their experiences, practices, perceptions, and beliefs regarding the arrangement of options. Objective measures show that the arrangement of options hardly affected performance, the only effect being that vertical condition with page break resulted in significantly longer response times. Subjective measures show that most students favored the vertical arrangement they consider more common but negatively perceived vertical condition with page break and considered squared arrangements (Z, inverted N) to facilitate visual exploration and comparison between options, as opposed to the vertical arrangement. Results suggest that the vertical arrangement does not offer clear advantages over squared arrangements for items with large-sized options.https://www.frontiersin.org/articles/10.3389/feduc.2024.1442047/fullmultiple-choiceitem-writing guidelinesitem formattingresponse optionsspatial arrangement |
| spellingShingle | Consuelo Soto Séverin Lions Séverin Lions Gabriel Ortega Melissa Arjona María Paz Blanco Pablo Dartnell Pablo Dartnell Pablo Dartnell The arrangement of response options in multiple-choice test items: verticality is not always better Frontiers in Education multiple-choice item-writing guidelines item formatting response options spatial arrangement |
| title | The arrangement of response options in multiple-choice test items: verticality is not always better |
| title_full | The arrangement of response options in multiple-choice test items: verticality is not always better |
| title_fullStr | The arrangement of response options in multiple-choice test items: verticality is not always better |
| title_full_unstemmed | The arrangement of response options in multiple-choice test items: verticality is not always better |
| title_short | The arrangement of response options in multiple-choice test items: verticality is not always better |
| title_sort | arrangement of response options in multiple choice test items verticality is not always better |
| topic | multiple-choice item-writing guidelines item formatting response options spatial arrangement |
| url | https://www.frontiersin.org/articles/10.3389/feduc.2024.1442047/full |
| work_keys_str_mv | AT consuelosoto thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT severinlions thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT severinlions thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT gabrielortega thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT melissaarjona thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT mariapazblanco thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT pablodartnell thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT pablodartnell thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT pablodartnell thearrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT consuelosoto arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT severinlions arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT severinlions arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT gabrielortega arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT melissaarjona arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT mariapazblanco arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT pablodartnell arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT pablodartnell arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter AT pablodartnell arrangementofresponseoptionsinmultiplechoicetestitemsverticalityisnotalwaysbetter |