Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries
Understanding factors that affect inconsistent responding in mixed-worded scales is crucial for ensuring the validity of survey score outcomes. This study investigated whether the assessment mode, defined as participating either in a digital or a paper-based achievement test, was associated with the...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-06-01
|
| Series: | Frontiers in Education |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/feduc.2025.1595648/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850130941468475392 |
|---|---|
| author | Evi Konstantinidou Evi Konstantinidou Michalis P. Michaelides |
| author_facet | Evi Konstantinidou Evi Konstantinidou Michalis P. Michaelides |
| author_sort | Evi Konstantinidou |
| collection | DOAJ |
| description | Understanding factors that affect inconsistent responding in mixed-worded scales is crucial for ensuring the validity of survey score outcomes. This study investigated whether the assessment mode, defined as participating either in a digital or a paper-based achievement test, was associated with the prevalence of inconsistent responding on a mixed-worded scale on a questionnaire administered immediately after. Data were used from 4th- and 8th-grade students from 16 countries participating in the 2019 Trends in International Mathematics and Science Study (TIMSS). The self-reported mixed-worded scale measured self-concept in mathematics and was administered in paper format to all students. The study employed the mean absolute difference (MAD) and factor mixture analysis (FMA) methods to identify inconsistent respondents. Although cross-cultural variation was observed, the 4th-graders who had taken the computer-based assessment prior to the questionnaire engaged in a slightly higher frequency of inconsistent responses. Among 8th-graders, the difference was smaller and, in several country samples, reversed. Larger prevalence of inconsistent responding was found with the FMA approach. The study emphasizes the need for further research on inconsistent responding across different assessment modes and contexts, suggesting implications for survey methodology and international studies. |
| format | Article |
| id | doaj-art-8acc63ebe77e43efae3a59fd0ad060f7 |
| institution | OA Journals |
| issn | 2504-284X |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Education |
| spelling | doaj-art-8acc63ebe77e43efae3a59fd0ad060f72025-08-20T02:32:33ZengFrontiers Media S.A.Frontiers in Education2504-284X2025-06-011010.3389/feduc.2025.15956481595648Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countriesEvi Konstantinidou0Evi Konstantinidou1Michalis P. Michaelides2Department of Psychology, University of Cyprus, Nicosia, CyprusDepartment of Education, University of Nicosia, Nicosia, CyprusDepartment of Psychology, University of Cyprus, Nicosia, CyprusUnderstanding factors that affect inconsistent responding in mixed-worded scales is crucial for ensuring the validity of survey score outcomes. This study investigated whether the assessment mode, defined as participating either in a digital or a paper-based achievement test, was associated with the prevalence of inconsistent responding on a mixed-worded scale on a questionnaire administered immediately after. Data were used from 4th- and 8th-grade students from 16 countries participating in the 2019 Trends in International Mathematics and Science Study (TIMSS). The self-reported mixed-worded scale measured self-concept in mathematics and was administered in paper format to all students. The study employed the mean absolute difference (MAD) and factor mixture analysis (FMA) methods to identify inconsistent respondents. Although cross-cultural variation was observed, the 4th-graders who had taken the computer-based assessment prior to the questionnaire engaged in a slightly higher frequency of inconsistent responses. Among 8th-graders, the difference was smaller and, in several country samples, reversed. Larger prevalence of inconsistent responding was found with the FMA approach. The study emphasizes the need for further research on inconsistent responding across different assessment modes and contexts, suggesting implications for survey methodology and international studies.https://www.frontiersin.org/articles/10.3389/feduc.2025.1595648/fullassessment modeinconsistent respondingmixed-worded scaleTIMSScross-cultural comparison |
| spellingShingle | Evi Konstantinidou Evi Konstantinidou Michalis P. Michaelides Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries Frontiers in Education assessment mode inconsistent responding mixed-worded scale TIMSS cross-cultural comparison |
| title | Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries |
| title_full | Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries |
| title_fullStr | Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries |
| title_full_unstemmed | Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries |
| title_short | Assessment mode and inconsistent responding on a mixed-worded scale: evidence from TIMSS 2019 across grades and countries |
| title_sort | assessment mode and inconsistent responding on a mixed worded scale evidence from timss 2019 across grades and countries |
| topic | assessment mode inconsistent responding mixed-worded scale TIMSS cross-cultural comparison |
| url | https://www.frontiersin.org/articles/10.3389/feduc.2025.1595648/full |
| work_keys_str_mv | AT evikonstantinidou assessmentmodeandinconsistentrespondingonamixedwordedscaleevidencefromtimss2019acrossgradesandcountries AT evikonstantinidou assessmentmodeandinconsistentrespondingonamixedwordedscaleevidencefromtimss2019acrossgradesandcountries AT michalispmichaelides assessmentmodeandinconsistentrespondingonamixedwordedscaleevidencefromtimss2019acrossgradesandcountries |