The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times
Several studies have been published on disengaged test respondents, and others have analyzed disengaged survey respondents separately. For many large-scale assessments, students answer questionnaire and test items in succession. This study examines the percentage of students who continuously engage...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
izzet kara
2021-09-01
|
| Series: | International Journal of Assessment Tools in Education |
| Subjects: | |
| Online Access: | https://dergipark.org.tr/tr/download/article-file/1269350 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850066948992270336 |
|---|---|
| author | Hatice Cigdem Bulut |
| author_facet | Hatice Cigdem Bulut |
| author_sort | Hatice Cigdem Bulut |
| collection | DOAJ |
| description | Several studies have been published on disengaged test respondents, and others have analyzed disengaged survey respondents separately. For many large-scale assessments, students answer questionnaire and test items in succession. This study examines the percentage of students who continuously engage in disengaged responding behaviors across sections in a low-stakes assessment. The effects on calculated scores of filtering students, based on their responding behaviors, are also analyzed. Data of this study came from the 2015 administration of PISA. For data analysis, frequencies and percentages of engaged students in the sessions were initially calculated using students' response times. To investigate the impact of filtering disengaged respondents on parameter estimation, three groups were created, namely engaged in both measures, engaged only in the test, and engaged only in the questionnaire. Next, several validity checks were performed on each group to verify the accuracy of the classifications and the impact of filtering student groups based on their responding behavior. The results indicate that students who are disengaged in tests tend to continue this behavior when responding to the questionnaire items in PISA. Moreover, the rate of continuity of disengaged responding is non-negligible as can be seen from the effect sizes. On the other hand, removing disengaged students in both measures led to higher or nearly the same performance ratings compared to the other groups. Researchers analyzing the dataset including achievement tests and survey items are recommended to review disengaged responses and filter out students who are continuously showing disengaged responding before performing further statistical analysis. |
| format | Article |
| id | doaj-art-c4dfb389b89f4eef8703deb446c11e1a |
| institution | DOAJ |
| issn | 2148-7456 |
| language | English |
| publishDate | 2021-09-01 |
| publisher | izzet kara |
| record_format | Article |
| series | International Journal of Assessment Tools in Education |
| spelling | doaj-art-c4dfb389b89f4eef8703deb446c11e1a2025-08-20T02:48:33Zengizzet karaInternational Journal of Assessment Tools in Education2148-74562021-09-018352754110.21449/ijate.789212618The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response TimesHatice Cigdem Bulut0https://orcid.org/0000-0003-2585-3686ÇUKUROVA ÜNİVERSİTESİSeveral studies have been published on disengaged test respondents, and others have analyzed disengaged survey respondents separately. For many large-scale assessments, students answer questionnaire and test items in succession. This study examines the percentage of students who continuously engage in disengaged responding behaviors across sections in a low-stakes assessment. The effects on calculated scores of filtering students, based on their responding behaviors, are also analyzed. Data of this study came from the 2015 administration of PISA. For data analysis, frequencies and percentages of engaged students in the sessions were initially calculated using students' response times. To investigate the impact of filtering disengaged respondents on parameter estimation, three groups were created, namely engaged in both measures, engaged only in the test, and engaged only in the questionnaire. Next, several validity checks were performed on each group to verify the accuracy of the classifications and the impact of filtering student groups based on their responding behavior. The results indicate that students who are disengaged in tests tend to continue this behavior when responding to the questionnaire items in PISA. Moreover, the rate of continuity of disengaged responding is non-negligible as can be seen from the effect sizes. On the other hand, removing disengaged students in both measures led to higher or nearly the same performance ratings compared to the other groups. Researchers analyzing the dataset including achievement tests and survey items are recommended to review disengaged responses and filter out students who are continuously showing disengaged responding before performing further statistical analysis.https://dergipark.org.tr/tr/download/article-file/1269350disengaged respondinginsufficient effort respondingvaliditylow-stakes assessmentsresponse timedisengaged respondinginsufficient effortrespondingvaliditylow-stakes assessments |
| spellingShingle | Hatice Cigdem Bulut The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times International Journal of Assessment Tools in Education disengaged responding insufficient effort responding validity low-stakes assessments response time disengaged responding insufficient effortresponding validity low-stakes assessments |
| title | The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times |
| title_full | The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times |
| title_fullStr | The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times |
| title_full_unstemmed | The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times |
| title_short | The Continuity of Students’ Disengaged Responding in Low-stakes Assessments: Evidence from Response Times |
| title_sort | continuity of students disengaged responding in low stakes assessments evidence from response times |
| topic | disengaged responding insufficient effort responding validity low-stakes assessments response time disengaged responding insufficient effortresponding validity low-stakes assessments |
| url | https://dergipark.org.tr/tr/download/article-file/1269350 |
| work_keys_str_mv | AT haticecigdembulut thecontinuityofstudentsdisengagedrespondinginlowstakesassessmentsevidencefromresponsetimes AT haticecigdembulut continuityofstudentsdisengagedrespondinginlowstakesassessmentsevidencefromresponsetimes |