Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing
Computer Adaptive Multistage Testing (ca-MST), which take the advantage of computer technology and adaptive test form, are widely used, and are now a popular issue of assessment and evaluation. This study aims at analyzing the effect of different panel designs, module lengths, and different sequence...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
izzet kara
2020-06-01
|
| Series: | International Journal of Assessment Tools in Education |
| Subjects: | |
| Online Access: | https://dergipark.org.tr/tr/download/article-file/1061808 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850067103454855168 |
|---|---|
| author | Melek Gülşah Şahin |
| author_facet | Melek Gülşah Şahin |
| author_sort | Melek Gülşah Şahin |
| collection | DOAJ |
| description | Computer Adaptive Multistage Testing (ca-MST), which take the advantage of computer technology and adaptive test form, are widely used, and are now a popular issue of assessment and evaluation. This study aims at analyzing the effect of different panel designs, module lengths, and different sequence of a parameter value across stages and change in b parameter range on measurement precision in ca-MST implementations. The study has been carried out as a simulation. MSTGen simulation software tool was used for that purpose. 5000 simulees derived from normal distribution (N (0,1)) were simulated. 60 different conditions (two panel designs (1-3-3; 1-2-2), three module lengths (10-15-20), 5 different a parameter sequences (“0.8; 0.8; 0.8” - “1.4; 0.8; 0.8”-“0.8;1.4; 0.8” - “0.8; 0.8;1,4” - “1.4; 1,4; 1.4”) and two b parameter difference (small; large) conditions) were taken into consideration during analysis. Correlation, RMSE and AAD values of conditions were calculated. Conditional RMSE values corresponding to each ability level are given in a graph. Dissimilar to other studies in the literature, this study examines b parameter difference condition in three-stage tests and its interaction with a parameter sequence. Study results show that measurement precision increases as the number and length of the modules increase. Errors in measurement decrease as item discrimination values increase in all stages. Including items with a high value of item discrimination in the second or last stage contributes to measurement precision. In extreme ability levels, large difficulty difference condition produces lower error values when compared to small difficulty difference condition. |
| format | Article |
| id | doaj-art-0af51a4c7ba64c2ca71e26a7fa0e3994 |
| institution | DOAJ |
| issn | 2148-7456 |
| language | English |
| publishDate | 2020-06-01 |
| publisher | izzet kara |
| record_format | Article |
| series | International Journal of Assessment Tools in Education |
| spelling | doaj-art-0af51a4c7ba64c2ca71e26a7fa0e39942025-08-20T02:48:30Zengizzet karaInternational Journal of Assessment Tools in Education2148-74562020-06-017219120610.21449/ijate.676947618Analyzing Different Module Characteristics in Computer Adaptive Multistage TestingMelek Gülşah Şahin0https://orcid.org/0000-0001-5139-9777GAZI UNIVERSITY, GAZİ FACULTY OF EDUCATIONComputer Adaptive Multistage Testing (ca-MST), which take the advantage of computer technology and adaptive test form, are widely used, and are now a popular issue of assessment and evaluation. This study aims at analyzing the effect of different panel designs, module lengths, and different sequence of a parameter value across stages and change in b parameter range on measurement precision in ca-MST implementations. The study has been carried out as a simulation. MSTGen simulation software tool was used for that purpose. 5000 simulees derived from normal distribution (N (0,1)) were simulated. 60 different conditions (two panel designs (1-3-3; 1-2-2), three module lengths (10-15-20), 5 different a parameter sequences (“0.8; 0.8; 0.8” - “1.4; 0.8; 0.8”-“0.8;1.4; 0.8” - “0.8; 0.8;1,4” - “1.4; 1,4; 1.4”) and two b parameter difference (small; large) conditions) were taken into consideration during analysis. Correlation, RMSE and AAD values of conditions were calculated. Conditional RMSE values corresponding to each ability level are given in a graph. Dissimilar to other studies in the literature, this study examines b parameter difference condition in three-stage tests and its interaction with a parameter sequence. Study results show that measurement precision increases as the number and length of the modules increase. Errors in measurement decrease as item discrimination values increase in all stages. Including items with a high value of item discrimination in the second or last stage contributes to measurement precision. In extreme ability levels, large difficulty difference condition produces lower error values when compared to small difficulty difference condition.https://dergipark.org.tr/tr/download/article-file/1061808ca-mstpanel designmodule lengthitem discrimination sequencedifficulty difference conditionca-mstpanel designmodule lengthitem discriminationsequencedifficulty differencecondition |
| spellingShingle | Melek Gülşah Şahin Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing International Journal of Assessment Tools in Education ca-mst panel design module length item discrimination sequence difficulty difference condition ca-mst panel design module length item discriminationsequence difficulty differencecondition |
| title | Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing |
| title_full | Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing |
| title_fullStr | Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing |
| title_full_unstemmed | Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing |
| title_short | Analyzing Different Module Characteristics in Computer Adaptive Multistage Testing |
| title_sort | analyzing different module characteristics in computer adaptive multistage testing |
| topic | ca-mst panel design module length item discrimination sequence difficulty difference condition ca-mst panel design module length item discriminationsequence difficulty differencecondition |
| url | https://dergipark.org.tr/tr/download/article-file/1061808 |
| work_keys_str_mv | AT melekgulsahsahin analyzingdifferentmodulecharacteristicsincomputeradaptivemultistagetesting |