How to score respondents? A Monte Carlo study comparing three different procedures
ABSTRACT This study aimed to empirically compare the effectiveness of Likert, Thurstonian, and Expected a Posteriori (EAP) scoring methods. A computational simulation of the two-parameter logistic model was employed under various conditions, including different sample sizes, number of items, scale l...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | Portuguese |
| Published: |
Instituto Brasileiro de Avaliação Psicológica (IBAP)
2025-06-01
|
| Series: | Avaliação Psicológica |
| Subjects: | |
| Online Access: | http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712025000100101&lng=en&tlng=en |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849333148055240704 |
|---|---|
| author | Víthor Rosa Franco Marie Wiberg |
| author_facet | Víthor Rosa Franco Marie Wiberg |
| author_sort | Víthor Rosa Franco |
| collection | DOAJ |
| description | ABSTRACT This study aimed to empirically compare the effectiveness of Likert, Thurstonian, and Expected a Posteriori (EAP) scoring methods. A computational simulation of the two-parameter logistic model was employed under various conditions, including different sample sizes, number of items, scale levels, item extremeness, and varying discriminations. Effectiveness was assessed through correlation with the true score, root mean squared errors, bias, and the accurate recovery of effect sizes. The results indicated that Likert scores exhibit greater bias than EAP and Thurstonian scores for extreme scores, however, they strongly correlate with both the true score and EAP scores. Likert scores were slightly more effective in recovering mean differences between two groups, correlation estimates, and regression parameters. Overall, Likert scores should be avoided when ordering or thresholding individuals at the extremes of scales is the primary objective. However, they are preferable in situations where Thurstonian and EAP scores may fail to converge. The study also recommends that future research explore conditions involving more complex data-generating processes. |
| format | Article |
| id | doaj-art-ed5c79c6391a413bb5f562ac0bae5e1d |
| institution | Kabale University |
| issn | 2175-3431 |
| language | Portuguese |
| publishDate | 2025-06-01 |
| publisher | Instituto Brasileiro de Avaliação Psicológica (IBAP) |
| record_format | Article |
| series | Avaliação Psicológica |
| spelling | doaj-art-ed5c79c6391a413bb5f562ac0bae5e1d2025-08-20T03:45:57ZporInstituto Brasileiro de Avaliação Psicológica (IBAP)Avaliação Psicológica2175-34312025-06-012410.15689/ap.2025.24.e22224How to score respondents? A Monte Carlo study comparing three different proceduresVíthor Rosa FrancoMarie WibergABSTRACT This study aimed to empirically compare the effectiveness of Likert, Thurstonian, and Expected a Posteriori (EAP) scoring methods. A computational simulation of the two-parameter logistic model was employed under various conditions, including different sample sizes, number of items, scale levels, item extremeness, and varying discriminations. Effectiveness was assessed through correlation with the true score, root mean squared errors, bias, and the accurate recovery of effect sizes. The results indicated that Likert scores exhibit greater bias than EAP and Thurstonian scores for extreme scores, however, they strongly correlate with both the true score and EAP scores. Likert scores were slightly more effective in recovering mean differences between two groups, correlation estimates, and regression parameters. Overall, Likert scores should be avoided when ordering or thresholding individuals at the extremes of scales is the primary objective. However, they are preferable in situations where Thurstonian and EAP scores may fail to converge. The study also recommends that future research explore conditions involving more complex data-generating processes.http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712025000100101&lng=en&tlng=enFactor scoresPsychometric theoryMonte Carlo simulation. |
| spellingShingle | Víthor Rosa Franco Marie Wiberg How to score respondents? A Monte Carlo study comparing three different procedures Avaliação Psicológica Factor scores Psychometric theory Monte Carlo simulation. |
| title | How to score respondents? A Monte Carlo study comparing three different procedures |
| title_full | How to score respondents? A Monte Carlo study comparing three different procedures |
| title_fullStr | How to score respondents? A Monte Carlo study comparing three different procedures |
| title_full_unstemmed | How to score respondents? A Monte Carlo study comparing three different procedures |
| title_short | How to score respondents? A Monte Carlo study comparing three different procedures |
| title_sort | how to score respondents a monte carlo study comparing three different procedures |
| topic | Factor scores Psychometric theory Monte Carlo simulation. |
| url | http://pepsic.bvsalud.org/scielo.php?script=sci_arttext&pid=S1677-04712025000100101&lng=en&tlng=en |
| work_keys_str_mv | AT vithorrosafranco howtoscorerespondentsamontecarlostudycomparingthreedifferentprocedures AT mariewiberg howtoscorerespondentsamontecarlostudycomparingthreedifferentprocedures |