Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs
Generalizability theory (GT) provides an all-encompassing framework for estimating accuracy of scores and effects of multiple sources of measurement error when using measures intended for either norm- or criterion-referencing purposes. Structural equation models (SEMs) can replicate results from GT-...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-03-01
|
| Series: | Mathematics |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2227-7390/13/6/1001 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850204713538027520 |
|---|---|
| author | Walter P. Vispoel Hyeryung Lee Tingting Chen |
| author_facet | Walter P. Vispoel Hyeryung Lee Tingting Chen |
| author_sort | Walter P. Vispoel |
| collection | DOAJ |
| description | Generalizability theory (GT) provides an all-encompassing framework for estimating accuracy of scores and effects of multiple sources of measurement error when using measures intended for either norm- or criterion-referencing purposes. Structural equation models (SEMs) can replicate results from GT-based ANOVA procedures while extending those analyses to account for scale coarseness, generate Monte Carlo-based confidence intervals for key parameters, partition universe score variance into general and group factor effects, and assess subscale score viability. We apply these techniques in R to univariate, multivariate, and bifactor designs using a novel indicator-mean approach to estimate absolute error. When representing responses to items from the shortened form of the Music Self-Perception Inventory (MUSPI-S) using 2-, 4-, and 8-point response metrics over two occasions, SEMs reproduced results from the ANOVA-based <i>mGENOVA</i> package for univariate and multivariate designs with score accuracy and subscale viability indices within bifactor designs comparable to those from corresponding multivariate designs. Adjusting for scale coarseness improved the accuracy of scores across all response metrics, with dichotomous observed scores least approximating truly continuous scales. Although general-factor effects were dominant, subscale viability was supported in all cases, with transient measurement error leading to the greatest reductions in score accuracy. Key implications are discussed. |
| format | Article |
| id | doaj-art-bb44800d882f45cbb641bebe5c80d51b |
| institution | OA Journals |
| issn | 2227-7390 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Mathematics |
| spelling | doaj-art-bb44800d882f45cbb641bebe5c80d51b2025-08-20T02:11:14ZengMDPI AGMathematics2227-73902025-03-01136100110.3390/math13061001Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor DesignsWalter P. Vispoel0Hyeryung Lee1Tingting Chen2Department of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA 52242, USADepartment of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA 52242, USADepartment of Psychological and Quantitative Foundations, University of Iowa, Iowa City, IA 52242, USAGeneralizability theory (GT) provides an all-encompassing framework for estimating accuracy of scores and effects of multiple sources of measurement error when using measures intended for either norm- or criterion-referencing purposes. Structural equation models (SEMs) can replicate results from GT-based ANOVA procedures while extending those analyses to account for scale coarseness, generate Monte Carlo-based confidence intervals for key parameters, partition universe score variance into general and group factor effects, and assess subscale score viability. We apply these techniques in R to univariate, multivariate, and bifactor designs using a novel indicator-mean approach to estimate absolute error. When representing responses to items from the shortened form of the Music Self-Perception Inventory (MUSPI-S) using 2-, 4-, and 8-point response metrics over two occasions, SEMs reproduced results from the ANOVA-based <i>mGENOVA</i> package for univariate and multivariate designs with score accuracy and subscale viability indices within bifactor designs comparable to those from corresponding multivariate designs. Adjusting for scale coarseness improved the accuracy of scores across all response metrics, with dichotomous observed scores least approximating truly continuous scales. Although general-factor effects were dominant, subscale viability was supported in all cases, with transient measurement error leading to the greatest reductions in score accuracy. Key implications are discussed.https://www.mdpi.com/2227-7390/13/6/1001structural equation modelsgeneralizability theorymultivariate analysisbifactor modelsR programmingreliability |
| spellingShingle | Walter P. Vispoel Hyeryung Lee Tingting Chen Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs Mathematics structural equation models generalizability theory multivariate analysis bifactor models R programming reliability |
| title | Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs |
| title_full | Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs |
| title_fullStr | Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs |
| title_full_unstemmed | Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs |
| title_short | Structural Equation Modeling Approaches to Estimating Score Dependability Within Generalizability Theory-Based Univariate, Multivariate, and Bifactor Designs |
| title_sort | structural equation modeling approaches to estimating score dependability within generalizability theory based univariate multivariate and bifactor designs |
| topic | structural equation models generalizability theory multivariate analysis bifactor models R programming reliability |
| url | https://www.mdpi.com/2227-7390/13/6/1001 |
| work_keys_str_mv | AT walterpvispoel structuralequationmodelingapproachestoestimatingscoredependabilitywithingeneralizabilitytheorybasedunivariatemultivariateandbifactordesigns AT hyeryunglee structuralequationmodelingapproachestoestimatingscoredependabilitywithingeneralizabilitytheorybasedunivariatemultivariateandbifactordesigns AT tingtingchen structuralequationmodelingapproachestoestimatingscoredependabilitywithingeneralizabilitytheorybasedunivariatemultivariateandbifactordesigns |