Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences
This article provides a comprehensive critique of psychology's overreliance on statistical modelling at the expense of epistemologically grounded measurement processes. It highlights that statistics deals with structural relations in data regardless of what these data represent, whereas measure...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Frontiers Media S.A.
2025-06-01
|
| Series: | Frontiers in Psychology |
| Subjects: | |
| Online Access: | https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1534270/full |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850163666288115712 |
|---|---|
| author | Jana Uher |
| author_facet | Jana Uher |
| author_sort | Jana Uher |
| collection | DOAJ |
| description | This article provides a comprehensive critique of psychology's overreliance on statistical modelling at the expense of epistemologically grounded measurement processes. It highlights that statistics deals with structural relations in data regardless of what these data represent, whereas measurement establishes traceable empirical relations between the phenomena studied and the data representing information about them. These crucial epistemic differences are elaborated using Rosen's general model of measurement, involving the coherent modelling of the (1) objects of research, (2) data generation (encoding), (3) formal manipulation (e.g., statistical analysis) and (4) result interpretation regarding the objects studied (decoding). This system of interrelated modelling relations is shown to underlie metrologists' approaches for tackling the problem of epistemic circularity in physical measurement, illustrated in the special cases of measurement coordination and calibration. The article then explicates psychology's challenges for establishing genuine analogues of measurement, which arise from the peculiarities of its study phenomena (e.g., higher-order complexity, non-ergodicity) and language-based methods (e.g., inbuilt semantics). It demonstrates that psychometrics cannot establish coordinated and calibrated modelling relations, thus generating only pragmatic quantifications with predictive power but precluding epistemically justified inferences on the phenomena studied. This epistemic gap is often overlooked, however, because many psychologists mistake their methods' inbuilt semantics—thus, descriptions of their study phenomena (e.g., in rating scales, item variables, statistical models)—for the phenomena described. This blurs the epistemically necessary distinction between the phenomena studied and those used as means of investigation, thereby confusing ontological with epistemological concepts—psychologists' cardinal error. Therefore, many mistake judgements of verbal statements for measurements of the phenomena described and overlook that statistics can neither establish nor analyze a model's relations to the phenomena explored. The article elaborates epistemological and methodological fundamentals to establish coherent modelling relations between real and formal study system and to distinguish the epistemic components involved, considering psychology's peculiarities. It shows that epistemically justified inferences necessitate methods for analysing individuals' unrestricted verbal responses, now advanced through artificial intelligence systems modelling natural language (e.g., NLP algorithms, LLMs). Their increasing use to generate standardised descriptions of study phenomena for rating scales and constructs, by contrast, will only perpetuate psychologists' cardinal error—and thus, psychology's crisis. |
| format | Article |
| id | doaj-art-ec0063479c094fdd8331fb9235a48b1e |
| institution | OA Journals |
| issn | 1664-1078 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | Frontiers Media S.A. |
| record_format | Article |
| series | Frontiers in Psychology |
| spelling | doaj-art-ec0063479c094fdd8331fb9235a48b1e2025-08-20T02:22:10ZengFrontiers Media S.A.Frontiers in Psychology1664-10782025-06-011610.3389/fpsyg.2025.15342701534270Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differencesJana UherThis article provides a comprehensive critique of psychology's overreliance on statistical modelling at the expense of epistemologically grounded measurement processes. It highlights that statistics deals with structural relations in data regardless of what these data represent, whereas measurement establishes traceable empirical relations between the phenomena studied and the data representing information about them. These crucial epistemic differences are elaborated using Rosen's general model of measurement, involving the coherent modelling of the (1) objects of research, (2) data generation (encoding), (3) formal manipulation (e.g., statistical analysis) and (4) result interpretation regarding the objects studied (decoding). This system of interrelated modelling relations is shown to underlie metrologists' approaches for tackling the problem of epistemic circularity in physical measurement, illustrated in the special cases of measurement coordination and calibration. The article then explicates psychology's challenges for establishing genuine analogues of measurement, which arise from the peculiarities of its study phenomena (e.g., higher-order complexity, non-ergodicity) and language-based methods (e.g., inbuilt semantics). It demonstrates that psychometrics cannot establish coordinated and calibrated modelling relations, thus generating only pragmatic quantifications with predictive power but precluding epistemically justified inferences on the phenomena studied. This epistemic gap is often overlooked, however, because many psychologists mistake their methods' inbuilt semantics—thus, descriptions of their study phenomena (e.g., in rating scales, item variables, statistical models)—for the phenomena described. This blurs the epistemically necessary distinction between the phenomena studied and those used as means of investigation, thereby confusing ontological with epistemological concepts—psychologists' cardinal error. Therefore, many mistake judgements of verbal statements for measurements of the phenomena described and overlook that statistics can neither establish nor analyze a model's relations to the phenomena explored. The article elaborates epistemological and methodological fundamentals to establish coherent modelling relations between real and formal study system and to distinguish the epistemic components involved, considering psychology's peculiarities. It shows that epistemically justified inferences necessitate methods for analysing individuals' unrestricted verbal responses, now advanced through artificial intelligence systems modelling natural language (e.g., NLP algorithms, LLMs). Their increasing use to generate standardised descriptions of study phenomena for rating scales and constructs, by contrast, will only perpetuate psychologists' cardinal error—and thus, psychology's crisis.https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1534270/fullmeasurementpsychometricslarge language models (LLMs)natural language processing (NLP)rating scalesmodelling relation |
| spellingShingle | Jana Uher Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences Frontiers in Psychology measurement psychometrics large language models (LLMs) natural language processing (NLP) rating scales modelling relation |
| title | Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences |
| title_full | Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences |
| title_fullStr | Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences |
| title_full_unstemmed | Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences |
| title_short | Statistics is not measurement: The inbuilt semantics of psychometric scales and language-based models obscures crucial epistemic differences |
| title_sort | statistics is not measurement the inbuilt semantics of psychometric scales and language based models obscures crucial epistemic differences |
| topic | measurement psychometrics large language models (LLMs) natural language processing (NLP) rating scales modelling relation |
| url | https://www.frontiersin.org/articles/10.3389/fpsyg.2025.1534270/full |
| work_keys_str_mv | AT janauher statisticsisnotmeasurementtheinbuiltsemanticsofpsychometricscalesandlanguagebasedmodelsobscurescrucialepistemicdifferences |