Evaluating Item Content and Scale Characteristics Using a Pretrained Neural Network Model
Multi-item scales are widely used in social research. The psychometric characteristics of a scale and the successful use of a scale in research depend in part on item wording. This article demonstrates a method for using natural language processing (NLP) tools to assist with the item development pr...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
European Survey Research Association
2024-08-01
|
| Series: | Survey Research Methods |
| Subjects: | |
| Online Access: | https://ojs.ub.uni-konstanz.de/srm/article/view/8240 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Multi-item scales are widely used in social research. The psychometric characteristics of a scale and the successful use of a scale in research depend in part on item wording. This article demonstrates a method for using natural language processing (NLP) tools to assist with the item development process, by showing that numeric embedding representations of items are useful in predicting the characteristics of a scale. NLP comprises a set of algorithmic techniques for analysing words, phrases, and larger units of written language. We used NLP tools to create and analyse semantic summaries of the item texts for n=386 previously published multi-item scales. Results showed that semantic representations of items connect to scale characteristics such as Cronbach's alpha internal consistency.
|
|---|---|
| ISSN: | 1864-3361 |