The promise of open survey questions-The validation of text-based job satisfaction measures.
Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psych...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2019-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0226408&type=printable |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850043452018917376 |
|---|---|
| author | Indy Wijngaards Martijn Burger Job van Exel |
| author_facet | Indy Wijngaards Martijn Burger Job van Exel |
| author_sort | Indy Wijngaards |
| collection | DOAJ |
| description | Recent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psychological constructs. In our study, we demonstrated the potential of CATA methods for the construction of text-based job satisfaction measures based on responses to a completely open and semi-open question. To do this, we employed three sentiment analysis techniques: Linguistic Inquiry and Word Count 2015, SentimentR and SentiStrength, and quantified the forms of measurement error they introduced: specific factor error, algorithm error and transient error. We conducted an initial test of the text-based measures' validity, assessing their convergence with closed-question job satisfaction measures. We adopted a time-lagged survey design (Nwave 1 = 996; Nwave 2 = 116) to test our hypotheses. In line with our hypotheses, we found that specific factor error is higher in the open question text-based measure than in the semi-open question text-based measure. As expected, algorithm error was substantial for both the open and semi-open question text-based measures. Transient error in the text-based measures was higher than expected, as it generally exceeded the transient error in the human-coded and the closed job satisfaction question measures. Our initial test of convergent and discriminant validity indicated that the semi-open question text-based measure is especially suitable for measuring job satisfaction. Our article ends with a discussion of limitations and an agenda for future research. |
| format | Article |
| id | doaj-art-5f2904fad3c240799d4de7fe7d5a96a1 |
| institution | DOAJ |
| issn | 1932-6203 |
| language | English |
| publishDate | 2019-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-5f2904fad3c240799d4de7fe7d5a96a12025-08-20T02:55:13ZengPublic Library of Science (PLoS)PLoS ONE1932-62032019-01-011412e022640810.1371/journal.pone.0226408The promise of open survey questions-The validation of text-based job satisfaction measures.Indy WijngaardsMartijn BurgerJob van ExelRecent advances in computer-aided text analysis (CATA) have allowed organizational scientists to construct reliable and convenient measures from open texts. As yet, there is a lack of research into using CATA to analyze responses to open survey questions and constructing text-based measures of psychological constructs. In our study, we demonstrated the potential of CATA methods for the construction of text-based job satisfaction measures based on responses to a completely open and semi-open question. To do this, we employed three sentiment analysis techniques: Linguistic Inquiry and Word Count 2015, SentimentR and SentiStrength, and quantified the forms of measurement error they introduced: specific factor error, algorithm error and transient error. We conducted an initial test of the text-based measures' validity, assessing their convergence with closed-question job satisfaction measures. We adopted a time-lagged survey design (Nwave 1 = 996; Nwave 2 = 116) to test our hypotheses. In line with our hypotheses, we found that specific factor error is higher in the open question text-based measure than in the semi-open question text-based measure. As expected, algorithm error was substantial for both the open and semi-open question text-based measures. Transient error in the text-based measures was higher than expected, as it generally exceeded the transient error in the human-coded and the closed job satisfaction question measures. Our initial test of convergent and discriminant validity indicated that the semi-open question text-based measure is especially suitable for measuring job satisfaction. Our article ends with a discussion of limitations and an agenda for future research.https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0226408&type=printable |
| spellingShingle | Indy Wijngaards Martijn Burger Job van Exel The promise of open survey questions-The validation of text-based job satisfaction measures. PLoS ONE |
| title | The promise of open survey questions-The validation of text-based job satisfaction measures. |
| title_full | The promise of open survey questions-The validation of text-based job satisfaction measures. |
| title_fullStr | The promise of open survey questions-The validation of text-based job satisfaction measures. |
| title_full_unstemmed | The promise of open survey questions-The validation of text-based job satisfaction measures. |
| title_short | The promise of open survey questions-The validation of text-based job satisfaction measures. |
| title_sort | promise of open survey questions the validation of text based job satisfaction measures |
| url | https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0226408&type=printable |
| work_keys_str_mv | AT indywijngaards thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT martijnburger thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT jobvanexel thepromiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT indywijngaards promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT martijnburger promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures AT jobvanexel promiseofopensurveyquestionsthevalidationoftextbasedjobsatisfactionmeasures |