Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture.
Student evaluations of teaching (SET) have been widely used by university staff to inform decisions on hiring and promotion. In recent years, an increasing body of research has revealed that student evaluations may be systemically affected by students' own conscious or unconscious biases. In th...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Public Library of Science (PLoS)
2025-01-01
|
| Series: | PLoS ONE |
| Online Access: | https://doi.org/10.1371/journal.pone.0324619 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849422530586083328 |
|---|---|
| author | Fiona Kim Xiongwen Ke Emma L Johnston Yanan Fan |
| author_facet | Fiona Kim Xiongwen Ke Emma L Johnston Yanan Fan |
| author_sort | Fiona Kim |
| collection | DOAJ |
| description | Student evaluations of teaching (SET) have been widely used by university staff to inform decisions on hiring and promotion. In recent years, an increasing body of research has revealed that student evaluations may be systemically affected by students' own conscious or unconscious biases. In this article, we study a data set from an Australian university, where both numerical and text survey responses were available in large quantities. Our study directly linked comments to numerical ratings, we developed approaches to convert text to quantitative data in the form of topics and sentiment scores, and make use of Bayesian ordinal regression techniques to identify drivers of SET scores. Our analysis of text identified 6 teaching dimensions that students discuss in their comments. Our findings suggest that students' SET ratings were correlated primarily with the personal characteristics of the lecturer (such as approachability, and being nice) than measures related to teaching dimensions such as course content and assessment. We found a positive gender effect towards the majority gender in a faculty, possibly reflecting students' gendered expectations. Finally we found that lecturers with a non-English language background were consistently rated lower by the student population, and this effect manifests strongly in local students. |
| format | Article |
| id | doaj-art-cb16698a6eea4de8913c8e8ece335702 |
| institution | Kabale University |
| issn | 1932-6203 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Public Library of Science (PLoS) |
| record_format | Article |
| series | PLoS ONE |
| spelling | doaj-art-cb16698a6eea4de8913c8e8ece3357022025-08-20T03:31:05ZengPublic Library of Science (PLoS)PLoS ONE1932-62032025-01-01206e032461910.1371/journal.pone.0324619Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture.Fiona KimXiongwen KeEmma L JohnstonYanan FanStudent evaluations of teaching (SET) have been widely used by university staff to inform decisions on hiring and promotion. In recent years, an increasing body of research has revealed that student evaluations may be systemically affected by students' own conscious or unconscious biases. In this article, we study a data set from an Australian university, where both numerical and text survey responses were available in large quantities. Our study directly linked comments to numerical ratings, we developed approaches to convert text to quantitative data in the form of topics and sentiment scores, and make use of Bayesian ordinal regression techniques to identify drivers of SET scores. Our analysis of text identified 6 teaching dimensions that students discuss in their comments. Our findings suggest that students' SET ratings were correlated primarily with the personal characteristics of the lecturer (such as approachability, and being nice) than measures related to teaching dimensions such as course content and assessment. We found a positive gender effect towards the majority gender in a faculty, possibly reflecting students' gendered expectations. Finally we found that lecturers with a non-English language background were consistently rated lower by the student population, and this effect manifests strongly in local students.https://doi.org/10.1371/journal.pone.0324619 |
| spellingShingle | Fiona Kim Xiongwen Ke Emma L Johnston Yanan Fan Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture. PLoS ONE |
| title | Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture. |
| title_full | Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture. |
| title_fullStr | Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture. |
| title_full_unstemmed | Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture. |
| title_short | Relationships between numerical score and free text comments in student evaluations of teaching: A sentiment topic analysis reveals the influence of gender and culture. |
| title_sort | relationships between numerical score and free text comments in student evaluations of teaching a sentiment topic analysis reveals the influence of gender and culture |
| url | https://doi.org/10.1371/journal.pone.0324619 |
| work_keys_str_mv | AT fionakim relationshipsbetweennumericalscoreandfreetextcommentsinstudentevaluationsofteachingasentimenttopicanalysisrevealstheinfluenceofgenderandculture AT xiongwenke relationshipsbetweennumericalscoreandfreetextcommentsinstudentevaluationsofteachingasentimenttopicanalysisrevealstheinfluenceofgenderandculture AT emmaljohnston relationshipsbetweennumericalscoreandfreetextcommentsinstudentevaluationsofteachingasentimenttopicanalysisrevealstheinfluenceofgenderandculture AT yananfan relationshipsbetweennumericalscoreandfreetextcommentsinstudentevaluationsofteachingasentimenttopicanalysisrevealstheinfluenceofgenderandculture |