Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study
Abstract BackgroundPatient-reported experience surveys allow administrators, clinicians, and researchers to quantify and improve health care by receiving feedback directly from patients. Existing research has focused primarily on quantitative analysis of survey items, but thes...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
JMIR Publications
2025-02-01
|
| Series: | JMIR Medical Informatics |
| Online Access: | https://medinform.jmir.org/2025/1/e63466 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850041697674723328 |
|---|---|
| author | Brian Steele Paul Fairie Kyle Kemp Adam G D'Souza Matthias Wilms Maria Jose Santana |
| author_facet | Brian Steele Paul Fairie Kyle Kemp Adam G D'Souza Matthias Wilms Maria Jose Santana |
| author_sort | Brian Steele |
| collection | DOAJ |
| description |
Abstract
BackgroundPatient-reported experience surveys allow administrators, clinicians, and researchers to quantify and improve health care by receiving feedback directly from patients. Existing research has focused primarily on quantitative analysis of survey items, but these measures may collect optional free-text comments. These comments can provide insights for health systems but may not be analyzed due to limited resources and the complexity of traditional textual analysis. However, advances in machine learning–based natural language processing provide opportunities to learn from this traditionally underused data source.
ObjectiveThis study aimed to apply natural language processing to model topics found in free-text comments of patient-reported experience surveys.
MethodsConsumer Assessment of Healthcare Providers and Systems–derived patient experience surveys were collected and linked to administrative inpatient records by the provincial health services organization responsible for inpatient care. Unsupervised topic modeling with automated labeling was performed with BERTopic. Sentiment analysis was performed to further assist in topic description.
ResultsBetween April 2016 and February 2020, 43.4% (43,522/100,272) adult patients and 46.9% (3501/7464) pediatric caregivers included free-text responses on completed patient experience surveys. Topic models identified 86 topics among adult survey responses and 35 topics among pediatric responses that included elements of care not currently surveyed by existing questionnaires. Frequent topics were generally positive.
ConclusionsWe found that with limited tuning, BERTopic identified care experience topics with interpretable automated labeling. Results are discussed in the context of person-centered care, patient safety, and health care quality improvement. Furthermore, we note the opportunity for the identification of temporal and site-specific trends as a method to identify patient care and safety concerns. As the use of patient experience measurement increases in health care, we discuss how machine learning can be leveraged to provide additional insight on patient experiences. |
| format | Article |
| id | doaj-art-d23c075d33a24509a44abb7f6bf01b00 |
| institution | DOAJ |
| issn | 2291-9694 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | JMIR Publications |
| record_format | Article |
| series | JMIR Medical Informatics |
| spelling | doaj-art-d23c075d33a24509a44abb7f6bf01b002025-08-20T02:55:42ZengJMIR PublicationsJMIR Medical Informatics2291-96942025-02-0113e63466e6346610.2196/63466Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling StudyBrian Steelehttp://orcid.org/0000-0002-0081-1079Paul Fairiehttp://orcid.org/0000-0003-2611-7680Kyle Kemphttp://orcid.org/0000-0003-0138-5013Adam G D'Souzahttp://orcid.org/0000-0002-4270-2065Matthias Wilmshttp://orcid.org/0000-0001-8845-360XMaria Jose Santanahttp://orcid.org/0000-0002-0202-5952 Abstract BackgroundPatient-reported experience surveys allow administrators, clinicians, and researchers to quantify and improve health care by receiving feedback directly from patients. Existing research has focused primarily on quantitative analysis of survey items, but these measures may collect optional free-text comments. These comments can provide insights for health systems but may not be analyzed due to limited resources and the complexity of traditional textual analysis. However, advances in machine learning–based natural language processing provide opportunities to learn from this traditionally underused data source. ObjectiveThis study aimed to apply natural language processing to model topics found in free-text comments of patient-reported experience surveys. MethodsConsumer Assessment of Healthcare Providers and Systems–derived patient experience surveys were collected and linked to administrative inpatient records by the provincial health services organization responsible for inpatient care. Unsupervised topic modeling with automated labeling was performed with BERTopic. Sentiment analysis was performed to further assist in topic description. ResultsBetween April 2016 and February 2020, 43.4% (43,522/100,272) adult patients and 46.9% (3501/7464) pediatric caregivers included free-text responses on completed patient experience surveys. Topic models identified 86 topics among adult survey responses and 35 topics among pediatric responses that included elements of care not currently surveyed by existing questionnaires. Frequent topics were generally positive. ConclusionsWe found that with limited tuning, BERTopic identified care experience topics with interpretable automated labeling. Results are discussed in the context of person-centered care, patient safety, and health care quality improvement. Furthermore, we note the opportunity for the identification of temporal and site-specific trends as a method to identify patient care and safety concerns. As the use of patient experience measurement increases in health care, we discuss how machine learning can be leveraged to provide additional insight on patient experiences.https://medinform.jmir.org/2025/1/e63466 |
| spellingShingle | Brian Steele Paul Fairie Kyle Kemp Adam G D'Souza Matthias Wilms Maria Jose Santana Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study JMIR Medical Informatics |
| title | Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study |
| title_full | Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study |
| title_fullStr | Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study |
| title_full_unstemmed | Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study |
| title_short | Identifying Patient-Reported Care Experiences in Free-Text Survey Comments: Topic Modeling Study |
| title_sort | identifying patient reported care experiences in free text survey comments topic modeling study |
| url | https://medinform.jmir.org/2025/1/e63466 |
| work_keys_str_mv | AT briansteele identifyingpatientreportedcareexperiencesinfreetextsurveycommentstopicmodelingstudy AT paulfairie identifyingpatientreportedcareexperiencesinfreetextsurveycommentstopicmodelingstudy AT kylekemp identifyingpatientreportedcareexperiencesinfreetextsurveycommentstopicmodelingstudy AT adamgdsouza identifyingpatientreportedcareexperiencesinfreetextsurveycommentstopicmodelingstudy AT matthiaswilms identifyingpatientreportedcareexperiencesinfreetextsurveycommentstopicmodelingstudy AT mariajosesantana identifyingpatientreportedcareexperiencesinfreetextsurveycommentstopicmodelingstudy |