Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network
Abstract Self-reported pain scores are often used for pain assessments and require effective communication. On the other hand, observer-based assessments are resource-intensive and require training. We developed an automated system to assess pain intensity in adult patients based on changes in facia...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-04-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-97885-5 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850181698076016640 |
|---|---|
| author | Chin Wen Tan Tiehua Du Jing Chun Teo Diana Xin Hui Chan Wai Ming Kong Ban Leong Sng |
| author_facet | Chin Wen Tan Tiehua Du Jing Chun Teo Diana Xin Hui Chan Wai Ming Kong Ban Leong Sng |
| author_sort | Chin Wen Tan |
| collection | DOAJ |
| description | Abstract Self-reported pain scores are often used for pain assessments and require effective communication. On the other hand, observer-based assessments are resource-intensive and require training. We developed an automated system to assess pain intensity in adult patients based on changes in facial expression. We recruited adult patients undergoing surgery or interventional pain procedures in two public healthcare institutions in Singapore. The patients’ facial expressions were videotaped from a frontal view with varying body poses using a customized mobile application. The collected videos were trimmed into multiple 1 s clips and categorized into three levels of pain: no pain, mild pain, or significant pain. A total of 468 facial key points were extracted from each video frame. A customized spatial temporal attention long short-term memory (STA-LSTM) deep learning network was trained and validated using the extracted keypoints to detect pain levels by analyzing facial expressions in both the spatial and temporal domains. Model performance was evaluated using accuracy, sensitivity, recall, and F1-score. Two hundred patients were recruited, with 2008 videos collected for further clipping into 10,274 1 s clips. Videos from 160 patients (7599 clips) were used for STA-LSTM training, while the remaining 40 patients’ videos (2675 clips) were set aside for validation. By differentiating the polychromous levels of pain (no pain versus mild pain versus significant pain requiring clinical intervention), we reported the optimal performance of STA-LSTM model, with accuracy, sensitivity, recall, and F1-score all at 0.8660. Our proposed solution has the potential to facilitate objective pain assessment in clinical settings through the developed STA-LSTM model, enabling healthcare professionals and caregivers to perform pain assessments effectively in both inpatient and outpatient settings. |
| format | Article |
| id | doaj-art-02cfab3631584b3ab0cce2964f955310 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-04-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-02cfab3631584b3ab0cce2964f9553102025-08-20T02:17:50ZengNature PortfolioScientific Reports2045-23222025-04-0115111310.1038/s41598-025-97885-5Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) networkChin Wen Tan0Tiehua Du1Jing Chun Teo2Diana Xin Hui Chan3Wai Ming Kong4Ban Leong Sng5Department of Women’s Anesthesia, KK Women’s and Children’s HospitalBiomedical Engineering and Materials Group, Nanyang PolytechnicDigital Integration Medical Innovation and Care Transformation, KK Women’s and Children’s HospitalAnesthesiology and Perioperative Sciences Academic Clinical Program, Duke-NUS Medical SchoolBiomedical Engineering and Materials Group, Nanyang PolytechnicDepartment of Women’s Anesthesia, KK Women’s and Children’s HospitalAbstract Self-reported pain scores are often used for pain assessments and require effective communication. On the other hand, observer-based assessments are resource-intensive and require training. We developed an automated system to assess pain intensity in adult patients based on changes in facial expression. We recruited adult patients undergoing surgery or interventional pain procedures in two public healthcare institutions in Singapore. The patients’ facial expressions were videotaped from a frontal view with varying body poses using a customized mobile application. The collected videos were trimmed into multiple 1 s clips and categorized into three levels of pain: no pain, mild pain, or significant pain. A total of 468 facial key points were extracted from each video frame. A customized spatial temporal attention long short-term memory (STA-LSTM) deep learning network was trained and validated using the extracted keypoints to detect pain levels by analyzing facial expressions in both the spatial and temporal domains. Model performance was evaluated using accuracy, sensitivity, recall, and F1-score. Two hundred patients were recruited, with 2008 videos collected for further clipping into 10,274 1 s clips. Videos from 160 patients (7599 clips) were used for STA-LSTM training, while the remaining 40 patients’ videos (2675 clips) were set aside for validation. By differentiating the polychromous levels of pain (no pain versus mild pain versus significant pain requiring clinical intervention), we reported the optimal performance of STA-LSTM model, with accuracy, sensitivity, recall, and F1-score all at 0.8660. Our proposed solution has the potential to facilitate objective pain assessment in clinical settings through the developed STA-LSTM model, enabling healthcare professionals and caregivers to perform pain assessments effectively in both inpatient and outpatient settings.https://doi.org/10.1038/s41598-025-97885-5FacialPain assessmentMachine learningArtificial intelligenceDeep learning |
| spellingShingle | Chin Wen Tan Tiehua Du Jing Chun Teo Diana Xin Hui Chan Wai Ming Kong Ban Leong Sng Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network Scientific Reports Facial Pain assessment Machine learning Artificial intelligence Deep learning |
| title | Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network |
| title_full | Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network |
| title_fullStr | Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network |
| title_full_unstemmed | Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network |
| title_short | Automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short-term memory (STA-LSTM) network |
| title_sort | automated pain detection using facial expression in adult patients with a customized spatial temporal attention long short term memory sta lstm network |
| topic | Facial Pain assessment Machine learning Artificial intelligence Deep learning |
| url | https://doi.org/10.1038/s41598-025-97885-5 |
| work_keys_str_mv | AT chinwentan automatedpaindetectionusingfacialexpressioninadultpatientswithacustomizedspatialtemporalattentionlongshorttermmemorystalstmnetwork AT tiehuadu automatedpaindetectionusingfacialexpressioninadultpatientswithacustomizedspatialtemporalattentionlongshorttermmemorystalstmnetwork AT jingchunteo automatedpaindetectionusingfacialexpressioninadultpatientswithacustomizedspatialtemporalattentionlongshorttermmemorystalstmnetwork AT dianaxinhuichan automatedpaindetectionusingfacialexpressioninadultpatientswithacustomizedspatialtemporalattentionlongshorttermmemorystalstmnetwork AT waimingkong automatedpaindetectionusingfacialexpressioninadultpatientswithacustomizedspatialtemporalattentionlongshorttermmemorystalstmnetwork AT banleongsng automatedpaindetectionusingfacialexpressioninadultpatientswithacustomizedspatialtemporalattentionlongshorttermmemorystalstmnetwork |