Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data
Ergonomic risk classification during manual lifting tasks is crucial for the prevention of workplace injuries. This study addresses the challenge of classifying lifting task risk levels (low, medium, and high risk, labeled as 0, 1, and 2) using multi-modal time-series data comprising raw facial land...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/15/4566 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849406460402860032 |
|---|---|
| author | Suyeon Kim Afrooz Shakeri Seyed Shayan Darabi Eunsik Kim Kyongwon Kim |
| author_facet | Suyeon Kim Afrooz Shakeri Seyed Shayan Darabi Eunsik Kim Kyongwon Kim |
| author_sort | Suyeon Kim |
| collection | DOAJ |
| description | Ergonomic risk classification during manual lifting tasks is crucial for the prevention of workplace injuries. This study addresses the challenge of classifying lifting task risk levels (low, medium, and high risk, labeled as 0, 1, and 2) using multi-modal time-series data comprising raw facial landmarks and bio-signals (electrocardiography [ECG] and electrodermal activity [EDA]). Classifying such data presents inherent challenges due to multi-source information, temporal dynamics, and class imbalance. To overcome these challenges, this paper proposes a Multi-Adaptive Functional Neural Network (Multi-AdaFNN), a novel method that integrates functional data analysis with deep learning techniques. The proposed model introduces a novel adaptive basis layer composed of micro-networks tailored to each individual time-series feature, enabling end-to-end learning of discriminative temporal patterns directly from raw data. The Multi-AdaFNN approach was evaluated across five distinct dataset configurations: (1) facial landmarks only, (2) bio-signals only, (3) full fusion of all available features, (4) a reduced-dimensionality set of 12 selected facial landmark trajectories, and (5) the same reduced set combined with bio-signals. Performance was rigorously assessed using 100 independent stratified splits (70% training and 30% testing) and optimized via a weighted cross-entropy loss function to manage class imbalance effectively. The results demonstrated that the integrated approach, fusing facial landmarks and bio-signals, achieved the highest classification accuracy and robustness. Furthermore, the adaptive basis functions revealed specific phases within lifting tasks critical for risk prediction. These findings underscore the efficacy and transparency of the Multi-AdaFNN framework for multi-modal ergonomic risk assessment, highlighting its potential for real-time monitoring and proactive injury prevention in industrial environments. |
| format | Article |
| id | doaj-art-ecb0f3030de647589a63cc4c901852ff |
| institution | Kabale University |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-ecb0f3030de647589a63cc4c901852ff2025-08-20T03:36:22ZengMDPI AGSensors1424-82202025-07-012515456610.3390/s25154566Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series DataSuyeon Kim0Afrooz Shakeri1Seyed Shayan Darabi2Eunsik Kim3Kyongwon Kim4Department of Statistics, Ewha Womans University, Seoul 03760, Republic of KoreaDepartment of Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON N9B 3P4, CanadaDepartment of Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON N9B 3P4, CanadaDepartment of Mechanical, Automotive, and Materials Engineering, University of Windsor, Windsor, ON N9B 3P4, CanadaDepartment of Applied Statistics, Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of KoreaErgonomic risk classification during manual lifting tasks is crucial for the prevention of workplace injuries. This study addresses the challenge of classifying lifting task risk levels (low, medium, and high risk, labeled as 0, 1, and 2) using multi-modal time-series data comprising raw facial landmarks and bio-signals (electrocardiography [ECG] and electrodermal activity [EDA]). Classifying such data presents inherent challenges due to multi-source information, temporal dynamics, and class imbalance. To overcome these challenges, this paper proposes a Multi-Adaptive Functional Neural Network (Multi-AdaFNN), a novel method that integrates functional data analysis with deep learning techniques. The proposed model introduces a novel adaptive basis layer composed of micro-networks tailored to each individual time-series feature, enabling end-to-end learning of discriminative temporal patterns directly from raw data. The Multi-AdaFNN approach was evaluated across five distinct dataset configurations: (1) facial landmarks only, (2) bio-signals only, (3) full fusion of all available features, (4) a reduced-dimensionality set of 12 selected facial landmark trajectories, and (5) the same reduced set combined with bio-signals. Performance was rigorously assessed using 100 independent stratified splits (70% training and 30% testing) and optimized via a weighted cross-entropy loss function to manage class imbalance effectively. The results demonstrated that the integrated approach, fusing facial landmarks and bio-signals, achieved the highest classification accuracy and robustness. Furthermore, the adaptive basis functions revealed specific phases within lifting tasks critical for risk prediction. These findings underscore the efficacy and transparency of the Multi-AdaFNN framework for multi-modal ergonomic risk assessment, highlighting its potential for real-time monitoring and proactive injury prevention in industrial environments.https://www.mdpi.com/1424-8220/25/15/4566functional data analysisneural networkclassification |
| spellingShingle | Suyeon Kim Afrooz Shakeri Seyed Shayan Darabi Eunsik Kim Kyongwon Kim Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data Sensors functional data analysis neural network classification |
| title | Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data |
| title_full | Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data |
| title_fullStr | Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data |
| title_full_unstemmed | Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data |
| title_short | Raw-Data Driven Functional Data Analysis with Multi-Adaptive Functional Neural Networks for Ergonomic Risk Classification Using Facial and Bio-Signal Time-Series Data |
| title_sort | raw data driven functional data analysis with multi adaptive functional neural networks for ergonomic risk classification using facial and bio signal time series data |
| topic | functional data analysis neural network classification |
| url | https://www.mdpi.com/1424-8220/25/15/4566 |
| work_keys_str_mv | AT suyeonkim rawdatadrivenfunctionaldataanalysiswithmultiadaptivefunctionalneuralnetworksforergonomicriskclassificationusingfacialandbiosignaltimeseriesdata AT afroozshakeri rawdatadrivenfunctionaldataanalysiswithmultiadaptivefunctionalneuralnetworksforergonomicriskclassificationusingfacialandbiosignaltimeseriesdata AT seyedshayandarabi rawdatadrivenfunctionaldataanalysiswithmultiadaptivefunctionalneuralnetworksforergonomicriskclassificationusingfacialandbiosignaltimeseriesdata AT eunsikkim rawdatadrivenfunctionaldataanalysiswithmultiadaptivefunctionalneuralnetworksforergonomicriskclassificationusingfacialandbiosignaltimeseriesdata AT kyongwonkim rawdatadrivenfunctionaldataanalysiswithmultiadaptivefunctionalneuralnetworksforergonomicriskclassificationusingfacialandbiosignaltimeseriesdata |