A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data
Pupillometry is commonly used to evaluate cognitive effort, attention, and facial expression response, offering valuable insights into human performance. The combination of eye tracking and facial expression data under the iMotions platform provides great opportunities for multimodal research. Howev...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-07-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/15/4737 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849770847836831744 |
|---|---|
| author | Jingxiang Ong Wenjing He Princess Maglanque Xianta Jiang Lawrence M. Gillman Ashley Vergis Krista Hardy |
| author_facet | Jingxiang Ong Wenjing He Princess Maglanque Xianta Jiang Lawrence M. Gillman Ashley Vergis Krista Hardy |
| author_sort | Jingxiang Ong |
| collection | DOAJ |
| description | Pupillometry is commonly used to evaluate cognitive effort, attention, and facial expression response, offering valuable insights into human performance. The combination of eye tracking and facial expression data under the iMotions platform provides great opportunities for multimodal research. However, there is a lack of standardized pipelines for managing pupillometry data on a multimodal platform. Preprocessing pupil data in multimodal platforms poses challenges like timestamp misalignment, missing data, and inconsistencies across multiple data sources. To address these challenges, the authors introduced a systematic preprocessing pipeline for pupil diameter measurements collected using iMotions 10 (version 10.1.38911.4) during an endoscopy simulation task. The pipeline involves artifact removal, outlier detection using advanced methods such as the Median Absolute Deviation (MAD) and Moving Average (MA) algorithm filtering, interpolation of missing data using the Piecewise Cubic Hermite Interpolating Polynomial (PCHIP), and mean pupil diameter calculation through linear regression, as well as normalization of mean pupil diameter and integration of the pupil diameter dataset with facial expression data. By following these steps, the pipeline enhances data quality, reduces noise, and facilitates the seamless integration of pupillometry other multimodal datasets. In conclusion, this pipeline provides a detailed and organized preprocessing method that improves data reliability while preserving important information for further analysis. |
| format | Article |
| id | doaj-art-4cb51539e72947b7add9c390ca5233ad |
| institution | DOAJ |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-4cb51539e72947b7add9c390ca5233ad2025-08-20T03:02:51ZengMDPI AGSensors1424-82202025-07-012515473710.3390/s25154737A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion DataJingxiang Ong0Wenjing He1Princess Maglanque2Xianta Jiang3Lawrence M. Gillman4Ashley Vergis5Krista Hardy6Department of Surgery, University of Manitoba Max Rady College of Medicine, Winnipeg, MB R3E 0W2, CanadaDepartment of Surgery, University of Manitoba Max Rady College of Medicine, Winnipeg, MB R3E 0W2, CanadaDepartment of Surgery, University of Manitoba Max Rady College of Medicine, Winnipeg, MB R3E 0W2, CanadaDepartment of Computer Science, Memorial University of Newfoundland, St. John’s, NL A1B 3X7, CanadaDepartment of Surgery, University of Manitoba Max Rady College of Medicine, Winnipeg, MB R3E 0W2, CanadaDepartment of Surgery, University of Manitoba Max Rady College of Medicine, Winnipeg, MB R3E 0W2, CanadaDepartment of Surgery, University of Manitoba Max Rady College of Medicine, Winnipeg, MB R3E 0W2, CanadaPupillometry is commonly used to evaluate cognitive effort, attention, and facial expression response, offering valuable insights into human performance. The combination of eye tracking and facial expression data under the iMotions platform provides great opportunities for multimodal research. However, there is a lack of standardized pipelines for managing pupillometry data on a multimodal platform. Preprocessing pupil data in multimodal platforms poses challenges like timestamp misalignment, missing data, and inconsistencies across multiple data sources. To address these challenges, the authors introduced a systematic preprocessing pipeline for pupil diameter measurements collected using iMotions 10 (version 10.1.38911.4) during an endoscopy simulation task. The pipeline involves artifact removal, outlier detection using advanced methods such as the Median Absolute Deviation (MAD) and Moving Average (MA) algorithm filtering, interpolation of missing data using the Piecewise Cubic Hermite Interpolating Polynomial (PCHIP), and mean pupil diameter calculation through linear regression, as well as normalization of mean pupil diameter and integration of the pupil diameter dataset with facial expression data. By following these steps, the pipeline enhances data quality, reduces noise, and facilitates the seamless integration of pupillometry other multimodal datasets. In conclusion, this pipeline provides a detailed and organized preprocessing method that improves data reliability while preserving important information for further analysis.https://www.mdpi.com/1424-8220/25/15/4737pupillometrymultimodal datapreprocessingmedian absolute deviation (MAD)moving average (MA) filterpiecewise cubic hermite interpolatingpolynomial (PCHIP) |
| spellingShingle | Jingxiang Ong Wenjing He Princess Maglanque Xianta Jiang Lawrence M. Gillman Ashley Vergis Krista Hardy A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data Sensors pupillometry multimodal data preprocessing median absolute deviation (MAD) moving average (MA) filter piecewise cubic hermite interpolatingpolynomial (PCHIP) |
| title | A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data |
| title_full | A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data |
| title_fullStr | A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data |
| title_full_unstemmed | A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data |
| title_short | A Preprocessing Pipeline for Pupillometry Signal from Multimodal iMotion Data |
| title_sort | preprocessing pipeline for pupillometry signal from multimodal imotion data |
| topic | pupillometry multimodal data preprocessing median absolute deviation (MAD) moving average (MA) filter piecewise cubic hermite interpolatingpolynomial (PCHIP) |
| url | https://www.mdpi.com/1424-8220/25/15/4737 |
| work_keys_str_mv | AT jingxiangong apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT wenjinghe apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT princessmaglanque apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT xiantajiang apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT lawrencemgillman apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT ashleyvergis apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT kristahardy apreprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT jingxiangong preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT wenjinghe preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT princessmaglanque preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT xiantajiang preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT lawrencemgillman preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT ashleyvergis preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata AT kristahardy preprocessingpipelineforpupillometrysignalfrommultimodalimotiondata |