Lightweight hybrid transformers-based dyslexia detection using cross-modality data
Abstract Early and precise diagnosis of dyslexia is crucial for implementing timely intervention to reduce its effects. Timely identification can improve the individual’s academic and cognitive performance. Traditional dyslexia detection (DD) relies on lengthy, subjective, restricted behavioral eval...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-05-01
|
| Series: | Scientific Reports |
| Subjects: | |
| Online Access: | https://doi.org/10.1038/s41598-025-01235-4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850273105254023168 |
|---|---|
| author | Abdul Rahaman Wahab Sait Yazeed Alkhurayyif |
| author_facet | Abdul Rahaman Wahab Sait Yazeed Alkhurayyif |
| author_sort | Abdul Rahaman Wahab Sait |
| collection | DOAJ |
| description | Abstract Early and precise diagnosis of dyslexia is crucial for implementing timely intervention to reduce its effects. Timely identification can improve the individual’s academic and cognitive performance. Traditional dyslexia detection (DD) relies on lengthy, subjective, restricted behavioral evaluations and interviews. Due to the limitations, deep learning (DL) models have been explored to improve DD by analyzing complex neurological, behavioral, and visual data. DL architectures, including convolutional neural networks (CNNs) and vision transformers (ViTs), encounter challenges in extracting meaningful patterns from cross-modality data. The lack of model interpretability and limited computational power restricts these models’ generalizability across diverse datasets. To overcome these limitations, we propose an innovative model for DD using magnetic resonance imaging (MRI), electroencephalography (EEG), and handwriting images. We introduce a model, leveraging hybrid transformer-based feature extraction, including SWIN-Linformer for MRI, LeViT-Performer for handwriting images, and graph transformer networks (GTNs) with multi-attention mechanisms for EEG data. A multi-modal attention-based feature fusion network was used to fuse the extracted features in order to guarantee the integration of key multi-modal features. We enhance Dartbooster XGBoost (DXB)-based classification using Bayesian optimization with Hyperband (BOHB) algorithm. In order to reduce computational overhead, we employ a quantization-aware training technique. The local interpretable model-agnostic explanations (LIME) technique and gradient-weighted class activation mapping (Grad-CAM) were adopted to enable model interpretability. Five public repositories were used to train and test the proposed model. The experimental outcomes demonstrated that the proposed model achieves an accuracy of 99.8% with limited computational overhead, outperforming baseline models. It sets a novel standard for DD, offering potential for early identification and timely intervention. In the future, advanced feature fusion and quantization techniques can be utilized to achieve optimal results in resource-constrained environments. |
| format | Article |
| id | doaj-art-fd2a510b86be4dc398dbc43824bc7492 |
| institution | OA Journals |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-fd2a510b86be4dc398dbc43824bc74922025-08-20T01:51:36ZengNature PortfolioScientific Reports2045-23222025-05-0115111810.1038/s41598-025-01235-4Lightweight hybrid transformers-based dyslexia detection using cross-modality dataAbdul Rahaman Wahab Sait0Yazeed Alkhurayyif1Department of Documents and Archive, Center of Documents and Administrative Communication, King Faisal UniversityDepartment of Computer Science, College of Computer Science, Shaqra UniversityAbstract Early and precise diagnosis of dyslexia is crucial for implementing timely intervention to reduce its effects. Timely identification can improve the individual’s academic and cognitive performance. Traditional dyslexia detection (DD) relies on lengthy, subjective, restricted behavioral evaluations and interviews. Due to the limitations, deep learning (DL) models have been explored to improve DD by analyzing complex neurological, behavioral, and visual data. DL architectures, including convolutional neural networks (CNNs) and vision transformers (ViTs), encounter challenges in extracting meaningful patterns from cross-modality data. The lack of model interpretability and limited computational power restricts these models’ generalizability across diverse datasets. To overcome these limitations, we propose an innovative model for DD using magnetic resonance imaging (MRI), electroencephalography (EEG), and handwriting images. We introduce a model, leveraging hybrid transformer-based feature extraction, including SWIN-Linformer for MRI, LeViT-Performer for handwriting images, and graph transformer networks (GTNs) with multi-attention mechanisms for EEG data. A multi-modal attention-based feature fusion network was used to fuse the extracted features in order to guarantee the integration of key multi-modal features. We enhance Dartbooster XGBoost (DXB)-based classification using Bayesian optimization with Hyperband (BOHB) algorithm. In order to reduce computational overhead, we employ a quantization-aware training technique. The local interpretable model-agnostic explanations (LIME) technique and gradient-weighted class activation mapping (Grad-CAM) were adopted to enable model interpretability. Five public repositories were used to train and test the proposed model. The experimental outcomes demonstrated that the proposed model achieves an accuracy of 99.8% with limited computational overhead, outperforming baseline models. It sets a novel standard for DD, offering potential for early identification and timely intervention. In the future, advanced feature fusion and quantization techniques can be utilized to achieve optimal results in resource-constrained environments.https://doi.org/10.1038/s41598-025-01235-4Cross-modalityMRIEEGHandwriting imagesFeature fusionMulti-attention mechanisms |
| spellingShingle | Abdul Rahaman Wahab Sait Yazeed Alkhurayyif Lightweight hybrid transformers-based dyslexia detection using cross-modality data Scientific Reports Cross-modality MRI EEG Handwriting images Feature fusion Multi-attention mechanisms |
| title | Lightweight hybrid transformers-based dyslexia detection using cross-modality data |
| title_full | Lightweight hybrid transformers-based dyslexia detection using cross-modality data |
| title_fullStr | Lightweight hybrid transformers-based dyslexia detection using cross-modality data |
| title_full_unstemmed | Lightweight hybrid transformers-based dyslexia detection using cross-modality data |
| title_short | Lightweight hybrid transformers-based dyslexia detection using cross-modality data |
| title_sort | lightweight hybrid transformers based dyslexia detection using cross modality data |
| topic | Cross-modality MRI EEG Handwriting images Feature fusion Multi-attention mechanisms |
| url | https://doi.org/10.1038/s41598-025-01235-4 |
| work_keys_str_mv | AT abdulrahamanwahabsait lightweighthybridtransformersbaseddyslexiadetectionusingcrossmodalitydata AT yazeedalkhurayyif lightweighthybridtransformersbaseddyslexiadetectionusingcrossmodalitydata |