ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification
SMS spam detection remains a critical challenge in digital communication systems, particularly in agriculture where farmers depend on SMS services for weather updates, crop prices, and government notifications. Traditional spam detection methods fail to handle evolving spam tactics and class imbalan...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-09-01
|
| Series: | Results in Engineering |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S259012302502794X |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849226912323338240 |
|---|---|
| author | B.S. Aparna Remya S Manu J. Pillai Somula Rama Subbareddy Yong Yun Cho |
| author_facet | B.S. Aparna Remya S Manu J. Pillai Somula Rama Subbareddy Yong Yun Cho |
| author_sort | B.S. Aparna |
| collection | DOAJ |
| description | SMS spam detection remains a critical challenge in digital communication systems, particularly in agriculture where farmers depend on SMS services for weather updates, crop prices, and government notifications. Traditional spam detection methods fail to handle evolving spam tactics and class imbalance effectively. This paper proposes a hybrid ensemble model that combines ALBERT transformer embeddings with BiLSTM networks enhanced by attention mechanisms and knowledge distillation. Experimental evaluation on the SMS Spam Collection dataset demonstrates superior performance with 98% accuracy, 0.98 precision, 0.97 recall, and 0.98 F1-score. The ensemble model significantly outperforms individual ALBERT (88% accuracy) and BiLSTM (54% accuracy) models. Knowledge distillation reduces model size from 425 MB to 67 MB, enabling real-time deployment with 0.045-second processing time per message. The system achieves 22.2 messages per second throughput, making it suitable for practical applications. Real-world testing in agricultural communication networks demonstrates significant improvements in spam filtering compared to traditional keyword-based filters. The hybrid approach provides an effective solution for SMS spam detection with demonstrated real-time performance and practical applicability in resource-constrained environments. |
| format | Article |
| id | doaj-art-0fffc5ff241f40afbabd865fad93434e |
| institution | Kabale University |
| issn | 2590-1230 |
| language | English |
| publishDate | 2025-09-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Results in Engineering |
| spelling | doaj-art-0fffc5ff241f40afbabd865fad93434e2025-08-24T05:14:20ZengElsevierResults in Engineering2590-12302025-09-012710672710.1016/j.rineng.2025.106727ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classificationB.S. Aparna0Remya S1Manu J. Pillai2Somula Rama Subbareddy3Yong Yun Cho4Department of CSE, TKM College of Engineering, Kollam, 691005, Kerala, IndiaDepartment of CSE, Amrita School of Computing, Amrita Vishwa Vidyapeetham, Amritapuri, 690525, Kerala, IndiaDepartment of CSE, TKM College of Engineering, Kollam, 691005, Kerala, IndiaDepartment of Computer Science and Engineering, Symbiosis Institute of Technology, Hyderabad Campus, Symbiosis International (Deemed University), 509217, Pune, IndiaDepartment of Information and Communication Engineering, Sunchon National University, Jeollanam-do, Suncheon, South Korea; Corresponding author.SMS spam detection remains a critical challenge in digital communication systems, particularly in agriculture where farmers depend on SMS services for weather updates, crop prices, and government notifications. Traditional spam detection methods fail to handle evolving spam tactics and class imbalance effectively. This paper proposes a hybrid ensemble model that combines ALBERT transformer embeddings with BiLSTM networks enhanced by attention mechanisms and knowledge distillation. Experimental evaluation on the SMS Spam Collection dataset demonstrates superior performance with 98% accuracy, 0.98 precision, 0.97 recall, and 0.98 F1-score. The ensemble model significantly outperforms individual ALBERT (88% accuracy) and BiLSTM (54% accuracy) models. Knowledge distillation reduces model size from 425 MB to 67 MB, enabling real-time deployment with 0.045-second processing time per message. The system achieves 22.2 messages per second throughput, making it suitable for practical applications. Real-world testing in agricultural communication networks demonstrates significant improvements in spam filtering compared to traditional keyword-based filters. The hybrid approach provides an effective solution for SMS spam detection with demonstrated real-time performance and practical applicability in resource-constrained environments.http://www.sciencedirect.com/science/article/pii/S259012302502794XSpam detectionBiLSTMALBERTKnowledge distillationDeep learningText classification |
| spellingShingle | B.S. Aparna Remya S Manu J. Pillai Somula Rama Subbareddy Yong Yun Cho ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification Results in Engineering Spam detection BiLSTM ALBERT Knowledge distillation Deep learning Text classification |
| title | ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification |
| title_full | ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification |
| title_fullStr | ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification |
| title_full_unstemmed | ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification |
| title_short | ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification |
| title_sort | albert bilstm cross attention network with progressive knowledge distillation for multi domain sms spam classification |
| topic | Spam detection BiLSTM ALBERT Knowledge distillation Deep learning Text classification |
| url | http://www.sciencedirect.com/science/article/pii/S259012302502794X |
| work_keys_str_mv | AT bsaparna albertbilstmcrossattentionnetworkwithprogressiveknowledgedistillationformultidomainsmsspamclassification AT remyas albertbilstmcrossattentionnetworkwithprogressiveknowledgedistillationformultidomainsmsspamclassification AT manujpillai albertbilstmcrossattentionnetworkwithprogressiveknowledgedistillationformultidomainsmsspamclassification AT somularamasubbareddy albertbilstmcrossattentionnetworkwithprogressiveknowledgedistillationformultidomainsmsspamclassification AT yongyuncho albertbilstmcrossattentionnetworkwithprogressiveknowledgedistillationformultidomainsmsspamclassification |