ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification

ALBERT-BiLSTM cross-attention network with progressive knowledge distillation for multi-domain SMS spam classification

SMS spam detection remains a critical challenge in digital communication systems, particularly in agriculture where farmers depend on SMS services for weather updates, crop prices, and government notifications. Traditional spam detection methods fail to handle evolving spam tactics and class imbalan...

Full description

Saved in:
Bibliographic Details
Main Authors: B.S. Aparna, Remya S, Manu J. Pillai, Somula Rama Subbareddy, Yong Yun Cho
Format: Article
Language:English
Published: Elsevier 2025-09-01
Series:Results in Engineering
Subjects:
Spam detection
BiLSTM
ALBERT
Knowledge distillation
Deep learning
Text classification
Online Access:http://www.sciencedirect.com/science/article/pii/S259012302502794X
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:SMS spam detection remains a critical challenge in digital communication systems, particularly in agriculture where farmers depend on SMS services for weather updates, crop prices, and government notifications. Traditional spam detection methods fail to handle evolving spam tactics and class imbalance effectively. This paper proposes a hybrid ensemble model that combines ALBERT transformer embeddings with BiLSTM networks enhanced by attention mechanisms and knowledge distillation. Experimental evaluation on the SMS Spam Collection dataset demonstrates superior performance with 98% accuracy, 0.98 precision, 0.97 recall, and 0.98 F1-score. The ensemble model significantly outperforms individual ALBERT (88% accuracy) and BiLSTM (54% accuracy) models. Knowledge distillation reduces model size from 425 MB to 67 MB, enabling real-time deployment with 0.045-second processing time per message. The system achieves 22.2 messages per second throughput, making it suitable for practical applications. Real-world testing in agricultural communication networks demonstrates significant improvements in spam filtering compared to traditional keyword-based filters. The hybrid approach provides an effective solution for SMS spam detection with demonstrated real-time performance and practical applicability in resource-constrained environments.
ISSN:2590-1230

Similar Items