Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model
Addressing the challenges of limited accuracy in anomaly detection within comprehensive environmental monitoring of industrial and mining enterprises, and the constraints posed by singular data modalities, this study proposes an integration of a multimodal Long Short-Term Memory (LSTM) model with th...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10852209/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832575585492140032 |
|---|---|
| author | Shuqin Wang Na Cheng Yan Hu |
| author_facet | Shuqin Wang Na Cheng Yan Hu |
| author_sort | Shuqin Wang |
| collection | DOAJ |
| description | Addressing the challenges of limited accuracy in anomaly detection within comprehensive environmental monitoring of industrial and mining enterprises, and the constraints posed by singular data modalities, this study proposes an integration of a multimodal Long Short-Term Memory (LSTM) model with the Contrastive Language-Image Pretraining (CLIP) model. The initial phase employs ResNet within the CLIP model for extracting image features, and a Transformer for encoding text features. Subsequently, feature vectors obtained from monitoring images and text are fused using a rudimentary concatenation method to generate a joint embedding representation. Principal Component Analysis (PCA) is then applied to diminish the dimensionality of the amalgamated features derived from environmental monitoring images, descriptive texts, and sensor data collected by industrial and mining enterprises. Finally, a multimodal LSTM model is leveraged to detect anomalies in the monitoring data by capturing long-term dependencies within time series information. The model was trained and evaluated using real-time data from a coal mining enterprise’s environmental monitoring system spanning March to September 2023. Results reveal that the multimodal LSTM-CLIP model achieved an anomaly detection accuracy of 0.98 in environmental monitoring, marking a 0.10 improvement over the unimodal LSTM model, with a response time of merely 110.25 milliseconds. These findings underscore the efficacy of the multimodal LSTM-CLIP model in integrating multimodal information, thereby significantly enhancing the accuracy of anomaly detection and the speed of environmental anomaly warnings, ultimately ensuring the safety of industrial and mining enterprises. |
| format | Article |
| id | doaj-art-604e1c12587c4187ab845aeed96477d7 |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-604e1c12587c4187ab845aeed96477d72025-01-31T23:05:16ZengIEEEIEEE Access2169-35362025-01-0113199641997810.1109/ACCESS.2025.353353710852209Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP ModelShuqin Wang0Na Cheng1Yan Hu2https://orcid.org/0009-0003-0823-8056School of Computer Engineering, Anhui Wenda University of Information Engineering, Hefei, ChinaSchool of Computer Engineering, Anhui Wenda University of Information Engineering, Hefei, ChinaSchool of Computer Engineering, Anhui Wenda University of Information Engineering, Hefei, ChinaAddressing the challenges of limited accuracy in anomaly detection within comprehensive environmental monitoring of industrial and mining enterprises, and the constraints posed by singular data modalities, this study proposes an integration of a multimodal Long Short-Term Memory (LSTM) model with the Contrastive Language-Image Pretraining (CLIP) model. The initial phase employs ResNet within the CLIP model for extracting image features, and a Transformer for encoding text features. Subsequently, feature vectors obtained from monitoring images and text are fused using a rudimentary concatenation method to generate a joint embedding representation. Principal Component Analysis (PCA) is then applied to diminish the dimensionality of the amalgamated features derived from environmental monitoring images, descriptive texts, and sensor data collected by industrial and mining enterprises. Finally, a multimodal LSTM model is leveraged to detect anomalies in the monitoring data by capturing long-term dependencies within time series information. The model was trained and evaluated using real-time data from a coal mining enterprise’s environmental monitoring system spanning March to September 2023. Results reveal that the multimodal LSTM-CLIP model achieved an anomaly detection accuracy of 0.98 in environmental monitoring, marking a 0.10 improvement over the unimodal LSTM model, with a response time of merely 110.25 milliseconds. These findings underscore the efficacy of the multimodal LSTM-CLIP model in integrating multimodal information, thereby significantly enhancing the accuracy of anomaly detection and the speed of environmental anomaly warnings, ultimately ensuring the safety of industrial and mining enterprises.https://ieeexplore.ieee.org/document/10852209/Industrial and mining enterprisesintegrated environmental monitoring systemmultimodal LSTM modelCLIP modelanomaly detection accuracy |
| spellingShingle | Shuqin Wang Na Cheng Yan Hu Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model IEEE Access Industrial and mining enterprises integrated environmental monitoring system multimodal LSTM model CLIP model anomaly detection accuracy |
| title | Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model |
| title_full | Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model |
| title_fullStr | Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model |
| title_full_unstemmed | Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model |
| title_short | Comprehensive Environmental Monitoring System for Industrial and Mining Enterprises Using Multimodal Deep Learning and CLIP Model |
| title_sort | comprehensive environmental monitoring system for industrial and mining enterprises using multimodal deep learning and clip model |
| topic | Industrial and mining enterprises integrated environmental monitoring system multimodal LSTM model CLIP model anomaly detection accuracy |
| url | https://ieeexplore.ieee.org/document/10852209/ |
| work_keys_str_mv | AT shuqinwang comprehensiveenvironmentalmonitoringsystemforindustrialandminingenterprisesusingmultimodaldeeplearningandclipmodel AT nacheng comprehensiveenvironmentalmonitoringsystemforindustrialandminingenterprisesusingmultimodaldeeplearningandclipmodel AT yanhu comprehensiveenvironmentalmonitoringsystemforindustrialandminingenterprisesusingmultimodaldeeplearningandclipmodel |