The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka
With the advent of Industry 4.0, the manufacturing industry is facing unprecedented data challenges. Sensors, PLCs, and various types of automation equipment in smart factories continue to generate massive amounts of heterogeneous data, but existing systems generally have bottlenecks in data collect...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/12/6862 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849418359999823872 |
|---|---|
| author | Daixing Lu Kun Wang Yubo Wang Ye Shen |
| author_facet | Daixing Lu Kun Wang Yubo Wang Ye Shen |
| author_sort | Daixing Lu |
| collection | DOAJ |
| description | With the advent of Industry 4.0, the manufacturing industry is facing unprecedented data challenges. Sensors, PLCs, and various types of automation equipment in smart factories continue to generate massive amounts of heterogeneous data, but existing systems generally have bottlenecks in data collection standardization, real-time processing capabilities, and system scalability, which make it difficult to meet the needs of efficient collaboration and dynamic decision making. This study proposes a multi-level industrial data processing framework based on edge computing that aims to improve the response speed and processing ability of manufacturing sites to data and to realize real-time decision making and lean management of intelligent manufacturing. At the edge layer, the OPC UA (OPC Unified Architecture) protocol is used to realize the standardized collection of heterogeneous equipment data, and a lightweight edge-computing algorithm is designed to complete the analysis and processing of data so as to realize a visualization of the manufacturing process and the inventory in a production workshop. In the storage layer, Apache Kafka is used to implement efficient data stream processing and improve the throughput and scalability of the system. The test results show that compared with the traditional workshop, the framework has excellent performance in improving the system throughput capacity and real-time response speed, can effectively support production process judgment and status analysis on the edge side, and can realize the real-time monitoring and management of the entire manufacturing workshop. This research provides a practical solution for the industrial data management system, not only helping enterprises improve the transparency level of manufacturing sites and the efficiency of resource scheduling but also providing a practical basis for further research on industrial data processing under the “edge-cloud collaboration” architecture in the academic community. |
| format | Article |
| id | doaj-art-6166e9e987634e60b66e2a6b3706286e |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-6166e9e987634e60b66e2a6b3706286e2025-08-20T03:32:27ZengMDPI AGApplied Sciences2076-34172025-06-011512686210.3390/app15126862The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and KafkaDaixing Lu0Kun Wang1Yubo Wang2Ye Shen3School of Mechanical Engineering, Shanghai Institute of Technology, Shanghai 201418, ChinaSchool of Mechanical Engineering, Shanghai Institute of Technology, Shanghai 201418, ChinaDepartment of Mechanical Engineering, RheinMain University of Applied Sciences, 65428 Rüsselsheim, GermanyHaina-Intelligent Manufacturing Industrial Software Research Center, Yangtze Delta Region Institute of Tsinghua University, Jiaxing 314006, ChinaWith the advent of Industry 4.0, the manufacturing industry is facing unprecedented data challenges. Sensors, PLCs, and various types of automation equipment in smart factories continue to generate massive amounts of heterogeneous data, but existing systems generally have bottlenecks in data collection standardization, real-time processing capabilities, and system scalability, which make it difficult to meet the needs of efficient collaboration and dynamic decision making. This study proposes a multi-level industrial data processing framework based on edge computing that aims to improve the response speed and processing ability of manufacturing sites to data and to realize real-time decision making and lean management of intelligent manufacturing. At the edge layer, the OPC UA (OPC Unified Architecture) protocol is used to realize the standardized collection of heterogeneous equipment data, and a lightweight edge-computing algorithm is designed to complete the analysis and processing of data so as to realize a visualization of the manufacturing process and the inventory in a production workshop. In the storage layer, Apache Kafka is used to implement efficient data stream processing and improve the throughput and scalability of the system. The test results show that compared with the traditional workshop, the framework has excellent performance in improving the system throughput capacity and real-time response speed, can effectively support production process judgment and status analysis on the edge side, and can realize the real-time monitoring and management of the entire manufacturing workshop. This research provides a practical solution for the industrial data management system, not only helping enterprises improve the transparency level of manufacturing sites and the efficiency of resource scheduling but also providing a practical basis for further research on industrial data processing under the “edge-cloud collaboration” architecture in the academic community.https://www.mdpi.com/2076-3417/15/12/6862intelligent manufacturingOPC UAedge computinghigh concurrencyreal time |
| spellingShingle | Daixing Lu Kun Wang Yubo Wang Ye Shen The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka Applied Sciences intelligent manufacturing OPC UA edge computing high concurrency real time |
| title | The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka |
| title_full | The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka |
| title_fullStr | The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka |
| title_full_unstemmed | The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka |
| title_short | The Proposal and Validation of a Distributed Real-Time Data Management Framework Based on Edge Computing with OPC Unified Architecture and Kafka |
| title_sort | proposal and validation of a distributed real time data management framework based on edge computing with opc unified architecture and kafka |
| topic | intelligent manufacturing OPC UA edge computing high concurrency real time |
| url | https://www.mdpi.com/2076-3417/15/12/6862 |
| work_keys_str_mv | AT daixinglu theproposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT kunwang theproposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT yubowang theproposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT yeshen theproposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT daixinglu proposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT kunwang proposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT yubowang proposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka AT yeshen proposalandvalidationofadistributedrealtimedatamanagementframeworkbasedonedgecomputingwithopcunifiedarchitectureandkafka |