Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology
With the explosive growth of data volume in the era of the digital economy, data platforms are facing increasing challenges in data storage, processing, and calculation. This paper proposes multiple Data Lakes under a unified Data Middle Platform, which is applied in industrial big data. The archite...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
China InfoCom Media Group
2025-01-01
|
| Series: | 大数据 |
| Subjects: | |
| Online Access: | http://www.j-bigdataresearch.com.cn/zh/article/109257467/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849433518529052672 |
|---|---|
| author | JIANG Nan |
| author_facet | JIANG Nan |
| author_sort | JIANG Nan |
| collection | DOAJ |
| description | With the explosive growth of data volume in the era of the digital economy, data platforms are facing increasing challenges in data storage, processing, and calculation. This paper proposes multiple Data Lakes under a unified Data Middle Platform, which is applied in industrial big data. The architecture, based on the integration of lake and warehouse technology, aims to build an efficient, flexible, and scalable data management platform to support cross-level, cross-organizational, and cross-regional data-sharing applications. With the centralized deployment of the three-level data lake and the middle platform deployed uniformly, the five core functions are constructed, forming the data acquisition capabilities of multi-source heterogeneous data, multi-mode data storage, data processing, data governance, data service, and data security protection. Through multi-data-lakes, collaborative interaction, and unified data control ability, to realize the overall management and open sharing of data of super-large group companies, application scenarios in various business fields are supported, and the value of data elements is released. |
| format | Article |
| id | doaj-art-aa1731342e6f470ca201f94dd1b13a64 |
| institution | Kabale University |
| issn | 2096-0271 |
| language | zho |
| publishDate | 2025-01-01 |
| publisher | China InfoCom Media Group |
| record_format | Article |
| series | 大数据 |
| spelling | doaj-art-aa1731342e6f470ca201f94dd1b13a642025-08-20T03:27:01ZzhoChina InfoCom Media Group大数据2096-02712025-01-01115109257467Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technologyJIANG NanWith the explosive growth of data volume in the era of the digital economy, data platforms are facing increasing challenges in data storage, processing, and calculation. This paper proposes multiple Data Lakes under a unified Data Middle Platform, which is applied in industrial big data. The architecture, based on the integration of lake and warehouse technology, aims to build an efficient, flexible, and scalable data management platform to support cross-level, cross-organizational, and cross-regional data-sharing applications. With the centralized deployment of the three-level data lake and the middle platform deployed uniformly, the five core functions are constructed, forming the data acquisition capabilities of multi-source heterogeneous data, multi-mode data storage, data processing, data governance, data service, and data security protection. Through multi-data-lakes, collaborative interaction, and unified data control ability, to realize the overall management and open sharing of data of super-large group companies, application scenarios in various business fields are supported, and the value of data elements is released.http://www.j-bigdataresearch.com.cn/zh/article/109257467/Data LakeLakehousedata middle platformindustrial big datadata governancedata service |
| spellingShingle | JIANG Nan Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology 大数据 Data Lake Lakehouse data middle platform industrial big data data governance data service |
| title | Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology |
| title_full | Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology |
| title_fullStr | Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology |
| title_full_unstemmed | Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology |
| title_short | Research and application practice of “Multi- Data Lake under One Middle-Platform” industrial big data platform based on Lakehouse technology |
| title_sort | research and application practice of multi data lake under one middle platform industrial big data platform based on lakehouse technology |
| topic | Data Lake Lakehouse data middle platform industrial big data data governance data service |
| url | http://www.j-bigdataresearch.com.cn/zh/article/109257467/ |
| work_keys_str_mv | AT jiangnan researchandapplicationpracticeofmultidatalakeunderonemiddleplatformindustrialbigdataplatformbasedonlakehousetechnology |