Dataflow model and its applications in big data processing
Unbounded,unordered and large scale datasets are increasingly common in recent years.Meanwhile,the processing requirements from data consumers are becoming more and more sophisticated,such as event time,window and latency.In order to deal with the evolved processing requirements on these unbounded,u...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
China InfoCom Media Group
2020-05-01
|
| Series: | 大数据 |
| Subjects: | |
| Online Access: | http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2020025 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850214406502219776 |
|---|---|
| author | Nifei BI Guangyao DING Qihang CHEN Chen XU Aoying ZHOU |
| author_facet | Nifei BI Guangyao DING Qihang CHEN Chen XU Aoying ZHOU |
| author_sort | Nifei BI |
| collection | DOAJ |
| description | Unbounded,unordered and large scale datasets are increasingly common in recent years.Meanwhile,the processing requirements from data consumers are becoming more and more sophisticated,such as event time,window and latency.In order to deal with the evolved processing requirements on these unbounded,unordered and large scale datasets,the dataflow model in big data processing was introduced.On one hand,the dataflow graph of the dataflow model in big data processing was analyzed from the level of execution engine.On other hand,the dataflow programming model of the dataflow model in big data processing was analyzed from the level of unified programming.Furthermore,the different implementations of dataflow graph and dataflow programming model in multiple execution engines were analyzed,including Spark,a batch processing engine,and Flink,a stream processing engine. |
| format | Article |
| id | doaj-art-e3140aba225e47b687731ea4829c290d |
| institution | OA Journals |
| issn | 2096-0271 |
| language | zho |
| publishDate | 2020-05-01 |
| publisher | China InfoCom Media Group |
| record_format | Article |
| series | 大数据 |
| spelling | doaj-art-e3140aba225e47b687731ea4829c290d2025-08-20T02:08:54ZzhoChina InfoCom Media Group大数据2096-02712020-05-0162020025159536616Dataflow model and its applications in big data processingNifei BIGuangyao DINGQihang CHENChen XUAoying ZHOUUnbounded,unordered and large scale datasets are increasingly common in recent years.Meanwhile,the processing requirements from data consumers are becoming more and more sophisticated,such as event time,window and latency.In order to deal with the evolved processing requirements on these unbounded,unordered and large scale datasets,the dataflow model in big data processing was introduced.On one hand,the dataflow graph of the dataflow model in big data processing was analyzed from the level of execution engine.On other hand,the dataflow programming model of the dataflow model in big data processing was analyzed from the level of unified programming.Furthermore,the different implementations of dataflow graph and dataflow programming model in multiple execution engines were analyzed,including Spark,a batch processing engine,and Flink,a stream processing engine.http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2020025dataflow model;big data processing;execution engine;unified programming |
| spellingShingle | Nifei BI Guangyao DING Qihang CHEN Chen XU Aoying ZHOU Dataflow model and its applications in big data processing 大数据 dataflow model;big data processing;execution engine;unified programming |
| title | Dataflow model and its applications in big data processing |
| title_full | Dataflow model and its applications in big data processing |
| title_fullStr | Dataflow model and its applications in big data processing |
| title_full_unstemmed | Dataflow model and its applications in big data processing |
| title_short | Dataflow model and its applications in big data processing |
| title_sort | dataflow model and its applications in big data processing |
| topic | dataflow model;big data processing;execution engine;unified programming |
| url | http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2020025 |
| work_keys_str_mv | AT nifeibi dataflowmodelanditsapplicationsinbigdataprocessing AT guangyaoding dataflowmodelanditsapplicationsinbigdataprocessing AT qihangchen dataflowmodelanditsapplicationsinbigdataprocessing AT chenxu dataflowmodelanditsapplicationsinbigdataprocessing AT aoyingzhou dataflowmodelanditsapplicationsinbigdataprocessing |