Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data
Cities in the big data era hold the massive urban data to create valuable information and digitally enhanced services. Sources of urban data are generally categorized as one of the three types: official, social, and sensorial, which are from the government and enterprises, social networks of citizen...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2020-01-01
|
| Series: | Complexity |
| Online Access: | http://dx.doi.org/10.1155/2020/8914757 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832568540553543680 |
|---|---|
| author | Hui Liu Jingqing Jiang Yaowei Hou Jie Song |
| author_facet | Hui Liu Jingqing Jiang Yaowei Hou Jie Song |
| author_sort | Hui Liu |
| collection | DOAJ |
| description | Cities in the big data era hold the massive urban data to create valuable information and digitally enhanced services. Sources of urban data are generally categorized as one of the three types: official, social, and sensorial, which are from the government and enterprises, social networks of citizens, and the sensor network. These types typically differ significantly from each other but are consolidated together for the smart urban services. Based on the sophisticated consolidation approaches, we argue that a new challenge, fragment complexity that represents a well-integrated data has appropriate but fragmentary schema and difficult to be queried, is ignored in the state-of-art urban data management. Comparing with predefined and rigid schema, fragmentary schema means a dataset contains millions of attributes but nonorthogonally distributed among tables, and of course, values of these attributes are even massive. As far as a query is concerned, locating where these attributes are being stored is the first encountered problem, while traditional value-based query optimization has no contributions. To address this problem, we propose an index on massive attributes as an attributes-oriented optimization, namely, attribute index. Attribute index is a secondary index for locating files in which the target attributes are stored. It contains three parts: ATree for searching keys, DTree for locating keys among files, and ADLinks as a mapping table between ATree and DTree. In this paper, the index architecture, logical structure and algorithms, the implementation details, the creation process, the integration to the existing key-value store, and the urban application scenario are described. Experiments show that, in comparison with B + -Tree, LSM-Tree, and AVL-Tree, the query time of ATree is 1.1x, 1.5x, and 1.2x faster, respectively. Finally, we integrate our proposition with HBase, namely, UrbanBase, whose query performance is 1.3x faster than the original HBase. |
| format | Article |
| id | doaj-art-f909d6295419488b891ae5706927a3f8 |
| institution | Kabale University |
| issn | 1076-2787 1099-0526 |
| language | English |
| publishDate | 2020-01-01 |
| publisher | Wiley |
| record_format | Article |
| series | Complexity |
| spelling | doaj-art-f909d6295419488b891ae5706927a3f82025-02-03T00:58:51ZengWileyComplexity1076-27871099-05262020-01-01202010.1155/2020/89147578914757Solving the Fragment Complexity of Official, Social, and Sensorial Urban DataHui Liu0Jingqing Jiang1Yaowei Hou2Jie Song3School of Metallurgy, Northeastern University, Shenyang, ChinaCollege of Computer Science and Technology, Inner Mongolia University for the Nationalities, Tongliao, ChinaSoftware College, Northeastern University, Shenyang, ChinaSoftware College, Northeastern University, Shenyang, ChinaCities in the big data era hold the massive urban data to create valuable information and digitally enhanced services. Sources of urban data are generally categorized as one of the three types: official, social, and sensorial, which are from the government and enterprises, social networks of citizens, and the sensor network. These types typically differ significantly from each other but are consolidated together for the smart urban services. Based on the sophisticated consolidation approaches, we argue that a new challenge, fragment complexity that represents a well-integrated data has appropriate but fragmentary schema and difficult to be queried, is ignored in the state-of-art urban data management. Comparing with predefined and rigid schema, fragmentary schema means a dataset contains millions of attributes but nonorthogonally distributed among tables, and of course, values of these attributes are even massive. As far as a query is concerned, locating where these attributes are being stored is the first encountered problem, while traditional value-based query optimization has no contributions. To address this problem, we propose an index on massive attributes as an attributes-oriented optimization, namely, attribute index. Attribute index is a secondary index for locating files in which the target attributes are stored. It contains three parts: ATree for searching keys, DTree for locating keys among files, and ADLinks as a mapping table between ATree and DTree. In this paper, the index architecture, logical structure and algorithms, the implementation details, the creation process, the integration to the existing key-value store, and the urban application scenario are described. Experiments show that, in comparison with B + -Tree, LSM-Tree, and AVL-Tree, the query time of ATree is 1.1x, 1.5x, and 1.2x faster, respectively. Finally, we integrate our proposition with HBase, namely, UrbanBase, whose query performance is 1.3x faster than the original HBase.http://dx.doi.org/10.1155/2020/8914757 |
| spellingShingle | Hui Liu Jingqing Jiang Yaowei Hou Jie Song Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data Complexity |
| title | Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data |
| title_full | Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data |
| title_fullStr | Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data |
| title_full_unstemmed | Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data |
| title_short | Solving the Fragment Complexity of Official, Social, and Sensorial Urban Data |
| title_sort | solving the fragment complexity of official social and sensorial urban data |
| url | http://dx.doi.org/10.1155/2020/8914757 |
| work_keys_str_mv | AT huiliu solvingthefragmentcomplexityofofficialsocialandsensorialurbandata AT jingqingjiang solvingthefragmentcomplexityofofficialsocialandsensorialurbandata AT yaoweihou solvingthefragmentcomplexityofofficialsocialandsensorialurbandata AT jiesong solvingthefragmentcomplexityofofficialsocialandsensorialurbandata |