An associated perception import method for big data
Against the background of the existing database to the large data migration，Apache introduced the Sqoop as the main tool for the relational database to the big data migration.Sqoop simply cut the data table and randomly store it on diffe rent nodes.Being aimed at the problem of low efficiency of the...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Beijing Xintong Media Co., Ltd
2016-03-01
|
| Series: | Dianxin kexue |
| Subjects: | |
| Online Access: | http://www.telecomsci.com/thesisDetails#10.11959/j.issn.1000-0801.2016044 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Against the background of the existing database to the large data migration，Apache introduced the Sqoop as the main tool for the relational database to the big data migration.Sqoop simply cut the data table and randomly store it on diffe rent nodes.Being aimed at the problem of low efficiency of the query of the relationship between the Hadoop，a method of data importing and preprocessing was designed.To reduce the network transmission delay and improve the performance of the system，the high correlation degree was kept in the adjacent nodes.The contrast experiment shows that the performance of the data query can be improved greatly by the same or adjacent nodes. |
|---|---|
| ISSN: | 1000-0801 |