A hot-update-aware optimization to the query of LSM-Tree
Key-value stores based on LSM-Tree have been widely used.LSM-Tree gains excellent write performance by collecting updated data in memory and then flushing data into storage in batches.However, in LSMTree-based key-value stores, old data generated by update operations will not be eliminated immediate...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
China InfoCom Media Group
2023-01-01
|
| Series: | 大数据 |
| Subjects: | |
| Online Access: | http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2022049 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850212298581344256 |
|---|---|
| author | Qingyin LIN Zhiguang CHEN |
| author_facet | Qingyin LIN Zhiguang CHEN |
| author_sort | Qingyin LIN |
| collection | DOAJ |
| description | Key-value stores based on LSM-Tree have been widely used.LSM-Tree gains excellent write performance by collecting updated data in memory and then flushing data into storage in batches.However, in LSMTree-based key-value stores, old data generated by update operations will not be eliminated immediately from the storage system, resulting in a large amount of invalid data accumulated in the entire storage system, which will eventually significantly reduce the read performance of key-value stores.For the above problems, an active compaction method was proposed.By recording the history information of updated key-value pairs, recognizing hot-updated keys, finding SSTables that contain a large amount of invalid data in the storage system, and triggering compaction as soon as possible to clear much more invalid data, the proposed method could reduce write amplification and improve the read performance of LSM-Tree based key-value stores.Experiments showed that this method could reduce the average read latency of LevelDB by 65.2%, 99% read tail latency by 69.4%, and write amplification by 71.4%. |
| format | Article |
| id | doaj-art-45231a436ef5479b91c19b69077033a7 |
| institution | OA Journals |
| issn | 2096-0271 |
| language | zho |
| publishDate | 2023-01-01 |
| publisher | China InfoCom Media Group |
| record_format | Article |
| series | 大数据 |
| spelling | doaj-art-45231a436ef5479b91c19b69077033a72025-08-20T02:09:22ZzhoChina InfoCom Media Group大数据2096-02712023-01-01912614059778170A hot-update-aware optimization to the query of LSM-TreeQingyin LINZhiguang CHENKey-value stores based on LSM-Tree have been widely used.LSM-Tree gains excellent write performance by collecting updated data in memory and then flushing data into storage in batches.However, in LSMTree-based key-value stores, old data generated by update operations will not be eliminated immediately from the storage system, resulting in a large amount of invalid data accumulated in the entire storage system, which will eventually significantly reduce the read performance of key-value stores.For the above problems, an active compaction method was proposed.By recording the history information of updated key-value pairs, recognizing hot-updated keys, finding SSTables that contain a large amount of invalid data in the storage system, and triggering compaction as soon as possible to clear much more invalid data, the proposed method could reduce write amplification and improve the read performance of LSM-Tree based key-value stores.Experiments showed that this method could reduce the average read latency of LevelDB by 65.2%, 99% read tail latency by 69.4%, and write amplification by 71.4%.http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2022049key-value stores;log-structured merge tree;read performance optimization;write amplification |
| spellingShingle | Qingyin LIN Zhiguang CHEN A hot-update-aware optimization to the query of LSM-Tree 大数据 key-value stores;log-structured merge tree;read performance optimization;write amplification |
| title | A hot-update-aware optimization to the query of LSM-Tree |
| title_full | A hot-update-aware optimization to the query of LSM-Tree |
| title_fullStr | A hot-update-aware optimization to the query of LSM-Tree |
| title_full_unstemmed | A hot-update-aware optimization to the query of LSM-Tree |
| title_short | A hot-update-aware optimization to the query of LSM-Tree |
| title_sort | hot update aware optimization to the query of lsm tree |
| topic | key-value stores;log-structured merge tree;read performance optimization;write amplification |
| url | http://www.j-bigdataresearch.com.cn/thesisDetails#10.11959/j.issn.2096-0271.2022049 |
| work_keys_str_mv | AT qingyinlin ahotupdateawareoptimizationtothequeryoflsmtree AT zhiguangchen ahotupdateawareoptimizationtothequeryoflsmtree AT qingyinlin hotupdateawareoptimizationtothequeryoflsmtree AT zhiguangchen hotupdateawareoptimizationtothequeryoflsmtree |