Compaction-Aware Flash Memory Remapping for Key–Value Stores
With the rapid development of big data and artificial intelligence, the demand for memory has exploded. As a key data structure in modern databases and distributed storage systems, the Log-Structured Merge Tree (LSM-tree) has been widely employed (such as LevelDB, RocksDB, etc.) in systems based on...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Micromachines |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2072-666X/16/6/699 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850168314551074816 |
|---|---|
| author | Jialin Wang Zhen Yang Yi Fan Yajuan Du |
| author_facet | Jialin Wang Zhen Yang Yi Fan Yajuan Du |
| author_sort | Jialin Wang |
| collection | DOAJ |
| description | With the rapid development of big data and artificial intelligence, the demand for memory has exploded. As a key data structure in modern databases and distributed storage systems, the Log-Structured Merge Tree (LSM-tree) has been widely employed (such as LevelDB, RocksDB, etc.) in systems based on key–value pairs due to its efficient writing performance. In LSM-tree-based KV stores, typically deployed on systems with DRAM-SSD storage, the KV items are first organized into MemTable as buffer for SSTables in main memory. When the buffer size exceeds the threshold, MemTable is flushed to the SSD and reorganized into an SSTable, which is then passed down level by level through compaction. However, the compaction degrades write performance and SSD endurance due to significant write amplification. To address this issue, recent proposals have mostly focused on redesigning the structure of LSM trees. We discover the prevalence of unchanged data blocks (UDBs) in the LSM-tree compaction process, i.e., UDBs are written back to SSD the same as they are read into memory, which induces extra write amplification and degrades I/O performance. In this paper, we propose a KV store design in SSD, called RemapCom, to exploit remapping on these UDBs. RemapCom first identifies UDBs with a lightweight state machine integrated into the compaction merge process. In order to increase the ratio of UDBs, RemapCom further designs a UDB retention method to further develop the benefit of remapping. Moreover, we implement a prototype of RemapCom on LevelDB by providing two primitives for the remapping. Compared to the state of the art, the evaluation results demonstrate that RemapCom can reduce write amplification by up to 53% and improve write throughput by up to 30%. |
| format | Article |
| id | doaj-art-ea155de65ee84472aadf5807fcc05874 |
| institution | OA Journals |
| issn | 2072-666X |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Micromachines |
| spelling | doaj-art-ea155de65ee84472aadf5807fcc058742025-08-20T02:20:59ZengMDPI AGMicromachines2072-666X2025-06-0116669910.3390/mi16060699Compaction-Aware Flash Memory Remapping for Key–Value StoresJialin Wang0Zhen Yang1Yi Fan2Yajuan Du3College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, ChinaSchool of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, ChinaSchool of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, ChinaSchool of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan 430070, ChinaWith the rapid development of big data and artificial intelligence, the demand for memory has exploded. As a key data structure in modern databases and distributed storage systems, the Log-Structured Merge Tree (LSM-tree) has been widely employed (such as LevelDB, RocksDB, etc.) in systems based on key–value pairs due to its efficient writing performance. In LSM-tree-based KV stores, typically deployed on systems with DRAM-SSD storage, the KV items are first organized into MemTable as buffer for SSTables in main memory. When the buffer size exceeds the threshold, MemTable is flushed to the SSD and reorganized into an SSTable, which is then passed down level by level through compaction. However, the compaction degrades write performance and SSD endurance due to significant write amplification. To address this issue, recent proposals have mostly focused on redesigning the structure of LSM trees. We discover the prevalence of unchanged data blocks (UDBs) in the LSM-tree compaction process, i.e., UDBs are written back to SSD the same as they are read into memory, which induces extra write amplification and degrades I/O performance. In this paper, we propose a KV store design in SSD, called RemapCom, to exploit remapping on these UDBs. RemapCom first identifies UDBs with a lightweight state machine integrated into the compaction merge process. In order to increase the ratio of UDBs, RemapCom further designs a UDB retention method to further develop the benefit of remapping. Moreover, we implement a prototype of RemapCom on LevelDB by providing two primitives for the remapping. Compared to the state of the art, the evaluation results demonstrate that RemapCom can reduce write amplification by up to 53% and improve write throughput by up to 30%.https://www.mdpi.com/2072-666X/16/6/699LSM-treememorycompactionSSDdata remapping |
| spellingShingle | Jialin Wang Zhen Yang Yi Fan Yajuan Du Compaction-Aware Flash Memory Remapping for Key–Value Stores Micromachines LSM-tree memory compaction SSD data remapping |
| title | Compaction-Aware Flash Memory Remapping for Key–Value Stores |
| title_full | Compaction-Aware Flash Memory Remapping for Key–Value Stores |
| title_fullStr | Compaction-Aware Flash Memory Remapping for Key–Value Stores |
| title_full_unstemmed | Compaction-Aware Flash Memory Remapping for Key–Value Stores |
| title_short | Compaction-Aware Flash Memory Remapping for Key–Value Stores |
| title_sort | compaction aware flash memory remapping for key value stores |
| topic | LSM-tree memory compaction SSD data remapping |
| url | https://www.mdpi.com/2072-666X/16/6/699 |
| work_keys_str_mv | AT jialinwang compactionawareflashmemoryremappingforkeyvaluestores AT zhenyang compactionawareflashmemoryremappingforkeyvaluestores AT yifan compactionawareflashmemoryremappingforkeyvaluestores AT yajuandu compactionawareflashmemoryremappingforkeyvaluestores |