FIDR: A scalable storage system for fine-grain inline data reduction with efficient memory handling

Storage systems play a critical role in modern servers which run highly data-intensive applications. To satisfy the high performance and capacity demands of such applications, storage systems now deploy an array of fast SSDs per server. To reduce the storage cost of employing many SSDs per server, storage systems actively perform inline data reduction (e.g., data deduplication, compression). Existing inline data reduction studies can achieve high performance and scalability by offloading computation-intensive data-reduction operations to dedicated hardware accelerators. However, such existing studies suffer from limited workload support and scalability. For example, they reduce only large data blocks, which incur many IO requests, leading to low data reduction rates, and their offloading overlooks memory-intensive operations, leading to the unoptimal scalability. In this paper, we propose FIDR, a highly scalable storage system to enable the inline data reduction of fine-grain data.We first identify key limitations of existing studies, and then set our scaling storage server design which effectively resolves the limitations by employing an optimal offloading mechanism. The key ideas of FIDR are to achieve high applicability by enabling fine-grain data reduction and high scalability by distributing data-reduction operations to optimal devices (e.g., host processor, accelerator, network interface card). The proposed offloading mechanism considers computation, memory capacity, and memory bandwidth requirements altogether. For evaluation, we implement an example FIDR system prototype using FPGAs. Our prototype system outperforms a current stateof- the-art data reduction system up to 3.3 times by significantly reducing both computation and memory resource requirements.