TY - GEN
T1 - Improving Access to HDFS using NVMeoF
AU - Han, Daegyu
AU - Nam, Beomseok
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/9
Y1 - 2019/9
N2 - In this extended abstract, we discuss how to improve access to HDFS by employing NVMe over Fabrics (NVMeoF), which has emerged as a new communication protocol between a host and a storage system. To address the legacy shortcomings of HDFS, we explore the opportunity of enabling all-to-all connections between NVMe SSDs and DataNodes rather than dedicating each NVMe to a single DataNode. Our experimental study shows that we can achieve up to 2.55 times higher I/O throughput than legacy HDFS by making HDFS leverage NVMeoF for remote data access.
AB - In this extended abstract, we discuss how to improve access to HDFS by employing NVMe over Fabrics (NVMeoF), which has emerged as a new communication protocol between a host and a storage system. To address the legacy shortcomings of HDFS, we explore the opportunity of enabling all-to-all connections between NVMe SSDs and DataNodes rather than dedicating each NVMe to a single DataNode. Our experimental study shows that we can achieve up to 2.55 times higher I/O throughput than legacy HDFS by making HDFS leverage NVMeoF for remote data access.
UR - https://www.scopus.com/pages/publications/85075267898
U2 - 10.1109/CLUSTER.2019.8890996
DO - 10.1109/CLUSTER.2019.8890996
M3 - Conference contribution
AN - SCOPUS:85075267898
T3 - Proceedings - IEEE International Conference on Cluster Computing, ICCC
BT - Proceedings - 2019 IEEE International Conference on Cluster Computing, CLUSTER 2019
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2019 IEEE International Conference on Cluster Computing, CLUSTER 2019
Y2 - 23 September 2019 through 26 September 2019
ER -