Depth hole filling based on deep learning for robust grasp detection

In current decades, object grasp detection of a diverse range of novel objects using vision systems has been developed. In order to achieve full performance, it requires high-quality depth images. However, commodity depth cameras often offer invalid depth pixels due to dark, shining surfaces and edges between the foreground and background of the scene. To address this problem, we propose a deep learning based depth hole filling method. The depth hole filling network learns to predict a ground truth depth map for a given sparse depth map. We generate the artificial sparse depth images from Dex-Net 2.0 by simulating the common situation of the depth hole generation in the commodity depth camera to train the network. The proposed model fills the depth hole with the RMSE value of 7.1 ± 4.1mm. The grasp detection performance using our model with the sparse depth image is comparable with the performance when using the ground truth image.