Hybrid Spatial and Frequency Network for Light Field Image Restoration

This paper proposes a novel hybrid light field (LF) restoration method based on a deep convolutional neural network (CNN) designed to capture the characteristics of LF images in both pixel and frequency domains. Restoring high-quality LF images from degraded versions is a complex task due to the high dimensionality of LF data. To address this, we leverage the geometric priors of LF images to design efficient restoration network components capable of effectively handling the 4D LF structure across both pixel and frequency domains. In the frequency restoration stage, where image artifacts often exhibit distinct frequency characteristics, we propose a 4D-DCT separated transform using 2D-DCT in spatial and angular pixel correlations. By decomposing transformed LF data into various frequency components, our frequency restoration network progressively recovers detailed information from each subband frequency component, enhancing performance in complex scenes and noisy images. For pixel restoration, we introduce the geometry-aware attention (GAM) mechanisms into spatial, angular, and epipolar dimensions of the 4D LF structure, helping to capture better global information in each LF embedding feature. Extensive experiments across diverse LF restoration tasks, including LF denoising, LF spatial super-resolution, and LF low-light enhancement, validate the effectiveness of our method compared to state-of-the-art approaches in both objective and subjective quality assessments.