Retinal Thickness Prediction from Multi-modal Fundus Photography

Retinal thickness map (RTM), generated from OCT volumes, provides a quantitative representation of the retina, which is then averaged into the ETDRS grid. The RTM and ETDRS grid are often used to diagnose and monitor retinal-related diseases that cause vision loss worldwide. However, OCT examinations can be available to limited patients because it is costly and time-consuming. Fundus photography (FP) is a 2D imaging technique for the retina that captures the reflection of a flash of light. However, current researches often focus on 2D patterns in FP, while its capacity of carrying thickness information is rarely explored. In this paper, we explore the capability of infrared fundus photography (IR-FP) and color fundus photography (C-FP) to provide accurate retinal thickness information. We propose a Multi-Modal Fundus photography enabled Retinal Thickness prediction network (M²FRT ). We predict RTM from IR-FP to overcome the limitation of acquiring RTM with OCT, which boosts mass screening with a cost-effective and efficient solution. We first introduce C-FP to provide IR-FP with complementary thickness information for more precise RTM prediction. The misalignment of images from the two modalities is tackled by the Transformer-CNN hybrid design in M ²FRT. Furthermore, we obtain the ETDRS grid prediction solely from C-FP using a lightweight decoder, which is optimized with the guidance of the RTM prediction task during the training phase. Our methodology utilizes the easily acquired C-FP, making it a valuable resource for providing retinal thickness quantification in clinical practice and telemedicine, thereby holding immense clinical significance.