GAN-based face identity feature recovery for image inpainting

In recent years, image inpainting techniques using GAN have yielded impressive results. However, repairing complete structure features and filling realistic textures in images with large-damaged areas is still a challenge. To solve this problem, we propose a refinement network to the baseline model to form a new two-step adversarial model, Semantic Guidance Refinement GAN (SGR-GAN). The first step of our model is the baseline model that consists of a pyramid structure encoder pSp-encoder and a pre-trained StyleGAN to generate a coarse result image with rich structural features. Then the second step is our proposed network, Facial Identity Preservation (FIP), for face identity preservation. To generate a more complete structural feature image, we add the semantic features of the coarse result into the FIP generator as semantic guidance. We have evaluated our model on the public dataset CelebA, and the experimental results also show that our model outperforms compared methods.