Gaze correction in video communication with single camera

In face-to-face video communication, we commonly use only single camera is placed on top of monitor screen. This general configuration occurs poor eye contact problem to decrease the feeling of natural conversation since the user stares at the monitor screen rather than directly staring the camera lens. In this paper we present a new approach for natural feeling in video communication using image-based modeling and rendering techniques. Our facial modeling approach has two components. The first component is to estimate the eye position from a input image to find the gaze-correction angle and generate the basic model to represent the user's facial shape. The second component is model-based shape approximation from motion which generate the user's simple facial shape to be used in rendering part from a few input frames which have facial motions. To render a good eye-contact image, we propose 3D mesh warping technique, a method to rotate input image with the correction angle and the facial model. Our approach is effective and convenient since it use the characteristic information about facial scene. Preliminary experimental result with real facial image shows enhanced naturalness which the face-to-face video communication has to offer.