Robust 3D face shape reconstruction from single images via two-fold coupled structure learning

In this paper, we propose a robust method for monocular face shape reconstruction (MFSR) using a sparse set of facial landmarks that are detected by most of the off-the-shelf landmark detectors. Different from the classical shape-from-shading framework, we formulate the MFSR problem as a Two-Fold Coupled Structure Learning (2FCSL) process, which consists of learning a regression between two subspaces spanned by 3D sparse landmarks and 2D sparse landmarks, and a coupled dictionary learned on 3D sparse and dense shape using K-SVD. To handle variations in face pose, we explicitly incorporate pose estimation in our method. Extensive experiments on both synthetic and real data from two challenging datasets using manual and automatic landmarks indicate that our method achieves promising performance and is robust to pose variations and landmark localization noise.