All About Faces
Interactive Generation of Realistic Facial Wrinkles from Sketchy Drawings
Hyeon-Joong Kim, A. Cengiz Öztireli, Il-Kyu Shin, Markus Gross, Soo-Mi Choi,
Corresponding Author
Soo-Mi Choi
Sejong University, Korea
Corresponding author ([email protected])Search for more papers by this authorHyeon-Joong Kim, A. Cengiz Öztireli, Il-Kyu Shin, Markus Gross, Soo-Mi Choi,
Corresponding Author
Soo-Mi Choi
Sejong University, Korea
Corresponding author ([email protected])Search for more papers by this authorAbstract
Synthesizing facial wrinkles has been tackled either by a long process of manual sculpting on 3D models, or using automatic methods that do not allow for user interaction or artistic expression. In this paper, we propose a method that accepts interactive sketchy drawings depicting wrinkle patterns, and synthesizes realistic looking wrinkles on faces. The method inherits the simplicity of sketching, making it possible for artists as well as novice users to generate realistic facial detail very efficiently, allowing fast preview for physical makeup, or aging simulations for fun and professional applications. All strokes are used to infer the wrinkles, retaining the expressiveness of the sketches and realism of the final result at the same time. This is achieved by designing novel multi-scale statistics tailored to the wrinkle geometry and coupled to the sketch interpretation method. The statistics capture the cross-sectional profiles of wrinkles at different scales and parts of a face. The strokes are augmented with the statistics extracted from given example face models, and applied to an input face model interactively. The interface gives the user control over the shapes and scales of wrinkles via sketching while adding extra details required for realism automatically.
Supporting Information
| Filename | Description |
|---|---|
| cgf12551-sup-0001-S1.mp427.2 MB | Supporting Information |
| cgf12551-sup-0002-S1.pdf21.4 MB | Supporting Information |
| cgf12551-sup-0003-S1.mp454.7 MB | Supporting Information |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- Akeo M., Hashimoto H., Kobayashi T., Shibusawa T.: Computer graphics system for reproducing three-dimensional shape from idea sketch. Comput. Graph. Forum 13, 3 (1994), 477–488. 3
10.1111/1467-8659.1330477 Google Scholar
- Alexander O., Rogers M., Lambeth W., Chiang J.-Y., MA W.-C., Wang C.-C., Debevec P.: The digital emily project: Achieving a photorealistic digital actor. IEEE Comput. Graph. & App. 30, 4 (2010), 20–31. 3
- Bickel B., Botsch M., Angst R., Matusik W., Otaduy M., Pfister H., Gross M.: Multi-scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3 (July 2007), 33:1–33:10. 2
- Beeler T., Bickel B., Beardsley P., Sumner B., Gross M.: High-quality single-shot capture of facial geometry. ACM Trans. Graph. (Proc. SIGGRAPH) 29, 3 (2010), 40:1–40:9. 11
- Bermano A.H., Bradley D., Beeler T., Zund F., NOWROUZEZAHRAI D., BARAN I., SORKINE-HORNUNG O., Pfister H., Sumner R.W., Bickel B., Gross M.: Facial performance enhancement using dynamic shape space analysis. ACM Trans. Graph. 33, 2 (Apr. 2014), 13:1–13:12. 3
- Beeler T., Bradley D., Zimmer H., Gross M.: Improved reconstruction of deforming surfaces by cancelling ambient occlusion. In Proc. ECCV 2012 (Berlin, Heidelberg, 2012), ECCV'12, Springer-Verlag, pp. 30–43. 11
- Bando Y., Kuratate T., Nishita T.: A simple method for modeling wrinkles on human skin. In Proc. Pacific Graphics ‘02 (Washington, DC, USA, 2002), PG ‘02, IEEE Computer Society, pp. 166–175. 2, 4
- Boissieux L., Kiss G., Thalmann N., Kalra P.: Simulation of skin aging and wrinkles with cosmetics insight. In Computer Animation and Simulation 2000 (2000), N. Magnenat-Thalmann, D. Thalmann, B. Arnaldi, (Eds.), Eurographics, Springer Vienna, pp. 15–27. 2
- Bickel B., Lang M., Botsch M., Otaduy M.A., Gross M.: Pose-space animation and transfer of facial details. In Proc. SCA 2008 (Aire-la-Ville, Switzerland, 2008), SCA ‘08, Eurographics Association, pp. 57–66. 3
- Blanz V., Vetter T.: A morphable model for the synthesis of 3d faces. In SIGGRAPH ‘99 (New York, NY, USA, 1999), SIGGRAPH ‘99, ACM Press/Addison-Wesley Publishing Co., pp. 187–194. 2
10.1145/311535.311556 Google Scholar
- Cutler L.D., Gershbein R., Wang X.C., Curtis C., Maigret E., Prasso L., Farson P.: An art-directed wrinkle system for cg character clothing. In Proc. SCA 2005 (New York, NY, USA, 2005), SCA ‘05, ACM, pp. 117–125. 2, 3
- Cutler L.D., Gershbein R., Wang X.C., Curtis C., Maigret E., Prasso L., Farson P.: An art-directed wrinkle system for cg character clothing and skin. Graphical Models 69, 5-6 (2007), 219–230. 2, 3
- Debreceni T.: Special Makeup Effects for Stage and Screen. Elsevier, 2009. 1, 2
- Dmitriev K., Makarov E.: Generating displacement from normal map for use in 3d games. In ACM SIGGRAPH 2011 Talks (New York, NY, USA, 2011), SIGGRAPH ‘11, ACM, pp. 91–9:1. 11
- Fu Y., GUO G., Huang T.S.: Age synthesis and estimation via faces: A survey. IEEE Trans. Pattern Analysis and Machine Intelligence 32, 11 (2010), 1955–1976. 1, 2
- Feng W.-W., Kim B.-U., YU Y.: Real-time data driven deformation using kernel canonical correlation analysis. ACM Trans. Graph. 27, 3 (Aug. 2008), 91:1–91:9. 2
- Gingold Y., Igarashi T., ZORIN D.: Structured annotations for 2d-to-3d modeling. ACM Trans. Graph. 28, 5 (December 2009), 148:1–148:9. 3
- Golovinskiy A., Matusik W., Pfister H., Rusinkiewicz S., Funkhouser T.: A statistical model for synthesis of detailed facial geometry. In ACM SIGGRAPH 2006 Papers (New York, NY, USA, 2006), SIGGRAPH ‘06, ACM, pp. 1025–1034. 2, 3, 4, 11
10.1145/1179352.1141988 Google Scholar
- Graham P., Tunwattanapong B., Busch J., YU X., Jones A., Debevec P., Ghosh A.: Measurement-based synthesis of facial microgeometry. Comput. Graph. Forum 32, 2 (2013), 335–344. 2
- Igarashi T., Matsuoka S., Tanaka H.: Teddy: a sketching interface for 3d freeform design. In Proc. SIGGRAPH ‘99 (New York, NY, USA, 1999), SIGGRAPH ‘99, ACM Press/Addison-Wesley Publishing Co., pp. 409–416. 3
10.1145/311535.311602 Google Scholar
- Igarashi T., Nishino K., Nayar S. K.: The appearance of human skin: A survey. Found. Trends. Comput. Graph. Vis. 3, 1 (Jan. 2007), 1–95. 2
10.1561/0600000013 Google Scholar
- Jimenez J., Echevarria J.I., Oat C., Gutierrez D.: GPU Pro 2. AK Peters Ltd., 2011, ch. Practical and Realistic Facial Wrinkles Animation, pp. 15–27. 11
10.1201/b11325-4 Google Scholar
- Jimenez J., Scully T., Barbosa N., Donner C., Alvarez X., Vieira T., Matts P., Orvalho V., Gutierrez D., Weyrich T.: A practical appearance model for dynamic facial color. ACM Trans. Graph. 29, 6 (Dec. 2010), 141:1–141:10. 11
- Kraevoy V., Sheffer A., van de Panne M.: Modeling from contour drawings. In Proc. SBIM ‘09 (New York, NY, USA, 2009), SBIM ‘09, ACM, pp. 37–44. 3
10.1145/1572741.1572749 Google Scholar
- Larboulette C., Cani M.-P.: Real-time dynamic wrinkles. In Proc. CGI ‘04 (Washington, DC, USA, 2004), CGI ‘04, IEEE Computer Society, pp. 522–525. 2
- LI P., Kry P. G.: Multi-layer skin simulation with adaptive constraints. In Proc. MIG ‘04 (New York, NY, USA, 2014), MIG ‘14, ACM, pp. 171–176. 2
- LI L., Liu F., LI C., Chen G.: Realistic wrinkle generation for 3d face modeling based on automatically extracted curves and improved shape control functions. Comput. & Graph. 35, 1 (2011), 175–184. Extended Papers from NPAR 2010. 2, 3, 4, 5, 10
- LI Y.-B., Xiao H., Zhang S.-Y.: The wrinkle generation method for facial reconstruction based on extraction of partition wrinkle line features and fractal interpolation. In Proc. ICIG ‘07 (Washington, DC, USA, 2007), ICIG ‘07, IEEE Computer Society, pp. 933–937. 2, 4
- Mancusi S.: The Police Composite Sketch. Springer, 2010. 2
10.1007/978-1-60761-832-4 Google Scholar
- MA W.-C., Jones A., Chiang J.-Y., Hawkins T., Frederiksen S., Peers P., Vukovic M., Ouhyoung M., Debevec P.: Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Trans. Graph. 27, 5 (Dec. 2008), 121:1–121:10. 3
- MA X., LE B.H., Deng Z.: Style learning and transferring for facial animation editing. In SCA (2009), D.W. Fellner, S.N. Spencer, (Eds.), ACM, pp. 123–132. 2
- Magnenat-Thalmann N., Kalra P., Luc Leveque J., Bazin R., Batisse D., Querleux B.: A computational skin model: fold and wrinkle formation. IEEE Trans. Inf. Technol. Biomed 6, 4 (2002), 317–323. 2
- Nealen A., Igarashi T., Sorkine O., Alexa M.: Fibermesh: designing freeform surfaces with 3d curves. ACM Trans. Graph. 26, 3 (July 2007), 41:1–41:10. 3
- Noris G., Sýkora D., Shamir A., Coros S., Whited B., Simmons M., Hornung A., Gross M., Sumner R.: Smart scribbles for sketch segmentation. Comput. Graph. Forum 31, 8 (2012), 2516–2527. 2, 3
- Oat C.: Animated wrinkle maps. In ACM SIGGRAPH 2007 Courses (New York, NY, USA, 2007), SIGGRAPH ‘07, ACM, pp. 33–37. 11
10.1145/1281500.1281667 Google Scholar
- Olsen L., Samavati F.F., Sousa M.C., Jorge J. A.: Sketch-based mesh augmentation. In Proc. SBIM ‘05 (Aug 28-29 2005), Eurographics, pp. 43–52. 3
- Olsen L., Samavati F.F., Sousa M.C., Jorge J.A.: Sketch-based modeling: A survey. Comput. & Graph. 33, 1 (2009), 85–103. 3, 11
- Pixologic: Zbrush, 2013. 3
- Paysan P., Knothe R., Amberg B., Romdhani S., Vetter T.: A 3d face model for pose and illumination invariant face recognition. In AVSS ‘09 (Sept 2009), pp. 296–301. 2
- Rohmer D., Popa T., Cani M.-P., Hahmann S., Sheffer A.: Animation wrinkling: augmenting coarse cloth simulations with realistic-looking wrinkles. In ACM SIGGRAPH Asia 2010 papers (New York, NY, USA, 2010), SIGGRAPH ASIA ‘10, ACM, pp. 157:1–157:8. 2
- Serup J., E. J. G. B., L. G. G.: Handbook of Non-Invasive Methods and the Skin, 2 ed. Taylor and Francis, 2006. 1, 2
10.3109/9781420003307 Google Scholar
- Scherbaum K., Sunkel M., Seidel H.-P., Blanz V.: Prediction of individual non-linear aging trajectories of faces. In EUROGRAPHICS 2007 (Prague, Czech Republic, 2007), vol. 26 of Computer Graphics Forum, The European Association for Computer Graphics, Blackwell, pp. 285–294. 2
- Sucontphunt T., Tunwattanapong B., Deng Z., Neumann U.: Crafting 3d faces using free form portrait sketching and plausible texture inference. In Proc. GI 2010 (Toronto, Ont., Canada, 2010), GI ‘10, Canadian Information Processing Society, pp. 209–216. 2
- Terzopoulos D., Waters K.: Physically-based facial modelling, analysis, and animation. The Journ. Visualization and Comput. Animat. 1, 2 (1990), 73–80. 2
10.1002/vis.4340010208 Google Scholar
- Venkataraman K., Lodha S., Raghavan R.: A kinematic-variational model for animating skin with wrinkles. Comput. & Graph. 29, 5 (2005), 756–770. 2
- Waters K.: A muscle model for animation three-dimensional facial expression. In Proc. SIGGRAPH ‘87 (New York, NY, USA, 1987), SIGGRAPH ‘87, ACM, pp. 17–24. 2
10.1145/37401.37405 Google Scholar
- Wu Y., Magnenat Thalmann N., Thalmann D.: A plastic-visco-elastic model for wrinkles in facial animation and skin aging. In Proc. PG ‘94 (River Edge, NJ, USA, 1994), Pacific Graphics ‘94, World Scientific Publishing Co., Inc., pp. 201–213. 2
- Wang Y., Wang C.C., Yuen M.M.: Fast energy-based surface wrinkle modeling. Comput. & Graph. 30, 1 (2006), 111–125. 2, 3
- Zahouani H., HUMBERT P.: Handbook of Non-Invasive Methods and the Skin, Second Edition. Taylor & Francis, 2006, ch. The Morphological Tree of the Cutaneous Network of Lines. 3, 11
- Zeleznik R.C., Herndon K.P., Hughes J.F.: Sketch: An interface for sketching 3d scenes. In ACM SIGGRAPH 2006 Courses (New York, NY, USA, 2006), SIGGRAPH ‘06, ACM. 3
- Zhou K., Huang J., Snyder J., Liu X., Bao H., Guo B., Shum H.-Y.: Large mesh deformation using the volumetric graph laplacian. ACM Trans. Graph. 24, 3 (July 2005), 496–503. 3
- Zhang Y., Prakash E., Sung E.: Real-time physically-based facial expression animation using mass-spring system. In Proc. CGI 01 (Washington, DC, USA, 2001), IEEE Computer Society, pp. 347–350. 2
- Zhang Y., Prakash E.C., Sung E.: A new physical model with multilayer architecture for facial expression animation using dynamic adaptive mesh. IEEE Trans. Visualization and Comput. Graph. 10, 3 (May 2004), 339–352. 2
- Zhang Y., Sim T.: Realistic and efficient wrinkle simulation using an anatomy-based face model with adaptive refinement. In Proc. CGI ‘05 (Washington, DC, USA, 2005), CGI ‘05, IEEE Computer Society, pp. 3–10. 2, 4
- Zhang Y., Sim T., Tan C.L., Sung E.: Anatomy-based face reconstruction for animation using multi-layer deformation. Journ. Visual Languages & Computing 17, 2 (2006), 126–160. 2
