Measurement-Based Synthesis of Facial Microgeometry
Paul Graham,
Borom Tunwattanapong,
Jay Busch,
Xueming Yu,
Andrew Jones,
Paul Debevec,
Abhijeet Ghosh,
Paul Graham
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorBorom Tunwattanapong
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorJay Busch
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorXueming Yu
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorAndrew Jones
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorPaul Debevec
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorAbhijeet Ghosh
USC Institute for Creative Technologies, Los Angeles, CA, USA
Currently at Imperial College London
Search for more papers by this authorPaul Graham,
Borom Tunwattanapong,
Jay Busch,
Xueming Yu,
Andrew Jones,
Paul Debevec,
Abhijeet Ghosh,
Paul Graham
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorBorom Tunwattanapong
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorJay Busch
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorXueming Yu
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorAndrew Jones
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorPaul Debevec
USC Institute for Creative Technologies, Los Angeles, CA, USA
Search for more papers by this authorAbhijeet Ghosh
USC Institute for Creative Technologies, Los Angeles, CA, USA
Currently at Imperial College London
Search for more papers by this authorAbstract
We present a technique for generating microstructure-level facial geometry by augmenting a mesostructure-level facial scan with detail synthesized from a set of exemplar skin patches scanned at much higher resolution. Additionally, we make point-source reflectance measurements of the skin patches to characterize the specular reflectance lobes at this smaller scale and analyze facial reflectance variation at both the mesostructure and microstructure scales. We digitize the exemplar patches with a polarization-based computational illumination technique which considers specular reflection and single scattering. The recorded microstructure patches can be used to synthesize full-facial microstructure detail for either the same subject or to a different subject. We show that the technique allows for greater realism in facial renderings including more accurate reproduction of skin's specular reflection effects.
Supporting Information
Please note: Blackwell Publishing are not responsible for the content or functionality of any supplementary materials supplied by the authors. Any queries (other than missing material) should be directed to the corresponding author for the article.
| Filename | Description |
|---|---|
| CGF_12053_sm_eg_2013_measurementbasedsynthesisofmicrogeomettry_1280x720.mp478.3 MB | Supporting info item |
| CGF_12053_sm_subject1_16k_image.jpg13 MB | Supporting info item |
| CGF_12053_sm_subject2_16k_image.jpg16.7 MB | Supporting info item |
| CGF_12053_sm_subject3_16k_image.jpg13.8 MB | Supporting info item |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- [ANCN10] Acevedo G., Nevshupov S., Cowely J., Norris K.: An accurate method for acquiring high resolution skin displacement maps. In ACM SIGGRAPH 2010 Talks ( New York , NY , USA , 2010), SIGGRAPH '10, ACM, pp. 4:1–4:1. 2.
- [APS00] Ashikhmin M., Premoze S., Shirley P. S.: A microfacet-based BRDF generator. In Proceedings of ACM SIGGRAPH 2000 (2000), pp. 65–74. 2.
- [Ash01]
Ashikhmin M.: Synthesizing natural textures. In
Proceedings of the 2001 symposium on Interactive 3D graphics (
New York
,
NY
,
USA
, 2001), I3D '01, ACM, pp.
217–226. 5.
10.1145/364338.364405 Google Scholar
- [BBB*10] Beeler T., Bickel B., Beardsley P., Sumner B., Gross M.: High-quality single-shot capture of facial geometry. ACM Trans. Graph. 29 (July 2010), 40:1–40:9. 2, 4.
- [BSFG09] Barnes C., Shechtman E., Finkelstein A., Goldman D. B.: Patchmatch: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28 (July 2009), 24:1–24:11. 8.
- [CDMR05] Cula O. G., Dana K. J., Murphy F. P., Rao B. K.: Skin texture modeling. International Journal of Computer Vision 62 (2005), 97–119. 10.1023/B:VISI.0000046591.79973.6f. 2.
- [CGS06] Chen T., Goesele M., Seidel H. P.: Mesostructure from specularities. In CVPR (2006), pp. 1825–1832. 2.
- [CT82] Cook R. L., Torrance K. E.: A reflectance model for computer graphics. ACM TOG 1, 1 (1982), 7–24. 2, 4.
- [DHT*00] Debevec P., Hawkins T., Tchou C., Duiker H.-P., Sarokin W., Sagar M.: Acquiring the reflectance field of a human face. In ACM SIGGRAPH (2000). 2.
- [DI11]
D'Eon E.,
Irving G.: A quantized-diffusion model for rendering translucent materials. In
ACM SIGGRAPH 2011 papers (
New York
,
NY
,
USA
, 2011), SIGGRAPH '11, ACM, pp.
56:1–56:14. 1.
10.1145/1964921.1964951 Google Scholar
- [DJ05] Donner C., Jensen H. W.: Light diffusion in multi-layered translucent materials. ACM TOG 24, 3 (2005), 1032–1039. 1.
- [DWT*10] Dong Y., Wang J., Tong X., Snyder J., Lan Y., Ben-Ezra M., Guo B.: Manifold bootstrapping for svbrdf capture. ACM Transactions on Graphics 29, 4 (July 2010), 98:1–98:10. 2.
- [EF01] Efros A. A., Freeman W. T.: Image quilting for texture synthesis and transfer. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques ( New York , NY , USA , 2001), SIGGRAPH '01, ACM, pp. 341–346. 2.
- [EL99] Efros A. A., Leung T. K.: Texture synthesis by non-parametric sampling. In Proceedings of the International Conference on Computer Vision-Volume 2 – Volume 2 (Washington, DC, USA, 1999), ICCV '99, IEEE Computer Society, pp. 1033–. 2.
- [FJP02] Freeman W., Jones T., Pasztor E.: Example-based super-resolution. Computer Graphics and Applications, IEEE 22, 2 (marapr 2002), 56–65. 2.
- [GFT*11] Ghosh A., Fyffe G., Tunwattanapong B., Busch J., Yu X., Debevec P.: Multiview face capture using polarized spherical gradient illumination. In Proceedings of the 2011 SIGGRAPH Asia Conference ( New York , NY , USA , 2011), SA '11, ACM, pp. 129:1–129:10. 2, 3, 4.
- [GHP*08] Ghosh A., Hawkins T., Peers P., Frederiksen S., Debevec P.: Practical modeling and acquisition of layered facial reflectance. ACM Transactions on Graphics 27, 5 (Dec 2008), 139:1–139:10. 2, 3, 6.
- [GMP*06] Golovinskiy A., Matusik W., Pfister H., Rusinkiewicz S., Funkhouser T.: A statistical model for synthesis of detailed facial geometry. ACM Transactions on Graphics 25, 3 (July 2006), 1025–1034. 3.
- [HB95] Heeger D. J., Bergen J. R.: Pyramid-based texture analysis/synthesis. In Proceedings of the 22nd annual conference on Computer graphics and interactive techniques ( New York , NY , USA , 1995), SIGGRAPH '95, ACM, pp. 229–238. 2.
- [HJO*01] Hertzmann A., Jacobs C. E., Oliver N., Curless B., Salesin D. H.: Image analogies. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques ( New York , NY , USA , 2001), SIGGRAPH '01, ACM, pp. 327–340. 2, 5.
- [HK93] Hanrahan P., Krueger W.: Reflection from layered surfaces due to subsurface scattering. In Proceedings of SIGGRAPH 93 (1993), pp. 165–174. 1.
- [HSRG07] Han C., Sun B., Ramamoorthi R., Grinspun E.: Frequency domain normal map filtering. ACM Trans. Graph. 26 (July 2007). 8.
- [JCRA11] Johnson M. K., Cole F., Raj A., Adelson E. H.: Microgeometry capture using an elastomeric sensor. ACM Trans. Graph. 30 (Aug 2011), 46:1–46:8. 2.
- [JMLH01] Jensen H. W., Marschner S. R., Levoy M., Hanrahan P.: A practical model for subsurface light transport. In Proceedings of ACM SIGGRAPH 2001 (2001), pp. 511–518. 1.
- [KP03] Koenderink J., Pont S.: The secret of velvety skin. Mach. Vision Appl. 14 (September 2003), 260–268. 8.
- [LH05] Lefebvre S., Hoppe H.: Parallel controllable texture synthesis. In ACM SIGGRAPH 2005 Papers ( New York , NY , USA , 2005), SIGGRAPH '05, ACM, pp. 777–786. 2.
- [LH06] Lefebvre S., Hoppe H.: Appearance-space texture synthesis. ACM Trans. Graph. 25 (July 2006), 541–548. 2, 5, 8.
- [LKG*03] Lensch H. P. A., Kautz J., Goesele M., Heidrich W., Seidel H.-P.: Image-based reconstruction of spatial appearance and geometric detail. ACM TOG 22, 2 (2003), 234–257. 2.
- [LSZ01] Liu C., Shum H.-Y., Zhang C.-S.: A two-step approach to hallucinating faces: global parametric model and local nonparametric model. In Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on (2001), vol. 1, pp. I–192 – I–198 vol. 1. 3.
- [LZS04] Liu Z., Zhang Z., Shan Y.: Image-based surface detail transfer. Computer Graphics and Applications, IEEE 24, 3 (mayjune 2004), 30–35. 3.
- [MGW01] Malzbender T., Gelb D., Wolters H.: Polynomial texture maps. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques ( New York , NY , USA , 2001), SIGGRAPH '01, ACM, pp. 519–528. 3.
- [MHP*07] Ma W.-C., Hawkins T., Peers P., Chabert C.-F., Weiss M., Debevec P.: Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Rendering Techniques (2007), pp. 183–194. 2, 6.
- [MWAM05] Marschner S. R., Westin S. H., Arbree A., Moon J. T.: Measuring and modeling the appearance of finished wood. ACM Trans. Graph. 24 (July 2005), 727–734. 3.
- [MWL*99] Marschner S. R., Westin S. H., Lafortune E. P. F., Torrance K. E., Greenberg D. P.: Image-based BRDF measurement including human skin. In Rendering Techniques (1999). 2.
- [RB07] Ramanarayanan G., Bala K.: Constrained texture synthesis via energy minimization. IEEE Transactions on Visualization and Computer Graphics 13 (January 2007), 167–178. 2.
- [RR08] Ramella-Roman J. C.: Out of plane polarimetric imaging of skin: Surface and subsurface effect. In Optical Waveguide Sensing and Imaging, W. J. Bock, I. Gannot, S. Tanev, (Eds.), NATO Science for Peace and Security Series B: Physics and Biophysics. Springer Netherlands, 2008, pp. 259–269. 10.1007978-1-4020-6952-9_12 2.
- [RSK09] Rump M., Sarlette R., Klein R.: Efficient resampling, compression and rendering of metallic and pearlescent paint. In Vision, Modeling, and Visualization (Nov 2009), M. Magnor, B. Rosenhahn, H. Theisel, (Eds.), pp. 11–18. 8.
- [RTG97] Rushmeier H., Taubin G., Guéziec A.: Applying shape from lighting variation to bump map capture. In Rendering Techniques (1997), pp. 35–44. 2.
- [TLQ*05] Tan P., Lin S., Quan L., Guo B., Shum H.-Y.: Multiresolution reflectance filtering. In Rendering Techniques 2005: 16th Eurographics Workshop on Rendering (June 2005), pp. 111–116. 8.
- [TOS*03] Tsumura N., Ojima N., Sato K., Shiraishi M., Shimizu H., Nabeshima H., Akazaki S., Hori K., Miyake Y.: Image-based skin color and texture analysis/synthesis by extracting hemoglobin and melanin information in the skin. ACM TOG 22, 3 (2003), 770–779. 2.
- [TS67] Torrance K. E., Sparrow E. M.: Theory of off-specular reflection from roughened surfaces. J. Opt. Soc. Am. 57 (1967), 1104–1114. 2, 4.
- [TZL*02] Tong X., Zhang J., Liu L., Wang X., Guo B., Shum H.-Y.: Synthesis of bidirectional texture functions on arbitrary surfaces. ACM Trans. Graph. 21 (July 2002), 665–672. 2.
- [WGP*10] Wilson C. A., Ghosh A., Peers P., Chiang J.-Y., Busch J., Debevec P.: Temporal upsampling of performance geometry using photometric alignment. ACM Trans. Graph. 29 (April 2010), 17:1–17:11. 3.
- [WL01] Wei L.-Y., Levoy M.: Texture synthesis over arbitrary manifold surfaces. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques ( New York , NY , USA , 2001), SIGGRAPH '01, ACM, pp. 355–360. 2.
- [WLKT09] Wei L.-Y., Lefebvre S., Kwatra V., Turk G.: State of the art in example-based texture synthesis. In Eurographics 2009, State of the Art Report, EG-STAR (2009), Eurographics Association. 2.
- [WM04] Wang L., Mueller K.: Generating sub-resolution detail in images and volumes using constrained texture synthesis. In Proceedings of the conference on Visualization '04 ( Washington , DC , USA , 2004), VIS '04, IEEE Computer Society, pp. 75–82. 2.
- [WMP*06] Weyrich T., Matusik W., Pfister H., Bickel B., Donner C., Tu C., McAndless J., Lee J., Ngan A., Jensen H. W., Gross M.: Analysis of human faces using a measurement-based skin reflectance model. ACM Transactions on Graphics 25, 3 (July 2006), 1013–1024. 2.
- [Woo78] Woodham R. J.: Photometric stereo: A reflectance map technique for determining surface orientation from image intensity. In Proc. SPIE's 22nd Annual Technical Symposium (1978), vol. 155. 2.
- [XYZ] XYZRGB: 3D laser scanning – XYZ RGB Inc. http://www.xyzrgb.com/. 2, 4.
- [YHBZ01] Ying L., Hertzmann A., Biermann H., Zorin D.: Texture and shape synthesis on surfaces. In Proceedings of the 12th Eurographics Workshop on Rendering Techniques ( London , UK , 2001), Springer-Verlag, pp. 301–312. 2.
- [ZJMB11]
Zhao S.,
Jakob W.,
Marschner S.,
Bala K.: Building volumetric appearance models of fabric using micro ct imaging.
ACM Trans. Graph.
30 (Aug 2011), 44:1–44:10. 3.
10.1145/2010324.1964939 Google Scholar
- [ZREB06] Zickler T., Ramamoorthi R., Enrique S., Belhumeur P. N.: Reflectance sharing: Predicting appearance from a sparse set of images of a known shape. PAMI 28, 8 (2006), 1287–1302. 2.
