Article
Style Invariant Locomotion Classification for Character Control
G.E. Boehs,
M.L.H. Vieira,
G.E. Boehs
Departamento de Expressao Grafica, Universidade Federal de Santa Catarina, Florianopolis, Brazil
Search for more papers by this authorM.L.H. Vieira
Departamento de Expressao Grafica, Universidade Federal de Santa Catarina, Florianopolis, Brazil
Search for more papers by this authorG.E. Boehs,
M.L.H. Vieira,
G.E. Boehs
Departamento de Expressao Grafica, Universidade Federal de Santa Catarina, Florianopolis, Brazil
Search for more papers by this authorM.L.H. Vieira
Departamento de Expressao Grafica, Universidade Federal de Santa Catarina, Florianopolis, Brazil
Search for more papers by this authorAbstract
We present a real-time system for character control that relies on the classification of locomotive actions in skeletal motion capture data. Our method is both progress dependent and style invariant. Two deep neural networks are used to correlate body shape and implicit dynamics to locomotive types and their respective progress. In comparison to related work, our approach does not require a setup step and enables the user to act in a natural, unconstrained manner. Also, our method displays better performance than the related work in scenarios where the actor performs sharp changes in direction and highly stylized motions while maintaining at least as good performance in other scenarios. Our motivation is to enable character control of non-bipedal characters in virtual production and live immersive experiences, where mannerisms in the actor's performance may be an issue for previous methods.
Supporting Information
| Filename | Description |
|---|---|
| cgf13590-sup-0001-SupMat.xlsx19.1 KB |
Data S1 |
| cgf13590-sup-0002-SupMat.xlsx27.3 KB |
Data S2 |
| cgf13590-sup-0003-video.mp46.1 MB |
Video S3 |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- [ABC96] Amaya K., Bruderlin A., Calvert T.: Emotion from motion. In Proceedings of the Conference on Graphics Interface '96 (Toronto, Ontario, Canada, 1996), GI '96, Canadian Information Processing Society, pp. 222–229.
- [AZS*17]
Aristidou A., Zeng Q., Stavrakis E., Yin K., Cohen-Or D., Chrysanthou Y., Chen B.: Emotion control of unstructured dance movements. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (New York, NY, USA, 2017), SCA '17, ACM, pp. 9:1–9:10.
10.1145/3099564.3099566 Google Scholar
- [BLCD02] Bregler C., Loeb L., Chuang E., Deshpande H.: Turning to the masters: Motion capturing cartoons. ACM Transactions on Graphics 21, 3 (2002), 399–407.
- [BVGP09] Baran I., Vlasic D., Grinspun E., Popovic J.: Semantic deformation transfer. ACM Transactions on Graphics 28, 3 (2009), 1–6.
- [CC13] Cho K., Chen X.: Classifying and visualizing motion capture sequences using deep neural networks. CoRR abs/1306.3874 (2013). URL: http://arxiv.org/abs/1306.3874, http://arxiv.org/abs/1306.3874. 2
- [CZW '07]
Chiu, B., Zordan, V., Wu, C.-C. (2007): State-annotated motion graphs. In Proceedings of the 2007 ACM Symposium on Virtual Reality Software and Technology (New York, NY, USA, 2007), ACM, pp. 73–76. http://doi.acm.org/10.1145/1315184.1315195
10.1145/1315184.1315195 Google Scholar
- [CIF12] Chen J., Izadi S., Fitzgibbon A.: KinÊtre: Animating the world with the human body. In ACM SIGGRAPH 2012 Talks (New York, NY, USA, 2012), SIGGRAPH '12, ACM, pp. 39:1–39:1.
- [CK15]
Chen X., Koskela M.: Skeleton-based action recognition with extreme learning machines. Neurocomputing 149, Part A (2015), 387–396.
10.1016/j.neucom.2013.10.046 Google Scholar
- [CMU17] CMU: Carnegie-Mellon mocap database (2017). URL: http://mocap.cs.cmu.edu/. Accessed November 2017.
- [COK*] Chaudhry R., Ofli F., Kurillo G., Bajcsy R., Vidal R.: Bio-inspired dynamic 3D discriminative skeletal features for human action recognition. In 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 471–478.
- [DYP03] Dontcheva M., Yngve G., Popovic Z.: Layered acting for character animation. ACM Transactions on Graphics 22, 3 (2003), 409–416.
- [HPP05] Hsu E., Pulli K., Popovic J.: Style translation for human motion. ACM Transactions on Graphics 24, 3 (2005), 1082–1089.
- [HSK16] Holden D., Saito J., Komura T.: A deep learning framework for character motion synthesis and editing. ACM Transactions on Graphics 35, 4 (2016), 1–11.
- [IWZL09] Ishigaki S., White T., Zordan V. B., Liu C. K.: Performance-based control interface for character animation. ACM Transactions on Graphics 28, 3 (2009), 1–8.
- [KB14]
Kruthiventi S. S., Babu R. V.: 3D action recognition by learning sequences of poses. In Proceedings of the 2014 Indian Conference on Computer Vision Graphics and Image Processing (New York, NY, USA, 2014), ICVGIP '14, ACM, pp. 23:1–23:7.
10.1145/2683483.2683506 Google Scholar
- [KG04] Kovar L., Gleicher M.: Automated extraction and parameterization of motions in large data sets. ACM Transactions on Graphics 23, 3 (2004), 559–568.
- [KGP08]
Kovar L., Gleicher M., Pighin F.: Motion graphs. In ACM SIGGRAPH 2008 Classes (New York, NY, USA, 2008), SIGGRAPH '08, ACM, pp. 51:1–51:10.
10.1145/1401132.1401202 Google Scholar
- [Lin06] Lin Y.: Efficient Motion Search in Large Motion Capture Databases. Springer Berlin Heidelberg, Berlin, Heidelberg, 2006, pp. 151–160.
- [LWH*12] Levine S., Wang J. M., Haraux A., Popovic Z., Koltun V.: Continuous character control with low-dimensional embeddings. ACM Transactions on Graphics 31, 4 (2012), 1–10.
- [Man13] Mankyu S.: Fast motion synthesis of quadrupedal animals using a minimum amount of motion capture data. ETRI Journal 35, 6 (2013), 1029–1037.
- [MC12] Min J., Chai J.: Motion graphs++: a compact generative model for semantic motion analysis and synthesis. ACM Transactions on Graphics 31, 6 (2012), 1–12.
- [MLC10]
Min J., Liu H., Chai J.: Synthesis and editing of personalized stylistic human motion. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (New York, NY, USA, 2010), I3D '10, ACM, pp. 39–46.
10.1145/1730804.1730811 Google Scholar
- [MN12]
Martin T., Neff M.: Interactive quadruped animation. In Motion in Games (Berlin, Heidelberg, 2012), Springer, pp. 208–219.
10.1007/978-3-642-34710-8_20 Google Scholar
- [PB00]
Pullen K., Bregler C.: Animating by multi-level sampling. In Proceedings Computer Animation 2000 (Washington, DC, USA, 2000), IEEE Computer Society, pp. 36–42.
10.1109/CA.2000.889031 Google Scholar
- [RKH11]
Raptis M., Kirovski D., Hoppe H.: Real-time classification of dance gestures from skeleton animation. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (New York, NY, USA, 2011), SCA '11, ACM, pp. 147–156.
10.1145/2019406.2019426 Google Scholar
- [RTK*14] Rhodin H., Tompkin J., Kim K. I., Varanasi K., Seidel H.-P., Theobalt C.: Interactive motion mapping for real-time character control. Computer Graphics Forum 33, 2 (2014), 273–282.
- [RTK*15] Rhodin H., Tompkin J., Kim K. I., Aguiar E. D., Pfister H., Seidel H.-P., Theobalt C.: Generalizing wave gestures from sparse examples for real-time character control. ACM Transactions on Graphics 34, 6 (2015), 1–12.
- [SOL13]
Seol Y., O'Sullivan C., Lee J.: Creature features: Online motion puppetry for non-human characters. In Proceedings of the 12th ACM SIGGRAPH/Eurographics Symposium on Computer Animation (New York, NY, USA, 2013), SCA '13, ACM, pp. 213–221.
10.1145/2485895.2485903 Google Scholar
- [VLBM08]
Vincent P., Larochelle H., Bengio Y., Manzagol P.-A.: Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning (New York, NY, USA, 2008), ICML '08, ACM, pp. 1096–1103.
10.1145/1390156.1390294 Google Scholar
- [WS14]
Wu D., Shao L.: Multimodal dynamic networks for gesture recognition. In Proceedings of the 22nd ACM International Conference on Multimedia (New York, NY, USA, 2014), MM '14, ACM, pp. 945–948.
10.1145/2647868.2654969 Google Scholar
- [XWCH15] Xia S., Wang C., Chai J., Hodgins J.: Realtime style transfer for unlabeled heterogeneous human motion. ACM Transactions on Graphics 34, 4 (2015), 1–10.
- [YAH10] Yamane K., Ariki Y., Hodgins J.: Animating non-humanoid characters with human motion data. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Goslar Germany, Germany, 2010), SCA '10, Eurographics Association, pp. 169–178.
- [YM16] Yumer M. E., Mitra N. J.: Spectral style transfer for human motion between independent actions. ACM Transactions on Graphics 35, 4 (2016), 1–8.
- [YWW]
Yong Du., Wang W., Wang L.: Hierarchical recurrent neural network for skeleton based action recognition. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), IEEE, pp. 1110–1118 https://doi.org/10.1109/CVPR.2015.7298714.
10.1109/CVPR.2015.7298714 Google Scholar
