Self-similarity Analysis for Motion Capture Cleaning
Abstract
Motion capture sequences may contain erroneous data, especially when the motion is complex or performers are interacting closely and occlusions are frequent. Common practice is to have specialists visually detect the abnormalities and fix them manually. In this paper, we present a method to automatically analyze and fix motion capture sequences by using self-similarity analysis. The premise of this work is that human motion data has a high-degree of self-similarity. Therefore, given enough motion data, erroneous motions are distinct when compared to other motions. We utilize motion-words that consist of short sequences of transformations of groups of joints around a given motion frame. We search for the K-nearest neighbors (KNN) set of each word using dynamic time warping and use it to detect and fix erroneous motions automatically. We demonstrate the effectiveness of our method in various examples, and evaluate by comparing to alternative methods and to manual cleaning.
Supporting Information
Filename | Description |
---|---|
cgf13362-sup-0002-S1.zip140.3 MB | Supporting Information |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- Aristidou A., Charalambous P., Chrysanthou Y.: Emotion analysis and classification: Understanding the performers’ emotions using the lma entities. Comput. Graph. Forum 34, 6 (Sept. 2015), 262–276. 2
- Aristidou A., Chrysanthou Y., Lasenby J.: Extending FABRIK with model constraints. Comput. Animat. Virtual Worlds 27, 1 (January 2016), 35–57. 3
- Aristidou A., Lasenby J.: Real-time marker prediction and CoR estimation in optical motion capture. Vis. Comput. 29, 1 (2013), 7–26. 1, 3
- Akhter I., Simon T., Khan S., Matthews I., Sheikh Y.: Bilinear spatiotemporal basis models. ACM Trans. Graph. 31, 2 (Apr. 2012), 17:1–17:12 3
- Buades A., Coll B., Morel J.-M.: A non-local algorithm for image denoising. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition - Volume 02 (Washington, DC, USA, 2005), CVPR ‘05, IEEE Computer Society, pp. 60–65. 1
- Beaudoin P., Coros S., van de Panne M., Poulin P.: Motion-motif graphs. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2008), SCA ‘08, Eurographics Association, pp. 117–126. 2
- Bellini R., Kleiman Y., Cohen-Or D.: Time-varying weathering in texture space. ACM Trans. Graph. 35, 4 (July 2016), 141:1–141:11 3
- Burke M., Lasenby J.: Estimating missing marker positions using low dimensional kalman smoothing. Journal of Biomechanics 49, 9 (2016), 1854–1858. 3, 9
- Chai J., Hodgins J. K.: Performance animation from low-dimensional control signals. ACM Trans. Graph. 24, 3 (July 2005), 686–696. 2, 8, 9
- CMU: Carnegie Mellon University MoCap Database: http://mocap.cs.cmu.edu/, 2017. 1, 6
- Forbes K., Fiume E.: An efficient search algorithm for motion data using weighted PCA. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (NY, USA, 2005), SCA ‘05, ACM, pp. 67–76. 6
- Feng Y., Ji M., Xiao J., Yang X., Zhang J. J., Zhuang Y., Li X.: Mining spatial-temporal patterns and structural sparsity for human motion data denoising. IEEE Transactions on Cybernetics 45, 12 (Dec 2015), 2693–2706. 3, 9, 10
- Gl⊘ersen Ø., Federolf P.: Predicting missing marker trajectories in human motion data using marker intercorrelations. PLOS ONE 11, 3 (2016), 1–14. 3, 10
- Herda L., Fua P., Plänkers R., Boulic R., Thalmann D.: Skeleton-based motion capture for robust reconstruction of human motion. In Proc. of the Computer Animation (Washington, DC, USA, 2000), CA ‘00, IEEE Computer Society, pp. 77–86. 3
- Hsu E., Gentry S., Popović J.: Example-based control of human motion. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2004), SCA ‘04, Eurographics Association, pp. 69–77. 3
- Ho E. S. L., Komura T., Tai C.-L.: Spatial relationship preserving character motion adaptation. ACM Trans. Graph. 29, 4 (July 2010), 33:1–33:8 2
- Holden D., Saito J., Komura T., Joyce T.: Learning motion manifolds with convolutional autoencoders. In SIGGRAPH Asia 2015 Technical Briefs (NY, USA, 2015), SA ‘15, ACM, pp. 18:1–18:4 3, 9, 10
- Ikemoto L., Arikan O., Forsyth D.: Quick transitions with cached multi-way blends. In Proc. of the Symposium on Interactive 3D Graphics and Games (NY, USA, 2007), I3D ‘07, ACM, pp. 145–151. 5
- iPi Soft: iPi Motion Capture: http://www.ipisoft.com/, 2017. 6
- Ilan S., Shamir A.: A survey on data-driven video completion. Comput. Graph. Forum 34, 6 (Sept. 2015), 60–85. 3
- Kapadia M., Chiang I.-k., Thomas T., Badler N. I., Kider Jr. J. T.: Efficient motion retrieval in large motion databases. In Proc. of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (NY, USA, 2013), I3D ‘13, ACM, pp. 19–28. 2
- Kovar L., Gleicher M.: Flexible automatic motion blending with registration curves. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2003), SCA ‘03, Eurographics Association, pp. 214–224. 5
- Kovar L., Gleicher M.: Automated extraction and parameterization of motions in large data sets. ACM Trans. Graph. 23, 3 (Aug. 2004), 559–568. 2
- Kovar L., Gleicher M., Pighin F.: Motion graphs. ACM Trans. Graph. 21, 3 (July 2002), 473–482. 2, 4
- Keogh E., Palpanas T., Zordan V. B., Gunopulos D., Cardle M.: Indexing large human-motion databases. In Proc. of the International Conference on Very Large Data Bases (2004), VLDB ‘04, pp. 780–791. 2
- Kim W., Rehg J. M.: Detection of unnatural movement using epitomic analysis. In Proc. of the Seventh International Conference on Machine Learning and Applications (Washington, DC, USA, 2008), ICMLA ‘08, IEEE Computer Society, pp. 271–276. 3
- Krüger B., Tautges J., Weber A., Zinke A.: Fast local and global similarity searches in large motion capture databases. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2010), SCA ‘10, Eurographics Association, pp. 1–10. 2
- Lou H., Chai J.: Example-based human motion denoising. IEEE Transactions on Visualization and Computer Graphics 16, 5 (Sept. 2010), 870–879. 3
- Liu X., Cheung Y.-m., Peng S.-J., Cui Z., Zhong B., Du J.-X.: Automatic motion capture data denoising via filtered sub-space clustering and low rank matrix approximation. Signal Process. 105 (2014), 350–362. 1, 3
- Lee J., Chai J., Reitsma P. S. A., Hodgins J. K., Pollard N. S.: Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3 (July 2002), 491–500. 4, 10
- Lv X., Chai J., Xia S.: Data-driven inverse dynamics for human motion. ACM Trans. Graph. 35, 6 (Nov. 2016), 163:1–163:12 2
- Liu G., McMillan L.: Estimation of missing markers in human motion capture. Vis. Comput. 22, 9 (Sept. 2006), 721–728. 3
- Li L., McCann J., Pollard N., Faloutsos C.: Bolero: A principled technique for including bone length constraints in motion capture occlusion filling. In Proc‥ of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2010), SCA ‘10, Eurographics Association, pp. 179–188. 1, 3
- Lyard E., Magnenat-Thalmann N.: A simple footskate removal method for virtual reality applications. Vis. Comput. 23, 9 (Aug. 2007), 689–695. 6
- Li Y., Wang T., Shum H.-Y.: Motion texture: A two-level statistical model for character motion synthesis. ACM Trans. Graph. 21, 3 (July 2002), 465–472. 4
- Müller M.: Dynamic Time Warping. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2007, pp. 69–84. 13
- Müller M., Röder T., Clausen M.: Efficient content-based retrieval of motion capture data. ACM Trans. Graph. 24, 3 (July 2005), 677–685. 2
- Mehta D., Sridhar S., Sotnychenko O., Rhodin H., Shafiei M., Seidel H.-P., Xu W., Casas D., Theobalt C.: VNect: Real-time 3d human pose estimation with a single rgb camera. ACM Trans. Graph. 36, 4 (July 2017), 44:1–44:14 1
- Park S. I., Hodgins J. K.: Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3 (July 2006), 881–889. 3
- PhaseSpace Inc.: Optical Motion Capture Systems: http://www.phasespace.com, 2017. 6
- Peng S.-J., He G.-F., Liu X., Wang H.-Z.: Hierarchical block-based incomplete human mocap data recovery using adaptive non-negative matrix factorization. Computer & Graphics 49, C (June 2015), 10–23. 1, 3
- Rose C., Cohen M. F., Bodenheimer B.: Verbs and adverbs: Multidimensional motion interpolation. IEEE Comput. Graph. Appl. 18, 5 (Sept. 1998), 32–40. 3
- Root-Motion: FINAL-IK: http://root-motion.com/, accessed 01/2017, 2017. 5
- Rothweiler J.: Polyphase quadrature filters - a new sub-band coding technique. In IEEE International Conference on Acoustics, Speech, and Signal Processing (1983), ICASSP ‘83, IEEE, p. 1280ǎĂŞ1283. 5
- Ren L., Patrick A., Efros A. A., Hodgins J. K., Rehg J. M.: A data-driven approach to quantifying natural human motion. ACM Trans. Graph. 24, 3 (July 2005), 1090–1097. 3
- Shen W., Deng K., Bai X., Leyvand T., Guo B., Tu Z.: Exemplar-based human action pose correction and tagging. In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (Washington, DC, USA, 2012), CVPR ‘12, IEEE Computer Society, pp. 1784–1791. 3
- Slyper R., Hodgins J. K.: Action capture with accelerometers. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2008), SCA ‘08, Eurographics Association, pp. 193–199. 3
- Shin H. J., Lee J., Shin S. Y., Gleicher M.: Computer puppetry: An importance-based approach. ACM Trans. Graph. 20, 2 (Apr. 2001), 67–94. 3
- Shotton J., Sharp T., Kipman A., Fitzgibbon A., Finocchio M., Blake A., Cook M., Moore R.: Real-time human pose recognition in parts from single depth images. Commun. ACM 56, 1 (Jan. 2013), 116–124. 1
- Trumble M., Gilbert A., Malleson C., Hilton A., Collomosse J.: Total Capture: 3D human pose estimation fusing video and inertial sensors. In Proc. of the 2017 British Machine Vision Conference (2017), BMVC ‘17. 3
- Taylor G. W., Hinton G. E., Roweis S.: Modeling human motion using binary latent variables. In Proc. of the 19th International Conference on Neural Information Processing Systems (Cambridge, MA, USA, 2006), NIPS'06, MIT Press, pp. 1345–1352. 3
- Tak S., Ko H.-S.: A physically-based motion retargeting filter. ACM Trans. Graph. 24, 1 (Jan. 2005), 98–117. 3
- Tautges J., Zinke A., Krüger B., Baumann J., Weber A., Helten T., Müller M., Seidel H.-P., Eberhardt B.: Motion reconstruction using sparse accelerometer data. ACM Trans. Graph. 30, 3 (May 2011), 18:1–18:12 1, 3
- Vondrak M., Sigal L., Hodgins J., Jenkins O.: Video-based 3d motion capture through biped control. ACM Trans. Graph. 31, 4 (July 2012), 27:1–27:12 2
- Wang J., Bodenheimer B.: An evaluation of a cost metric for selecting transitions between motion segments. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2003), SCA ‘03, Eurographics Association, pp. 232–238. 4
- Won J., Lee K., O'Sullivan C., Hodgins J. K., Lee J.: Generating and ranking diverse multi-character interactions. ACM Trans. Graph. 33, 6 (Nov. 2014), 219:1–219:12 6
- Xiao J., Feng Y., Ji M., Yang X., Zhang J. J., Zhuang Y.: Sparse motion bases selection for human motion denoising. Signal Process. 110, C (May 2015), 108–122. 3
- Xia G., Sun H., Zhang G., Feng L.: Human motion recovery jointly utilizing statistical and kinematic information. Information Sciences 339, C (Apr. 2016), 189–205. 3
- Zordan V. B., Van Der Horst N. C.: Mapping optical motion capture data to skeletal motion using a physical model. In Proc. of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (Aire-la-Ville, CH, 2003), SCA ‘03, Eurographics Association, pp. 245–250. 3