Learning class-specific descriptors for deformable shapes using localized spectral convolutional networks
D. Boscaini
Institute of Computational Science, Faculty of Informatics, University of Lugano (USI), Switzerland
Search for more papers by this authorJ. Masci
Institute of Computational Science, Faculty of Informatics, University of Lugano (USI), Switzerland
Search for more papers by this authorS. Melzi
Department of Informatics, University of Verona, Italy
Search for more papers by this authorM. M. Bronstein
Institute of Computational Science, Faculty of Informatics, University of Lugano (USI), Switzerland
Perceptual Computing, Intel, Israel
Search for more papers by this authorU. Castellani
Department of Informatics, University of Verona, Italy
Search for more papers by this authorP. Vandergheynst
Department of Electrical Engineering, EPFL, Lausanne, Switzerland
Search for more papers by this authorD. Boscaini
Institute of Computational Science, Faculty of Informatics, University of Lugano (USI), Switzerland
Search for more papers by this authorJ. Masci
Institute of Computational Science, Faculty of Informatics, University of Lugano (USI), Switzerland
Search for more papers by this authorS. Melzi
Department of Informatics, University of Verona, Italy
Search for more papers by this authorM. M. Bronstein
Institute of Computational Science, Faculty of Informatics, University of Lugano (USI), Switzerland
Perceptual Computing, Intel, Israel
Search for more papers by this authorU. Castellani
Department of Informatics, University of Verona, Italy
Search for more papers by this authorP. Vandergheynst
Department of Electrical Engineering, EPFL, Lausanne, Switzerland
Search for more papers by this authorAbstract
In this paper, we propose a generalization of convolutional neural networks (CNN) to non-Euclidean domains for the analysis of deformable shapes. Our construction is based on localized frequency analysis (a generalization of the windowed Fourier transform to manifolds) that is used to extract the local behavior of some dense intrinsic descriptor, roughly acting as an analogy to patches in images. The resulting local frequency representations are then passed through a bank of filters whose coefficient are determined by a learning procedure minimizing a task-specific cost. Our approach generalizes several previous methods such as HKS, WKS, spectral CNN, and GPS embeddings. Experimental results show that the proposed approach allows learning class-specific shape descriptors significantly outperforming recent state-of-the-art methods on standard benchmarks.
References
- Anguelov D., et al.: SCAPE: Shape completion and animation of people. TOG 24, 3 (2005), 408–416. 7
- Aubry M., Schlickewei U., Cremers D.: The wave kernel signature: A quantum mechanical approach to shape analysis. In Proc. ICCV (2011). 1, 3, 7
- Bergstra J., et al.: Theano: a CPU and GPU math expression compiler. In Proc. SciPy (June 2010). 7
- Bérard P., Besson G., Gallot S.: Embedding riemannian manifolds by their heat kernel. Geometric & Functional Analysis 4, 4 (1994), 373–398. 1, 3
- Ben-Chen M., Gotsman C.: Characterizing shape using conformal factors. In Proc. 3DOR (2008). 1
- Belongie S., Malik J., Puzicha J.: Shape context: A new descriptor for shape matching and object recognition. In Proc. NIPS (2000). 1
- Bogo F., Romero J., Loper M., Black M.J.: FAUST: Dataset and evaluation for 3D mesh registration. In Proc. CVPR (2014). 7
- Bruna J., Zaremba W., Szlam A., LeCun Y.: Spectral networks and locally connected networks on graphs. In Proc. ICLR (2014). 2, 7
- Coifman R.R., Lafon S.: Diffusion maps. Applied and Computational Harmonic Analysis 21, 1 (2006), 5–30. 1, 3, 4
- Corman E., Ovsjanikov M., Chambolle A.: Supervised descriptor learning for non-rigid shape matching. In Proc. NORDIA (2014). 2
- Elad A., Kimmel R.: On bending invariant signatures for surfaces. PAMI 25, 10 (2003), 1285–1295. 1
- Gebal K., Bærentzen J. A., Anæs H., Larsen R.: Shape analysis using the auto diffusion function. CGF 28, 5 (2009), 1405–1413. 1, 3
- Hamza A.B., Krim H.: Geodesic object representation and recognition. In Proc. DGCI (2003). 1
- Huang Q., Su H., Guibas L.: Fine-grained semi-supervised labeling of large shape collections. TOG (2013). 1
- Johnson A.E., Hebert M.: Using spin images for efficient object recognition in cluttered 3D scenes. PAMI 21, 5 (1999), 433–449. 1
- Kokkinos I., Bronstein M.M., Litman R., Bronstein A.M.: Intrinsic shape context descriptors for deformable shapes. In Proc. CVPR (2012). 1, 2, 7
- Kalogerakis E., Hertzmann A., Singh K.: Learning 3D mesh segmentation and labeling. TOG 29, 3 (2010). 1
- Kim V.G., Lipman Y., Funkhouser T.: Blended intrinsic maps. TOG 30, 4 (2011), 1–12. 8
10.1145/2010324.1964974 Google Scholar
- Lian Z., et al.: A comparison of methods for non-rigid 3D shape retrieval. Pattern Recognition 46, 1 (2013), 449–461. 1
- Litman R., Bronstein A.M.: Learning spectral descriptors for deformable shape correspondence. PAMI 36, 1 (2014), 170–180. 2, 3, 6, 7, 8
- Litman R., Bronstein A., Bronstein M., Castellani U.: Supervised learning of bag-of-features shape descriptors using sparse coding. CGF 33, 5 (2014), 127–136. 1, 2
- LeCun Y., Boser B., Denker J.S., Henderson D., Howard R.E., Hubbard W., Jackel L.D.: Backpropagation applied to handwritten zip code recognition. Neural Computation 1, 4 (1989), 541–551. 2, 5
- Lévy B.: Laplace-Beltrami eigenfunctions towards an algorithm that understands geometry. In Proc. SMI (2006). 1
- Lowe D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60, 2 (2004), 91–110. 1
- Masci J., Boscaini D., Bronstein M.M., Vandergheynst P.: ShapeNet: Convolutional neural networks on non-Euclidean manifolds. arXiv:1501.06297 (2015). 2, 6, 7, 8
- Manay S., Cremers D., Hong B.-W., Yezzi A.J., Soatto S.: Integral invariants for shape matching. PAMI 28, 10 (2006), 1602–1618. 1
- Meyer M., Desbrun M., Schröder P., Barr A.H.: Discrete differential-geometry operators for triangulated 2-manifolds. Visualization&Mathematics (2003), 35–57. 5
- Mateus D., Horaud R., Knossow D., Cuzzolin F., Boyer E.: Articulated shape matching using Laplacian eigenfunctions and unsupervised point registration. In Proc. CVPR (2008). 3
- Neumann T., Varanasi K., Theobalt C., Magnor M., Wacker M.: Compressed manifold modes for mesh processing. In Computer Graphics Forum (2014), vol. 33, pp. 35–44. 4
- Osada R., Funkhouser T., Chazelle B., Dobkin D.: Shape distributions. TOG 21, 4 (2002), 807–832. 1
- Ovsjanikov M., Mérigot Q., Mémoli F., Guibas L.: One point isometric matching with the heat kernel. Computer Graphics Forum 29, 5 (2010), 1555–1564. 1
- Pauly M., Keiser R., Gross M.: Multi-scale feature extraction on point-sampled surfaces. CGF 22, 3 (2003), 281–289. 1
- Pinkall U., Polthier K.: Computing discrete minimal surfaces and their conjugates. Experimental Mathematics 2, 1 (1993), 15–36. 5
10.1080/10586458.1993.10504266 Google Scholar
- Rodolà E., Rota Bulò S., Windheuser T., Vestner M., Cremers D.: Dense non-rigid shape correspondence using random forests. In Proc. CVPR (2014). 1
- Rustamov R.M.: Laplace-Beltrami eigenfunctions for deformation invariant shape representation. In Proc. SGP (2007). 1, 3
- Reuter M., Wolter F.-E., Peinecke N.: Laplace–beltrami spectra as ‘shape-dna’ of surfaces and solids. Computer-Aided Design 38, 4 (2006), 342–366. 1
- Sipiran I., Bustos B.: Harris 3D: a robust extension of the harris operator for interest point detection on 3D meshes. Visual Computer 27, 11 (2011), 963–976. 1
- Sun J., Ovsjanikov M., Guibas L.J.: A concise and provably informative multi-scale signature based on heat diffusion. CGF 28, 5 (2009), 1383–1392. 1, 3, 7
- Shuman D.I., Ricaud B., Vandergheynst P.: Vertex-frequency analysis on graphs. arXiv:1307.5708 (2013). 2, 4
- Wu Z., Song S., Khosla A., Yu F., Zhang L., Tang X., Xiao J.: 3d shapenets: A deep representation for volumetric shape modeling. In Proc. CVPR (2015), vol. 1, p. 3. 2
- Zeiler M.D.: ADADELTA: An adaptive learning rate method. arXiv:1212.5701 (2012). 7