SurfNet: Learning Surface Representations via Graph Convolutional Network
Abstract
For scientific visualization applications, understanding the structure of a single surface (e.g., stream surface, isosurface) and selecting representative surfaces play a crucial role. In response, we propose SurfNet, a graph-based deep learning approach for representing a surface locally at the node level and globally at the surface level. By treating surfaces as graphs, we leverage a graph convolutional network to learn node embedding on a surface. To make the learned embedding effective, we consider various pieces of information (e.g., position, normal, velocity) for network input and investigate multiple losses. Furthermore, we apply dimensionality reduction to transform the learned embeddings into 2D space for understanding and exploration. To demonstrate the effectiveness of SurfNet, we evaluate the embeddings in node clustering (node-level) and surface selection (surface-level) tasks. We compare SurfNet against state-of-the-art node embedding approaches and surface selection methods. We also demonstrate the superiority of SurfNet by comparing it against a spectral-based mesh segmentation approach. The results show that SurfNet can learn better representations at the node and surface levels with less training time and fewer training samples while generating comparable or better clustering and selection results.
Supporting Information
Filename | Description |
---|---|
cgf14526-sup-0001-S1.pdf25.1 MB | Supporting Information |
cgf14526-sup-0001-S2.wmv51.3 MB | Supporting Information |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- Abu-El-Haija S., Perozzi B., Al-Rfou R., Alemi A. A.: Watch your step: Learning node embeddings via graph attention. In Proceedings of Advances in Neural Information Processing Systems (2018), pp. 9198–9208. 6
- Ahmed A., Shervashidze N., Narayanamurthy S., Josifovski V., Smola A. J.: Distributed large-scale natural graph factorization. In Proceedings of ACM International Conference on World Wide Web (2013), pp. 37–48. 4
- Bronstein M. M., Bruna J., LeCun Y., Szlam A., Vandergheynst P.: Geometric deep learning: Going beyond Euclidean data. IEEE Signal Processing Magazine 34, 4 (2017), 18–42. 1, 2
- Bai Y., Ding H., Qiao Y., Marinovic A., Gu K., Chen T., Sun Y., Wang W.: Unsupervised inductive graph-level representation learning via graph-graph proximity. In Proceedings of International Joint Conferences on Artificial Intelligence (2019), pp. 1988–1994. 3
- Berger M., Li J., Levine J. A.: A generative model for volume rendering. IEEE Transactions on Visualization and Computer Graphics 25, 4 (2019), 1636–1650. 2
- Bruckner S., Möller T.: Isosurface similarity maps. Computer Graphics Forum 29, 3 (2010), 773–782. 8, 10
- Barrow H. G., Tenenbaum J. M., Bolles R. C., Wolf H. C.: Parametric correspondence and chamfer matching: Two new techniques for image matching. In Proceedings of International Joint Conference on Artificial Intelligence (1977), pp. 659–663. 5
- Cheng H.-C., Cardone A., Jain S., Krokos E., Narayan K., Subramaniam S., Varshney A.: Deep-learning-assisted volume visualization. IEEE Transactions on Visualization and Computer Graphics 25, 2 (2019), 1378–1391. 2
- Corso G., Ying R., Pándy M., Veličković P., Leskovec J., Liò P.: Neural distance embeddings for biological sequences. In Proceedings of Advances in Neural Information Processing Systems (2021). 4
- Ester M., Kriegel H.-P., Sander J., Xu X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (1996), pp. 226–231. 3
- Fofonov A., Molchanov V., Linsen L.: Visual analysis of multi-run spatio-temporal simulations using isocontour similarity for projected views. IEEE Transactions on Visualization and Computer Graphics 22, 8 (2015), 2037–2050. 5
- Garland M., Heckbert P. S.: Surface simplification using quadric error metrics. In Proceedings of ACM SIGGRAPH Conference (1997), pp. 209–216. 3
- Gu P., Han J., Chen D. Z., Wang C.: Reconstructing unsteady flow data from representative streamlines via diffusion and deep learning based denoising. IEEE Computer Graphics and Applications 41, 6 (2021), 111–121. 2
- Gu P., Han J., Chen D. Z., Wang C.: Scalar2Vec: Translating scalar fields to vector fields via deep learning. In Proceedings of IEEE Pacific Visualization Symposium (2022). Accepted. 2
- Guo L., Ye S., Han J., Zheng H., Gao H., Chen D. Z., Wang J.-X., Wang C.: SSR-VFD: Spatial super-resolution for vector field data analysis and visualization. In Proceedings of IEEE Pacific Visualization Symposium (2020), pp. 71–80. 2
- Hanocka R., Hertz A., Fish N., Giryes R., Fleishman S., Cohen-Or D.: MeshCNN: A network with an edge. ACM Transactions on Graphics 38, 4 (2019), 90:1–90:12. 1, 3, 8
- Huang J., Li Z., Li N., Liu S., Li G.: AttPool: Towards hierarchical feature representation in graph convolutional networks via attention mechanism. In Proceedings of IEEE International Conference on Computer Vision (2019), pp. 6480–6489. 6
- Hong F., Liu C., Yuan X.: DNN-VolVis: Interactive volume visualization supported by deep neural network. In Proceedings of IEEE Pacific Visualization Symposium (2019), pp. 282–291. 2
- Han J., Tao J., Wang C.: FlowNet: A deep learning framework for clustering and selection of streamlines and stream surfaces. IEEE Transactions on Visualization and Computer Graphics 26, 4 (2020), 1732–1744. 1, 2, 3, 5, 6, 8
- Han J., Tao J., Zheng H., Guo H., Chen D. Z., Wang C.: Flow field reduction via reconstructing vector data from 3D streamlines using deep learning. IEEE Computer Graphics and Applications 39, 4 (2019), 54–67. 2
- Han J., Wang C.: SSR-TVD: Spatial super-resolution for time-varying data analysis and visualization. IEEE Transactions on Visualization and Computer Graphics (2020). Accepted. 2
- Han J., Wang C.: TSR-TVD: Temporal super-resolution for time-varying data analysis and visualization. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 205–215. 2
- Han J., Wang C.: TSR-VFD: Generating temporal super-resolution for unsteady vector field data. Computers & Graphics 103 (2022), 168–179. 2
- He W., Wang J., Guo H., Wang K.-C., Shen H.-W., Raj M., Nashed Y. S. G., Peterka T.: InSituNet: Deep image synthesis for parameter space exploration of ensemble simulations. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2020), 23–33. 2
- Hamilton W., Ying Z., Leskovec J.: Inductive representation learning on large graphs. In Proceedings of Advances in Neural Information Processing Systems (2017), pp. 1024–1034. 4
- Han J., Zheng H., Chen D. Z., Wang C.: STNet: An end-to-end generative framework for synthesizing spatiotemporal super-resolution volumes. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2022), 270–280. 2
- He K., Zhang X., Ren S., Sun J.: Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. In Proceedings of IEEE International Conference on Computer Vision (2015), pp. 1026–1034. 5
- He K., Zhang X., Ren S., Sun J.: Deep residual learning for image recognition. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778. 4
- Han J., Zheng H., Xing Y., Chen D. Z., Wang C.: V2V: A deep learning approach to variable-to-variable selection and translation for multivariate time-varying data. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2021), 1290–1300. 2
- Hong F., Zhang J., Yuan X.: Access pattern learning with long short-term memory for parallel particle tracing. In Proceedings of IEEE Pacific Visualization Symposium (2018), pp. 76–85. 2
- Jakob J., Gross M., Günther T.: A fluid flow data set for machine learning and its application to neural flow map interpolation. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2021), 1279–1289. 2
- Jiang B., Zhang Z., Lin D., Tang J., Luo B.: Semi-supervised learning with graph learning-convolutional networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2019), pp. 11313–11320. 6
- Kingma D., Ba J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). 5
- Kostrikov I., Jiang Z., Panozzo D., Zorin D., Bruna J.: Surface networks. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 2540–2548. 3
- Kipf T. N., Welling M.: Semi-supervised classification with graph convolutional networks. In Proceedings of International Conference for Learning Representations (2017). 6
- Litany O., Bronstein A., Bronstein M., Makadia A.: Deformable shape completion with graph convolutional autoen-coders. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 1886–1895. 2
- Lin J.: Divergence measures based on the Shannon entropy. IEEE Transactions on Information theory 37, 1 (1991), 145–151. 5
- Liu C., Ji H., Qiu A.: Convolutional neural network on semi-regular triangulated meshes and its application to brain image data. arXiv preprint arXiv:1903.08828 (2019). 2, 3
- Liu R., Zhang H.: Segmentation of 3D meshes through spectral clustering. In Proceedings of Pacific Conference on Computer Graphics and Applications (2004), pp. 298–305. 7, 8
- Monti F., Boscaini D., Masci J., Rodola E., Svoboda J., Bronstein M. M.: Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 5115–5124. 2, 3, 6
- Nair V., Hinton G. E.: Rectified linear units improve restricted Boltzmann machines. In Proceedings of International Conference on Machine Learning (2010), pp. 807–814. 4
- Qi C. R., Su H., Niessner M., Dai A., Yan M., Guibas L. J.: Volumetric and multi-view CNNs for object classification on 3D data. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 5648–5656. 1
- Ranjan A., Bolkart T., Sanyal S., Black M. J.: Generating 3D faces using convolutional mesh autoencoders. In Proceedings of European Conference on Computer Vision (2018), pp. 704–720. 2
- Smith E., Fujimoto S., Romero A., Meger D.: GEO-Metrics: Exploiting geometric structure for graph-encoded objects. In Proceedings of International Conference on Machine Learning (2019), pp. 5866–5876. 3
- Shamir A.: A survey on mesh segmentation techniques. Computer Graphics Forum 27, 6 (2008), 1539–1556. 10
- Su H., Maji S., Kalogerakis E., Learned-Miller E.: Multi-view convolutional neural networks for 3D shape recognition. In Proceedings of IEEE International Conference on Computer Vision (2015), pp. 945–953. 1
- Shu Z., Qi C., Xin S., Hu C., Wang L., Zhang Y., Liu L.: Unsupervised 3D shape segmentation and co-segmentation via deep learning. Computer Aided Geometric Design 43 (2016), 39–52. 2
- Shi N., Xu J., Wurster S. W., Guo H., Woodring J., Van Roekel L. P., Shen H.-W.: Gnn-Surrogate: A hierarchical and adaptive graph neural network for parameter space exploration of unstructured-mesh ocean simulations. IEEE Transactions on Visualization and Computer Graphics (2022). Accepted. 2
- Tkachev G., Frey S., Ertl T.: Local prediction models for spatiotemporal volume visualization. IEEE Transactions on Visualization and Computer Graphics 27, 7 (2021), 3091–3108. 2
- Tkachev G., Frey S., Ertl T.: S4: Self-supervised learning of spatiotemporal similarity. IEEE Transactions on Visualization and Computer Graphics (2021). Accepted. 2, 4
- Tao J., Wang C.: Semi-automatic generation of stream surfaces via sketching. IEEE Transactions on Visualization and Computer Graphics 24, 9 (2018), 2622–2635. 2
- van der Maaten L. J. P., Hinton G. E.: Visualizing high-dimensional data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579–2605. 3
- Weiss S., Chu M., Thuerey N., Westermann R.: Volumetric isosurface rendering with deep learning-based super-resolution. IEEE Transactions on Visualization and Computer Graphics 27, 6 (2021), 3064–3078. 2
- Weiss S., Işik M., Thies J., Westermann R.: Learning adaptive sampling and reconstruction for volume visualization. IEEE Transactions on Visualization and Computer Graphics (2020). Accepted. 2
- Wu Z., Song S., Khosla A., Yu F., Zhang L., Tang X., Xiao J.: 3D ShapeNets: A deep representation for volumetric shapes. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 1912–1920. 1
- Wang Y., Sun Y., Liu Z., Sarma S. E., Bronstein M. M., Solomon J. M.: Dynamic graph CNN for learning on point clouds. ACM Transactions on Graphics 38, 5 (2019), 146:1–146:12. 3, 6, 8
- Wang N., Zhang Y., Li Z., Fu Y., Liu W., Jiang Y.-G.: Pixel2Mesh: Generating 3D mesh models from single RGB images. In Proceedings of European Conference on Computer Vision (2018), pp. 52–67. 3
- Wang M., Zheng D., Ye Z., Gan Q., Li M., Song X., Zhou J., Ma C., Yu L., Gai Y., Xiao T., He T., Karypis G., Li J., Zhang Z.: Deep Graph Library: A graph-centric, highly-performant package for graph neural networks. arXiv preprint arXiv:1909.01315 (2019). 5
- Xu C., Prince J. L.: Gradient vector flow: A new external force for snakes. In Proceedings of IEEE International Conference on Computer Vision (1997), pp. 66–71. 6
- Xu K., Zhang M., Jegelka S., Kawaguchi K.: Optimization of graph neural networks: Implicit acceleration by skip connections and more depth. In Proceedings of International Conference on Machine Learning (2021), pp. 11592–11602. 4
- Ying R., He R., Chen K., Eksombatchai P., Hamilton W. L., Leskovec J.: Graph convolutional neural networks for web-scale recommender systems. In Proceedings of ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2018), pp. 974–983. 3, 4
- Yi L., Su H., Guo X., Guibas L. J.: SyncSpecCNN: Synchronized spectral CNN for 3D shape segmentation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 2282–2290. 2, 3
- Ying Z., You J., Morris C., Ren X., Hamilton W., Leskovec J.: Hierarchical graph representation learning with differentiable pooling. In Proceedings of Advances in Neural Information Processing Systems (2018), pp. 4800–4810. 1
- Yao G., Yuan Y., Shao T., Zhou K.: Mesh guided one-shot face reenactment using graph convolutional networks. In Proceedings of ACM International Conference on Multimedia (2020), pp. 1773–1781. 3
- Zheng H., Perrine S. M. M., Pitirri M. K., Kawasaki K., Wang C., Richtsmeier J. T., Chen D. Z.: Cartilage segmentation in high-resolution 3D micro-CT images via uncertainty-guided self-training with very sparse annotation. In Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention (2020), pp. 802–812. 8