VisFM: Visual Analysis of Image Feature Matchings
Chenhui Li
School of Computer Science and Software Engineering, East China Normal University, China
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, P.R. China
Search for more papers by this authorGeorge Baciu
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, P.R. China
Search for more papers by this authorChenhui Li
School of Computer Science and Software Engineering, East China Normal University, China
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, P.R. China
Search for more papers by this authorGeorge Baciu
Department of Computing, The Hong Kong Polytechnic University, Hong Kong, P.R. China
Search for more papers by this authorAbstract
Feature matching is the most basic and pervasive problem in computer vision and it has become a primary component in big data analytics. Many tools have been developed for extracting and matching features in video streams and image frames. However, one of the most basic tools, that is, a tool for simply visualizing matched features for the comparison and evaluation of computer vision algorithms is not generally available, especially when dealing with a large number of matching lines. We introduce VisFM, an integrated visual analysis system for comprehending and exploring image feature matchings. VisFM presents a matching view with an intuitive line bundling to provide useful insights regarding the quality of matched features. VisFM is capable of showing a summarization of the features and matchings through group view to assist domain experts in observing the feature matching patterns from multiple perspectives. VisFM incorporates a series of interactions for exploring the feature data. We demonstrate the visual efficacy of VisFM by applying it to three scenarios. An informal expert feedback, conducted by our collaborator in computer vision, demonstrates how VisFM can be used for comparing and analysing feature matchings when the goal is to improve an image retrieval algorithm.
Supporting Information
| Filename | Description |
|---|---|
| cgf13391-sup-0001-VisFM-Demo-Video.mp46.5 MB | Data Video S1 |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- [ASS*12] Achanta R., Shaji A., Smith K., Lucchi A., Fua P., Susstrunk S.: Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis and Machine Intelligence 34, 11 (2012), 2274–2282.
- [BOH11] Bostock M., Ogievetsky V., Heer J.: D3 data-driven documents. IEEE Transactions on Visualization and Computer Graphics 17, 12 (2011), 2301–2309.
- [BRH*17] Bach B., Riche N. H., Hurter C., Marriott K., Dwyer T.: Towards unambiguous edge bundling: Investigating confluent drawings for network visualization. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 541–550.
- [BTVG06] Bay H., Tuytelaars T., Van Gool L.: Surf: Speeded up robust features. In Proceedings of European Conference on Computer Vision (2006), Springer, pp. 404–417.
10.1007/11744023_32 Google Scholar
- [CCRV08] Caldwell B., Cooper M., Reid L. G., Vanderheiden G.: Web content accessibility guidelines (wcag) 2.0.
- [DMM*14] Dwyer T., Mears C., Morgan K., Niven T., Marriott K., Wallace M.: Improved optimal and approximate power graph compression for clearer visualisation of dense graphs. In Proceedings of 2014 IEEE Pacific Visualization Symposium (PacificVis) (2014), IEEE, pp. 105–112.
10.1109/PacificVis.2014.46 Google Scholar
- [ED07] Ellis G., Dix A.: A taxonomy of clutter reduction for information visualisation. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1216–1223.
- [EHP*11] Ersoy O., Hurter C., Paulovich F., Cantareiro G., Telea A.: Skeleton-based edge bundling for graph visualization. IEEE Transactions on Visualization and Computer Graphics 17, 12 (2011), 2364–2373.
- [EKS*96] Ester M., Kriegel H.-P., Sander J., Xu X.: A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of KDD (1996), vol. 96, pp. 226–231.
- [FR91] Fruchterman T. M., Reingold E. M.: Graph drawing by force-directed placement. Software: Practice and Experience 21, 11 (1991), 1129–1164.
- [HET12] Hurter C., Ersoy O., Telea A.: Graph bundling by kernel density estimation. Computer Graphics Forum 31, 3pt1 (2012), 865–874.
- [HET13] Hurter C., Ersoy O., Telea A.: Smooth bundling of large streaming and sequence graphs. In Proceedings of 2013 IEEE Pacific Visualization Symposium (PacificVis) (2013), pp. 41–48.
10.1109/PacificVis.2013.6596126 Google Scholar
- [Hol06] Holten D.: Hierarchical edge bundles: Visualization of adjacency relations in hierarchical data. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 741–748.
- [HVW09] Holten D., Van Wijk J. J.: Force-directed edge bundling for graph visualization. Computer Graphics Forum 28, 3 (2009), 983–990.
- [KMN*02] Kanungo T., Mount D., Netanyahu N., Piatko C., Silverman R., Wu A.: An efficient k-means clustering algorithm: Analysis and implementation. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 7 (2002), 881–892.
- [KSH12] Krizhevsky A., Sutskever I., Hinton G. E.: Imagenet classification with deep convolutional neural networks. In Proceedings of the International Conference on Advances in Neural Information Processing Systems (2012), pp. 1097–1105.
- [Low04] Lowe D. G.: Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2 (2004), 91–110.
- [LRFH13] Lin S., Ritchie D., Fisher M., Hanrahan P.: Probabilistic color-by-numbers: Suggesting pattern colorizations using factor graphs. ACM Transactions on Graphics 32, 4 (2013), 37:1–37:12.
- [Mac67] MacQueen J.: Some methods for classification and analysis of multivariate observations. In Proceedings of the 5th Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Statistics (Berkeley, CA, USA, 1967), University of California Press, pp. 281–297.
- [MG13] Mayorga A., Gleicher M.: Splatterplots: Overcoming overdraw in scatter plots. IEEE Transactions on Visualization and Computer Graphics 19, 9 (2013), 1526–1538.
- [ML09] Muja M., Lowe D. G.: Fast approximate nearest neighbors with automatic algorithm configuration. In Proceedings of the International Conference on Computer Vision Theory and Application (VISSAPP'09) (2009), pp. 331–340.
- [MS07] Moreira A., Santos M. Y.: Concave hull: A k-nearest neighbours approach for the computation of the region occupied by a set of points. In Proceedings of the International Conference on Computer Graphics Theory and Applications (2007), pp. 61–68.
- [QZH15] Qian X., Zhao Y., Han J.: Image location estimation by salient region matching. IEEE Transactions on Image Processing 24, 11 (2015), 4348–4358.
- [SAA*11] Sebastian B., Andrienko G., Andrienko N., Schreck T., Landesberger T. V.: Interactive analysis of object group changes over time. In Proceedings of International Workshop on Visual Analytics (EuroVA 2011) (2011), pp. 41–44.
- [Sil86] Silverman B. W.: Density Estimation for Statistics and Data Analysis, vol. 26. CRC Press, Boca Raton, FL, 1986.
- [SMNR16] Sun M., Mi P., North C., Ramakrishnan N.: Biset: Semantic edge bundling with biclusters for sensemaking. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2016), 310–319.
- [TE10] Telea A., Ersoy O.: Image-based edge bundles: Simplified visualization of large graphs. Computer Graphics Forum 29, 3 (2010), 843–852.
- [VKP*16] Vondrick C., Khosla A., Pirsiavash H., Malisiewicz T., Torralba A.: Visualizing object detection features. International Journal of Computer Vision 119, 2 (2016), 145–158.
- [vL07] von Luxburg U.: A tutorial on spectral clustering. Statistics and Computing 17, 4 (2007), 395–416.
- [WZL13] Wang X.-J., Zhang L., Liu C.: Duplicate discovery on 2 billion internet images. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2013), pp. 429–436.
10.1109/CVPRW.2013.71 Google Scholar
- [XLXJ11] Xu L., Lu C., Xu Y., Jia J.: Image smoothing via l0 gradient minimization. ACM Transactions on Graphics 30, 6 (2011), 174:1–174:12.
- [XYXJ12] Xu L., Yan Q., Xia Y., Jia J.: Structure extraction from texture via relative total variation. ACM Transactions on Graphics 31, 6 (2012), 139:1–139:10.
- [YDGM17] Yang Y., Dwyer T., Goodwin S., Marriott K.: Many-to-many geographically-embedded flow visualisation: An evaluation. IEEE Transactions on Visualization and Computer Graphics 23, 1 (2017), 411–420.
- [ZF14] Zeiler M. D., Fergus R.: Visualizing and understanding convolutional networks. In Proceedings of European Conference on Computer Vision (2014), Springer, pp. 818–833.
10.1007/978-3-319-10590-1_53 Google Scholar
- [ZXYQ13] Zhou H., Xu P., Yuan X., Qu H.: Edge bundling in information visualization. Tsinghua Science and Technology 18, 2 (2013), 145–156.
- [ZYCA06] Zhu Q., Yeh M.-C., Cheng K.-T., Avidan S.: Fast human detection using a cascade of histograms of oriented gradients. In Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2006), vol. 2, IEEE, pp. 1491–1498.
- [ZYS09] Zhou H., Yuan Y., Shi C.: Object tracking using sift features and mean shift. Computer Vision and Image Understanding 113, 3 (2009), 345–352.
