Article

ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics

Authors:

Nicolai Marquardt,

Andrea BianchiAuthors Info & Claims

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

Article No.: 727, Pages 1 - 14

https://doi.org/10.1145/3411764.3445434

Published: 07 May 2021 Publication History

Editorial Notes

The authors have requested minor, non-substantive changes to the VoR and, in accordance with ACM policies, a Corrected VoR was published on June 3, 2021. For reference purposes the VoR may still be accessed via the Supplemental Material section on this page.

Abstract

Computational notebooks help data analysts analyze and visualize datasets, and share analysis procedures and outputs. However, notebooks typically combine code (e.g., Python scripts), notes, and outputs (e.g., tables, graphs). The combination of disparate materials is known to hinder the comprehension of notebooks, making it difficult for analysts to collaborate with other analysts unfamiliar with the dataset. To mitigate this problem, we introduce ToonNote, a JupyterLab extension that enables the conversion of notebooks into “data comics.” ToonNote provides a simplified view of a Jupyter notebook, highlighting the most important results while supporting interactive and free exploration of the dataset. This paper presents the results of a formative study that motivated the system, its implementation, and an evaluation with 12 users, demonstrating the effectiveness of the produced comics. We discuss how our findings inform the future design of interfaces for computational notebooks and features to support diverse collaborators.

Supplementary Material

3445434-vor (3445434-vor.pdf)

Version of Record for "ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics" by Kang et al., Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21).

Download
18.99 MB

Supplementary Materials (3411764.3445434_supplementalmaterials.zip)

Download
92.26 KB

References

[1]

2020. Kaggle. https://www.kaggle.com/.

[2]

Paul A Aleixo and Krystina Sumner. 2017. Memory for biopsychology material presented in comic book format. Journal of Graphic Novels and Comics 8, 1 (2017), 79–88.

[3]

Benjamin Bach, Natalie Kerracher, Kyle Wm. Hall, Sheelagh Carpendale, Jessie Kennedy, and Nathalie Henry Riche. 2016. Telling Stories about Dynamic Networks with Graph Comics. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 3670–3682. https://doi.org/10.1145/2858036.2858387

Digital Library

[4]

Benjamin Bach, Nathalie Henry Riche, Sheelagh Carpendale, and Hanspeter Pfister. 2017. The Emerging Genre of Data Comics. IEEE Computer Graphics and Applications 37, 3 (May 2017), 6–13. https://doi.org/10.1109/MCG.2017.33Conference Name: IEEE Computer Graphics and Applications.

Digital Library

[5]

Benjamin Bach, Zezhong Wang, Matteo Farinella, Dave Murray-Rust, and Nathalie Henry Riche. 2018. Design Patterns for Data Comics. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173612

Digital Library

[6]

M. Brehmer, B. Lee, B. Bach, N. H. Riche, and T. Munzner. 2017. Timelines Revisited: A Design Space and Considerations for Expressive Storytelling. IEEE Transactions on Visualization and Computer Graphics 23, 9(2017), 2151–2164.

Digital Library

[7]

Souti Chattopadhyay, Ishita Prasad, Austin Z. Henley, Anita Sarma, and Titus Barik. 2020. What’s Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376729

Digital Library

[8]

Andy Cockburn, Amy Karlson, and Benjamin B. Bederson. 2009. A Review of Overview+detail, Zooming, and Focus+context Interfaces. ACM Comput. Surv. 41, 1, Article 2 (Jan. 2009), 31 pages. https://doi.org/10.1145/1456650.1456652

Digital Library

[9]

Kristin A Cook and James J Thomas. 2005. Illuminating the Path: The Research and Development Agenda for Visual Analytics. (5 2005).

[10]

Ian Drosos, Titus Barik, Philip J. Guo, Robert DeLine, and Sumit Gulwani. 2020. Wrex: A Unified Programming-by-Example Interaction for Synthesizing Readable Code for Data Scientists. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376442

Digital Library

[11]

Ana Figueiras. 2014. Narrative visualization: A case study of how to incorporate narrative elements in existing visualizations. In 2014 18th International Conference on Information Visualisation. IEEE, 46–52.

[12]

Nahum Gershon and Ward Page. 2001. What Storytelling Can Do for Information Visualization. Commun. ACM 44, 8 (Aug. 2001), 31–37. https://doi.org/10.1145/381641.381653

Digital Library

[13]

Philip Jia Guo. 2012. Software tools to facilitate research programming. Ph.D. Dissertation. Stanford University Stanford, CA.

[14]

Andrew Head, Fred Hohman, Titus Barik, Steven M. Drucker, and Robert DeLine. 2019. Managing Messes in Computational Notebooks. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300500

Digital Library

[15]

John T. Van Stan II Jan Friesen and Skander Elleuche. 2018. Communicating Science through Comics: A Method. Publications 6(2018), 2,3.

[16]

Jeffery.M.Perkel. [n.d.]. Why Jupyter is data scientists’ computational notebook of choice. https://www.nature.com/articles/d41586-018-07196-1. Accessed: 2020-09-03.

[17]

Mary Beth Kery, Marissa Radensky, Mahima Arya, Bonnie E. John, and Brad A. Myers. 2018. The Story in the Notebook: Exploratory Data Science Using a Literate Programming Tool. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3173574.3173748

Digital Library

[18]

Miryung Kim, Thomas Zimmermann, Robert DeLine, and Andrew Begel. 2016. The emerging role of data scientists on software development teams. In 2016 IEEE/ACM 38th International Conference on Software Engineering (ICSE). IEEE, 96–107.

Digital Library

[19]

N. W. Kim, B. Bach, H. Im, S. Schriber, M. Gross, and H. Pfister. 2018. Visualizing Nonlinear Narratives with Story Curves. IEEE Transactions on Visualization and Computer Graphics 24, 1(2018), 595–604.

[20]

Nam Wook Kim, Nathalie Henry Riche, Benjamin Bach, Guanpeng Xu, Matthew Brehmer, Ken Hinckley, Michel Pahud, Haijun Xia, Michael J. McGuffin, and Hanspeter Pfister. 2019. DataToon: Drawing Dynamic Network Comics With Pen + Touch Interaction. In CHI. ACM, 105.

[21]

N. W. Kim, E. Schweickart, Z. Liu, M. Dontcheva, W. Li, J. Popovic, and H. Pfister. 2017. Data-Driven Guides: Supporting Expressive Design for Information Graphics. IEEE Transactions on Visualization and Computer Graphics 23, 1(2017), 491–500.

Digital Library

[22]

B. Lee, N. H. Riche, P. Isenberg, and S. Carpendale. 2015. More Than Telling a Story: Transforming Data into Visually Shared Stories. IEEE Computer Graphics and Applications 35, 5 (2015), 84–90.

Digital Library

[23]

Yaoli Mao, Dakuo Wang, Michael Muller, Kush R Varshney, Ioana Baldini, Casey Dugan, and Aleksandra Mojsilović. 2019. How Data ScientistsWork Together With Domain Experts in Scientific Collaborations: To Find The Right Answer Or To Ask The Right Question?Proceedings of the ACM on Human-Computer Interaction 3, GROUP(2019), 1–23.

[24]

Andreas Mathisen, Tom Horak, Clemens Nylandsted Klokmose, Kaj Grønbæk, and Niklas Elmqvist. 2019. InsideInsights: Integrating Data-Driven Reporting in Collaborative Visual Analytics. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 649–661.

[25]

Kate Matsudaira. 2015. The science of managing data science. Queue 13, 4 (2015), 30–34.

Digital Library

[26]

Richard E Mayer and Joan K Gallini. 1990. When is an illustration worth ten thousand words?Journal of educational psychology 82, 4 (1990), 715.

[27]

Scott McCloud. 1993. Understanding comics: The invisible art. Northampton.

[28]

Adam Rule, Ian Drosos, Aurélien Tabard, and James D Hollan. 2018. Aiding collaborative reuse of computational notebooks with annotated cell folding. Proceedings of the ACM on Human-Computer Interaction 2, CSCW(2018), 1–12.

Digital Library

[29]

Adam Rule, Amanda Birmingham Id, Cristal Zuniga Id, Ilkay Altintas Id, Cheng Huang Id, Rob Knight, Niema Moshiri Id, and Mai H Nguyen. 2019. Ten simple rules for writing and sharing computational analyses in Jupyter Notebooks. (2019), 1–8.

[30]

Adam Rule, Aurélien Tabard, and James D. Hollan. 2018. Exploration and Explanation in Computational Notebooks. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3173606

Digital Library

[31]

Ali Sarvghad and Melanie Tory. 2015. Exploiting analysis history to support collaborative data analysis. In Proceedings of the 41st Graphics Interface Conference. 123–130.

Digital Library

[32]

Andreas Schreiber and Regina Struminski. 2017. Visualizing Provenance using Comics. In 9th USENIX Workshop on the Theory and Practice of Provenance (TaPP 2017). USENIX Association, Seattle, WA. https://www.usenix.org/conference/tapp17/workshop-program/presentation/schreiber

Digital Library

[33]

E. Segel and J. Heer. 2010. Narrative Visualization: Telling Stories with Data. IEEE Transactions on Visualization and Computer Graphics 16, 6(2010), 1139–1148.

Digital Library

[34]

Aurélien Tabard, Wendy E. Mackay, and Evelyn Eastmond. 2008. From Individual to Collaborative: The Evolution of Prism, a Hybrid Laboratory Notebook. In Proceedings of the 2008 ACM Conference on Computer Supported Cooperative Work (San Diego, CA, USA) (CSCW ’08). Association for Computing Machinery, New York, NY, USA, 569–578. https://doi.org/10.1145/1460563.1460653

Digital Library

[35]

Chao Tong, Richard Roberts, Rita Borgo, Sean Walton, Robert S Laramee, Kodzo Wegba, Aidong Lu, Yun Wang, Huamin Qu, Qiong Luo, 2018. Storytelling and visualization: An extended survey. Information 9, 3 (2018), 65.

[36]

April Yi Wang, Zihan Wu, Christopher Brooks, and Steve Oney. 2020. Callisto: Capturing the ”Why” by Connecting Conversations with Computational Narratives. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376740

Digital Library

[37]

Zezhong Wang, Shunming Wang, Matteo Farinella, Dave Murray-Rust, Nathalie Henry Riche, and Benjamin Bach. 2019. Comparing Effectiveness and Engagement of Data Comics and Infographics. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3290605.3300483

Digital Library

[38]

Amy X. Zhang, Michael Muller, and Dakuo Wang. 2020. How do Data Science Workers Collaborate? Roles, Workflows, and Tools. 1, 1 (2020), 1–23. arxiv:2001.06684http://arxiv.org/abs/2001.06684

[39]

Zhenpeng Zhao, Rachael Marr, and Niklas Elmqvist. 2015. Data comics: Sequential art for data-driven storytelling. tech. report (2015).

Cited By

Ouyang YShen LWang YLi Q(2024)NotePlayer: Engaging Computational Notebooks for Dynamic Presentation of Analytical ProcessesProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676410(1-20)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676410
Milesi MAlfredo REcheverria VYan LZhao LTsai YMartinez-Maldonado R(2024)"It's Really Enjoyable to See Me Solve the Problem like a Hero": GenAI-enhanced Data Comics as a Learning Analytics ToolExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651111(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651111
In SKrokos EWhitley KNorth CYang Y(2024)Evaluating Navigation and Comparison Performance of Computational Notebooks on Desktop and in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642932(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642932
Show More Cited By

Index Terms

ToonNote: Improving Communication in Computational Notebooks Using Interactive Data Comics
1. Human-centered computing
  1. Collaborative and social computing
  2. Human computer interaction (HCI)

Index terms have been assigned to the content through auto-classification.

Recommendations

What's Wrong with Computational Notebooks? Pain Points, Needs, and Design Opportunities
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Computational notebooks - such as Azure, Databricks, and Jupyter - are a popular, interactive paradigm for data scientists to author code, analyze data, and interleave visualizations, all within a single document. Nevertheless, as data scientists ...
How Data Scientists Use Computational Notebooks for Real-Time Collaboration

Effective collaboration in data science can leverage domain expertise from each team member and thus improve the quality and efficiency of the work. Computational notebooks give data scientists a convenient interactive solution for sharing and keeping ...
StickyLand: Breaking the Linear Presentation of Computational Notebooks
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

How can we better organize code in computational notebooks? Notebooks have become a popular tool among data scientists, as they seamlessly weave text and code together, supporting users to rapidly iterate and document code experiments. However, it is ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

May 2021

10862 pages

ISBN:9781450380966

DOI:10.1145/3411764

General Chairs:
Yoshifumi Kitamura
Tohoku University, Japan
,
Aaron Quigley
University of New South Wales, Australia
,
Program Chairs:
Katherine Isbister
University of California Santa Cruz, USA
,
Takeo Igarashi
The University of Tokyo, Japan
,
Publications Chairs:
Pernille Bjørn
University of Copenhagen, Denmark
,
Steven Drucker
Microsoft Research, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article
Research
Refereed limited

Funding Sources

National Research Foundation of Korea(NRF)

Conference

CHI '21

Sponsor:

SIGCHI

CHI '21: CHI Conference on Human Factors in Computing Systems

May 8 - 13, 2021

Yokohama, Japan

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
833
Total Downloads

Downloads (Last 12 months)153
Downloads (Last 6 weeks)6

Reflects downloads up to 24 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ouyang YShen LWang YLi Q(2024)NotePlayer: Engaging Computational Notebooks for Dynamic Presentation of Analytical ProcessesProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676410(1-20)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676410
Milesi MAlfredo REcheverria VYan LZhao LTsai YMartinez-Maldonado R(2024)"It's Really Enjoyable to See Me Solve the Problem like a Hero": GenAI-enhanced Data Comics as a Learning Analytics ToolExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3651111(1-7)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613905.3651111
In SKrokos EWhitley KNorth CYang Y(2024)Evaluating Navigation and Comparison Performance of Computational Notebooks on Desktop and in Virtual RealityProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642932(1-15)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642932
Wang FLin YYang LLi HGu MZhu MQu H(2024)OutlineSpark: Igniting AI-powered Presentation Slides Creation from Computational Notebooks through OutlinesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642865(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642865
Li HWang YQu H(2024)Where Are We So Far? Understanding Data Storytelling Tools from the Perspective of Human-AI CollaborationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642726(1-19)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642726
Zhou THuang JChan G(2024)Epigraphics: Message-Driven Infographics AuthoringProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642172(1-18)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642172
Kim JLee HNguyen DShin MKwon BKo SElmqvist N(2024)DG Comics: Semi-Automatically Authoring Graph Comics for Dynamic GraphsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2024.345634031:1(973-983)Online publication date: 10-Sep-2024
https://dl.acm.org/doi/10.1109/TVCG.2024.3456340
Chen QCao SWang JCao N(2024)How Does Automation Shape the Process of Narrative Visualization: A Survey of ToolsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.326132030:8(4429-4448)Online publication date: Aug-2024
https://doi.org/10.1109/TVCG.2023.3261320
Riyantoko PFunabiki NWai KAung SMuhaimin ATrimono (2024)A Proposal of Python Programming Exercise Problems for Basic Statistics Learning2024 Seventh International Conference on Vocational Education and Electrical Engineering (ICVEE)10.1109/ICVEE63912.2024.10824036(289-295)Online publication date: 30-Oct-2024
https://doi.org/10.1109/ICVEE63912.2024.10824036
Zou YShan XTan SZhou S(2024)Can We Do Better with What We Have Done? Unveiling the Potential of ML Pipeline in Notebooks2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00052(499-511)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00052
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten