research-article

Simplified Audio Production in Asynchronous Voice-Based Discussions

Authors:

Venkatesh Sivaraman,

Piotr MitrosAuthors Info & Claims

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

Pages 1045 - 1054

https://doi.org/10.1145/2858036.2858416

Published: 07 May 2016 Publication History

Abstract

Voice communication adds nuance and expressivity to virtual discussions, but its one-shot nature tends to discourage collaborators from utilizing it. However, text-based interfaces have made voice editing much easier, especially with recent advancements enabling live, time-aligned speech transcription. We introduce SimpleSpeech, an easy-to-use platform for asynchronous audio communication (AAC) with lightweight tools for inserting content, adjusting pauses, and correcting transcript errors. Qualitative and quantitative results suggest that novice audio producers, such as high school students, experience decreased mental workload when using SimpleSpeech to produce audio messages than without editing. The linguistic formality in SimpleSpeech messages was also studied, and found to form a middle ground between oral and written media. Our findings on editable voice messages show new implications for the optimal design and use cases of AAC systems.

Supplementary Material

suppl.mov (pn1888-file3.mp4)

Supplemental video

Download
20.35 MB

References

[1]

Stephen Ades and Daniel C Swinehart. 1986. Voice annotation and editing in a workstation environment. XEROX Corporation, Palo Alto Research Center.

[2]

Barry Arons. 1993. SpeechSkimmer: Interactively Skimming Recorded Speech. In Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology (UIST '93). ACM, NY, NY, USA, 187--196.

Digital Library

[3]

Floraine Berthouzoz, Wilmot Li, and Maneesh Agrawala. 2012. Tools for Placing Cuts and Transitions in Interview Video. ACM Trans. Graph. 31, 4, Article 67 (July 2012), 8 pages.

Digital Library

[4]

Steven Bird, Ewan Klein, and Edward Loper. 2009. Natural Language Processing with Python. O'Reilly Media.

Digital Library

[5]

Thomas Cho. 2010. Linguistic Features of Electronic Mail in the Workplace: A Comparison with Memoranda. Language@Internet 7, 3 (2010).

[6]

Juan Casares et al. 2002a. Simplifying Video Editing Using Metadata. In DIS '02 Proceedings. London, 157--166.

Digital Library

[7]

Janet M. Baker et al. 2009. Developments and directions in speech recognition and understanding, Part 1. Signal Processing, IEEE 26, 3 (2009).

[8]

Steve Whittaker et al. 2002b. SCANMail: a voicemail interface that makes speech browsable, readable and searchable. CHI Letters 4, 1 (2002).

[9]

Alec Go, Richa Bhayani, and Lei Huang. 2009. Twitter Sentiment Classification using Distant Supervision. Technical Report. Stanford.

[10]

Jonathan Grudin. 1988. Why CSCW applications fail: Problems in the design and evaluation of organizational interfaces. ACM Conference on Computer-Supported Cooperative Work (1988).

Digital Library

[11]

C Halverson, D Horn, C Karat, and John Karat. 1999. The beauty of errors: Patterns of error correction in desktop speech systems. In Proceedings of INTERACT99. 133--140.

[12]

Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Human Mental Workload, P. A. Hancock and N. Meshkati (Eds.). North Holland Press, Amsterdam.

[13]

Francis Heylighen and Jean-Marc Dewaele. 2002. Variation in the contextuality of language: an empirical measure. Foundations of Science 7 (2002), 239--340.

[14]

Debby Hindus and Chris Schmandt. 1992. Ubiquitous Audio: Capturing Spontaneous Collaboration. In Proceedings of the 1992 ACM Conference on Computer-supported Cooperative Work (CSCW '92). ACM, NY, NY, USA, 210--217.

Digital Library

[15]

Philip Ice, Reagan Curtis, Perry Phillips, and John Wells. 2007. Using asynchronous audio feedback to enhance teaching presence and students' sense of community. Journal of Asynchronous Learning Networks 11, 2 (2007).

[16]

Sara Kiesler, Jane Siegel, and Timothy W. McGuire. 1984. Social Psychological Aspects of Computer-Mediated Communication. Amer. Psychologist 39, 10 (1984), 1123--1134.

[17]

Henry Kucera and W. Nelson Francis. 1967. Computational Analysis of Present-Day American English. Brown University Press, Providence.

[18]

Jennifer Lai and John Vergo. 1997. MedSpeak: Report Creation with Continuous Speech Recognition. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI '97). ACM, NY, NY, USA, 431--438.

Digital Library

[19]

Philip Marriott. 2002. Voice vs text-based discussion forums: An implementation of Wimba Voice Boards. In World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, Vol. 2002. 640--646.

[20]

Philip Marriott and Jane Hiscock. 2002. Voice vs Text-based Discussion Forums: An Implementation of Wimba Voice Boards. In Proc. E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, M. Driscoll and T. Reeves (Eds.). Chesapeake, VA.

[21]

Jody Oomen-Early, Mary Bold, Kristin L. Wiginton, Tara L. Gallien, and Nancy Anderson. 2008. Using asynchronous audio communication (AAC) in the online classroom: a comparative study. Journal of Online Learning and Teaching 4, 3 (2008).

[22]

Steve Rubin, Floraine Berthouzoz, Gautham J. Mysore, Wilmot Li, and Maneesh Agrawala. 2013. Content-Based Tools for Editing Audio Stories. In UIST '13. 113--122.

Digital Library

[23]

George Saon, Hong-Kwang J. Kuo, Steven Rennie, and Michael Picheny. 2015. The IBM 2015 English Conversational Telephone Speech Recognition System. Interspeech (2015).

[24]

Christopher Schmandt. 1981. The Intelligent Ear: A Graphical Interface to Digital Audio. In Proceedings, IEEE International Conference on Cybernetics and Society, IEEE.

[25]

Hagen Soltau, George Saon, and Tara N. Sainath. 2014. Joint training of convolutional and non-convolutional neural networks. In Proceedings of the IEEE Intl. Conference on Acoustic, Speech and Signal Processing. Florence, 5572--5576.

[26]

Will Styler. 2011. The EnronSent Corpus. Technical Report 01--2011. University of CO at Boulder Institute of Cognitive Science, Boulder, CO.

[27]

Chi-Hsiung Tu and Marina McIsaac. 2002. The Relationship of Social Presence and Interaction in Online Classes. Amer. Journal of Distance Education 16, 3 (2002).

[28]

Sunil Vemuri, Philip DeCamp, Walter Bender, and Chris Schmandt. 2004a. Improving Speech Playback Using Time-compression and Speech Recognition. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, NY, NY, USA, 295--302.

Digital Library

[29]

Sunil Vemuri, Philip DeCamp, Walter Bender, and Chris Schmandt. 2004b. Improving Speech Playback Using Time-Compression and Speech Recognition. CHI Letters 6, 1 (2004).

[30]

Joseph B. Walther. 1995. Relational Aspects of Computer-Mediated Communication: Experimental Observations over Time. Organization Science 6, 2 (1995), 186--203.

Digital Library

[31]

Steve Whittaker and Brian Amento. 2004. Semantic Speech Editing. CHI Letters (2004), 527--534.

Digital Library

[32]

Lynn Wilcox, Ian Smith, and Marcia Bush. 1992. Wordspotting for Voice Editing and Audio Indexing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '92). ACM, NY, NY, USA, 655--656.

Digital Library

[33]

Dongwook Yoon, Nicholas Chen, Francois Guimbreti'ere, and Abigail Sellen. 2014. RichReview: blending ink, speech, and gesture to support collaborative document review. In Proceedings of the 27th annual ACM symposium on User interface software and technology. ACM, 481--490.

Digital Library

[34]

Dongwook Yoon, Nicholas Chen, Bernie Randles, Amy Cheatle, Corinna E. Loeckenhoff, Steven J. Jackson, Abigail Sellen, and Francois Guimbreti'ere. 2016. Deployment of a Collaborative Multi-Modal Annotation System for Instructor Feedback and Peer Discussion. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM.

Digital Library

Cited By

Li ZJiang YLiu XZhao YYu CShi Y(2022)Enhancing Revisitation in Touchscreen Reading for Visually Impaired People with Semantic Navigation DesignProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35502906:3(1-22)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550290
McMillan DJaber R(2021)Leaving the Butler Behind: The Future of Role Reproduction in CUIProceedings of the 3rd Conference on Conversational User Interfaces10.1145/3469595.3469606(1-4)Online publication date: 27-Jul-2021
https://dl.acm.org/doi/10.1145/3469595.3469606
Khan TYoon DMcGrenere JBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Designing an Eyes-Reduced Document Skimming App for Situational ImpairmentsProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376641(1-14)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376641
Show More Cited By

Index Terms

Simplified Audio Production in Asynchronous Voice-Based Discussions
1. Human-centered computing
  1. Human computer interaction (HCI)

Recommendations

Semantic speech editing
CHI '04: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

Editing speech data is currently time-consuming and error-prone. Speech editors rely on acoustic waveform representations, which force users to repeatedly sample the underlying speech to identify words and phrases to edit. Instead we developed a ...
Singing Voice Database
Speech and Computer
Abstract
The first publicly available singing voice database, which was first released in 2012, is presented in this paper. This database contains recordings of professional singers including one Grammy Award winner. The database includes so-called plain ...
Content-based tools for editing audio stories
UIST '13: Proceedings of the 26th annual ACM symposium on User interface software and technology

Audio stories are an engaging form of communication that combine speech and music into compelling narratives. Existing audio editing tools force story producers to manipulate speech and music tracks via tedious, low-level waveform editing. In contrast, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems

May 2016

6108 pages

ISBN:9781450333627

DOI:10.1145/2858036

General Chairs:
Jofish Kaye
Yahoo
,
Allison Druin
University of Maryland / National Park Service
,
Program Chairs:
Cliff Lampe
University of Michigan
,
Dan Morris
Microsoft
,
Juan Pablo Hourcade
University of Iowa

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 May 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI'16

Sponsor:

SIGCHI

CHI'16: CHI Conference on Human Factors in Computing Systems

May 7 - 12, 2016

California, San Jose, USA

Acceptance Rates

CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
408
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)3

Reflects downloads up to 14 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Li ZJiang YLiu XZhao YYu CShi Y(2022)Enhancing Revisitation in Touchscreen Reading for Visually Impaired People with Semantic Navigation DesignProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35502906:3(1-22)Online publication date: 7-Sep-2022
https://dl.acm.org/doi/10.1145/3550290
McMillan DJaber R(2021)Leaving the Butler Behind: The Future of Role Reproduction in CUIProceedings of the 3rd Conference on Conversational User Interfaces10.1145/3469595.3469606(1-4)Online publication date: 27-Jul-2021
https://dl.acm.org/doi/10.1145/3469595.3469606
Khan TYoon DMcGrenere JBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Designing an Eyes-Reduced Document Skimming App for Situational ImpairmentsProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376641(1-14)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376641
Paay JKjeldskov JHansen KJørgensen TOvergaard K(2020)Digital ethnography of home use of digital personal assistantsBehaviour & Information Technology10.1080/0144929X.2020.183462041:4(740-758)Online publication date: 17-Oct-2020
https://doi.org/10.1080/0144929X.2020.1834620
Chow KCoyiuto CNguyen CYoon D(2019)Challenges and Design Considerations for Multimodal Asynchronous Collaboration in VRProceedings of the ACM on Human-Computer Interaction10.1145/33591423:CSCW(1-24)Online publication date: 7-Nov-2019
https://dl.acm.org/doi/10.1145/3359142
Rainey JMontague KBriggs PAnderson RNappey TOlivier PBrewster SFitzpatrick GCox AKostakos V(2019)GabberProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300607(1-12)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300607
Clark LDoyle PGaraialde DGilmartin ESchlögl SEdlund JAylett MCabral JMunteanu CEdwards JR Cowan B(2019)The State of Speech in HCI: Trends, Themes and ChallengesInteracting with Computers10.1093/iwc/iwz01631:4(349-371)Online publication date: 11-Sep-2019
https://doi.org/10.1093/iwc/iwz016
Nestsiarovich KPons D(2018)Interaction Diagrams: Development of a Method for Observing Group InteractionsBehavioral Sciences10.3390/bs90100059:1(5)Online publication date: 30-Dec-2018
https://doi.org/10.3390/bs9010005
Long KBakewell LMcNaney RVasileiou KAtkinson MBarreto MBarnett JWilson MLawson SVines JMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)Connecting Those That CareProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025715(1339-1351)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025715
Vashistha ASethi PAnderson RMark GFussell SLampe Cschraefel mHourcade JAppert CWigdor D(2017)RespeakProceedings of the 2017 CHI Conference on Human Factors in Computing Systems10.1145/3025453.3025640(1855-1866)Online publication date: 2-May-2017
https://dl.acm.org/doi/10.1145/3025453.3025640
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents