research-article

EmailValet: managing email overload through private, accountable crowdsourcing

Authors:

Nicolas Kokkalis,

Michael S. Bernstein,

Scott R. KlemmerAuthors Info & Claims

CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative work

Pages 1291 - 1300

https://doi.org/10.1145/2441776.2441922

Published: 23 February 2013 Publication History

Abstract

This paper introduces privacy and accountability techniques for crowd-powered systems. We focus on email task management: tasks are an implicit part of every inbox, but the overwhelming volume of incoming email can bury important requests. We present EmailValet, an email client that recruits remote assistants from an expert crowdsourcing marketplace. By annotating each email with its implicit tasks, EmailValet's assistants create a task list that is automatically populated from emails in the user's inbox. The system is an example of a valet approach to crowdsourcing, which aims for parsimony and transparency in access con-trol for the crowd. To maintain privacy, users specify rules that define a sliding-window subset of their inbox that they are willing to share with assistants. To support accountability, EmailValet displays the actions that the assistant has taken on each email. In a weeklong field study, participants completed twice as many of their email-based tasks when they had access to crowdsourced assistants, and they became increasingly comfortable sharing their inbox with assistants over time.

Supplementary Material

suppl.mov (cscw0512-file3.mp4)

Supplemental video

Download
70.29 MB

References

[1]

Allow Someone Else to Manage Your Mail and Calendar. http://office.microsoft.com/en-us/outlook-help/allow-someone-else-to-manage-your-mail-and-calendar-HA010355554.aspx.

[2]

ClearContext tames Outlook. http://news.cnet.com/8301-17939_109-9947065-2.html.

[3]

Create Tasks and To-Do Items. http://office.microsoft.com/en-us/outlook-help/create-tasks-and-to-do-items-HA001229302.aspx.

[4]

New in Labs: Tasks. http://gmailblog.blogspot.com/2008/12/new-in-labs-tasks.html.

[5]

Set up Mail Delegation. http://support.google.com/mail/bin/answer.py?hl=en&answer=138350.

[6]

Barley, S. R., D. Meyerson, and S. Grodal. E-mail as a Source and Symbol of Stress. Organization Science 22, 4. 2011.

Digital Library

[7]

Bekkerman, R., A. McCallum, and G. Huang. Automatic Categorization of Email into Folders: Benchmark Experiments on Enron and SRI Corpora. Technical Report IR-418. 2004.

[8]

Bellotti, V., N. Ducheneaut, M. Howard, and I. Smith. Taking Email to Task: The Design and Evaluation of a Task Management Centered Email Tool. Proc. CHI 2003. 2003.

Digital Library

[9]

Bennett, P. N. and J. Carbonell. Detecting Action-Items in E-mail. Proc. SIGIR 2005. 2005.

Digital Library

[10]

Bernstein, M. S., G. Little, R. C. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, and K. Panovich. Soylent: A Word Processor with a Crowd Inside. Proc. UIST 2010. 2010.

Digital Library

[11]

Boone, G. Concept Features in Re:Agent, an Intelligent Email Agent. Proc. AAMAS 1998. 1998.

Digital Library

[12]

Corston-Oliver, S., E. Ringger, M. Gamon and R. Campbell. Task-focused summarization of email. ACL 2004 Workshop: Text Summarization Branches Out. 2004.

[13]

Crawford, A. B. Corporate Electronic Mail - A Communication-Intensive Application of Information Technology. MIS Quarterly 6, 3. 1982.

Digital Library

[14]

Dow. S., A. Kulkarni, S. R. Klemmer, and B. Hartmann. Shepherding the Crowd Yields Better Work. Proc. CSCW 2012. 2012.

Digital Library

[15]

Erickson, T., C. Davis, W. Kellogg, and M. Helander. Assistance: The Work Practices of Human Administrative Assistants and their Implications for IT and Organizations. Proc. CSCW 2008. 2008.

Digital Library

[16]

Faulring, A., B. Myers, K. Mohnkern, B. Schmerl, A. Steinfeld, J. Zimmerman, A. Smailagic, J. Hansen, and D. Siewiorek. Agent-assisted task management that reduces email overload. Proc. IUI 2010. 2010.

Digital Library

[17]

Feldman, M. S. Constraints on Communication and Electronic Mail. Proc. CSCW 1986. 1986.

Digital Library

[18]

Fisher, D., A. J. Brush, E. Gleave, M. A. Smith. Revisiting Whittaker & Sidner's "email overload" ten years later. Proc. CSCW 2006. 2006.

Digital Library

[19]

Freed, M., J. Carbonell, G. Gordon, J. Hayes, B. Myers, D. Siewiorek, S. Smith, A. Steinfeld, and A. Tomasic. RADAR: A Personal Assistant that Learns to Reduce Email Overload. In Proc. AAAI 2008. 2008.

Digital Library

[20]

Gwizdka, J. TaskView - Design and Evaluation of a Task-based Email Interface. Proc. CASCON 2002. 2002.

Digital Library

[21]

Horvitz, E., A. Jacobs, D. Hovel. Attention-sensitive alerting. Proc. UAI 1999. 1999.

Digital Library

[22]

Jackson, T., R. Dawson, and D. Wilson. The Cost of Email Interruption. Journal of Systems and Information Technology 5, 1. 2001.

[23]

Kiritchenko, S. and Matwin, S. Email Classification with Co-Training. Proc. CASCON 2001. 2001.

Digital Library

[24]

Kokkalis, N., T. Köhn, J. Huebner, M. Lee, F. Schulze, S. R. Klemmer. TaskGenies: Automatically Providing Action Plans Helps People Complete Tasks. Technical Report CSTR 2012-04. 2012.

[25]

Lampert, A., R. Dale, C. Paris. Detecting Emails Containing Requests for Action. Proc. ACL 2010. 2010.

Digital Library

[26]

Mackay, W. E. More than Just a Communication System: Diversity in the Use of Electronic Mail. Proc. CSCW 1988. 1988.

Digital Library

[27]

Muller, M. J., and Gruen, D. M. Working Together Inside an Emailbox. Proc. ECSCW 2005. 2005.

Digital Library

[28]

Palen, L., and P. Dourish. Unpacking "Privacy" for a Networked World. Proc. CHI 2003. 2003.

Digital Library

[29]

Rambow, O., L. Shrestha, J. Chen, and C. Lauridsen. Summarizing Email Threads. Proc. HLT-NAACL 2004. 2004.

Digital Library

[30]

Sahami, M., S. Dumais, D. Heckerman, and E. Horvitz. A Bayesian Approach to Filtering Junk E-mail. AAAI Workshop: Learning for Text Categorization. 1998.

[31]

Samiei, M., J. Dill, and A. Kirkpatrick. EzMail: Using Information Visualization Techniques to Help Manage Email. Proc. IV 2004. 2004.

Digital Library

[32]

Segal, R. B. and Kephart, J. O. MailCat: An Intelligent Assistant for Organizing E-Mail. Proc. AAMAS 1999. 1999.

Digital Library

[33]

Shen, J., L. Li, T. G. Dietterich, and J. L. Herlocker. A Hybrid Learning System for Recognizing User Tasks from Desktop Activities and Email Messages. Proc. IUI 2006. 2006.

Digital Library

[34]

Takkinen, J. and N. Shahmehri. CAFE: A Conceptual Model for Managing Information in Electronic Mail. Proc. HICSS 1998. 1998.

Digital Library

[35]

Venolia, G., L. Dabbish, J. J. Cadiz, and A. Goopta. Supporting email workflow. Technical Report MSR-TR-2001-88. 2001.

[36]

Weber, R. The Grim Reaper: The Curse of E-Mail. MIS Quarterly 28, 3. 2004.

Digital Library

[37]

Weiser, M. The Computer for the 21st Century. Scientific American. 1991.

[38]

Whittaker, S. and C. Sidner. Email Overload: Exploring Personal Information Management of Email. Proc. CHI 1996. 1996.

Digital Library

[39]

Winograd, T., and F. Flores. Understanding Computers and Cognition: A New Foundation for Design. Ablex, 1986.

Digital Library

Cited By

Morrison KIqbal SHorvitz E(2024)AI-Powered Reminders for Collaborative Tasks: Experiences and FuturesProceedings of the ACM on Human-Computer Interaction10.1145/36537018:CSCW1(1-20)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3653701
Park SVishwabhan SMuller MKarger D(2024)“I Really Need Your Help with This Work...”: A System for Navigating the Tricky Terrain of Managing Up by Leveraging One’s Motivation to Get Things DoneACM Transactions on Computer-Human Interaction10.1145/365260331:4(1-36)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3652603
McInnis BPindus RKareem DGamboa SNebeker C(2024)Exploring the Future of Informed Consent: Applying a Service Design ApproachProceedings of the ACM on Human-Computer Interaction10.1145/36373308:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637330
Show More Cited By

Index Terms

EmailValet: managing email overload through private, accountable crowdsourcing
1. Social and professional topics
  1. Professional topics
    1. Computing and business

Recommendations

Overload is overloaded: email in the age of Gmail
CHI '14: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems

The term email overload has two definitions: receiving a large volume of incoming email, and having emails of different status types (to do, to read, etc). Whittaker and Sidner proposed the latter definition in 1996, noticing that email inboxes were far ...
Agent-assisted task management that reduces email overload
IUI '10: Proceedings of the 15th international conference on Intelligent user interfaces

RADAR is a multiagent system with a mixed-initiative user interface designed to help office workers cope with email overload. RADAR agents observe experts to learn models of their strategies and then use the models to assist other people who are working ...
Revisiting Whittaker & Sidner's "email overload" ten years later
CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work

Ten years ago, Whittaker and Sidner [8] published research on email overload, coining a term that would drive a research area that continues today. We examine a sample of 600 mailboxes collected at a high-tech company to compare how users organize their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '13: Proceedings of the 2013 conference on Computer supported cooperative work

February 2013

1594 pages

ISBN:9781450313315

DOI:10.1145/2441776

General Chairs:
Amy Bruckman
Georgia Institute of Technology, USA
,
Scott Counts
Microsoft Research, USA
,
Program Chairs:
Cliff Lampe
University of Michigan, USA
,
Loren Terveen
University of Minnesota, USA

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 February 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CSCW '13

Sponsor:

SIGCHI

CSCW '13: Computer Supported Cooperative Work

February 23 - 27, 2013

Texas, San Antonio, USA

Acceptance Rates

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

25
Total Citations
View Citations
552
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Morrison KIqbal SHorvitz E(2024)AI-Powered Reminders for Collaborative Tasks: Experiences and FuturesProceedings of the ACM on Human-Computer Interaction10.1145/36537018:CSCW1(1-20)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3653701
Park SVishwabhan SMuller MKarger D(2024)“I Really Need Your Help with This Work...”: A System for Navigating the Tricky Terrain of Managing Up by Leveraging One’s Motivation to Get Things DoneACM Transactions on Computer-Human Interaction10.1145/365260331:4(1-36)Online publication date: 19-Sep-2024
https://dl.acm.org/doi/10.1145/3652603
McInnis BPindus RKareem DGamboa SNebeker C(2024)Exploring the Future of Informed Consent: Applying a Service Design ApproachProceedings of the ACM on Human-Computer Interaction10.1145/36373308:CSCW1(1-31)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3637330
Reif JCzakert JSpieß EBerger R(2023)Digitalisierungsangst als Gefahr für ein digitales Mindset?Digitale Mindsets10.1007/978-3-658-41104-6_4(43-67)Online publication date: 20-Jul-2023
https://doi.org/10.1007/978-3-658-41104-6_4
Sannon SSun BCosley D(2022)Privacy, Surveillance, and Power in the Gig EconomyProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502083(1-15)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502083
Pfaffinger KReif JSpieß ECzakert JBerger R(2022)Using digital interventions to reduce digitalization-related stress: does it work?International Journal of Occupational Safety and Ergonomics10.1080/10803548.2022.211523429:3(1196-1211)Online publication date: 27-Sep-2022
https://doi.org/10.1080/10803548.2022.2115234
Feldman MMcInnis B(2021)How We Write with CrowdsProceedings of the ACM on Human-Computer Interaction10.1145/34329284:CSCW3(1-31)Online publication date: 5-Jan-2021
https://dl.acm.org/doi/10.1145/3432928
Hayat ONgah RKaleem ZHashim SRodrigues J(2020)A Survey on Security and Privacy Challenges in Device Discovery for Next-Generation SystemsIEEE Access10.1109/ACCESS.2020.29914598(84584-84603)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2991459
Mahar KZhang AKarger DMandryk RHancock MPerry MCox A(2018)SquadboxProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3174160(1-13)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3174160
Zhong SZhong HHuang XYang PShi JXie LWang KZhong SZhong HHuang XYang PShi JXie LWang K(2018)Connecting Human to Cyber-World: Security and Privacy Issues in Mobile Crowdsourcing NetworksSecurity and Privacy for Next-Generation Wireless Networks10.1007/978-3-030-01150-5_4(65-100)Online publication date: 23-Nov-2018
https://doi.org/10.1007/978-3-030-01150-5_4
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents