research-article

Public Access

Characterizing and Predicting Enterprise Email Reply Behavior

Authors:

Susan T. Dumais,

Paul N. Bennett,

Ahmed Hassan AwadallahAuthors Info & Claims

SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 235 - 244

https://doi.org/10.1145/3077136.3080782

Published: 07 August 2017 Publication History

Abstract

Email is still among the most popular online activities. People spend a significant amount of time sending, reading and responding to email in order to communicate with others, manage tasks and archive personal information. Most previous research on email is based on either relatively small data samples from user surveys and interviews, or on consumer email accounts such as those from Yahoo! Mail or Gmail. Much less has been published on how people interact with enterprise email even though it contains less automatically generated commercial email and involves more organizational behavior than is evident in personal accounts. In this paper, we extend previous work on predicting email reply behavior by looking at enterprise settings and considering more than dyadic communications. We characterize the influence of various factors such as email content and metadata, historical interaction features and temporal features on email reply behavior. We also develop models to predict whether a recipient will reply to an email and how long it will take to do so. Experiments with the publicly-available Avocado email collection show that our methods outperform all baselines with large gains. We also analyze the importance of different features on reply behavior predictions. Our findings provide new insights about how people interact with enterprise email and have implications for the design of the next generation of email clients.

References

[1]

Douglas Aberdeen, Ondrej Pacovsky, and Andrew Slater. The Learning behind Gmail Priority Inbox. In NIPS 2010 Workshop on Learning on Cores, Clusters and Clouds.

[2]

Victoria Bellotti, Nicolas Ducheneaut, Mark Howard, and Ian Smith. Taking Email to Task: The Design and Evaluation of a Task Management Centered Email Tool CHI '03.

[3]

Paul N. Bennett and Jaime Carbonell. Detecting Action-items in E-mail. In SIGIR '05.

[4]

Vitor R. Carvalho and William W. Cohen. On the Collective Classification of Email "Speech Acts" SIGIR '05.

[5]

Marta E. Cecchinato, Abigail Sellen, Milad Shokouhi, and Gavin Smyth. Finding Email in a Multi-Account, Multi-Device World CHI'16.

[6]

Michael Chui, James Manyika, Jacques Bughin, Richard Dobbs, Charles Roxburgh, Hugo Sarrazin, Geoffrey Sands, and Magdalena Westergren 2012. The social economy: Unlocking value and productivity through social technologies. (2012). A report by McKinsey Global Institute.

[7]

William W. Cohen, Vitor R. Carvalho, and Tom M. Mitchell. Learning to Classify Email into Speech Acts. In EMNLP '04.

[8]

Simon Corston-Oliver, Eric Ringger, Michael Gamon, and Richard Campbell. Integration of Email and Task Lists. In First Conference on Email and Anti-Spam.

[9]

Laura A. Dabbish and Robert E. Kraut. Email Overload at Work: An Analysis of Factors Associated with Email Strain CSCW '06.

[10]

Laura A. Dabbish, Robert E. Kraut, Susan Fussell, and Sara Kiesler. Understanding Email Use: Predicting Action on a Message CHI '05.

[11]

Dotan Di Castro, Zohar Karnin, Liane Lewin-Eytan, and Yoelle Maarek. You've Got Mail, and Here is What You Could Do With It!: Analyzing and Predicting Actions on Email Messages. In WSDM '16.

[12]

Danyel Fisher, A. J. Brush, Eric Gleave, and Marc A. Smith. Revisiting Whittaker & Sidner's "Email Overload" Ten Years Later CSCW '06.

[13]

Michael Freed, Jaime G. Carbonell, Geoffrey J. Gordon, Jordan Hayes, Brad A. Myers, Daniel P. Siewiorek, Stephen F. Smith, Aaron Steinfeld, and Anthony Tomasic. RADAR: A Personal Assistant that Learns to Reduce Email Overload AAAI '08.

[14]

David Graus, David van Dijk, Manos Tsagkias, Wouter Weerkamp, and Maarten de Rijke. Recipient Recommendation in Enterprises Using Communication Graphs and Email Content SIGIR '14.

[15]

Mihajlo Grbovic, Guy Halawi, Zohar Karnin, and Yoelle Maarek. How Many Folders Do You Really Need? Classifying Email into a Handful of Categories CIKM '14.

[16]

Ido Guy, Michal Jacovi, Noga Meshulam, Inbal Ronen, and Elad Shahar. Public vs. Private: Comparing Public Social Network Information with Email CSCW '08.

[17]

Minqing Hu and Bing Liu. Mining and Summarizing Customer Reviews. In KDD '04.

Digital Library

[18]

Anjuli Kannan, Karol Kurach, Sujith Ravi, Tobias Kaufmann, Andrew Tomkins, Balint Miklos, Greg Corrado, Laszlo Lukacs, Marina Ganea, Peter Young, and Vivek Ramavajjala. Smart Reply: Automated Response Suggestion for Email KDD '16.

[19]

Bryan Klimt and Yiming Yang. The Enron Corpus: A New Dataset for Email Classification Research ECML'04.

[20]

Farshad Kooti, Luca Maria Aiello, Mihajlo Grbovic, Kristina Lerman, and Amin Mantrach. Evolution of Conversations in the Age of Email Overload WWW '15.

[21]

Andrew Lampert, Robert Dale, and Cecile Paris. Detecting Emails Containing Requests for Action. HLT '10.

[22]

Carman Neustaedter, A. J. Bernheim Brush, and Marc A. Smith. Beyond "From" and "Received": Exploring the Dynamics of Email Triage CHI EA '05.

[23]

Byung-Won On, Ee-Peng Lim, Jing Jiang, Amruta Purandare, and Loo-Nin Teow. Mining Interaction Behaviors for Email Reply Order Prediction ASONAM '10.

[24]

Ashequl Qadir, Michael Gamon, Patrick Pantel, and Ahmed Hassan Awadallah. Activity Modeling in Email. In NAACL-HLT '16.

[25]

S. Radicati. 2014. Email statistics report, 2014--2018. (2014).

[26]

Maya Sappelli, Gabriella Pasi, Suzan Verberne, Maaike de Boer, and Wessel Kraaij 2016. Assessing E-mail intent and tasks in E-mail messages. Inf. Sci. Vol. 358--359 (2016), 1--17.

Digital Library

[27]

Michael Gamon Richard Campbell Simon H. Corston-Oliver, Eric Ringger. Task-focused Summarization of Email. In ACL'04.

[28]

Joshua R. Tyler and John C. Tang. When Can I Expect an Email Response? A Study of Rhythms in Email Usage ECSCW'03.

[29]

Steve Whittaker and Candace Sidner. Email Overload: Exploring Personal Information Management of Email CHI '96.

[30]

Yiming Yang and Xin Liu. A Re-examination of Text Categorization Methods. SIGIR '99.

[31]

Ji Zhu, Hui Zou, Saharon Rosset, and Trevor Hastie. 2009. Multi-class AdaBoost. Statistics and Its Interface (2009).

Cited By

Shah HJaidka KUngar LFagan JGrosser T(2023)Building a Multimodal Classifier of Email Behavior: Towards a Social Network Understanding of Organizational CommunicationInformation10.3390/info1412066114:12(661)Online publication date: 14-Dec-2023
https://doi.org/10.3390/info14120661
Chitare NCoventry LNicholson J(2023)“It may take ages”: Understanding Human-Centred Lateral Phishing Attack Detection in OrganisationsProceedings of the 2023 European Symposium on Usable Security10.1145/3617072.3617116(344-355)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3617072.3617116
Liu JNeville JSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Stationary Algorithmic Balancing For Dynamic Email Re-Ranking ProblemProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599909(4527-4538)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599909
Show More Cited By

Index Terms

Characterizing and Predicting Enterprise Email Reply Behavior
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User models
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Clustering and classification
    2. Users and interactive retrieval

Recommendations

Evolution of Conversations in the Age of Email Overload
WWW '15: Proceedings of the 24th International Conference on World Wide Web

Email is a ubiquitous communications tool in the workplace and plays an important role in social interactions. Previous studies of email were largely based on surveys and limited to relatively small populations of email users within organizations. In ...
Characterizing and Predicting Email Deferral Behavior
WSDM '19: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining

Email triage involves going throughunhandled emails and deciding what to do with them. This familiar process can become increasingly challenging as the number of unhandled email grows. During a triage session, users commonlydefer handling emails that ...
Email Reply Prediction: A Machine Learning Approach
Proceedings of the Symposium on Human Interface 2009 on Human Interface and the Management of Information. Information and Interaction. Part II: Held as part of HCI International 2009

Email has now become the most-used communication tool in the world and has also become the primary business productivity applications for most organizations and individuals. With the ever increasing popularity of emails, email over-load and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval

August 2017

1476 pages

ISBN:9781450350228

DOI:10.1145/3077136

General Chairs:
Noriko Kando
National Institute of Informatics
,
Tetsuya Sakai
Waseda University
,
Hideo Joho
University of Tsukuba
,
Program Chairs:
Hang Li
Huawei Noah's Ark Lab
,
Arjen P. de Vries
Radboud University
,
Ryen W. White
Microsoft Cortana

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 August 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation

Conference

SIGIR '17

Sponsor:

SIGIR

SIGIR '17: The 40th International ACM SIGIR conference on research and development in Information Retrieval

August 7 - 11, 2017

Tokyo, Shinjuku, Japan

Acceptance Rates

SIGIR '17 Paper Acceptance Rate 78 of 362 submissions, 22%;

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

53
Total Citations
View Citations
1,434
Total Downloads

Downloads (Last 12 months)235
Downloads (Last 6 weeks)50

Reflects downloads up to 16 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Shah HJaidka KUngar LFagan JGrosser T(2023)Building a Multimodal Classifier of Email Behavior: Towards a Social Network Understanding of Organizational CommunicationInformation10.3390/info1412066114:12(661)Online publication date: 14-Dec-2023
https://doi.org/10.3390/info14120661
Chitare NCoventry LNicholson J(2023)“It may take ages”: Understanding Human-Centred Lateral Phishing Attack Detection in OrganisationsProceedings of the 2023 European Symposium on Usable Security10.1145/3617072.3617116(344-355)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3617072.3617116
Liu JNeville JSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Stationary Algorithmic Balancing For Dynamic Email Re-Ranking ProblemProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599909(4527-4538)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599909
Hasso HGroßer KAymaz IGeppert HJürjens J(2023)Enhanced abbreviation–expansion pair detection for glossary term extractionInformation and Software Technology10.1016/j.infsof.2023.107203159:COnline publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107203
Frattini JFucci DMendez DSpínola RMandić VTaušan NAhmad MGonzalez-Huerta J(2023)An initial theory to understand and manage requirements engineering debt in practiceInformation and Software Technology10.1016/j.infsof.2023.107201159:COnline publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107201
Jia HYang WShen CPan MZhou Y(2023)Git command recommendations using crowd-sourced knowledgeInformation and Software Technology10.1016/j.infsof.2023.107199159:COnline publication date: 10-May-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107199
Siebert J(2023)Applications of statistical causal inference in software engineeringInformation and Software Technology10.1016/j.infsof.2023.107198159:COnline publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107198
Yousefifeshki FLi HKhomh F(2023)Studying the challenges of developing hardware description language programsInformation and Software Technology10.1016/j.infsof.2023.107196159:COnline publication date: 1-Jul-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107196
Stradowski SMadeyski L(2023)Industrial applications of software defect prediction using machine learningInformation and Software Technology10.1016/j.infsof.2023.107192159:COnline publication date: 10-May-2023
https://dl.acm.org/doi/10.1016/j.infsof.2023.107192
Jach A(2023)Artificial Intelligence Methods in Email Marketing—A SurveyDependable Computer Systems and Networks10.1007/978-3-031-37720-4_8(85-94)Online publication date: 11-Aug-2023
https://doi.org/10.1007/978-3-031-37720-4_8
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents