research-article

A Picture is Worth a Thousand Words: Improving Mobile Messaging with Real-time Autonomous Image Suggestion

Authors:

Sung-Ju LeeAuthors Info & Claims

HotMobile '16: Proceedings of the 17th International Workshop on Mobile Computing Systems and Applications

Pages 51 - 56

https://doi.org/10.1145/2873587.2873602

Published: 23 February 2016 Publication History

Get Access

Abstract

Text messaging on smartphones has become one of the most popular communication methods. With many smartphone chat applications, text messaging no longer is only "text"; users send emoticons to express emotions or share pictures stored on their phones. We believe that providing more visuals in chat applications by autonomously suggesting proper images from the Internet (i.e., "auto complete" with images), based on the chat content, is the next evolution of mobile messaging. Realizing this simple vision however, is a difficult task due to the intrinsic nature of mobile chat and resource limitations of smartphones. We identify these challenges and to overcome them, we suggest integrating solutions from the field of mobile computing, natural language processing, sentiment analysis, machine learning, storage, human computer interaction, networking, and systems. We present MilliCat, a lightweight mobile messaging service that autonomously suggests images based on chat context to improve emotion expression, nuance delivery, and information delivery of a conversation. Experimental results from our preliminary prototype implementation show promises that real-time autonomous image suggestion can provide timely, proper images while only incurring manageable networking and energy overhead.

References

[1]

J. R. Bach, C. Fuller, A. Gupta, A. Hampapur, B. Horowitz, R. Humphrey, R. C. Jain, and C.-F. Shu. Virage image search engine: an open framework for image management. In Proc. SPIE 2670, Storage and Retrieval for Still Image and Video Databases IV, 76, 1996.

Google Scholar

[2]

B. D. Higgins, J. Flinn, T. J. Giuli, B. Noble, C. Peplin, and D. Watson. Informed mobile prefetching. In Proceedings of the 10th International ACM Conference on Mobile Systems, Applications, and Services (MobiSys '12).

Digital Library

Google Scholar

[3]

B. Liu. Sentiment analysis: A multifaceted problem. IEEE Intelligent Systems, (3):76--80, 2010.

Google Scholar

[4]

C. D. Manning, M. Surdeanu, J. Bauer, J. Finkel, S. J. Bethard, and D. McClosky. The Stanford CoreNLP natural language processing toolkit. In Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 55--60, 2014.

Crossref

Google Scholar

[5]

NLTK Project. NLTK 3.0 documentation. Available at: http://www.nltk.org/, Oct. 2015.

Google Scholar

[6]

O. Owoputi, B. O'Connor, C. Dyer, K. Gimpel, N. Schneider, and N. A. Smith. Improved part-of-speech tagging for online conversational text with word clusters. In Proc. of Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT 2013).

Google Scholar

[7]

A. Parate, M. Böhmer, D. Chu, D. Ganesan, and B. M. Marlin. Practical prediction and prefetch for faster access to applications on mobile phones. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13).

Digital Library

Google Scholar

[8]

A. Smith. U.S. Smartphone Use in 2015. PEW Research Center Report. Available at: http://www.pewinternet.org/2015/04/01/us-smartphone-use-in-2015/, Apr. 2015.

Google Scholar

[9]

R. Socher, A. Perelygin, J. Wu, J. Chuang, C. Manning, A. Ng, and C. Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of Conference on Empirical Methods in Natural Language Processing (EMNLP 2013).

Google Scholar

[10]

The Trustees of Princeton University. WordNet: A Lexical Database for English. . Available at: https://wordnet.princeton.edu/, Mar. 2015.

Google Scholar

Cited By

View all

Jahanshahi HKazmi SCevik M(2022)Auto Response Generation in Online Medical Chat ServicesJournal of Healthcare Informatics Research10.1007/s41666-022-00118-x6:3(344-374)Online publication date: 15-Jul-2022
https://doi.org/10.1007/s41666-022-00118-x
Kim JGong THan KKim JKo JLee S(2020)Messaging Beyond Texts with Real-time Image Suggestions22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403553(1-12)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403553
Chen FXia KDhabalia KHong JBrewster SFitzpatrick GCox AKostakos V(2019)MessageOnTapProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300805(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300805
Show More Cited By

Index Terms

A Picture is Worth a Thousand Words: Improving Mobile Messaging with Real-time Autonomous Image Suggestion
1. Human-centered computing
  1. Ubiquitous and mobile computing
    1. Ubiquitous and mobile computing theory, concepts and paradigms
      1. Mobile computing

Recommendations

Demo: MilliCat: Real-Time Autonomous Image Suggestion for Mobile Messaging
MobiSys '16 Companion: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services Companion

Text based communication has become the most popular form of communication using smartphones. We believe that suggesting images while typing is will innovate the mobile text communication. We suggest real-time autonomous image suggestion system based on ...
A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation
MM '21: Proceedings of the 29th ACM International Conference on Multimedia

A creative image-and-text generative AI system mimics humans' extraordinary abilities to provide users with diverse and comprehensive caption suggestions, as well as rich image creations. In this work, we demonstrate such an AI creation system to produce ...
Is a picture really worth a thousand words?: - on the role of images in e-commerce
WSDM '14: Proceedings of the 7th ACM international conference on Web search and data mining

In online peer-to-peer commerce places where physical examination of the goods is infeasible, textual descriptions, images of the products, reputation of the participants, play key roles. Visual image is a powerful channel to convey crucial information ...

Comments

Information & Contributors

Information

Published In

HotMobile '16: Proceedings of the 17th International Workshop on Mobile Computing Systems and Applications

February 2016

120 pages

ISBN:9781450341455

DOI:10.1145/2873587

General Chair:
David Chu
Microsoft Research - Redmond WA, USA
,
Program Chair:
Prabal Dutta
University of Michigan, USA

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 February 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HotMobile '16

Sponsor:

SIGMOBILE

HotMobile '16: The 17th International Workshop on Mobile Computing Systems and Applications

February 23 - 24, 2016

Florida, St. Augustine, USA

Acceptance Rates

HotMobile '16 Paper Acceptance Rate 18 of 55 submissions, 33%;

Overall Acceptance Rate 96 of 345 submissions, 28%

Upcoming Conference

HOTMOBILE '25

Sponsor:
sigmobile

The 26th International Workshop on Mobile Computing Systems and Applications

February 26 - 27, 2025

La Quinta , CA , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
453
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)2

Reflects downloads up to 22 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Jahanshahi HKazmi SCevik M(2022)Auto Response Generation in Online Medical Chat ServicesJournal of Healthcare Informatics Research10.1007/s41666-022-00118-x6:3(344-374)Online publication date: 15-Jul-2022
https://doi.org/10.1007/s41666-022-00118-x
Kim JGong THan KKim JKo JLee S(2020)Messaging Beyond Texts with Real-time Image Suggestions22nd International Conference on Human-Computer Interaction with Mobile Devices and Services10.1145/3379503.3403553(1-12)Online publication date: 5-Oct-2020
https://dl.acm.org/doi/10.1145/3379503.3403553
Chen FXia KDhabalia KHong JBrewster SFitzpatrick GCox AKostakos V(2019)MessageOnTapProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300805(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300805
Chen DFreeman DBalakrishnan RBrewster SFitzpatrick GCox AKostakos V(2019)Integrating Multimedia Tools to Enrich Interactions in Live Streaming for Language LearningProceedings of the 2019 CHI Conference on Human Factors in Computing Systems10.1145/3290605.3300668(1-14)Online publication date: 2-May-2019
https://dl.acm.org/doi/10.1145/3290605.3300668
Li YChen FLi TGuo YHuang GFredrikson MAgarwal YHong J(2017)PrivacyStreamsProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31309411:3(1-26)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3130941
Kim JKo JLee SBalan RMisra ACox LArakawa YZhou XLiKamWa R(2016)DemoProceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services Companion10.1145/2938559.2938585(108-108)Online publication date: 25-Jun-2016
https://dl.acm.org/doi/10.1145/2938559.2938585

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Demo: MilliCat: Real-Time Autonomous Image Suggestion for Mobile Messaging

A Picture is Worth a Thousand Words: A Unified System for Diverse Captions and Rich Images Generation

Is a picture really worth a thousand words?: - on the role of images in e-commerce