skip to main content
survey

User Intent in Multimedia Search: A Survey of the State of the Art and Future Challenges

Published: 13 August 2016 Publication History

Abstract

Today's multimedia search engines are expected to respond to queries reflecting a wide variety of information needs from users with different goals. The topical dimension (“what” the user is searching for) of these information needs is well studied; however, the intent dimension (“why” the user is searching) has received relatively less attention. Specifically, intent is the “immediate reason, purpose, or goal” that motivates a user to query a search engine. We present a thorough survey of multimedia information retrieval research directed at the problem of enabling search engines to respond to user intent. The survey begins by defining intent, including a differentiation from related, often-confused concepts. It then presents the key conceptual models of search intent. The core is an overview of intent-aware approaches that operate at each stage of the multimedia search engine pipeline (i.e., indexing, query processing, ranking). We discuss intent in conventional text-based search wherever it provides insight into multimedia search intent or intent-aware approaches. Finally, we identify and discuss the most important future challenges for intent-aware multimedia search engines. Facing these challenges will allow multimedia information retrieval to recognize and respond to user intent and, as a result, fully satisfy the information needs of users.

References

[1]
Rakesh Agrawal, Sreenivas Gollapudi, Alan Halverson, and Samuel Ieong. 2009. Diversifying search results. In Proceedings of the 2nd ACM International Conference on Web Search and Data Mining (WSDM'09). ACM, 5--14.
[2]
Morgan Ames and Mor Naaman. 2007. Why we tag: Motivations for annotation in mobile and online media. In Proceedings of the SIGCHI Conference on Human Factors in Computer Systems (CHI'07). ACM, 971--980.
[3]
Paul André, Edward Cutrell, Desney S. Tan, and Greg Smith. 2009. Designing novel image search interfaces by understanding unique characteristics and usage. In Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part II (INTERACT'09). Springer-Verlag, 340--353.
[4]
Ricardo Baeza-Yates, Liliana Calderón-Benavides, and Cristina González-Caro. 2006. The intention behind web queries. In Proceedings of the 13th International Conference on String Processing and Information Retrieval (SPIRE'06). Springer-Verlag, 98--109.
[5]
Jana Besser, Martha Larson, and Katja Hofmann. 2010. Podcast search: User goals and retrieval technologies. Online Information Review 34, 3 (2010), 395--419.
[6]
Jingwen Bian, Zheng-Jun Zha, Hanwang Zhang, Qi Tian, and Tat-Seng Chua. 2012. Visual query attributes suggestion. In Proceedings of the 20th ACM International Conference on Multimedia (MM'12). ACM, 869--872.
[7]
Nis Bornoe and Louise Barkhuus. 2010. Video microblogging: Your 12 seconds of fame. In CHI'10 Extended Abstracts on Human Factors in Computing Systems (CHI EA'10). ACM, 3325--3330.
[8]
Andrei Broder. 2002. A taxonomy of web search. SIGIR Forum 36, 2 (Sept. 2002), 3--10.
[9]
Stefan Buettcher, Charles L. A. Clarke, and Gordon V. Cormack. 2010. Information Retrieval: Implementing and Evaluating Search Engines. MIT Press.
[10]
John Burgess and Joshua Green. 2009. YouTube: Online Video and Participatory Culture. Polity Press.
[11]
Marco Campanella and Jettie Hoonhout. 2008. Understanding behaviors and needs for home videos. In Proceedings of the 22nd British HCI Group Conference on People and Computers: Culture, Creativity, Interaction - Volume 2 (BCS-HCI'08). British Computer Society, 23--26.
[12]
Huanhuan Cao, Daxin Jiang, Jian Pei, Qi He, Zhen Liao, Enhong Chen, and Hang Li. 2008. Context-aware query suggestion by mining click-through and session data. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'08). ACM, 875--883.
[13]
Shih-Fu Chang. 2011. Content based multimedia retrieval: Lessons learned from two decades of research. In Proceedings of the 19th ACM International Conference on Multimedia (MM'11). ACM, 1--2.
[14]
Yao-Sheng Chang, Kuan-Yu He, Scott Yu, and Wen-Hsiang Lu. 2006. Identifying user goals from web search results. In Proceedings of the 2006 IEEE/WIC/ACM International Conference on Web Intelligence (WI'06). IEEE Computer Society, 1038--1041.
[15]
Tao Chen, Dongyuan Lu, Min-Yen Kan, and Peng Cui. 2013. Understanding and classifying image tweets. In Proceedings of the 21st ACM International Conference on Multimedia (MM'13). ACM, 781--784.
[16]
Xu Cheng, Cameron Dale, and Jiangchuan Liu. 2007. Understanding the characteristics of internet short video sharing: Youtube as a case study. CoRR abs/0707.3670 (2007).
[17]
Zhicong Cheng, Bin Gao, and Tie-Yan Liu. 2010. Actively predicting diverse search intent from user browsing behaviors. In Proceedings of the 19th International Conference on World Wide Web (WWW'10). ACM, 221--230.
[18]
Youngok Choi and Edie M. Rasmussen. 2003. Searching for images: The analysis of users' queries for image retrieval in American history. Journal of the American Society for Information Science and Technology 54, 6 (April 2003), 498--511.
[19]
EunKyung Chung and JungWon Yoon. 2012. Analysis of multimedia needs and searching features: An exploratory study. Proceedings of the American Society for Information Science and Technology 49, 1 (2012), 1--5.
[20]
Karen Church and Barry Smyth. 2009. Understanding the intent behind mobile information needs. In Proceedings of the 14th International Conference on Intelligent User Interfaces (IUI'09). ACM, 247--256.
[21]
Charles L. A. Clarke, Maheedhar Kolla, Gordon V. Cormack, Olga Vechtomova, Azin Ashkan, Stefan Büttcher, and Ian MacKinnon. 2008. Novelty and diversity in information retrieval evaluation. In Proceedings of the 31st International ACM Conference on Information Retrieval (SIGIR'08). ACM, 659--666.
[22]
Steve Cronen-Townsend, Yun Zhou, and W. Bruce Croft. 2002. Predicting query performance. In Proceedings of the 25th Internatioal ACM Conference on Information Retrieval (SIGIR'02). ACM, 299--306.
[23]
Jingyu Cui, Fang Wen, and Xiaoou Tang. 2008. IntentSearch: Interactive on-line image search re-ranking. In Proceedings of the 16th ACM International Conference on Multimedia (MM'08). ACM, 997--998.
[24]
Peng Cui, Shao-Wei Liu, Wen-Wu Zhu, Huan-Bo Luan, Tat-Seng Chua, and Shi-Qiang Yang. 2014. Social-sensed image search. ACM Transactions on Information Systems 32, 2, Article 8 (April 2014), 23 pages.
[25]
Sally Jo Cunningham and David M. Nichols. 2008. How people find videos. In Proceedings of the 8th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL'08). ACM, 201--210.
[26]
Na Dai, Xiaoguang Qi, and Brian D. Davison. 2011. Bridging link and query intent to enhance web search. In Proceedings of the 22nd ACM Conference on Hypertext and Hypermedia (HT'11). ACM, 17--26.
[27]
Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z. Wang. 2008. Image retrieval: Ideas, influences, and trends of the new age. ACM Computer Surveys 40, 2, Article 5 (May 2008), 60 pages.
[28]
Andrew Demetriou, Martha Larson, and Cynthia Liem. 2016. On cultural, textual and experiential aspects of music mood. In Proceedings of the 17th International Society for Music Information Retrieval Conference, ISMIR 2016. to appear.
[29]
Doug Downey, Susan Dumais, Dan Liebling, and Eric Horvitz. 2008. Understanding the relationship between searchers' queries and information goals. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM'08). ACM, 449--458.
[30]
Carsten Eickhoff, Wen Li, and Arjen P. de Vries. 2013. Exploiting user comments for audio-visual content indexing and retrieval. In Proceedings of the 35th European Conference on Advances in Informationa Retrieval (ECIR'13). Springer-Verlag, 38--49.
[31]
Joseph Ellis, Weisi Lin, Ching-Yung Lin, and Shih-Fu Chang. 2014. Predicting evoked emotions in video. In Proceedings of the 2014 IEEE International Symposium on Multimedia (ISM). 287--294.
[32]
Bailan Feng, Juan Cao, Zhineng Chen, Yongdong Zhang, and Shouxun Lin. 2010. Multi-modal query expansion for web video search. In Proceedings of the 33rd International ACM Conference on Information Retrieval (SIGIR'10). ACM, 721--722.
[33]
Raya Fidel. 1997. The image retrieval task: Implications for the design and evaluation of image databases. The New Review of Hypermedia and Multimedia 3 (1997).
[34]
Giovanni Gardelli and Ingmar Weber. 2012. Why do you ask this? In Proceedings of the 21st International Conference Companion on World Wide Web (WWW'12 Companion). ACM, 815--822.
[35]
Jian Guan and Guoping Qiu. 2007. Learning user intention in relevance feedback using optimization. In Proceedings of the International Workshop on Workshop on Multimedia Information Retrieval (MIR'07). ACM, 41--50.
[36]
Fan Guo, Lei Li, and Christos Faloutsos. 2009. Tailoring click models to user goals. In Proceedings of the 2009 Workshop on Web Search Click Data (WSCD'09). ACM, 88--92.
[37]
Jiafeng Guo, Xueqi Cheng, Gu Xu, and Xiaofei Zhu. 2011. Intent-aware query similarity. In Proceedings of the 20th ACM International Conference on Information and Knowledge Management (CIKM'11). ACM, 259--268.
[38]
Martin J. Halvey and Mark T. Keane. 2007. Analysis of online video search and sharing. In Proceedings of the 18th Conference on Hypertext and Hypermedia (HT'07). ACM, 217--226.
[39]
Alan Hanjalic. 2013. Multimedia retrieval that matters. ACM Transactions on Multimedia Computing, Communications, and Applications 9, 1s, Article 44 (Oct. 2013), 5 pages.
[40]
Alan Hanjalic, Christoph Kofler, and Martha Larson. 2012. Intent and its discontents: The user at the wheel of the online video search engine. In Proceedings of the 20th ACM International Conference on Multimedia (MM'12). ACM, 1239--1248.
[41]
Mauro Rojas Herrera, Edleno Silva de Moura, Marco Cristo, Thomaz Philippe Silva, and Altigran Soares da Silva. 2010. Exploring features for the automatic identification of user goals in web search. Information Processing & Management 46, 2 (2010), 131--142.
[42]
Enamul Hoque, Orland Hoeber, and Minglun Gong. 2013. CIDER: Concept-based image diversification, exploration, and retrieval. Information Processing & Management 49, 5 (Sept. 2013), 1122--1138.
[43]
Jian Hu, Gang Wang, Fred Lochovsky, Jian-tao Sun, and Zheng Chen. 2009. Understanding user's query intent with wikipedia. In Proceedings of the 18th International Conference on World Wide Web (WWW'09). ACM, 471--480.
[44]
Xian-Sheng Hua, Linjun Yang, Jingdong Wang, Jing Wang, Ming Ye, Kuansan Wang, Yong Rui, and Jin Li. 2013. Clickage: Towards bridging semantic and intent gaps via mining click logs of search engines. In Proceedings of the 21st ACM International Conference on Multimedia (MM'13). ACM, 243--252.
[45]
Ramesh Jain and Pinaki Sinha. 2010. Content without context is meaningless. In Proceedings of the International Conference on Multimedia (MM'10). ACM, 1259--1268.
[46]
Vidit Jain and Manik Varma. 2011. Learning to re-rank: Query-dependent image re-ranking using click data. In Proceedings of the 20th International Conference on World Wide Web (WWW'11). ACM, 277--286.
[47]
Bernard J. Jansen, Danielle L. Booth, and Amanda Spink. 2008. Determining the informational, navigational, and transactional intent of web queries. Information Processing & Management 44, 3 (May 2008), 1251--1266.
[48]
Akshay Java, Xiaodan Song, Tim Finin, and Belle Tseng. 2007. Why we twitter: Understanding microblogging usage and communities. In Proceedings of the 9th WebKDD and 1st SNA-KDD 2007 Workshop on Web Mining and Social Network Analysis (WebKDD/SNA-KDD'07). ACM, 56--65.
[49]
Beth St. Jean, Soo Young Rieh, Yong-Mi Kim, and Ji Yeon Yang. 2012. An analysis of the information behaviors, goals, and intentions of frequent internet users: Findings from online activity diaries. First Monday 17, 2 (2012).
[50]
Yu-Gang Jiang, Subhabrata Bhattacharya, Shih-Fu Chang, and Mubarak Shah. 2013. High-level event recognition in unconstrained videos. International Journal of Multimedia Information Retrieval 2, 2 (2013), 73--101.
[51]
Rosie Jones and Kristina Lisa Klinkner. 2008. Beyond the session timeout: Automatic hierarchical segmentation of search topics in query logs. In Proceedings of the 17th ACM Conference on Information and Knowledge Management (CIKM'08). ACM, 699--708.
[52]
In-Ho Kang and GilChang Kim. 2003. Query type classification for web document retrieval. In Proceedings of the 26th International ACM Conference on Research and Development in Information Retrieval (SIGIR'03). ACM, 64--71.
[53]
Lyndon S. Kennedy, Apostol (Paul) Natsev, and Shih-Fu Chang. 2005. Automatic discovery of query-class-dependent models for multimodal search. In Proceedings of the 13th ACM International Conference on Multimedia (MULTIMEDIA'05). ACM, 882--891.
[54]
Eugene Kharitonov, Craig Macdonald, Pavel Serdyukov, and Iadh Ounis. 2013. Intent models for contextualising and diversifying query suggestions. In Proceedings of the 22nd ACM International Conference on Information & Knowledge Management (CIKM'13). ACM, 2303--2308.
[55]
Tim Kindberg, Mirjana Spasojevic, Rowanne Fleck, and Abigail Sellen. 2005. The ubiquitous camera: An in-depth study of camera phone use. IEEE Pervasive Computing 4, 2 (April 2005), 42--50.
[56]
Christoph Kofler, Subhabrata Bhattacharya, Martha Larson, Tao Chen, Alan Hanjalic, and Shih-Fu Chang. 2015. Uploader intent for online video: Typology, inference and applications. IEEE Transactions on Multimedia (To Appear) (2015).
[57]
Christoph Kofler, Martha Larson, and Alan Hanjalic. 2014. Intent-aware video search result optimization. IEEE Transactions on Multimedia 16, 5 (Aug 2014), 1421--1433.
[58]
Christoph Kofler and Mathias Lux. 2009a. An exploratory study on the explicitness of user intentions in digital photo retrieval. In Proceedings of the International Conference on Knowledge Technologies and Data-driven Business (I-KNOW'09). 7.
[59]
Christoph Kofler and Mathias Lux. 2009b. Dynamic presentation adaptation based on user intent classification. In Proceedings of the 17th ACM International Conference on Multimedia (MM'09). ACM, 1117--1118.
[60]
Marian Kogler and Mathias Lux. 2011. Pursuing the holy grail by interrelating user intentions and bag of visual words to perform retrieval adaptation. In Proceedings of the 2011 ACM Workshop on Social and Behavioural Networked Media Access (SBNMA'11). ACM, 3--8.
[61]
Christoph Lagger, Mathias Lux, and Oge Marques. 2011. Which video do you want to watch now? Development of a prototypical intention-based interface for video retrieval. In Proceedings of the 2011 Workshop on Multimedia on the Web (MMWeb). 45--48.
[62]
Christoph Lagger, Mathias Lux, and Oge Marques. 2012. What makes people watch online videos: An exploratory study. Computer Entertainment (2012).
[63]
Jin Ha Lee. 2010. Analysis of user needs and information features in natural language queries seeking music information. Journal of the American Society for Information Science and Technology 61, 5 (2010), 1025--1045.
[64]
Uichin Lee, Zhenyu Liu, and Junghoo Cho. 2005. Automatic identification of user goals in web search. In Proceedings of the 14th International Conference on World Wide Web (WWW'05). ACM, 391--400.
[65]
Lusong Li and Jing Li. 2011. MQSS: Multimodal query suggestion and searching for video search. Multimedia Tools and Applications 54, 1 (2011), 55--68.
[66]
Xiao Li, Ye-Yi Wang, Dou Shen, and Alex Acero. 2010. Learning with click graph for query intent classification. ACM Transactions on Information Systems 28, 3, Article 12 (July 2010), 20 pages.
[67]
Zhiwei Li, Shuming Shi, and Lei Zhang. 2008. Improving relevance judgment of web search results with image excerpts. In Proceedings of the 17th International Conference on World Wide Web (WWW'08). ACM, 21--30.
[68]
Shaowei Liu, Peng Cui, Huanbo Luan, Wenwu Zhu, Shiqiang Yang, and Qi Tian. 2014. Social-oriented visual image search. Computer Vision and Image Understanding 118, 0 (2014), 30--39.
[69]
Yiqun Liu, Junwei Miao, Min Zhang, Shaoping Ma, and Liyun Ru. 2011. How do users describe their information need: Query recommendation based on snippet click model. Expert Systems with Applications 38, 11 (2011), 13847--13856.
[70]
David G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2 (Nov. 2004), 91--110.
[71]
Zheng Lu, Xiaokang Yang, Weiyao Lin, Hongyuan Zha, and Xiaolin Chen. 2014. Inferring user image-search goals under the implicit guidance of users. IEEE Transactions on Circuits and Systems for Video Technology 24, 3 (March 2014), 394--406.
[72]
Mathias Lux and Jochen Huber. 2012. Why did you record this video? An exploratory study on user intentions for video production. In Image Analysis for Multimedia Interactive Services (WIAMIS), 2012 13th Int'l Workshop on. 1--4.
[73]
Mathias Lux, Christoph Kofler, and Oge Marques. 2010a. A classification scheme for user intentions in image search. In CHI'10 Extended Abstracts on Human Factors in Computing Systems (CHI EA'10). ACM, 3913--3918.
[74]
Mathias Lux, Marian Kogler, and Manfred del Fabro. 2010b. Why did you take this photo: A study on user intentions in digital photo productions. In Proceedings of the 2010 ACM Workshop on Social, Adaptive and Personalized Multimedia Interaction and Access (SAPMIA'10). ACM, 41--44.
[75]
Mathias Lux, Mario Taschwer, and Oge Marques. 2012. A closer look at photographers' intentions: A test dataset. In Proceedings of the ACM Multimedia 2012 Workshop on Crowdsourcing for Multimedia (CrowdMM'12). ACM, 17--18.
[76]
Amy Madden, Ian Ruthven, and David McMenemy. 2013. A classification scheme for content analyses of YouTube video comments. Journal of Documentation 69, 5 (2013), 693--714.
[77]
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press.
[78]
Cameron Marlow, Mor Naaman, Danah Boyd, and Marc Davis. 2006. HT06, tagging paper, taxonomy, flickr, academic article, to read. In Proceedings of the 17th Conference on Hypertext and Hypermedia (HYPERTEXT'06). ACM, 31--40.
[79]
Tao Mei and Xian-Sheng Hua. 2005. Intention-based home video browsing. In Proceedings of the 13th Annual ACM International Conference on Multimedia (MULTIMEDIA'05). ACM, New York, 221--222.
[80]
Tao Mei, Yong Rui, Shipeng Li, and Qi Tian. 2014. Multimedia search reranking: A literature survey. ACM Computer Surveys 46, 3, Article 38 (Jan. 2014), 38 pages.
[81]
Julie B. Morrison, Peter Pirolli, and Stuart K. Card. 2001. A taxonomic analysis of what world wide web activities significantly impact people's decisions and actions. In CHI'01 Extended Abstracts on Human Factors in Computing Systems (CHI EA'01). ACM, 163--164.
[82]
Yashar Moshfeghi and Joemon M. Jose. 2013. On cognition, emotion, and interaction aspects of search tasks with different search intentions. In Proceedings of the 22nd International Conference on World Wide Web (WWW'13). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, Switzerland, 931--942. http://dl.acm.org/citation.cfm?id=2488388.2488469
[83]
Bonnie A. Nardi, Diane J. Schiano, Michelle Gumbrecht, and Luke Swartz. 2004. Why we blog. Communications of the ACM 47, 12 (Dec. 2004), 41--46.
[84]
Paul Over. 1996. TREC-5 interactive report. In Proceedings of the 5th Text Retrieval Conference (TREC-5). NIST Special Publication 500-238.
[85]
Yingwei Pan, Ting Yao, Tao Mei, Houqiang Li, Chong-Wah Ngo, and Yong Rui. 2014. Click-through-based cross-view learning for image search. In Proceedings of the 37th International ACM Conference on Information Retrieval (SIGIR'14). ACM, 717--726.
[86]
Jaimie Y. Park, Neil O'Hare, Rossano Schifanella, Alejandro Jaimes, and Chin-Wan Chung. 2015. A large-scale study of user image search behavior on the web. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI'15). ACM, New York, 985--994.
[87]
Namkee Park, Younbo Jung, and Kwan Min Lee. 2011. Intention to upload video content on the internet: The role of social norms and ego-involvement. Computers in Human Behavior 27, 5 (2011), 1996--2004.
[88]
Jerome Picault, Myriam Ribiere, and Yann Gaste. 2013. Indexing video segments using microblogs. In Proceedings of the 11th International Workshop on Content-Based Multimedia Indexing (CBMI 2013), 155--160.
[89]
Michael Riegler, Martha Larson, Mathias Lux, and Christoph Kofler. 2014. How ‘how' reflects what's what: Content-based exploitation of how users frame social images. In Proceedings of the ACM International Conference on Multimedia (MM'14). ACM, 397--406.
[90]
Daniel E. Rose and Danny Levinson. 2004. Understanding user goals in web search. In Proceedings of the 13th International Conference on World Wide Web (WWW'04). ACM, 13--19.
[91]
Stevan Rudinac, Martha Larson, and Alan Hanjalic. 2012. Leveraging visual concepts and query performance prediction for semantic-theme-based video retrieval. International Journal of Multimedia Information Retrieval 1, 4 (2012), 263--280.
[92]
Gordon Rugg and Peter McGeorge. 1997. The sorting techniques: A tutorial paper on card sorts, picture sorts and item sorts. Expert Systems 14, 2 (1997), 80--93.
[93]
Tetsuya Sakai. 2012. Evaluation with informational and navigational intents. In Proceedings of the 21st International Conference on World Wide Web (WWW'12). ACM, 499--508.
[94]
Rodrygo L. T. Santos, Craig Macdonald, and Iadh Ounis. 2011. Intent-aware search result diversification. In Proceedings of the 34th International ACM Conference on Information Retrieval (SIGIR'11). ACM, 595--604.
[95]
Dan Siroker and Pete Koomen. 2013. A/B Testing: The Most Powerful Way to Turn Clicks into Customers (1st ed.). Wiley Publishing.
[96]
Mette Skov and Marianne Lykke. 2012. Unlocking radio broadcasts: User needs in sound retrieval. In Proceedings of the 4th Information Interaction in Context Symposium (IIIX'12). ACM, 298--301.
[97]
Cees G. M. Snoek and Marcel Worring. 2009. Concept-based video retrieval. Foundations and Trends in Information Retrieval 2, 4 (April 2009), 215--322.
[98]
Amanda Spink and Bernhard J. Jansen. 2005. Web Search: Public Searching of the Web -- Multimedia Searching. Information Science and Knowledge Management, Vol. 6. Springer Netherlands. 161--177 pages.
[99]
Markus Strohmaier. 2008. Purpose tagging: Capturing user intent to assist goal-oriented social search. In Proceedings of the 2008 ACM Workshop on Search in Social Media (SSM'08). ACM, 35--42.
[100]
Markus Strohmaier and Mark Kröll. 2012. Acquiring knowledge about human goals from search query logs. Information Processing & Management 48, 1 (2012), 63--82.
[101]
Markus Strohmaier, Mark Kröll, and Christian Körner. 2009. Intentional query suggestion: Making user goals more explicit during search. In Proceedings of the 2009 Workshop on Web Search Click Data (WSCD'09). ACM, 68--74.
[102]
Markus Strohmaier, Mathias Lux, Michael Granitzer, Peter Scheir, Sotirios Liaskos, and Eric Yu. 2007. How do users express goals on the web? In Web Information Systems Engineering WISE 2007 Workshops. Lecture Notes in Computer Science, Vol. 4832. Springer Berlin, 67--78.
[103]
S. Taheri-Panah and A. MacFarlane. 2004. Music information retrieval systems: Why do individuals use them and what are their needs. In Proceedings of the International Symposium on Music Information Retrieval. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.115.7318&rank=2.
[104]
Jaime Teevan, Susan T. Dumais, and Daniel J. Liebling. 2008. To personalize or not to personalize: Modeling queries with variation in user intent. In Proceedings of the 31st International ACM Conference on Information Retrieval (SIGIR'08). ACM, 163--170.
[105]
Bart Thomee and Michael S. Lew. 2012. Interactive search in image retrieval: A survey. International Jouranl of Multimedia Information Retrieval 1, 2 (2012), 71--86.
[106]
Wang Ting-Xuan and Lu Wen-Hsiang. 2011. Identifying popular search goals behind search queries to improve web search ranking. In Information Retrieval Technology. Lecture Notes in Computer Science, Vol. 7097. Springer Berlin Heidelberg, 250--262.
[107]
Dian Tjondronegoro, Amanda Spink, and Bernard J. Jansen. 2009. A study and comparison of multimedia web searching: 1997-2006. Journal of the American Society for Information and Science Technology 60, 9 (Sept. 2009), 1756--1768.
[108]
Michele Trevisiol, Luca Chiarandini, Luca Maria Aiello, and Alejandro Jaimes. 2012. Image ranking based on user browsing behavior. In Proceedings of the 35th International ACM Conference on Informationl Retriveal (SIGIR'12). ACM, 445--454.
[109]
Kosetsu Tsukuda, Tetsuya Sakai, Zhicheng Dou, and Katsumi Tanaka. 2013. Estimating intent types for search result diversification. In Informational Retrieval Technology. Lecture Notes in Computer Science, Vol. 8281. Springer Berlin Heidelberg, 25--37.
[110]
Kazutoshi Umemoto, Takehiro Yamamoto, Satoshi Nakamura, and Katsumi Tanaka. 2012. Search intent estimation from user's eye movements for supporting information seeking. In Proceedings of the International Working Conference on Advanced Visual Interfaces (AVI'12). ACM, 349--356.
[111]
Nancy A. Van House. 2007. Flickr and public image-sharing: Distant closeness and photo exhibition. In CHI'07 Extended Abstracts on Human Factors in Computing Systems (CHI EA'07). ACM, 2717--2722.
[112]
Reinier H. van Leuken, Lluis Garcia, Ximena Olivares, and Roelof van Zwol. 2009. Visual diversification of image search results. In Proceedings of the 18th International Conference on World Wide Web (WWW'09). ACM, 341--350.
[113]
Roelof van Zwol, Vanessa Murdock, Lluis Garcia Pueyo, and Georgina Ramirez. 2008. Diversifying image search with user generated content. In Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval (MIR'08). ACM, 67--74.
[114]
Raynor Vliegendhart, Babak Loni, Martha Larson, and Alan Hanjalic. 2013. How do we deep-link? Leveraging user-contributed time-links for non-linear video access. In Proceedings of the 21st ACM International Conference on Multimedia (MM'13). ACM, 517--520.
[115]
Jingdong Wang and Xian-Sheng Hua. 2011. Interactive image search by color map. ACM Transactions on Intelligent Systems Technology 3, 1, Article 12 (Oct. 2011), 23 pages.
[116]
Dayong Wu, Yu Zhang, Shiqi Zhao, and Ting Liu. 2010. Identification of web query intent based on query text and web knowledge. In Proceedings of the 1st International Conference on Pervasive Computing Signal Processing and Applications (PCSPA'10), 128--131.
[117]
Peng Xu and Martha Larson. 2014. Users tagging visual moments: Timed tags in social video. In Proceedings of the 2014 International ACM Workshop on Crowdsourcing for Multimedia (CrowdMM'14). ACM, 57--62.
[118]
Linjun Yang, Bo Geng, Alan Hanjalic, and Xian-Sheng Hua. 2012. A unified context model for web image retrieval. ACM Transactions on Multimedia Computing, Communications, and Applications 8, 3, Article 28 (Aug. 2012), 19 pages.
[119]
Linjun Yang and Alan Hanjalic. 2010. Supervised reranking for web image search. In Proceedings of the International Conference on Multimedia (MM'10). ACM, 183--192.
[120]
Xiaopeng Yang, Yongdong Zhang, Ting Yao, Chong-Wah Ngo, and Tao Mei. 2014. Click-boosting multi-modality graph-based reranking for image search. Multimedia Systems (2014), 1--11.
[121]
Yi-Hsuan Yang and W. H. Hsu. 2008. Video search reranking via online ordinal reranking. In Proceedings of the 2008 IEEE International Conference and Expo on Multimedia. 285--288.
[122]
Soungwoong Yoon, Adam Jatowt, and Katsumi Tanaka. 2009. Intent-based categorization of search results using questions from web q&a corpus. In Web Information Systems Engineering - WISE 2009. Lecture Notes in Computer Science, Vol. 5802. Springer Berlin, 145--158.
[123]
Xiaojie Yuan, Zhicheng Dou, Lu Zhang, and Fang Liu. 2008. Automatic user goals identification based on anchor text and click-through data. Wuhan University Journal of Natural Sciences 13, 4 (2008), 495--500.
[124]
Zheng-Jun Zha, Linjun Yang, Tao Mei, Meng Wang, Zengfu Wang, Tat-Seng Chua, and Xian-Sheng Hua. 2010. Visual query suggestion: Towards capturing user intent in internet image search. ACM Transactions on Multimedia Computing, Communications, and Applications 6, 3, Article 13 (Aug. 2010), 19 pages.
[125]
Hanwang Zhang, Zheng-Jun Zha, Yang Yang, Shuicheng Yan, Yue Gao, and Tat-Seng Chua. 2013. Attribute-augmented semantic hierarchy: Towards bridging semantic gap and intention gap in image retrieval. In Proceedings of the 21st ACM International Conference on Multimedia (MM'13). ACM, 33--42.

Cited By

View all
  1. User Intent in Multimedia Search: A Survey of the State of the Art and Future Challenges

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Computing Surveys
    ACM Computing Surveys  Volume 49, Issue 2
    June 2017
    747 pages
    ISSN:0360-0300
    EISSN:1557-7341
    DOI:10.1145/2966278
    • Editor:
    • Sartaj Sahni
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 August 2016
    Accepted: 01 May 2016
    Revised: 01 January 2016
    Received: 01 July 2015
    Published in CSUR Volume 49, Issue 2

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. User intent
    2. multimedia indexing
    3. multimedia information retrieval
    4. multimedia search
    5. retrieval algorithms

    Qualifiers

    • Survey
    • Research
    • Refereed

    Funding Sources

    • Dutch national program COMMIT/
    • Google Europe Doctoral Fellowship in Video Search
    • Google Fellowship

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)96
    • Downloads (Last 6 weeks)7
    Reflects downloads up to 18 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media