article

Social-oriented visual image search

Authors:

Qi TianAuthors Info & Claims

Computer Vision and Image Understanding, Volume 118

Pages 30 - 39

https://doi.org/10.1016/j.cviu.2013.06.011

Published: 01 January 2014 Publication History

Abstract

Many research have been focusing on how to match the textual query with visual images and their surrounding texts or tags for Web image search. The returned results are often unsatisfactory due to their deviation from user intentions, particularly for queries with heterogeneous concepts (such as ''apple'', ''jaguar'') or general (non-specific) concepts (such as ''landscape'', ''hotel''). In this paper, we exploit social data from social media platforms to assist image search engines, aiming to improve the relevance between returned images and user intentions (i.e., social relevance). Facing the challenges of social data sparseness, the tradeoff between social relevance and visual relevance, and the complex social and visual factors, we propose a community-specific Social-Visual Ranking (SVR) algorithm to rerank the Web images returned by current image search engines. The SVR algorithm is implemented by PageRank over a hybrid image link graph, which is the combination of an image social-link graph and an image visual-link graph. By conducting extensive experiments, we demonstrated the importance of both visual factors and social factors, and the advantages of social-visual ranking algorithm for Web image search.

References

[1]

Yan, R., Hauptmann, A. and Jin, R., Multimedia search with pseudo-relevance feedback. In: Bakker, E., Lew, M., Huang, T., Sebe, N., Zhou, X. (Eds.), Lecture Notes in Computer Science, vol. 2728. Springer, Berlin/Heidelberg. pp. 649-654.

[2]

Yang, Y., Nie, F., Xu, D., Luo, J., Zhuang, Y. and Pan, Y., A multimedia retrieval framework based on semi-supervised ranking and relevance feedback. IEEE Trans. Pattern Anal. Mach. Intell. v34 i4. 723-742.

[3]

Yang, Y., Zhuang, Y.-T., Wu, F. and Pan, Y.-H., Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans. Multimedia. v10 i3. 437-446.

[4]

Hsu, W.H., Kennedy, L.S. and Chang, S.-F., Video search reranking via information bottleneck principle. In: Proceedings of the 14th Annual ACM International Conference on Multimedia, MULTIMEDIA '06, ACM, New York, NY, USA. pp. 35-44.

[5]

Jing, Y. and Baluja, S., VisualRank: applying pagerank to large-scale image search. IEEE Trans. Pattern Anal. Mach. Intell. v30 i11. 1877-1890.

[6]

Liu, J., Lai, W., Hua, X.-S., Huang, Y. and Li, S., Video search re-ranking via multi-graph propagation. In: Proceedings of the 15th International Conference on Multimedia, MULTIMEDIA '07, ACM, New York, NY, USA. pp. 208-217.

[7]

Hsu, W.H., Kennedy, L.S. and Chang, S.-F., Video search reranking through random walk over document-level context graph. In: Proceedings of the 15th international conference on Multimedia, MULTIMEDIA '07, ACM, New York, NY, USA. pp. 971-980.

[8]

R.I. Kondor, J. Lafferty, Diffusion kernels on graphs and other discrete structures, in: Proceedings of the ICML, 2002, pp. 315-322.

[9]

Tian, X., Yang, L., Wang, J., Yang, Y., Wu, X. and Hua, X.-S., Bayesian video search reranking. In: Proceedings of the 16th ACM International Conference on Multimedia, MM '08, ACM, New York, NY, USA. pp. 131-140.

[10]

H. Zitouni, S. Sevil, D. Ozkan, P. Duygulu, Re-ranking of web image search results using a graph algorithm, in: ICPR 2008. 19th International Conference on Pattern Recognition, 2008, pp. 1-4. (ISSN 1051-4651).

[11]

Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., Tomkins, A. and Wiener, J., Graph structure in the Web. Comput. Netw. v33 i1C6. 309-320.

[12]

Geng, B., Yang, L., Xu, C., Hua, X.-S. and Li, S., The role of attractiveness in web image search. In: Proceedings of the 19th ACM International Conference on Multimedia, MM '11, ACM, New York, NY, USA. pp. 63-72.

[13]

Järvelin, K. and Kekäläinen, J., IR evaluation methods for retrieving highly relevant documents. In: Proceedings of the 23rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '00, ACM, New York, NY, USA. pp. 41-48.

[14]

Shiliang, Zhang, Q., Tian, G., Hua, Q., Huang, S. and Li, Descriptive visual words and visual phrases for image applications. In: Proceedings of the 17th ACM International Conference on Multimedia, MM '09, ACM, New York, NY, USA. pp. 75-84.

[15]

Zhou, W., Tian, Q., Lu, Y., Yang, L. and Li, H., Latent visual context learning for web image applications. Pattern Recognit. v44 i10C11. 2263-2273.

[16]

Zhou, X., Zhuang, X., Yan, S., Chang, S.-F., Hasegawa-Johnson, M. and Huang, T.S., SIFT-Bag kernel for video event analysis. In: Proceedings of the 16th ACM International Conference on Multimedia, MM '08, ACM, New York, NY, USA. pp. 229-238.

[17]

Yang, Y.H., Wu, P.T., Lee, C.W., Lin, K.H., Hsu, W.H. and Chen, H.H., ContextSeer: context search and recommendation at query time for shared consumer photos. In: Proceedings of the 16th ACM International Conference on Multimedia, MM '08, ACM, New York, NY, USA. pp. 199-208.

[18]

L. Page, S. Brin, R. Motwani, T. Winograd, The PageRank Citation Ranking: Bringing Order to the Web, Technical Report 1999-66, Stanford InfoLab, Previous, 1999.

[19]

Li, X., Snoek, C.G. and Worring, M., Learning tag relevance by neighbor voting for social image retrieval. In: Proceedings of the 1st ACM International Conference on Multimedia Information Retrieval, MIR '08, ACM, New York, NY, USA. pp. 180-187.

[20]

Larson, M., Kofler, C. and Hanjalic, A., Reading between the tags to predict real-world size-class for visually depicted objects in images. In: Proceedings of the 19th ACM International Conference on Multimedia, MM '11, ACM, New York, NY, USA. pp. 273-282.

[21]

Sun, A. and Bhowmick, S.S., Quantifying tag representativeness of visual content of social images. In: Proceedings of the International Conference on Multimedia, MM '10, ACM, New York, NY, USA. pp. 471-480.

[22]

Ames, M. and Naaman, M., Why we tag: motivations for annotation in mobile and online media. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI '07, ACM, New York, NY, USA. pp. 971-980.

[23]

Sang, J., Liu, J. and Xu, C., Exploiting user information for image tag refinement. In: Proceedings of the 19th ACM International Conference on Multimedia, MM '11, ACM, New York, NY, USA. pp. 1129-1132.

[24]

Park, L.A.F. and Ramamohanarao, K., Mining web multi-resolution community-based popularity for information retrieval. In: Proceedings of the Sixteenth ACM Conference on Information and Knowledge Management, CIKM '07, ACM, New York, NY, USA. pp. 545-554.

[25]

Zhuang, J., Mei, T., Hoi, S.C., Hua, X.-S. and Li, S., Modeling social strength in social media community via kernel-based learning. In: Proceedings of the 19th ACM International Conference on Multimedia, MM '11, ACM, New York, NY, USA. pp. 113-122.

[26]

Bu, J., Tan, S., Chen, C., Wang, C., Wu, H., Zhang, L. and He, X., Music recommendation by unified hypergraph: combining social media information and music content. In: Proceedings of the International Conference on Multimedia, ACM. pp. 391-400.

[27]

D. Liu, G. Ye, C.-T. Chen, S. Yan, S.-F. Chang, Hybrid Social Media Network.

[28]

Negoescu, R.A. and Gatica-Perez, D., Analyzing flickr groups. In: Proceedings of the 2008 International Conference on Content-based Image and Video Retrieval, ACM. pp. 417-426.

[29]

Topic-sensitive pagerank: a context-sensitive ranking algorithm for web search. IEEE Trans. Knowl. Data Eng. v15 i4. 784-796.

[30]

D. Nister, H. Stewenius, Scalable recognition with a vocabulary tree, in: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, 2006, pp. 2161-2168. (ISSN 1063-6919).

[31]

Chakrabarti, S., Dynamic personalized pagerank in entity-relation graphs. In: Proceedings of the 16th International Conference on World Wide Web, ACM. pp. 571-580.

[32]

Wang, L., Yang, L. and Tian, X., Query aware visual similarity propagation for image search reranking. In: Proceedings of the 17th ACM International Conference on Multimedia, MM '09, ACM, New York, NY, USA. pp. 725-728.

Cited By

Zhang DWu XYu J(2021)Label Consistent Flexible Matrix Factorization Hashing for Efficient Cross-modal RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/344677417:3(1-18)Online publication date: 22-Jul-2021
https://dl.acm.org/doi/10.1145/3446774
Kofler CLarson MHanjalic A(2016)User Intent in Multimedia SearchACM Computing Surveys10.1145/295493049:2(1-37)Online publication date: 13-Aug-2016
https://dl.acm.org/doi/10.1145/2954930
Wang JXu ZOssowski S(2016)A graph theory-based online keywords model for image semantic extractionProceedings of the 31st Annual ACM Symposium on Applied Computing10.1145/2851613.2851633(67-72)Online publication date: 4-Apr-2016
https://dl.acm.org/doi/10.1145/2851613.2851633
Show More Cited By

Recommendations

Optimizing social image search with multiple criteria: Relevance, diversity, and typicality

The explosive growth and wide-spread accessibility of community-contributed multimedia contents on the Internet have led to a surging research activity in social image search. However, the existing tag-based search methods frequently return irrelevant ...
Towards a Relevant and Diverse Search of Social Images

Recent years have witnessed the great success of social media websites. Tag-based image search is an important approach to accessing the image content on these websites. However, the existing ranking methods for tag-based image search frequently return ...
Image Search Reranking with Relevance, Diversity and Topic Coverage
ICIMCS'16: Proceedings of the International Conference on Internet Multimedia Computing and Service

Image search reranking has recently been proposed to improve image search results. Most of the conventional reranking methods cannot leverage both relevance and diversity of the search results simultaneously. In addition, they usually ignore the latent ...

Comments

Information & Contributors

Information

Published In

cover image Computer Vision and Image Understanding

Computer Vision and Image Understanding Volume 118, Issue

January, 2014

229 pages

ISSN:1077-3142

Issue’s Table of Contents

Copyright © Elsevier Inc. © 2013.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 January 2014

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 12 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang DWu XYu J(2021)Label Consistent Flexible Matrix Factorization Hashing for Efficient Cross-modal RetrievalACM Transactions on Multimedia Computing, Communications, and Applications10.1145/344677417:3(1-18)Online publication date: 22-Jul-2021
https://dl.acm.org/doi/10.1145/3446774
Kofler CLarson MHanjalic A(2016)User Intent in Multimedia SearchACM Computing Surveys10.1145/295493049:2(1-37)Online publication date: 13-Aug-2016
https://dl.acm.org/doi/10.1145/2954930
Wang JXu ZOssowski S(2016)A graph theory-based online keywords model for image semantic extractionProceedings of the 31st Annual ACM Symposium on Applied Computing10.1145/2851613.2851633(67-72)Online publication date: 4-Apr-2016
https://dl.acm.org/doi/10.1145/2851613.2851633
Zhu SAloufi SYang JEl Saddik A(2016)On the learning of image social relevance from heterogeneous social networkNeurocomputing10.1016/j.neucom.2015.08.133210:C(269-282)Online publication date: 19-Oct-2016
https://dl.acm.org/doi/10.1016/j.neucom.2015.08.133
Ou MCui PWang FWang JZhu WCao LZhang CJoachims TWebb GMargineantu DWilliams G(2015)Non-transitive Hashing with Latent Similarity ComponentsProceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining10.1145/2783258.2783283(895-904)Online publication date: 10-Aug-2015
https://dl.acm.org/doi/10.1145/2783258.2783283
Wang ZCui PXie LZhu WRui YYang S(2014)Bilateral Correspondence Model for Words-and-Pictures Association in Multimedia-Rich MicroblogsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/261138810:4(1-21)Online publication date: 4-Jul-2014
https://dl.acm.org/doi/10.1145/2611388
Cui PLiu SZhu WLuan HChua TYang S(2014)Social-Sensed Image SearchACM Transactions on Information Systems10.1145/259097432:2(1-23)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.1145/2590974
Yang YCui PZhu WZhao HShi YYang SKankanhalli MRueger SManmatha RJose Jvan Rijsbergen K(2014)Emotionally Representative Image Discovery for Social EventsProceedings of International Conference on Multimedia Retrieval10.1145/2578726.2578749(177-184)Online publication date: 1-Apr-2014
https://dl.acm.org/doi/10.1145/2578726.2578749
Wang YTang SZhang YLi JWang D(2014)Representative selection based on sparse modelingNeurocomputing10.1016/j.neucom.2014.02.013139(423-431)Online publication date: 1-Sep-2014
https://dl.acm.org/doi/10.1016/j.neucom.2014.02.013

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents