research-article

Leveraging high-level and low-level features for multimedia event detection

Authors:

Alexander G. Hauptmann,

Guang XiangAuthors Info & Claims

MM '12: Proceedings of the 20th ACM international conference on Multimedia

Pages 449 - 458

https://doi.org/10.1145/2393347.2393412

Published: 29 October 2012 Publication History

Abstract

This paper addresses the challenge of Multimedia Event Detection by proposing a novel method for high-level and low-level features fusion based on collective classification. Generally, the method consists of three steps: training a classifier from low-level features; encoding high-level features into graphs; and diffusing the scores on the established graph to obtain the final prediction. The final prediction is derived from multiple graphs each of which corresponds to a high-level feature. The paper investigates two graph construction methods using logarithmic and exponential loss functions, respectively and two collective classification algorithms, i.e. Gibbs sampling and Markov random walk. The theoretical analysis demonstrates that the proposed method converges and is computationally scalable and the empirical analysis on TRECVID 2011 Multimedia Event Detection dataset validates its outstanding performance compared to state-of-the-art methods, with an added benefit of interpretability.

References

[1]

Laptev, T. Lindeberg. Space-time interest points. In ICCV, pages 432--439, Nice, France, 2003.

Digital Library

[2]

Li-Jia Li, Hao Su, Eric Xing, Fei-Fei Li. Object bank: a high-level image representation for scene classification and semantic feature sparsification. In NIPS, pages 1378--1386, Vancouver, Canada, 2010.

[3]

C. Snoek, M. Worring, A. W. M. Smeulders. Early versus late fusion in semantic video analysis. In ACM Multimedia, pages 399--402, Singapore, 2005.

Digital Library

[4]

T. Pham, N. Maillot, J. Lim, J. Chevallet. Latent semantic fusion model for image retrieval and annotation. In CIKM, pages 439--444, Lisbon, Portugal, 2007.

Digital Library

[5]

H. Escalante, C. Hernández, L. Sucar, M. Montes. Late fusion of heterogeneous methods for multimedia image retrieval. In ACM MIR, pages 172--179, Vancouver, Canada, 2008.

Digital Library

[6]

J. Kludas, E. Bruno, S. Marchand-Maillet. Information fusion in multimedia information retrieval. In Adaptive Multimedia Retrieval, pages 147--159, Paris, France, 2007.

[7]

L. Bao et al. Informedia@TRECVID 2011. In Trecvid Video Retrieval Evaluation Workshop, NIST, Gaitherburg, USA, 2011.

[8]

H. Eldardiry, J. Neville. Across-Model collective ensemble classification. In AAAI, to appear, San Francisco, USA, 2011.

[9]

P. Sen, G. Namata, M. Bilgic, L. Getoor, B. Gallagher, and T. Eliassi-Rad. Collective classification in network data. AI Magazine, 29(3):93--106, 2008.

Digital Library

[10]

S. Macskassy, and F. Provost. Classification in networked data: A toolkit and a univariate case study. JMLR, 8:935--983, 2007.

Digital Library

[11]

S. Lazebnik, C. Schmid, and J. Ponce. Beyond bags of features: Spatial pyramid matching for recognizing natural scene categories. In CVPR, pages 2169--2178, New York, USA, 2006.

Digital Library

[12]

A. Bosch, A. Zisserman, and X. Munoz. Representing shape with a spatial pyramid kernel. In CIVR, pages 401--408, Amsterdam, Netherlands, 2007.

Digital Library

[13]

Y. Wu, E. Y. Chang, K. C. Chang, J. R. Smith. Optimal multimodal fusion for multimedia data analysis. In ACM Multimedia, pages 572--579, New York, USA, 2004.

Digital Library

[14]

N. Rasiwasia, JC. Pereira, E. Coviello, G. Doyle, G. Lanckriet, R. Levy, N. Vasconcelos. A new approach to cross-modal multimedia retrieval. In ACM Multimedia, pages 251--260, Firenze, Italy, 2010.

Digital Library

[15]

L. K. McDowell, K.M. Gupta, D.W. Aha. Cautious inference in collective classification. In AAAI, pages 596--601, Vancouver, Canada, 2007.

Digital Library

[16]

W. R. Gilks,S. Richardson and D. J. Spiegelhalter. Markov chain Monte Carlo in Practice. Chapman Hall/CRC Interdisciplinary Statistics, 1996.

[17]

J. Gemert, J. Geusebroek, C. Veenman, A. Smeulders. Kernel codebooks for scene categorization. In ECCV, pages 696--709, Marseille, France, 2008.

Digital Library

[18]

H. Hotelling. Relations between two sets of variates. Biometrika, 28:321--377, 1936.

[19]

P. Over, G. Awad, J. Fiscus, B. Antonishek, and M. Michel. Trecvid 2010 - an overview of the goals, tasks, data, evaluation mechanisms and metrics. In Trecvid Video Retrieval Evaluation Workshop, NIST, Gaitherburg, USA, 2010.

[20]

Doeblin, W. Exposé sur la théorie des chaînes simples constantes de Markoff à un nombre fini d'états. Rev. Math. Union Interbalkanique, 2:77--105, 1938.

Cited By

Zhou RZhang WYuan ZRong XLiu WFu KSun X(2022)Weakly Supervised Semantic Segmentation in Aerial Imagery via Explicit Pixel-Level ConstraintsIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2022.322447760(1-17)Online publication date: 2022
https://doi.org/10.1109/TGRS.2022.3224477
Karbalaie AAbtahi FSjöström M(2022)Event detection in surveillance videos: a reviewMultimedia Tools and Applications10.1007/s11042-021-11864-281:24(35463-35501)Online publication date: 25-Jun-2022
https://doi.org/10.1007/s11042-021-11864-2
Wang HZhou CYang CYang HHe J(2021)Controllable Gradient Item RetrievalProceedings of the Web Conference 202110.1145/3442381.3449963(768-777)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449963
Show More Cited By

Index Terms

Leveraging high-level and low-level features for multimedia event detection
1. Information systems
  1. Information systems applications

Recommendations

Layout-driven RTL binding techniques for high-level synthesis
ISSS '96: Proceedings of the 9th international symposium on System synthesis

The importance of effective and efficient accounting of layout effects is well-established in high-level synthesis (HLS), since it allows more realistic exploration of the design space and the generation of solutions with predictable metrics. This ...
High-Level Test Synthesis: A Survey from Synthesis Process Flow Perspective

High-level test synthesis is a special class of high-level synthesis having testability as one of the important components. This article presents a detailed survey on recent developments in high-level test synthesis from a synthesis process flow ...
High-Level Synthesis: Past, Present, and Future

Editor's note:This article presents the history and evolution of HLS from research to industry adoption. The authors offer insights on why earlier attempts to gain industry adoption were not successful, why current HLS tools are finally seeing adoption, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '12: Proceedings of the 20th ACM international conference on Multimedia

October 2012

1584 pages

ISBN:9781450310895

DOI:10.1145/2393347

General Chairs:
Noboru Babaguchi
Osaka University, Japan
,
Kiyoharu Aizawa
The University of Tokyo, Japan
,
John Smith
IBM, USA
,
Program Chairs:
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Thomas Plagemann
University of Oslo, Norway
,
Xian-Sheng Hua
Microsoft, USA
,
Rong Yan
Facebook, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '12

Sponsor:

SIGMM

MM '12: ACM Multimedia Conference

October 29 - November 2, 2012

Nara, Japan

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

56
Total Citations
View Citations
445
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhou RZhang WYuan ZRong XLiu WFu KSun X(2022)Weakly Supervised Semantic Segmentation in Aerial Imagery via Explicit Pixel-Level ConstraintsIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2022.322447760(1-17)Online publication date: 2022
https://doi.org/10.1109/TGRS.2022.3224477
Karbalaie AAbtahi FSjöström M(2022)Event detection in surveillance videos: a reviewMultimedia Tools and Applications10.1007/s11042-021-11864-281:24(35463-35501)Online publication date: 25-Jun-2022
https://doi.org/10.1007/s11042-021-11864-2
Wang HZhou CYang CYang HHe J(2021)Controllable Gradient Item RetrievalProceedings of the Web Conference 202110.1145/3442381.3449963(768-777)Online publication date: 19-Apr-2021
https://dl.acm.org/doi/10.1145/3442381.3449963
Duan BTang HWang WZong ZYang GYan Y(2021)Audio-Visual Event Localization via Recursive Fusion by Joint Co-Attention2021 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV48630.2021.00406(4012-4021)Online publication date: Jan-2021
https://doi.org/10.1109/WACV48630.2021.00406
Luo MChang XGong C(2021)Reliable shot identification for complex event detection via visual-semantic embeddingComputer Vision and Image Understanding10.1016/j.cviu.2021.103300213:COnline publication date: 1-Dec-2021
https://dl.acm.org/doi/10.1016/j.cviu.2021.103300
Li JFeng XFan H(2020)Saliency Consistency-Based Image Re-Colorization for Color BlindnessIEEE Access10.1109/ACCESS.2020.29933008(88558-88574)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2993300
Ahmad KConci N(2019)How Deep Features Have Improved Event Recognition in MultimediaACM Transactions on Multimedia Computing, Communications, and Applications10.1145/330624015:2(1-27)Online publication date: 5-Jun-2019
https://dl.acm.org/doi/10.1145/3306240
Gong MLi HMeng DMiao QLiu J(2019)Decomposition-Based Evolutionary Multiobjective Optimization to Self-Paced LearningIEEE Transactions on Evolutionary Computation10.1109/TEVC.2018.285076923:2(288-302)Online publication date: Apr-2019
https://doi.org/10.1109/TEVC.2018.2850769
Chen ZLu JFeng JZhou J(2018)Nonlinear Structural Hashing for Scalable Video SearchIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2017.266909528:6(1421-1433)Online publication date: 1-Jun-2018
https://dl.acm.org/doi/10.1109/TCSVT.2017.2669095
Liu HZheng QLi ZQin TZhu L(2018)An efficient multi-feature SVM solver for complex event detectionMultimedia Tools and Applications10.1007/s11042-017-5166-z77:3(3509-3532)Online publication date: 1-Feb-2018
https://dl.acm.org/doi/10.1007/s11042-017-5166-z
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents