Article

On the detection of semantic concepts at TRECVID

Authors:

Milind R. Naphade,

John R. SmithAuthors Info & Claims

MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia

Pages 660 - 667

https://doi.org/10.1145/1027527.1027680

Published: 10 October 2004 Publication History

Abstract

Semantic multimedia management is necessary for the effective and widespread utilization of multimedia repositories and realizing the potential that lies untapped in the rich multimodal information content. This challenge has driven researchers to devise new algorithms and systems that enable automatic or semi-automatic tagging of large scale multimedia content with rich semantics. An emerging research area is the detection of a predetermined set of semantic concepts that can act as semantic filters and aid in search, and manipulation. The NIST TRECVID benchmark has responded by creating a task that has evaluated the performance of concept detection. Within the scope of this benchmark task, this paper studies trends in the emerging concept detection systems, architectures and algorithms. It also analyzes strategies that have yielded reasonable success, and challenges and gaps that lie ahead.

References

[1]

A. Amir, M. Berg, S. F. Chang, G. Iyengar, C. Lin, M. Naade, A. Natsev, C. Neti, H. Nock, W. Hsu, I. Sachdev, J. Smith, B. Tseng, Y. Wu, and D. Zhang, "IBM research trecvid-2003 video retrieval system," Nov 2003, NIST TRECVID 2003.

[2]

A. Hauptmann, R. Baron, M. Chen, M. Christel, P. Duygulu, C. Huang, R. jin, W. Lin, T. Ng, N. Moraveji, N. Papernick, C. Snoek, G~Tzanetakis, J. Yang, R. Yan, and H. Wactlar, "Informedia at TRECVID 2003: Analyzing and searching broadcast news video," Nov 2003, NIST TRECVID 2003.

[3]

L. Wu, Y. Guo, X. Qiu, Z. Feng, J. Rong, W. Jin, D. Zhou, R. Wang, and M. Jin, "Fudan university at TRECVID 2003," Nov 2003, NIST TRECVID 2003.

[4]

M. Rautiainen, J. Pebttila, P. Peterila, K. Noponen, M. Hosio, T. Koskela, S. Makela, J. Peltola, J. Liu, T. Ojala, and T. Seppanen, "TRECVID 2003 experiments at mediaTeam Oulu and VTT," Nov 2003, NIST TRECVID 2003.

[5]

Y. Zhai, Z. Rasheed, and M. Shah, "University of central florida at TRECVID 2003," Nov 2003, NIST TRECVID 2003.

[6]

X. Huang, G. Wei, and V. Petrushin, "Shot boundary detection and high-level features extraction for the TREC video evaluation 2003," Nov 2003, NIST TRECVID 2003.

[7]

F. Souvannavong, B. Merialdo, and B. Huet, "Latent semantic indexing for video content modeling and analysis," Nov 2003, NIST TRECVID 2003.

[8]

W. H. Adams, A. Amir, C. Dorai, S. Ghoshal, G. Iyengar, A. Jaimes, C. Lang, C. Y. Lin, M. R. Naade, A. Natsev, C. Neti, H. J. Nock, H. Permutter, R. Singh, S. Srinivasan, J. R. Smith, B. L. Tseng, A. T. Varadaraju, and D. Zhang, "IBM research TREC-2002 video retrieval system," in Proc. Text Retrieval Conference (TREC), Gaithersburg, MD, Nov 2002, pp. 289--298.

[9]

J. Smith, S. Srinivasan, A. Amir, S. Basu, G. Iyengar, C. Lin, M. Naade, D. Ponceleon, and B. Tseng, "Integrating features, models, and semantics for content-based retrieval," NIST video-TEC notebook, 2001.

[10]

A. Hauptmann, R. Yan, Y. Qi, R. Jin, M. Christel, M. Derthick, M. Chen, R. Baron, W. Lin, and T. Ng, "Video classification and retrieval with the informedia digital video library system," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 119--127.

[11]

L. Wu, X. Huang, J. Niu, Y. Xia, Z. Feng, and Y. Zhou, "FDU at TREC 2002: Filtering, q&a and video tasks," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 232--247.

[12]

M. Rautiainen, J. Pebttila, P. Peterila, D. Vorobiev, K. Noponen, M. Hosio, E. Matinmikko, S. Makela, J. Peltola, T. Ojala, and T. Seppanen, "TRECVID 2002 experiments at MediaTeam Oulu and VTT," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 417--428.

[13]

G. Quenot, D. Moraru, L. Besacier, and P. Muthem, "Clips at trec 11: Experiments in video retrieval," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 181--187.

[14]

F. Souvannavong, B. Merialdo, and B. Huet, "Semantic feature extraction using mpeg macro-block classification," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 227--231.

[15]

A. Smeaton, "TRECVID 2003- an introduction," Nov 2003, NIST TRECVID 2003.

[16]

A. Smeaton and P. Over, "The TREC-2002 video track report," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 69--85.

[17]

M. Naade, S. Basu, J. Smith, C. Lin, and B. Tseng, "Modeling semnatic concepts to support query by keywords in video," in IEEE International Confernce on Image Processing, Rochester, NY, Sep 2002.

[18]

C. Lin, B. Tseng, and J. Smith, "Video collaborative annotation forum: Establishing ground-truth labels on large multimedia datasets," in Proc. Text Retrieval Conference (TREC), Gaithersburg, MD, Nov 2003.

[19]

M. Naade, T. Kristjansson, B. Frey, and T. S. Huang, "Probabilistic multimedia objects (multijects): A novel approach to indexing and retrieval in multimedia systems," in Proceedings of IEEE International Conference on Image Processing, Chicago, IL, Oct. 1998, vol. 3, pp. 536--540.

[20]

A. Vailaya, A. Jain, and H. Zhang, "On image classification: City images vs. landscapes," Pattern Recognition, vol. 31, pp. 1921--1936, Dec. 1998.

[21]

NIST TREC10, "Common evaluation measures," http://trec.nist.gov/pubs/trec10/appendices/measures.pdf.

[22]

E. Voorhees, "The ilosoy of information retrieval evaluation," http://www.itl.nist.gov/iaui/894.02/~works/papers/eval_ilosoy.ps.

[23]

J. Smith, S. Srinivasan, A. Amir, S. Basu, G. Iyengar, C. Lin, M. Naade, D. Ponceleon, and B. Tseng, "Integrating features, models, and semantics for content-based retrieval," NIST video-TEC notebook, 2001.

[24]

Milind R. Naade and John R. Smith, "A hybrid framework for detecting the semantics of concepts and context," in Lecture Notes in Computer Science: Image and Video Retrieval, M. Lew, N. Sebe, and J. Eakins, Eds. Springer, 2003.

Digital Library

[25]

J. Vendrig, J. Hartog, D. Leeuwen, I. Patras, S. Raaijmakers, J. Best, C. Snoek, and M. Worring, "TREC feature extraction by active learning," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 429--438.

[26]

P. Browne, C. Czirjek, C. Gurrin, R. Jarina, H. Lee, S. Markow, K. McDonald, N. Mury, N. O'Connor, A. Smeaton, and J. Ye, "Dublin city university video track experiments for TREC 2002," in The Eleventh Text Retrieval Conference, TREC 2002, Gaithersburg, MD, Nov 2002, pp. 217--226.

[27]

A. Hauptmann and H. Wactlar, "Indexing and search of multimodal information," in IEEE International Conference on Acoustics, Speech and Signal Processing, Munich, Germany, 1997.

Digital Library

Cited By

Saravanan DUrkude S(2023)Confiscate Boisterous from Color-Based Images Using Rule-Based TechniqueProceedings of the International Conference on Cognitive and Intelligent Computing10.1007/978-981-19-2358-6_47(503-514)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-981-19-2358-6_47
Ravi AVenugopal HPaul STizhoosh H(2018)A Dataset and Preliminary Results for Umpire Pose Detection Using SVM Classification of Deep Features2018 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2018.8628877(1396-1402)Online publication date: Nov-2018
https://doi.org/10.1109/SSCI.2018.8628877
Hauptmann A(2018)Video Content AnalysisEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_1018(4381-4388)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-1-4614-8265-9_1018
Show More Cited By

Index Terms

On the detection of semantic concepts at TRECVID
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision

Recommendations

Semantic concept detection for user-generated video content using a refined image folksonomy
MMM'10: Proceedings of the 16th international conference on Advances in Multimedia Modeling

The automatic detection of semantic concepts is a key technology for enabling efficient and effective video content management. Conventional techniques for semantic concept detection in video content still suffer from several interrelated issues: the ...
Association and Temporal Rule Mining for Post-Filtering of Semantic Concept Detection in Video

Automatic semantic concept detection in video is important for effective content-based video retrieval and mining and has gained great attention recently. In this paper, we propose a general post-filtering framework to enhance robustness and accuracy of ...
Discriminative model fusion for semantic concept detection and annotation in video
MULTIMEDIA '03: Proceedings of the eleventh ACM international conference on Multimedia

In this paper we describe a general information fusion algorithm that can be used to incorporate multimodal cues in building user-defined semantic concept models. We compare this technique with a Bayesian Network-based approach on a semantic concept ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia

October 2004

1028 pages

ISBN:1581138938

DOI:10.1145/1027527

General Chairs:
Henning Schulzrinne
Columbia University
,
Nevenka Dimitrova
Philips Research
,
Program Chairs:
Angela Sasse
UCL
,
Sue Moon
KAIST
,
Rainer Lienhart
U Augsburg

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM04

Sponsor:

MM04: 2004 12th Annual ACM International Conference on Multimedia

October 10 - 16, 2004

NY, New York, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

87
Total Citations
View Citations
987
Total Downloads

Downloads (Last 12 months)8
Downloads (Last 6 weeks)2

Reflects downloads up to 01 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Saravanan DUrkude S(2023)Confiscate Boisterous from Color-Based Images Using Rule-Based TechniqueProceedings of the International Conference on Cognitive and Intelligent Computing10.1007/978-981-19-2358-6_47(503-514)Online publication date: 1-Jan-2023
https://doi.org/10.1007/978-981-19-2358-6_47
Ravi AVenugopal HPaul STizhoosh H(2018)A Dataset and Preliminary Results for Umpire Pose Detection Using SVM Classification of Deep Features2018 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2018.8628877(1396-1402)Online publication date: Nov-2018
https://doi.org/10.1109/SSCI.2018.8628877
Hauptmann A(2018)Video Content AnalysisEncyclopedia of Database Systems10.1007/978-1-4614-8265-9_1018(4381-4388)Online publication date: 7-Dec-2018
https://doi.org/10.1007/978-1-4614-8265-9_1018
Hauptmann A(2017)Video Content AnalysisEncyclopedia of Database Systems10.1007/978-1-4899-7993-3_1018-2(1-8)Online publication date: 19-Jan-2017
https://doi.org/10.1007/978-1-4899-7993-3_1018-2
Jiang LBourdeau JHendler JNkambou RHorrocks IZhao B(2016)Web-scale Multimedia Search for Internet Video ContentProceedings of the 25th International Conference Companion on World Wide Web10.1145/2872518.2888599(311-316)Online publication date: 11-Apr-2016
https://dl.acm.org/doi/10.1145/2872518.2888599
Shirahama KGrzegorzek M(2016)Towards large-scale multimedia retrieval enriched by knowledge about human interpretationMultimedia Tools and Applications10.1007/s11042-014-2292-875:1(297-331)Online publication date: 1-Jan-2016
https://dl.acm.org/doi/10.1007/s11042-014-2292-8
Sun FXu MLi HHao S(2016)Social video annotation by combining features with a tri-adaptation approachMultimedia Systems10.1007/s00530-014-0405-x22:4(413-422)Online publication date: 1-Jul-2016
https://dl.acm.org/doi/10.1007/s00530-014-0405-x
Shen JWang MChua T(2016)Accurate online video tagging via probabilistic hybrid modelingMultimedia Systems10.1007/s00530-014-0399-422:1(99-113)Online publication date: 1-Feb-2016
https://dl.acm.org/doi/10.1007/s00530-014-0399-4
Jiang LYu SMeng DYang YMitamura THauptmann AZhou XSmeaton ATian QBulterman DShen HMayer-Patel KYan S(2015)Fast and Accurate Content-based Semantic Search in 100M Internet VideosProceedings of the 23rd ACM international conference on Multimedia10.1145/2733373.2806237(49-58)Online publication date: 13-Oct-2015
https://dl.acm.org/doi/10.1145/2733373.2806237
Duong TNguyen NTruong HNguyen V(2015)A collaborative algorithm for semantic video annotation using a consensus-based social network analysisExpert Systems with Applications: An International Journal10.1016/j.eswa.2014.07.04642:1(246-258)Online publication date: 1-Jan-2015
https://dl.acm.org/doi/10.1016/j.eswa.2014.07.046
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten