short-paper

A Solution to Multi-modal Ads Video Tagging Challenge

Authors:

Zhonglin ZuAuthors Info & Claims

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

Pages 4808 - 4812

https://doi.org/10.1145/3474085.3479204

Published: 17 October 2021 Publication History

Abstract

In this paper, we present our solution to the Multi-modal Ads Video Tagging Challenge of Tencent Advertising Algorithm Competition in ACM Multimedia 2021 Grand Challenges. We extend the baseline model by redesigning the visual feature extraction procedure and we modify the loss function to cope with sparse positive targets. Moreover, we propose Semi-supervised Learning with Negative Masking to leverage both labeled data and unlabeled data from the preliminary contest which effectively enhances the training process. We further utilize Cross-Class Relevance Learning to boost the performance. We achieve 0.8237 GAP score via model ensemble and rank the second place among all submissions in the challenge.

Supplementary Material

MP4 File (MM21-gch3317.mp4)

In this video, we present our solution to the Multi-modal Ads Video Tagging Challenge in ACM Multimedia 2021 Grand Challenges. We extend the baseline model by redesigning the visual feature extraction procedure and we modify the loss function to cope with sparse positive targets. Moreover, we propose Semi-supervised Learning with Negative Masking to leverage both labeled data and unlabeled data from the preliminary contest which effectively enhances the training process. We further utilize Cross-Class Relevance Learning to boost the performance. We achieve 0.8237 GAP score via model ensemble and rank the second place among all submissions in the challenge.

Download
53.80 MB

References

[1]

Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, and Sudheendra Vijayanarasimhan. 2016. Youtube-8m: A large-scale video classification benchmark. arXiv preprint arXiv:1609.08675 (2016).

[2]

Emanuel Ben-Baruch, Tal Ridnik, Nadav Zamir, Asaf Noy, Itamar Friedman, Matan Protter, and Lihi Zelnik-Manor. 2020. Asymmetric loss for multi-label classification. arXiv preprint arXiv:2009.14119 (2020).

[3]

Amanda Clare and Ross D King. 2001. Knowledge discovery in multi-label phenotype data. In European conference on principles of data mining and knowledge discovery. Springer, 42--53.

Digital Library

[4]

Shaoguo Cui, Lei Mao, Jingfeng Jiang, Chang Liu, and Shuyu Xiong. 2018. Automatic semantic segmentation of brain gliomas from MRI images using a deep cascaded neural network. Journal of healthcare engineering, Vol. 2018 (2018).

[5]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).

[6]

Deepti Ghadiyaram, Du Tran, and Dhruv Mahajan. 2019. Large-scale weakly-supervised pre-training for video action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 12046--12055.

[7]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.

[8]

Shawn Hershey, Sourish Chaudhuri, Daniel PW Ellis, Jort F Gemmeke, Aren Jansen, R Channing Moore, Manoj Plakal, Devin Platt, Rif A Saurous, Bryan Seybold, et almbox. 2017. CNN architectures for large-scale audio classification. In 2017 ieee international conference on acoustics, speech and signal processing (icassp). IEEE, 131--135.

[9]

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015).

[10]

Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7132--7141.

[11]

Fabian Isensee, Philipp Kickingereder, Wolfgang Wick, Martin Bendszus, and Klaus H Maier-Hein. 2017. Brain tumor segmentation and radiomics survival prediction: Contribution to the brats 2017 challenge. In International MICCAI Brainlesion Workshop. Springer, 287--297.

[12]

Fabian Isensee, Philipp Kickingereder, Wolfgang Wick, Martin Bendszus, and Klaus H Maier-Hein. 2018. No new-net. In International MICCAI Brainlesion Workshop. Springer, 234--244.

[13]

Robert A Jacobs, Michael I Jordan, Steven J Nowlan, and Geoffrey E Hinton. 1991. Adaptive mixtures of local experts. Neural computation, Vol. 3, 1 (1991), 79--87.

[14]

Konstantinos Kamnitsas, Wenjia Bai, Enzo Ferrante, Steven McDonagh, Matthew Sinclair, Nick Pawlowski, Martin Rajchl, Matthew Lee, Bernhard Kainz, Daniel Rueckert, et al. 2017. Ensembles of multiple models and architectures for robust brain tumour segmentation. In International MICCAI brainlesion workshop. Springer, 450--462.

[15]

Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).

[16]

Rongcheng Lin, Jing Xiao, and Jianping Fan. 2018. Nextvlad: An efficient neural network to aggregate frame-level features for large-scale video classification. In Proceedings of the European Conference on Computer Vision (ECCV) Workshops. 0--0.

[17]

Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision. 2980--2988.

[18]

Junwei Ma, Satya Krishna Gorti, Maksims Volkovs, Ilya Stanevich, and Guangwei Yu. 2019. Cross-Class Relevance Learning for Temporal Concept Localization. arXiv preprint arXiv:1911.08548 (2019).

[19]

Q. Meng. 2018. LightGBM: A Highly Efficient Gradient Boosting Decision Tree. (2018).

[20]

Jesse Read, Bernhard Pfahringer, Geoff Holmes, and Eibe Frank. 2011. Classifier chains for multi-label classification. Machine learning, Vol. 85, 3 (2011), 333--359.

Digital Library

[21]

Du Tran, Heng Wang, Lorenzo Torresani, and Matt Feiszli. 2019. Video classification with channel-separated convolutional networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5552--5561.

[22]

Grigorios Tsoumakas, Ioannis Katakis, and Ioannis Vlahavas. 2010. Random k-labelsets for multilabel classification. IEEE transactions on knowledge and data engineering, Vol. 23, 7 (2010), 1079--1089.

Digital Library

[23]

Yixin Wang, Yao Zhang, Feng Hou, Yang Liu, Jiang Tian, Cheng Zhong, Yang Zhang, and Zhiqiang He. 2020. Modality-Pairing Learning for Brain Tumor Segmentation. arXiv preprint arXiv:2010.09277 (2020).

[24]

Hao Wu, Jiangchao Yao, Jiajie Wang, Yinru Chen, Ya Zhang, and Yanfeng Wang. 2019. Collaborative Label Correction via Entropy Thresholding. In 2019 IEEE International Conference on Data Mining (ICDM). 1390--1395. https://doi.org/10.1109/ICDM.2019.00179

[25]

Hao Wu, Jiangchao Yao, Ya Zhang, and Yanfeng Wang. 2021. Cooperative Learning for Noisy Supervision. In 2021 IEEE International Conference on Multimedia and Expo (ICME). https://doi.org/10.1109/ICME51207.2021.9428133

[26]

Qizhe Xie, Minh-Thang Luong, Eduard Hovy, and Quoc V Le. 2020. Self-training with noisy student improves imagenet classification. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 10687--10698.

[27]

Min-Ling Zhang and Zhi-Hua Zhou. 2007. ML-KNN: A lazy learning approach to multi-label learning. Pattern recognition, Vol. 40, 7 (2007), 2038--2048.

Digital Library

Cited By

Wu HWang JZu Z(2023)Interaction-Assisted Multi-Modal Representation Learning for RecommendationICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095080(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10095080
Nama VDeepak G(2023)DTagRecPLS: Diversification of Tag Recommendation for Videos Using Preferential Learning and Differential SemanticsProceedings of the 14th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2022)10.1007/978-3-031-27524-1_87(887-898)Online publication date: 28-Mar-2023
https://doi.org/10.1007/978-3-031-27524-1_87
Wang XGan TWei YWu JMeng DNie LMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Micro-video Tagging via Jointly Modeling Social Influence and Tag RelationProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548098(4478-4486)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548098
Show More Cited By

Index Terms

A Solution to Multi-modal Ads Video Tagging Challenge
1. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification

Recommendations

Multi-modal Contextual Prompt Learning for Multi-label Classification with Partial Labels
ICMLC '24: Proceedings of the 2024 16th International Conference on Machine Learning and Computing

Multi-label classification is a task with diverse applications, but current algorithms heavily rely on accurately labeled data, leading to time-consuming and labor-intensive data collection. However, multi-label classification with partial labels ...
Self-paced multi-label co-training
Abstract
Multi-label learning aims to solve classification problems where instances are associated with a set of labels. In reality, it is generally easy to acquire unlabeled data but expensive or time-consuming to label them, and this ...
Semi-supervised multi-label image classification based on nearest neighbor editing

Semi-supervised multi-label classification has been applied to many real-world applications such as image classification, document classification and so on. In semi-supervised learning, unlabeled samples are added to the training set for enhancing the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '21: Proceedings of the 29th ACM International Conference on Multimedia

October 2021

5796 pages

ISBN:9781450386517

DOI:10.1145/3474085

General Chairs:
Heng Tao Shen
University of Electronic Science&Technology of China, China
,
Yueting Zhuang
Zhejiang University, China
,
John R. Smith
IBM, USA
,
Program Chairs:
Yang Yang
University of Electronic Science and Technology of China, China
,
Pablo Cesar
CWI&TU Delft, The Netherlands
,
Florian Metze
FACEBOOK, Inc., USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

MM '21

Sponsor:

SIGMM

MM '21: ACM Multimedia Conference

October 20 - 24, 2021

Virtual Event, China

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
176
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)4

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wu HWang JZu Z(2023)Interaction-Assisted Multi-Modal Representation Learning for RecommendationICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP49357.2023.10095080(1-5)Online publication date: 4-Jun-2023
https://doi.org/10.1109/ICASSP49357.2023.10095080
Nama VDeepak G(2023)DTagRecPLS: Diversification of Tag Recommendation for Videos Using Preferential Learning and Differential SemanticsProceedings of the 14th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2022)10.1007/978-3-031-27524-1_87(887-898)Online publication date: 28-Mar-2023
https://doi.org/10.1007/978-3-031-27524-1_87
Wang XGan TWei YWu JMeng DNie LMagalhães Jdel Bimbo ASatoh SSebe NAlameda-Pineda XJin QOria VToni L(2022)Micro-video Tagging via Jointly Modeling Social Influence and Tag RelationProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548098(4478-4486)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3503161.3548098
Wu HYao J(2022)PEAR: Photographic Embedding for Aesthetic RatingICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP43922.2022.9747287(4038-4042)Online publication date: 23-May-2022
https://doi.org/10.1109/ICASSP43922.2022.9747287

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents