The SoccerSum Dataset for Automated Detection, Segmentation, and Tracking of Objects on the Soccer Pitch
Authors: 
Mehdi Houshmand Sarkhoosh, 
Sushant Gautam, Cise Midoglu, Saeed Shafiee Sabet, Thomas Torjusen, 
Pål HalvorsenAuthors Info & Claims
Mehdi Houshmand Sarkhoosh, Sushant Gautam, Cise Midoglu, Saeed Shafiee Sabet, Thomas Torjusen, Pål HalvorsenAuthors Info & ClaimsPages 353 - 359
Abstract
This paper introduces SoccerSum, a novel dataset aimed at enhancing object detection and segmentation in video frames depicting the soccer pitch, using footage from the Norwegian Eliteserien league across 2021-2023. With the goal of detecting elements beyond common entities in existing datasets, such as the soccer ball, players and referees, this dataset includes additional annotations for the goal net, corner flag posts, and the penalty mark. SoccerSum also includes the segmentation of key pitch areas such as the penalty and goal boxes for the same frame sequences. Comprising 750 frames annotated with 10 classes for advanced analysis, SoccerSum offers compatibility with existing frameworks, providing a rich dataset for the development of computer vision algorithms. This dataset not only serves as a resource for improving sports analytics, but also introduces a new application for automatic game summarization, enabling the generation of detailed and engaging content for fans and professionals. The SoccerSum dataset is accessible on Zenodo: https://zenodo.org/records/10612084.
References
[1]
Daniel Berjón, Carlos Cuevas, and Narciso García. 2023. Soccer line mark segmentation and classification with stochastic watershed transform. Signal Process. Image Commun. 118 (Oct. 2023), 117014.
[2]
Anthony Cioppa, Silvio Giancola, Adrien Deliège, Le Kang, Xin Zhou, Zhiyu Cheng, Bernard Ghanem, and Marc Van Droogenbroeck. 2022. SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videos. In Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 19--20.
[3]
Adrien Deliège, Anthony Cioppa, Silvio Giancola, Meisam J. Seikavandi, Jacob V. Dueholm, Kamal Nasrollahi, Bernard Ghanem, Thomas B. Moeslund, and Marc Van Droogenbroeck. 2021. SoccerNet-v2: A Dataset and Benchmarks for Holistic Understanding of Broadcast Soccer Videos. In Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). IEEE, 19--25.
[4]
Fédération internationale de football association (FIFA). 2023. Pitch Dimensions and Surrounding Areas | FIFA Publications. https://publications.fifa.com/en/football-stadiums-guidelines/technical-guideline/stadium-guidelines/pitch-dimensions-and-surrounding-areas [Online; accessed 15. Mar. 2024].
[5]
Sushant Gautam. 2022. AI-based Soccer Game Summarization: From Video Highlights to Dynamic Text Summaries. Master's Thesis, Tribhuvan University. https://www.researchgate.net/publication/363857936_AI-based_Soccer_Game_Summarization_From_Video_Highlights_to_Dynamic_Text_Summaries
[6]
Sushant Gautam. 2023. Bridging Multimedia Modalities: Enhanced Multimodal AI Understanding and Intelligent Agents. In Proc. of the International Conference on Multimodal Interaction (ICMI). 695--699. https://doi.org/10.1145/3577190.3614225
[7]
Sushant Gautam, Cise Midoglu, Saeed Shafiee Sabet, Dinesh Baniya Kshatri, and Pål Halvorsen. 2022. Assisting soccer game summarization via audio intensity analysis of game highlights. In Proc. of IOE Graduate Conference, Vol. 12. Institute of Engineering, Tribhuvan University, Nepal, 25 - 32. https://conference.ioe.edu.np/publications/ioegc12/IOEGC-12-004-12009.pdf
[8]
Sushant Gautam, Cise Midoglu, Saeed Shafiee Sabet, Dinesh Baniya Kshatri, and Pål Halvorsen. 2022. Soccer Game Summarization using Audio Commentary, Metadata, and Captions. In Proc. of the Workshop on User-centric Narrative Summarization of Long Videos (NarSUM) (2022-10-10). 13--22. https://doi.org/10.1145/3552463.3557019
[9]
Silvio Giancola, Mohieddine Amine, Tarek Dghaily, and Bernard Ghanem. 2018. SoccerNet: A Scalable Dataset for Action Spotting in Soccer Videos. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 18--22.
[10]
Silvio Giancola, Anthony Cioppa, Adrien Deliège, Floriane Magera, Vladimir Somers, Le Kang, et al. 2022. SoccerNet 2022 Challenges Results. In Proc. of the International ACM Workshop on Multimedia Content Analysis in Sports. ACM.
[11]
Chakradhar Guntuboina, Aditya Porwal, Preet Jain, and Hansa Shingrakhia. 2021. Deep Learning Based Automated Sports Video Summarization Using Yolo. ELCVIA 20, 1 (May 2021), 99--116.
[12]
Mehdi Houshmand Sarkhoosh. 2024. SoccerSum: Dataset for Automatic Summarization of Soccer Videos. https://doi.org/10.5281/zenodo.10612084.
[13]
Mehdi Houshmand Sarkhoosh, Sushant Gautam, Cise Midoglu, Saeed Shafiee Sabet, and Pål Halvorsen. 2024. Multimodal AI-Based Summarization and Storytelling for Soccer on Social Media. In Proc. of the ACM Multimedia Systems Conference (MMSYS).
[14]
Kuan-Hao Huang, Chen Li, and Kai-Wei Chang. 2020. Generating Sports News from Live Commentary: A Chinese Dataset for Sports Game Summarization. ACL Anthology (Dec. 2020), 609--615. https://aclanthology.org/2020.aacl-main.61
[15]
Glenn Jocher, Ayush Chaurasia, and Jing Qiu. 2023. Ultralytics YOLOv8. https://github.com/ultralytics/ultralytics.
[16]
Abdullah Aman Khan, Jie Shao, Waqar Ali, and Saifullah Tumrani. 2020. Content-Aware Summarization of Broadcast Sports Videos: An Audio--Visual Feature Extraction Approach. Neural Process. Lett. 52, 3 (Dec. 2020), 1945--1968.
[17]
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual Instruction Tuning. arXiv (April 2023). arXiv:2304.08485
[18]
Jonathon Luiten, Aljosa Osep, Patrick Dendorfer, Philip Torr, Andreas Geiger, Laura Leal-Taixe, and Bastian Leibe. 2021. HOTA: A Higher Order Metric for Evaluating Multi-object Tracking. Int. J. Comput. Vision 129, 2 (Feb. 2021), 548--578.
[19]
Wenhan Luo, Junliang Xing, Anton Milan, Xiaoqin Zhang, Wei Liu, and Tae-Kyun Kim. 2021. Multiple object tracking: A literature review. Artif. Intell. 293 (April 2021), 103448.
[20]
Miguel Santos Marques, Ricardo Gomes Faria, Pedro Santos, and José Henrique Brito. 2023. Soccer Pitch Areas Segmentation with Hierarchical U-Net on the SoccerNet Dataset. In Proc. of the IEEE International Conference on Electronics, Computers and Artificial Intelligence (ECAI). 29--30.
[21]
Cise Midoglu, Saeed Shafiee Sabet, Mehdi Houshmand Sarkhoosh, Mohammad Majidi, Sushant Gautam, Håkon Maric Solberg, Tomas Kupka, and Pål Halvorsen. 2024. AI-Based Sports Highlight Generation for Social Media. In Proc. of the ACM Mile-High Video Conference (MHV). 7--13.
[22]
Hassan Mkhallati, Anthony Cioppa, Silvio Giancola, Bernard Ghanem, and Marc Van Droogenbroeck. 2023. SoccerNet-Caption: Dense Video Captioning for Soccer Broadcasts Commentaries. In Proc. of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. 5074--5085.
[23]
Ji Qi, Jifan Yu, Teng Tu, Kunyu Gao, Yifan Xu, Xinyu Guan, Xiaozhi Wang, Bin Xu, Lei Hou, Juanzi Li, et al. 2023. GOAL: A Challenging Knowledge-grounded Video Captioning Benchmark for Real-time Soccer Commentary Generation. In Proc. of the ACM International Conference on Information and Knowledge Management (CIKM). 5391--5395.
[24]
Mehdi Houshmand Sarkhoosh, Sayed Mohammad Majidi Dorcheh, Cise Midoglu, Saeed Shafiee Sabet, Tomas Kupka, Dag Johansen, Michael A. Riegler, and Pål Halvorsen. 2024. AI-Based Cropping of Soccer Videos for Different Social Media Representations. In Proc. of the International Conference on MultiMedia Modeling (MMM). 279--287.
[25]
Chris van der Lee, Emiel Krahmer, and Sander Wubben. 2017. PASS: A Dutch data-to-text system for soccer, targeted towards specific audiences. ACL Anthology (Sept. 2017), 95--104.
[26]
Bastien Vanderplaetse and Stéphane Dupont. 2020. Improved Soccer Action Spotting using both Audio and Video Streams. In Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). 14--19.
[27]
Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. 2022. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv preprint arXiv:2207.02696 (2022).
[28]
Chien-Yao Wang and Hong-Yuan Mark Liao. 2024. YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information. (2024).
[29]
Jiaan Wang, Zhixu Li, Tingyi Zhang, Duo Zheng, Jianfeng Qu, An Liu, Lei Zhao, and Zhigang Chen. 2022. Knowledge Enhanced Sports Game Summarization. In Proc. of the ACM International Conference on Web Search and Data Mining (WSDM). 1045--1053.
[30]
Jiaan Wang, Tingyi Zhang, and Haoxiang Shi. 2022. GOAL: Towards Benchmarking Few-Shot Sports Game Summarization. arXiv (July 2022). arXiv:2207.08635
[31]
Cheng Yan, Xin Li, and Guoqiang Li. 2024. A New Action Recognition Framework for Video Highlights Summarization in Sporting Events. In Proc. of the IEEE International Conference on Computer Science & Education (ICCSE). 17--21.
Index Terms
- The SoccerSum Dataset for Automated Detection, Segmentation, and Tracking of Objects on the Soccer Pitch
Recommendations
Multimodal AI-Based Summarization and Storytelling for Soccer on Social Media
MMSys '24: Proceedings of the 15th ACM Multimedia Systems ConferenceThe rapid advancement of technology has been revolutionizing the field of sports media, where there is a growing need for sophisticated data processing methods. Current methodologies for extracting information from soccer broadcast videos to generate ...
Soccer Athlete Data Visualization and Analysis with an Interactive Dashboard
MultiMedia ModelingAbstractSoccer is among the most popular and followed sports in the world. As its popularity increases, it becomes highly professionalized. Even though research on soccer makes up for a big part in classic sports science, there is a greater potential for ...
Self-Supervised Small Soccer Player Detection and Tracking
MMSports '20: Proceedings of the 3rd International Workshop on Multimedia Content Analysis in SportsIn a soccer game, the information provided by detecting and tracking brings crucial clues to further analyze and understand some tactical aspects of the game, including individual and team actions. State-of-the-art tracking algorithms achieve impressive ...
Comments
Information & Contributors
Information
Published In
April 2024
557 pages
ISBN:9798400704123
DOI:10.1145/3625468
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 April 2024
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
MMSys '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 176 of 530 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 332Total Downloads
- Downloads (Last 12 months)332
- Downloads (Last 6 weeks)56
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in