ImageCLEF 2022: Multimedia Retrieval in Medical, Nature, Fusion, and Internet Applications
Authors: 
Alba G. Seco de Herrera, 
Bogdan Ionescu, Henning Müller, Renaud Péteri, Asma Ben Abacha, Christoph M. Friedrich, Johannes Rückert, Louise Bloch, Raphael Brüngel, Ahmad Idrissi-Yaghir, Henning Schäfer, Serge Kozlovski, Yashin Dicente Cid, Vassili Kovalev, Jon Chamberlain, Adrian Clark, Antonio Campello, Hugo Schindler, Jérôme Deshayes, Adrian Popescu, Liviu-Daniel Ştefan, Mihai Gabriel Constantin, Mihai DogariuAuthors Info & Claims
Alba G. Seco de Herrera, Bogdan Ionescu, Henning Müller, Renaud Péteri, Asma Ben Abacha, Christoph M. Friedrich, Johannes Rückert, Louise Bloch, Raphael Brüngel, Ahmad Idrissi-Yaghir, Henning Schäfer, Serge Kozlovski, Yashin Dicente Cid, Vassili Kovalev, Jon Chamberlain, Adrian Clark, Antonio Campello, Hugo Schindler, Jérôme Deshayes, Adrian Popescu, Liviu-Daniel Ştefan, Mihai Gabriel Constantin, Mihai DogariuAuthors Info & ClaimsPages 382 - 389
Abstract
ImageCLEF s part of the Conference and Labs of the Evaluation Forum (CLEF) since 2003. CLEF 2022 will take place in Bologna, Italy. ImageCLEF is an ongoing evaluation initiative which promotes the evaluation of technologies for annotation, indexing, and retrieval of visual data with the aim of providing information access to large collections of images in various usage scenarios and domains. In its 20th edition, ImageCLEF will have four main tasks: (i) a Medical task addressing concept annotation, caption prediction, and tuberculosis detection; (ii) a Coral task addressing the annotation and localisation of substrates in coral reef images; (iii) an Aware task addressing the prediction of real-life consequences of online photo sharing; and (iv) a new Fusion task addressing late fusion techniques based on the expertise of the pool of classifiers. In 2021, over 100 research groups registered at ImageCLEF with 42 groups submitting more than 250 runs. These numbers show that, despite the COVID-19 pandemic, there is strong interest in the evaluation campaign.
References
[1]
Azcona, D., Moreu, E., Hu, F., Ward, T.E., Smeaton, A.F.: Predicting media memorability using ensemble models. In: Working Notes Proceedings of the MediaEval 2019 Workshop. CEUR Workshop Proceedings, vol. 2670. CEUR-WS.org (2019)
[2]
Chamberlain, J., Campello, A., Wright, J.P., Clift, L.G., Clark, A., Seco de Herrera, A.G.: Overview of ImageCLEF coral 2019 task. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2019). CEUR Workshop Proceedings, vol. 2380. CEUR-WS.org (2019)
[3]
Chamberlain, J., Campello, A., Wright, J.P., Clift, L.G., Clark, A., Seco de Herrera, A.G.: Overview of the ImageCLEF coral 2020 task: automated coral reef image annotation. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2020). CEUR Workshop Proceedings, vol. 2696. CEUR-WS.org (2020)
[4]
Chamberlain, J., Seco de Herrera, A.G., Campello, A., Clark, A., Oliver, T.A., Moustahfid, H.: Overview of the ImageCLEF coral 2021 task: coral reef image annotation of a 3D environment. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2021). CEUR Workshop Proceedings, vol. 2936. CEUR-WS.org (2021)
[5]
Constantin, M.G., Ştefan, L.-D., Ionescu, B.: DeepFusion: deep ensembles for domain independent system fusion. In: Lokoč, J., et al. (eds.) MMM 2021. LNCS, vol. 12572, pp. 240–252. Springer, Cham (2021).
[6]
Constantin, M.G., Ştefan, L.D., Ionescu, B., Duong, N.Q., Demarty, C.H., Sjöberg, M.: Visual interestingness prediction: a benchmark framework and literature review. Int. J. Comput. Vis. 1–25 (2021)
[7]
Dai, Q., et al.: Fudan-Huawei at MediaEval 2015: detecting violent scenes and affective impact in movies with deep learning. In: Working Notes Proceedings of the MediaEval 2015 Workshop. CEUR Workshop Proceedings, vol. 1436. CEUR-WS.org (2015)
[8]
Demarty, C.H., Sjöberg, M., Ionescu, B., Do, T.T., Gygli, M., Duong, N.: MediaEval 2017 predicting media interestingness task. In: Working Notes Proceedings of the MediaEval 2017 Workshop. CEUR Workshop Proceedings, vol. 1984. CEUR-WS.org (2017)
[9]
Cid, Y.D., Kalinovsky, A., Liauchuk, V., Kovalev, V., Müller, H.: Overview of Image CLEF tuberculosis 2017 - predicting tuberculosis type and drug resistances. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2017). CEUR Workshop Proceedings, vol. 1866. CEUR-WS.org (2017)
[10]
Cid, Y.D., Liauchuk, V., Klimuk, D., Tarasau, A., Kovalev, V., Müller, H.: Overview of image CLEF tuberculosis 2019 - automatic CT-based report generation and tuberculosis severity assessment. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2019). CEUR Workshop Proceedings, vol. 2380. CEUR-WS.org (2019)
[11]
Cid, Y.D., Liauchuk, V., Kovalev, V., Müller, H.: Overview of Image CLEF tuberculosis 2018 - detecting multi-drug resistance, classifying tuberculosis type, and assessing severity score. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2018). CEUR Workshop Proceedings, vol. 2125. CEUR-WS.org (2018)
[12]
Eickhoff, C., Schwall, I., Seco de Herrera, A.G., Müller, H.: Overview of Image CLEF caption 2017 - the image caption prediction and concept extraction tasks to understand biomedical images. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2017). CEUR Workshop Proceedings, vol. 1866. CEUR-WS.org (2017)
[13]
Seco De Herrera, A.G., Eickhof, C., Andrearczyk, V., Müller, H.: Overview of the Image CLEF 2018 caption prediction tasks. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2018). CEUR Workshop Proceedings, vol. 2125. CEUR-WS.org (2018)
[14]
Seco De Herrera, A.G., et al.: Overview of MediaEval 2020 predicting media memorability task: what makes a video memorable? Working Notes Proceedings of the MediaEval 2020 Workshop (2020)
[15]
Goutte C and Gaussier E Losada DE and Fernández-Luna JM A probabilistic interpretation of precision, recall and F-score, with implication for evaluation Advances in Information Retrieval 2005 Heidelberg Springer 345-359
[16]
Kalpathy-Cramer, J., Seco de Herrera, A.G., Demner-Fushman, D., Antani, S., Bedrick, S., Müller, H.: Evaluating performance of biomedical image retrieval systems: overview of the medical image retrieval task at image CLEF 2004–2014. Comput. Med. Imaging Graph. 39, 55–61 (2015)
[17]
Kozlovski, S., Liauchuk, V., Cid, Y.D., Kovalev, V., Müller, H.: Overview of Image CLEF tuberculosis 2021 - CT-based tuberculosis type classification. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2021). CEUR Workshop Proceedings, vol. 2936. CEUR-WS.org (2021)
[18]
Kozlovski, S., Liauchuk, V., Cid, Y.D., Tarasau, A., Kovalev, V., Müller, H.: Overview of Image CLEF tuberculosis 2020 - automatic CT-based report generation. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2020). CEUR Workshop Proceedings, vol. 2696. CEUR-WS.org (2020)
[19]
Müller, H., Clough, P., Deselaers, T., Caputo, B. (eds.): Image CLEF - Experimental Evaluation in Visual Information Retrieval, The Springer International Series On Information Retrieval, vol. 32. Springer, Berlin Heidelberg (2010).
[20]
Müller, H., Kalpathy-Cramer, J., Seco de Herrera, A.G.: Experiences from the image CLEF medical retrieval and annotation tasks. In: Information Retrieval Evaluation in a Changing World, pp. 231–250. Springer (2019).
[21]
Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002), pp. 311–318 (2002)
[22]
Pelka, O., et al.: Overview of the Image CLEFmed 2021 concept & caption prediction task. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2021). CEUR Workshop Proceedings, vol. 2936. CEUR-WS.org (2021)
[23]
Pelka, O., Friedrich, C.M., Seco de Herrera, A.G., Müller, H.: Overview of the Image CLEFmed 2019 concept detection task. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2019). CEUR Workshop Proceedings, vol. 2380. CEUR-WS.org (2019)
[24]
Pelka, O., Friedrich, C.M., Seco de Herrera, A.G., Müller, H.: Overview of the Image CLEFmed 2020 concept prediction task: medical image understanding. In: Working Notes of Conference and Labs of the Evaluation Forum (CLEF 2020). CEUR Workshop Proceedings, vol. 2696. CEUR-WS.org (2020)
[25]
Pelka, O., Koitka, S., Rückert, J., Nensa, F., Friedrich, C.M.: Radiology objects in COntext (ROCO): a multimodal image dataset. In: Intravascular Imaging and Computer Assisted Stenting and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, pp. 180–189. Springer (2018).
[26]
Ştefan, L.D., Constantin, M.G., Ionescu, B.: System fusion with deep ensembles. In: Proceedings of the 2020 International Conference on Multimedia Retrieval (ICMR 2020), pp. 256–260. Association for Computing Machinery (ACM) (2020)
[27]
Sudhakaran, S., Escalera, S., Lanz, O.: Gate-shift networks for video action recognition. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2020), pp. 1102–1111 (2020)
[28]
Tsikrika T, de Herrera AGS, and Müller H Forner P, Gonzalo J, Kekäläinen J, Lalmas M, and de Rijke M Assessing the scholarly impact of ImageCLEF Multilingual and Multimodal Information Access Evaluation 2011 Heidelberg Springer 95-106
[29]
Tsikrika, T., Larsen, B., Müller, H., Endrullis, S., Rahm, E.: The scholarly impact of CLEF (2000–2009). In: Information Access Evaluation. Multilinguality, Multimodality, and Visualization, pp. 1–12. Springer (2013).
[30]
Wang, S., Chen, S., Zhao, J., Jin, Q.: Video interestingness prediction based on ranking model. In: Proceedings of the Joint Workshop of the 4th Workshop on Affective Social Multimedia Computing and First Multi-Modal Affective Computing of Large-Scale Multimedia Data (ASMMC-MMAC 2018), pp. 55–61. Association for Computing Machinery (ACM) (2018)
[31]
Zaharieva, M., Ionescu, B., Gînsca, A.L., Santos, R.L., Müller, H.: Retrieving diverse social images at MediaEval 2017: challenges, dataset and evaluation. In: Working Notes Proceedings of the MediaEval 2017 Workshop. CEUR Workshop Proceedings, vol. 1984. CEUR-WS.org (2017)
Recommendations
The 2021 ImageCLEF Benchmark: Multimedia Retrieval in Medical, Nature, Internet and Social Media Applications
Advances in Information RetrievalAbstractThis paper presents the ideas for the 2021 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum—CLEF Labs 2021 in Bucharest, Romania. ImageCLEF is an ongoing evaluation initiative (active since 2003) that ...
ImageCLEF 2023 Highlight: Multimedia Retrieval in Medical, Social Media and Content Recommendation Applications
Advances in Information RetrievalAbstractIn this paper, we provide an overview of the upcoming ImageCLEF campaign. ImageCLEF is part of the CLEF Conference and Labs of the Evaluation Forum since 2003. ImageCLEF, the Multimedia Retrieval task in CLEF, is an ongoing evaluation initiative ...
ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications
Advances in Information RetrievalAbstractThis paper presents an overview of the 2020 ImageCLEF lab that will be organized as part of the Conference and Labs of the Evaluation Forum—CLEF Labs 2020 in Thessaloniki, Greece. ImageCLEF is an ongoing evaluation initiative (run since 2003) ...
Comments
Information & Contributors
Information
Published In
Apr 2022
629 pages
ISBN:978-3-030-99738-0
DOI:10.1007/978-3-030-99739-7
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2022.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 10 April 2022
Author Tags
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025