From Technical to Aesthetics Quality Assessment and Beyond: Challenges and Potential
Authors: 
Vlad Hosu, Dietmar Saupe, Bastian Goldluecke, Weisi Lin, Wen-Huang Cheng, John See, Lai-Kuan WongAuthors Info & Claims
Vlad Hosu, Dietmar Saupe, Bastian Goldluecke, Weisi Lin, Wen-Huang Cheng, John See, Lai-Kuan WongAuthors Info & ClaimsPages 19 - 20
Abstract
Every day 1.8+ billion images are being uploaded to Facebook, Instagram, Flickr, Snapchat, and WhatsApp [6]. The exponential growth of visual media has made quality assessment become increasingly important for various applications, from image acquisition, synthesis, restoration, and enhancement, to image search and retrieval, storage, and recognition.
There have been two related but different classes of visual quality assessment techniques: image quality assessment (IQA) and image aesthetics assessment (IAA). As perceptual assessment tasks, subjective IQA and IAA share some common underlying factors that affect user judgments. Moreover, they are similar in methodology (especially NR-IQA in-the-wild and IAA). However, the emphasis for each is different: IQA focuses on low-level defects e.g. processing artefacts, noise, and blur, while IAA puts more emphasis on abstract and higher-level concepts that capture the subjective aesthetics experience, e.g. established photographic rules encompassing lighting, composition, and colors, and personalized factors such as personality, cultural background, age, and emotion.
IQA has been studied extensively over the last decades [3, 14, 22]. There are three main types of IQA methods: full-reference (FR), reduced-reference (RR), and no-reference (NR). Among these, NRIQA is the most challenging as it does not depend on reference images or impose strict assumptions on the distortion types and level. NR-IQA techniques can be further divided into those that predict the global image score [1, 2, 10, 17, 26] and patch-based IQA [23, 25], naming a few of the more recent approaches.
References
[1]
Shahrukh Athar and Zhou Wang. 2019. A Comprehensive Performance Evaluation of Image Quality Assessment Algorithms. Ieee Access 7 (2019), 140030--140070.
[2]
Simone Bianco, Luigi Celona, Paolo Napoletano, and Raimondo Schettini. 2018. On the use of deep learning for blind image quality assessment. Signal, Image and Video Processing 12, 2 (2018), 355--362.
[3]
Damon M Chandler. 2013. Seven challenges in image quality assessment: past, present, and future research. International Scholarly Research Notices 2013 (2013).
[4]
Ritendra Datta, Dhiraj Joshi, Jia Li, and James Z Wang. 2006. Studying aesthetics in photographic images using a computational approach. In European conference on computer vision. Springer, 288--301.
[5]
Yubin Deng, Chen Change Loy, and Xiaoou Tang. 2017. Image aesthetic assessment: An experimental survey. IEEE Signal Processing Magazine 34, 4 (2017), 80--106.
[6]
Jim Edwards. 2014 (accessed September 15, 2020). PLANET SELFIE: We're Now Posting A Staggering 1.8 Billion Photos Every Day. https: //www.businessinsider.com/were-now-posting-a-staggering-18-billionphotos- to-social-media-every-day-2014--5
[7]
Yong-Lian Hii, John See, Magzhan Kairanbay, and Lai-KuanWong. 2017. Multigap: Multi-pooled inception network with text augmentation for aesthetic prediction of photographs. In 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 1722--1726.
[8]
Vlad Hosu, Bastian Goldlucke, and Dietmar Saupe. 2019. Effective aesthetics prediction with multi-level spatially pooled features. In Proceedings of the IEEE conference on computer vision and pattern recognition. 9375--9383.
[9]
Vlad Hosu, Hanhe Lin, and Dietmar Saupe. 2018. Expertise screening in crowdsourcing image quality. In QoMEX 2018: Tenth International Conference on Quality of Multimedia Experience.
[10]
V. Hosu, H. Lin, T. Sziranyi, and D. Saupe. 2020. KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment. IEEE Transactions on Image Processing 29 (2020), 4041--4056.
[11]
Mohsen Jenadeleh, Mohammad M Masaeli, and Mohsen E Moghaddam. 2017. Blind image quality assessment based on aesthetic and statistical quality-aware features. Journal of Electronic Imaging 26, 4 (2017), 043018.
[12]
Yan Ke, Xiaoou Tang, and Feng Jing. 2006. The design of high-level features for photo quality assessment. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06), Vol. 1. IEEE, 419--426.
[13]
Leida Li, Hancheng Zhu, Sicheng Zhao, Guiguang Ding, and Weisi Lin. 2020. Personality-Assisted Multi-Task Learning for Generic and Personalized Image Aesthetics Assessment. IEEE Transactions on Image Processing 29 (2020), 3898-- 3910.
[14]
Weisi Lin and C-C Jay Kuo. 2011. Perceptual visual quality metrics: A survey. Journal of visual communication and image representation 22, 4 (2011), 297--312.
[15]
Xin Lu, Zhe Lin, Hailin Jin, Jianchao Yang, and James Z Wang. 2014. Rapid: Rating pictorial aesthetics using deep learning. In Proceedings of the 22nd ACM international conference on Multimedia. 457--466.
[16]
Yiwen Luo and Xiaoou Tang. 2008. Photo and video quality evaluation: Focusing on the subject. In European Conference on Computer Vision. Springer, 386--399.
[17]
Kede Ma,Wentao Liu, Tongliang Liu, ZhouWang, and Dacheng Tao. 2017. dipIQ: Blind image quality assessment by learning-to-rank discriminable image pairs. IEEE Transactions on Image Processing 26, 8 (2017), 3951--3964.
[18]
Luca Marchesotti, Florent Perronnin, Diane Larlus, and Gabriela Csurka. 2011. Assessing the aesthetic quality of photographs using generic image descriptors. In 2011 international conference on computer vision. IEEE, 1784--1791.
[19]
Judith A Redi and Ingrid Heynderickx. 2012. Image integrity and aesthetics: towards a more encompassing definition of visual quality. In Human Vision and Electronic Imaging XVII, Vol. 8291. International Society for Optics and Photonics, 829115.
[20]
Jian Ren, Xiaohui Shen, Zhe Lin, Radomir Mech, and David J Foran. 2017. Personalized image aesthetics. In Proceedings of the IEEE International Conference on Computer Vision. 638--647.
[21]
Hossein Talebi and Peyman Milanfar. 2018. NIMA: Neural image assessment. IEEE Transactions on Image Processing 27, 8 (2018), 3998--4011.
[22]
ZhouWang and Alan C Bovik. 2006. Modern image quality assessment. Synthesis Lectures on Image, Video, and Multimedia Processing 2, 1 (2006), 1--156.
[23]
Oliver Wiedemann, Vlad Hosu, Hanhe Lin, and Dietmar Saupe. 2018. Disregarding the Big Picture: Towards Local Image Quality Assessment. In 10th International Conference on Quality of Multimedia Experience(QoMEX). IEEE. http://database.mmsp-kn.de
[24]
Lai-Kuan Wong and Kok-Lim Low. 2009. Saliency-enhanced image aesthetics class prediction. In 2009 16th IEEE International Conference on Image Processing (ICIP). IEEE, 997--1000.
[25]
Zhenqiang Ying, Haoran Niu, Praful Gupta, Dhruv Mahajan, Deepti Ghadiyaram, and Alan Bovik. 2020. From patches to pictures (PaQ-2-PiQ): Mapping the perceptual space of picture quality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 3575--3585.
[26]
Weixia Zhang, Kede Ma, and Xiaokang Yang. 2019. Learning to Blindly Assess Image Quality in the Laboratory and Wild. arXiv preprint arXiv:1907.00516 (2019).
Index Terms
- From Technical to Aesthetics Quality Assessment and Beyond: Challenges and Potential
Recommendations
Technical Quality-Assisted Image Aesthetics Quality Assessment
Pattern Recognition and Computer VisionAbstractImage aesthetics assessment (IAA) aims at predicting the perceived aesthetic quality of images. Intuitively, the technical quality of an image has significant impact on its aesthetic quality, e.g., an image with noticeable distortions is not ...
An improved blind/referenceless image spatial quality evaluator algorithm for image quality assessment
Image quality assessment (IQA) methods are generally studied in the spatial or transform domain. Due to the BRISQUE algorithm evaluating the quality of an image only based on its natural scene statistics of the spatial domain, the frequency features that ...
Detection of images degraded by rain using image quality assessment
AbstractVarious weather conditions degrade images, and hence the quality of the images is compromised to a large extent. Atmospheric conditions like Rain, Fog, Haze, Mist, etc., degrade scenes, and the scene’s acquisition results in noisy images. The ...
Comments
Information & Contributors
Information
Published In
October 2020
38 pages
ISBN:9781450381543
DOI:10.1145/3423268
Copyright © 2020 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 October 2020
Check for updates
Author Tags
Qualifiers
- Abstract
Funding Sources
- Deutsche Forschungsgemeinschaft (DFG German Research Foundation)
Conference
MM '20
Sponsor:
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 184Total Downloads
- Downloads (Last 12 months)4
- Downloads (Last 6 weeks)0
Reflects downloads up to 29 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in