research-article

Affective Analysis of Professional and Amateur Abstract Paintings Using Statistical Analysis and Art Theory

Authors:

Andreza Sartori,

Victoria Yanulevskaya,

Almila Akdag Salah,

Jasper Uijlings,

Nicu SebeAuthors Info & Claims

ACM Transactions on Interactive Intelligent Systems (TiiS), Volume 5, Issue 2

Article No.: 8, Pages 1 - 27

https://doi.org/10.1145/2768209

Published: 30 June 2015 Publication History

Abstract

When artists express their feelings through the artworks they create, it is believed that the resulting works transform into objects with “emotions” capable of conveying the artists' mood to the audience. There is little to no dispute about this belief: Regardless of the artwork, genre, time, and origin of creation, people from different backgrounds are able to read the emotional messages. This holds true even for the most abstract paintings. Could this idea be applied to machines as well? Can machines learn what makes a work of art “emotional”? In this work, we employ a state-of-the-art recognition system to learn which statistical patterns are associated with positive and negative emotions on two different datasets that comprise professional and amateur abstract artworks. Moreover, we analyze and compare two different annotation methods in order to establish the ground truth of positive and negative emotions in abstract art. Additionally, we use computer vision techniques to quantify which parts of a painting evoke positive and negative emotions. We also demonstrate how the quantification of evidence for positive and negative emotions can be used to predict which parts of a painting people prefer to focus on. This method opens new opportunities of research on why a specific painting is perceived as emotional at global and local scales.

References

[1]

D. E. Berlyne. 1971. Aesthetics and Psychobiology. Appleton-Century-Crofts, New York.

[2]

P. Blank, C. Massey, H. Gardner, and E. Winner. 1984. Perceiving what paintings express. Cognitive Processes in the Perception of Art (1984), 127--143.

[3]

D. H. Brainard. 1997. The psychophysics toolbox. Spatial Vision 10, 4 (1997).

[4]

N. Bruce and J. Tsotsos. 2009. Saliency, attention, and visual search: An information theoretic approach. Journal of Vision 9, 3 (2009).

[5]

M. Csíkszentmihályi and R. E. Robinson. 1990. The Art of Seeing: An Interpretation of the Aesthetic Encounter. J. Paul Getty Museum.

[6]

G. Csurka, C. Dance, L. Fan, J. Willamowski, and C. Bray. 2004. Visual categorization with bags of keypoints. In Proceedings of the Workshop Statistical Learning in Computer Vision (ECCV'04) 1. 1--2.

[7]

G. C. Cupchik, O. Vartanian, A. Crawley, and D. J. Mikulis. 2009. Viewing artworks: Contributions of cognitive control and perceptual facilitation to aesthetic experience. Brain and Cognition 70, 1 (2009), 84--91.

[8]

A. R. Damasio. 1999. The Feeling of What Happens: Body and Emotion in the Making of Consciousness.

[9]

E. H. Gombrich. 1961. How to read a painting. Saturday Evening Post 234, 30 (1961), 20--21.

[10]

H. Hagtvedt, R. Hagtvedt, and V. Patrick. 2008. The perception and evaluation of visual art. Empirical Studies of the Arts 26, 2 (2008), 197--218.

[11]

R. Herbrich, T. Minka, and T. Graepel. 2007. TrueSkill(TM): A bayesian skill rating system. In Advances in Neural Information Processing Systems. 569--576.

[12]

P. Isola, J. Xiao, A. Torralba, and A. Oliva. 2011. What makes an image memorable? In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 145--152.

Digital Library

[13]

D. Joshi, R. Datta, E. A. Fedorovskaya, Quang-Tuan Luong, J. Z. Wang, Jia Li, and Jiebo Luo. 2011. Aesthetics and emotions in images. IEEE Signal Processing Magazine 28, 5 (2011), 94--115.

[14]

P. J. Lang, M. M. Bradley, and B. N. Cuthbert. 1999. International affective picture system (IAPS): Technical manual and affective ratings. (1999).

[15]

H. Leder, G. Gerger, D. Brieber, and N. Schwarz. 2014. What makes an art expert? Emotion and evaluation in art appreciation. Cognition & Emotion 28, 6 (2014), 1137--1147.

[16]

H. Leder, G. Gerger, S. G. Dressler, and A. Schabmann. 2012. How art is appreciated. Psychology of Aesthetics, Creativity, and the Arts. 6, 1 (2012), 2--10.

[17]

H. Leder, P. P. Tinio, and M. Bar. 2011. Emotional valence modulates the preference for curved objects. Perception 40, 6, 649--655 (2011).

[18]

H. Liu, Min Xu, X. He, and J. Wang. 2014. Estimate gaze density by incorporating emotion. In Proceedings of the ACM International Conference on Multimedia. 1113--1116.

Digital Library

[19]

P. Locher, E. A. Krupinski, C. Mello-Thoms, and C. F. Nodine. 2007. Visual interest in pictorial art during an aesthetic experience. Spatial Vision 21, 1--2 (2007), 55--77.

[20]

D. G. Lowe. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2 (2004), 91--110.

Digital Library

[21]

X. Lu, P. Suryanarayan, R. B. Adams Jr., J. L. M. G. Newman, and J. Z. Wang. 2012. On shape and the computability of emotions. In Proceedings of the ACM International Conference on Multimedia. 229--238.

Digital Library

[22]

J. Machajdik and A. Hanbury. 2010. Affective image classification using features inspired by psychology and art theory. In Proceedings of the ACM International Conference on Multimedia. 83--92.

Digital Library

[23]

S. Maji, A. C. Berg, and J. Malik. 2008. Classification using intersection kernel support vector machines is efficient. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'08). 1--8.

[24]

L. Marchesotti, F. Perronnin, D. Larlus, and G. Csurka. 2011. Assessing the aesthetic quality of photographs using generic image descriptors. In Proceedings of the IEEE International Conference on Computer Vision (ICCV'11). 1784--1791.

Digital Library

[25]

G. Mather. 2012. Aesthetic judgement of orientation in modern art. i-Perception 3, 1 (2012), 18--24.

[26]

I. C. McManus, K. Stöver, and D. Kim. 2011. Arnheim's Gestalt theory of visual balance: Examining the compositional structure of art photographs and abstract images. i-Perception 2 (2011), 615--647. Issue 6.

[27]

D. Melcher and F. Bacci. 2012. Chapter 10 - Perception of emotion in abstract artworks: A multidisciplinary approach. In The Fine Arts, Neurology, and Neuroscience New Discoveries and Changing Landscapes. 204. 191--216.

[28]

L. Moholy-Nagy. 1945. In defense of “Abstract” art. The Journal of Aesthetics and Art Criticism 4, 2 (1945), 74--76.

[29]

J. Moser. 2010. True Skill Library. Retrieved from https://github.com/moserware/Skills/.

[30]

M. Pelowski and F. Akiba. 2011. A model of art perception, evaluation and emotion in transformative aesthetic experience. New Ideas in Psychology 29 (2011), 80--97. Issue 2.

[31]

R. W. Picard. 1995. Affective computing. Perceptual Computing Section, Media Laboratory, Massachusetts Institute of Technology (1995).

Digital Library

[32]

E. Pihko, A. Virtanen, V. Saarinen, S. Pannasch, L. Hirvenkari, T. Tossavainen, A. Haapala, and R. Hari. 2011. Experiencing art: The influence of expertise and painting abstraction level. Frontiers in Human Neuroscience 5 (2011).

[33]

P. Reinagel and A. Zador. 1999. Natural scene statistics at the centre of gaze. Network: Computation in Neural Systems 10, 4 (1999).

[34]

J. A. Russell. 1980. A circumplex model of affect. Journal of Personality and Social Psychology 39, 6 (1980), 1161--1178.

[35]

M. Schapiro. 1937. Nature of abstract art. American Marxist Association.

[36]

N. Sebe, I. Cohen, and T. S. Huang. 2005. Multimodal emotion recognition. Handbook of Pattern Recognition and Computer Vision 4 (2005), 387--419.

[37]

P. J. Silvia. 2005. Emotional responses to art: From collation and arousal to cognition and emotion. Review of General Psychology 9, 4 (2005), 342--357.

[38]

P. J. Silvia. 2006. Artistic training and interest in visual art: Applying the appraisal model of aesthetic emotions. Empirical Studies of the Arts 24, 2 (2006), 139--161.

[39]

J. Sivic and A. Zisserman. 2003. Video Google: A text retrieval approach to object matching in videos. In Proceedings of the International Conference on Computer Vision (ICCV). 1470--1477.

Digital Library

[40]

R. Subramanian, H. Katti, N. Sebe, M. S. Kankanhalli, and C. Tat-Seng. 2010. An eye fixation database for saliency detection in images. In Proceedings of the European Conference on Computer Vision (ECCV). 30--43.

Digital Library

[41]

R. Subramanian, V. Yanulevskaya, and N. Sebe. 2011. Can computers learn from humans to see better? Inferring scene semantics from viewers' eye movements. In Proceedings of the ACM International Conference on Multimedia. 33--42.

Digital Library

[42]

R. Szeliski. 2010. Computer Vision: Algorithms and Applications. Springer Science & Business Media.

Digital Library

[43]

S. Takahashi and Y. Ejima. 2013. Contextual information processing of brain in art appreciation. Behavioral and Brain Sciences 36, 2 (2013), 158--159.

[44]

B. W. Tatler, R. J. Baddeley, and I. D. Gilchrist. 2005. Visual correlates of fixation selection: Effects of scale and time. Vision Research 45, 5 (2005), 643--659.

[45]

J. R. R. Uijlings, A.W. M. Smeulders, and R. J. H. Scha. 2010. Real-time visual concept classification. IEEE Transactions on Multimedia 12, 7 (2010), 665--681.

Digital Library

[46]

J. R. R. Uijlings, A. W. M. Smeulders, and R. J. H. Scha. 2012. The visual extent of an object. International Journal of Computer Vision 96, 1 (2012), 46--63.

Digital Library

[47]

K. E. A. van de Sande, T. Gevers, and C. G. M. Snoek. 2010. Evaluating color descriptors for object and scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 32, 9 (2010), 1582--1596.

Digital Library

[48]

A. Vedaldi and B. Fulkerson. 2010. VLFeat: An open and portable library of computer vision algorithms. In Proceedings of the ACM International Conference on Multimedia. 1469--1472.

Digital Library

[49]

L. Veronesi. 1968. Luigi Veronesi. Con una antologia di scritti. Torino.

[50]

E. Winner, P. Blank, C. Massey, and H. Gardner. 1983. Children's sensitivity to aesthetic properties of line drawings. In D. Rogers and J. A. Sloboda (Eds.), The Acquisition of Symbolic Skills. Springer, 97--104.

[51]

O. Wu, Y. Chen, B. Li, and W. Hu. 2011. Evaluating the visual quality of web pages using a computational aesthetic approach. In Proceedings of the ACM Conference on Web Search and Data Mining. 337--346.

Digital Library

[52]

V. Yanulevskaya, J. V. Gemert, K. Roth, A. Herbold, N. Sebe, and J. Geusebroek. 2008. Emotional valence categorization using holistic image features. In Proceedings of the IEEE International Conference on Image Processing. 101--104.

[53]

V. Yanulevskaya, J. B. Marsman, F. Cornelissen, and J. M. Geusebroek. 2011. An image statistics based model for fixation prediction. Cognitive Computation 3, 1 (2011), 94--104.

[54]

V. Yanulevskaya, J. Uijlings, E. Bruni, A. Sartori, E. Zamboni, F. Bacci, D. Melcher, and N. Sebe. 2012. In the eye of the beholder: Employing statistical analysis and eye tracking for analyzing abstract paintings. In Proceedings of the ACM International Conference on Multimedia. 349--358.

Digital Library

[55]

S. Zeki. 1999. Art and the brain. Journal of Consciousness Studies 6, 6--7 (1999), 76--96.

[56]

Semir Zeki and John Nash. 1999. Inner Vision: An Exploration of Art and the Brain. Vol. 415. Oxford University Press Oxford.

[57]

Q. Zhao and C. Koch. 2011. Learning a saliency map using fixated locations in natural scenes. Journal of Vision 11, 3 (2011), 9.

[58]

S. Zhao, Y. Gao, X. Jiang, H. Yao, T. Chua, and X. Sun. 2014. Exploring principles-of-art features for image emotion recognition. In Proceedings of the ACM International Conference on Multimedia. 47--56.

Digital Library

Cited By

Ruyi B(2024)A general image orientation detection method by feature fusionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02782-540:1(287-302)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00371-023-02782-5
Manju Singh SKumar S(2024)IoT-Based Smart Indoor Navigation System with Voice Assistance for MuseumsCommunication and Intelligent Systems10.1007/978-981-97-2053-8_34(449-457)Online publication date: 21-Jun-2024
https://doi.org/10.1007/978-981-97-2053-8_34
Kim SAn CCha JKim DPark E(2023)D-ViSA: A Dataset for Detecting Visual Sentiment from Art Images2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)10.1109/ICCVW60793.2023.00328(3043-3051)Online publication date: 2-Oct-2023
https://doi.org/10.1109/ICCVW60793.2023.00328
Show More Cited By

Index Terms

Affective Analysis of Professional and Amateur Abstract Paintings Using Statistical Analysis and Art Theory

Recommendations

Who's Afraid of Itten: Using the Art Theory of Color Combination to Analyze Emotions in Abstract Paintings
MM '15: Proceedings of the 23rd ACM international conference on Multimedia

Color plays an essential role in everyday life and is one of the most important visual cues in human perception. In abstract art, color is one of the essential means to convey the artist's intention and to affect the viewer emotionally. However, colors ...
Affective Analysis of Abstract Paintings Using Statistical Analysis and Art Theory
ICMI '14: Proceedings of the 16th International Conference on Multimodal Interaction

A novel approach to the emotion classification of abstract paintings is proposed. Based on a user study, we employ computer vision techniques to understand what makes an abstract artwork emotional. Our aim is to identify and quantify which are the ...
Affective image classification using features inspired by psychology and art theory
MM '10: Proceedings of the 18th ACM international conference on Multimedia

Images can affect people on an emotional level. Since the emotions that arise in the viewer of an image are highly subjective, they are rarely indexed. However there are situations when it would be helpful if images could be retrieved based on their ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Interactive Intelligent Systems

ACM Transactions on Interactive Intelligent Systems Volume 5, Issue 2

Special Issue on Behavior Understanding for Arts and Entertainment (Part 1 of 2)

July 2015

144 pages

ISSN:2160-6455

EISSN:2160-6463

DOI:10.1145/2799389

Editors:
Anthony Jameson
German Research Center for Artificial Intelligence (DFKI), Germany
,
Krzysztof Gajos
Harvard University, U.S.A.

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 June 2015

Accepted: 01 February 2015

Revised: 01 December 2014

Received: 01 April 2014

Published in TIIS Volume 5, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

FIRB S-PATTERNS projects
Italian Ministry of University and Research (MIUR) through the “Active Ageing at Home”

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

29
Total Citations
View Citations
601
Total Downloads

Downloads (Last 12 months)37
Downloads (Last 6 weeks)1

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ruyi B(2024)A general image orientation detection method by feature fusionThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-023-02782-540:1(287-302)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1007/s00371-023-02782-5
Manju Singh SKumar S(2024)IoT-Based Smart Indoor Navigation System with Voice Assistance for MuseumsCommunication and Intelligent Systems10.1007/978-981-97-2053-8_34(449-457)Online publication date: 21-Jun-2024
https://doi.org/10.1007/978-981-97-2053-8_34
Kim SAn CCha JKim DPark E(2023)D-ViSA: A Dataset for Detecting Visual Sentiment from Art Images2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)10.1109/ICCVW60793.2023.00328(3043-3051)Online publication date: 2-Oct-2023
https://doi.org/10.1109/ICCVW60793.2023.00328
Huang JQing LHan LLiao JGuo LPeng Y(2023)A collaborative perception method of human-urban environment based on machine learning and its application to the case areaEngineering Applications of Artificial Intelligence10.1016/j.engappai.2022.105746119(105746)Online publication date: Mar-2023
https://doi.org/10.1016/j.engappai.2022.105746
Sun MGong XNie HIqbal MXie B(2022)SRAFE: Siamese Regression Aesthetic Fusion Evaluation for Chinese Calligraphic CopyCAAI Transactions on Intelligence Technology10.1049/cit2.12095Online publication date: 6-May-2022
https://doi.org/10.1049/cit2.12095
Sugii KNagai TKayama M(2022)A Basic Study on Educational Growth Indicators Based on Quantitative Evaluation of Strokes Quality in Drawing WorksArtificial Intelligence in Education10.1007/978-3-031-11644-5_40(490-501)Online publication date: 27-Jul-2022
https://dl.acm.org/doi/10.1007/978-3-031-11644-5_40
Ji HQing LHan LWang ZCheng YPeng Y(2021)A New Data-Enabled Intelligence Framework for Evaluating Urban Space PerceptionISPRS International Journal of Geo-Information10.3390/ijgi1006040010:6(400)Online publication date: 9-Jun-2021
https://doi.org/10.3390/ijgi10060400
Gang LWeishang G(2021)The Effectiveness of Pictorial Aesthetics Based on Multiview Parallel Neural Networks in Art-Oriented TeachingComputational Intelligence and Neuroscience10.1155/2021/37351042021Online publication date: 1-Jan-2021
https://dl.acm.org/doi/10.1155/2021/3735104
Zhang JMiao YYu J(2021)A Comprehensive Survey on Computational Aesthetic Evaluation of Visual Art Images: Metrics and ChallengesIEEE Access10.1109/ACCESS.2021.30830759(77164-77187)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3083075
Rollins J(2021)References‘Purpose-built’ Art in Hospitals: Art with Intent10.1108/978-1-83909-680-820211022(205-227)Online publication date: 18-May-2021
https://doi.org/10.1108/978-1-83909-680-820211022
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents