research-article

System for matching paintings with music based on emotions

Authors:

Kyunghyun YoonAuthors Info & Claims

SA '16: SIGGRAPH ASIA 2016 Technical Briefs

Article No.: 31, Pages 1 - 4

https://doi.org/10.1145/3005358.3005366

Published: 28 November 2016 Publication History

Abstract

People experience various emotions when they interact with artistic content such as music and visual art in the form of paintings. Thus, painters and composers use features in music and paintings to influence people emotionally. An analysis of methods employed to create features to influence people using paintings and music indicated that people apparently do not find it difficult to understand artistic content. When people view paintings, listening to music that creates a mood similar to that portrayed by the paintings could be helpful to understand the painter's intention.

In this work, we extract the emotions from music and paintings depending on their features. Based on these extracted emotions, the proposed system suggests the most appropriate music to accompany a given image, and vice versa. In addition, based on our algorithm, we developed a mobile application that could assist people to enjoy music and paintings emotionally.

Supplementary Material

ZIP File (a31-lee.zip)

Supplemental files.

Download
3.69 MB

References

[1]

Kim, Y. E., Schmidt, E. M., Migneco, R., Morton, B. G., Richardson, P., Scott, J., Speck, J. A., and Turnbull, D. 2010. Music emotion recognition: A state of the art review, In Proceedings of the International Conference on Music Information Retrieval, 255--266.

[2]

Kobayashi, S. 1981, The aim and method of the color image scale, Color research and application, 6(2), 93--107.

[3]

Kobayashi S. 1991, Color Image Scale, Kosdansha International.

[4]

Lartillot, O. and Toiviainen, P. 2007. A Matlab toolbox for musical feature extraction from audio, In Proceedings of the International Conference on Digital Audio Effects, 237--244.

[5]

Lidy, T. and Rauber, A. 2005, Evaluation of feature extractors and psycho-acoustic transformations for music genre classification, In Proceedings of the International Conference on Music Information Retrieval, 34--41.

[6]

Mckay, C., Fujinaga, I., and Depalle, P. 2005. jAudio: A feature extraction library, In Proceedings of the International Conference on Music Information Retrieval, 600--603.

[7]

Russell, J., Noble, B., and Sneddon, I. N. 1980 A Circumplex Model of Affect, Journal of Personality and Social Psychology, 1161--1178.

[8]

Sasaki, S., Hirai, T., Ohya, H., and Morishima, S. 2013. Affective music recommendation system reflecting the mood of input image, In Proceedings of the International Conference on Culture and Computing, 153--154.

Digital Library

[9]

Shim, H. O., Kang, D. W., and Yoon, K. H., 2015. Extracting emotional adjectives from paintings using color combinations, In Proceedings of the Computer Graphics International.

[10]

Tzanetakis, G. and Cook, P. 2000. Marsyas: A framework for audio analysis, Organised Sound, 4, 3, 169--175.

Digital Library

[11]

Wagner, J., Kim, J., and Andre, E. 2005. From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification, IEEE International Conference on Multimedia and Expo, 940--943.

[12]

Warriner, A. B., Kuperman, V., and Brysbaert, M. 2013. Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods., 1--17

[13]

Yang, Y.-H., Lin, Y.-H., Su, Y.-F., and Chen, H. H. 2008, A regression approach to music emotion recognition, IEEE Transactions on Audio, Speech, and Language Processing, 16, 2, 448--457

Digital Library

[14]

Yanulevskaya, V., Van Germert, J. C., Roth, K. Herbold, A. K., Sebe, N., and Geusebroek, J. M., 2008. Emotional valence categorization using holistic image features. In IEEE International Conference on Image Processing.

[15]

Zujovic, J., Gandy, l., Friedman, S., Pardo, B., and Pappas, T. N., 2009. Classifying paintings by artistic genre: An analysis of features and classifiers. Multimedia Signal Processing, 2009, MMSP'09 IEEE International Workshop on.

Cited By

Urakawa KWang Y(2024)Enhancing Cooking Recipe Browsing Experience through Integration of Recipes and Music2024 International Conference on Future Technologies for Smart Society (ICFTSS)10.1109/ICFTSS61109.2024.10690273(117-122)Online publication date: 7-Aug-2024
https://doi.org/10.1109/ICFTSS61109.2024.10690273
Miyamoto TWang Y(2023)Generating Appreciation Contents of Artworks by Combining Images and Music2023 IEEE 12th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE59613.2023.10315309(952-956)Online publication date: 10-Oct-2023
https://doi.org/10.1109/GCCE59613.2023.10315309
Lee NLee T(2021)The Study for Emotion Analysis from Images depend on Machine Learning ModelsJournal of Digital Contents Society10.9728/dcs.2021.22.8.129522:8(1295-1302)Online publication date: 31-Aug-2021
https://doi.org/10.9728/dcs.2021.22.8.1295
Show More Cited By

Index Terms

System for matching paintings with music based on emotions
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        Computational photography
  2. Computer graphics
    1. Image manipulation

Recommendations

Acoustic Features for Music Emotion Recognition and System Building
ICIT '17: Proceedings of the 2017 International Conference on Information Technology

We are faced with a massive growth of musical data in the form of digital files. Accurate metadata labeling of music archives is necessary in order to make digital music searchable and to be efficiently organized, not only by file name or song title but ...
The Role of Time in Music Emotion Recognition: Modeling Musical Emotions from Time-Varying Music Features
CMMR 2012: Revised Selected Papers of the 9th International Symposium on From Sounds to Music and Emotions - Volume 7900

Music is widely perceived as expressive of emotion. However, there is no consensus on which factors in music contribute to the expression of emotions, making it difficult to find robust objective predictors for music emotion recognition MER. Currently, ...
Transformer-Based Approach Towards Music Emotion Recognition from Lyrics
Advances in Information Retrieval
Abstract
The task of identifying emotions from a given music track has been an active pursuit in the Music Information Retrieval (MIR) community for years. Music emotion recognition has typically relied on acoustic features, social tags, and other metadata ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SA '16: SIGGRAPH ASIA 2016 Technical Briefs

November 2016

124 pages

ISBN:9781450345415

DOI:10.1145/3005358

Conference Chairs:
Johannes Kopf
Facebook
,
Chi-Wing (Philip) Fu
The Chinese University of Hong Kong, Hong Kong

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SA '16

Sponsor:

SIGGRAPH

SA '16: SIGGRAPH Asia 2016

December 5 - 8, 2016

Macau

Acceptance Rates

Overall Acceptance Rate 178 of 869 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
361
Total Downloads

Downloads (Last 12 months)11
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Urakawa KWang Y(2024)Enhancing Cooking Recipe Browsing Experience through Integration of Recipes and Music2024 International Conference on Future Technologies for Smart Society (ICFTSS)10.1109/ICFTSS61109.2024.10690273(117-122)Online publication date: 7-Aug-2024
https://doi.org/10.1109/ICFTSS61109.2024.10690273
Miyamoto TWang Y(2023)Generating Appreciation Contents of Artworks by Combining Images and Music2023 IEEE 12th Global Conference on Consumer Electronics (GCCE)10.1109/GCCE59613.2023.10315309(952-956)Online publication date: 10-Oct-2023
https://doi.org/10.1109/GCCE59613.2023.10315309
Lee NLee T(2021)The Study for Emotion Analysis from Images depend on Machine Learning ModelsJournal of Digital Contents Society10.9728/dcs.2021.22.8.129522:8(1295-1302)Online publication date: 31-Aug-2021
https://doi.org/10.9728/dcs.2021.22.8.1295
Bramah OCheng XMorreale F(2021)The Singing GalleryProceedings of the 16th International Audio Mostly Conference10.1145/3478384.3478396(117-120)Online publication date: 1-Sep-2021
https://dl.acm.org/doi/10.1145/3478384.3478396
Zhao SLi YYao XNie WXu PYang JKeutzer KWen Chen CCucchiara RHua XQi GRicci EZhang ZZimmermann R(2020)Emotion-Based End-to-End Matching Between Image and Music in Valence-Arousal SpaceProceedings of the 28th ACM International Conference on Multimedia10.1145/3394171.3413776(2945-2954)Online publication date: 12-Oct-2020
https://dl.acm.org/doi/10.1145/3394171.3413776
Lee TLee NSeo SKang D(2020)A Study on the Prediction of Emotion from Image by Time-flow depend on Color Analysis2020 International Conference on Computational Science and Computational Intelligence (CSCI)10.1109/CSCI51800.2020.00141(747-749)Online publication date: Dec-2020
https://doi.org/10.1109/CSCI51800.2020.00141
Lee JChoi JSeo S(2020)Emotion-Inspired Painterly RenderingIEEE Access10.1109/ACCESS.2020.29994608(104565-104578)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.2999460
Lee TSeo S(2019)Painting-Emotion Matching Technology Learning System through RepetitionSustainability10.3390/su1116450711:16(4507)Online publication date: 20-Aug-2019
https://doi.org/10.3390/su11164507
Ranjgar BKhoshlahjeh Azar MSadeghi-Niaraki AChoi S(2019)A Novel Method for Emotion Extraction From Paintings Based on Luscher’s Psychological Color Test: Case Study Iranian-Islamic PaintingsIEEE Access10.1109/ACCESS.2019.29368967(120857-120871)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2936896
Kang DSeo S(2019)Personalized smart home audio system with automatic music selection based on emotionMultimedia Tools and Applications10.1007/s11042-018-6733-778:3(3267-3276)Online publication date: 1-Feb-2019
https://dl.acm.org/doi/10.1007/s11042-018-6733-7

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents