research-article

Machine Recognition of Music Emotion: A Review

Authors:

Homer H. ChenAuthors Info & Claims

ACM Transactions on Intelligent Systems and Technology (TIST), Volume 3, Issue 3

Article No.: 40, Pages 1 - 30

https://doi.org/10.1145/2168752.2168754

Published: 01 May 2012 Publication History

Abstract

The proliferation of MP3 players and the exploding amount of digital music content call for novel ways of music organization and retrieval to meet the ever-increasing demand for easy and effective information access. As almost every music piece is created to convey emotion, music organization and retrieval by emotion is a reasonable way of accessing music information. A good deal of effort has been made in the music information retrieval community to train a machine to automatically recognize the emotion of a music signal. A central issue of machine recognition of music emotion is the conceptualization of emotion and the associated emotion taxonomy. Different viewpoints on this issue have led to the proposal of different ways of emotion annotation, model training, and result visualization. This article provides a comprehensive review of the methods that have been proposed for music emotion recognition. Moreover, as music emotion recognition is still in its infancy, there are many open issues. We review the solutions that have been proposed to address these issues and conclude with suggestions for further research.

References

[1]

Ali, S. O. and Peynircioǧu;, Z. F. 2006. Songs and emotions: Are lyrics and melodies equal partners. Psychol. Music 34, 4, 511--534.

[2]

Allamanche, E., Herre, J., Helmuth, O., Fröba, B., Kasten, T., and Cremer, M. 2001. Content-based identification of audio material using MPEG-7 low level description. In Proceedings of the International Conference on Music on Information Retrieval. 197--204.

[3]

Anderson, K. and McOwan, P. W. 2006. A real-time automated system for the recognition of human facial expressions. IEEE Trans. Syst. Man Cyber. 36, 1, 96--105.

Digital Library

[4]

Arifin, S. and Cheung, P. Y. K. 2008. Affective level video segmentation by utilizing the pleasure-arousal-dominance information. IEEE Trans. Multimedia 10, 7, 1325--1341.

Digital Library

[5]

Benetos, E., Kotti, M., and Kotropoulos, C. 2007. Large scale musical instrument identification. In Proceedings of the International Conference on Music Information Retrieval. http://www.ifs.tuwien.ac.at/mir/muscle/del/audio_tools.html#SoundDescrToolbox.

[6]

Bigand, E., Vieillard, S., Madurell, F., Marozeau, J., and Dacquet, A. 2005. Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts. Cognition Emotion 19, 8, 1113--1139.

[7]

Bischoff, K., Firan, C. S., Paiu, R., Nejdl, W., Laurier, C., and Sordo, M. 2009. Music mood and theme classification a hybrid approach. In Proceedings of the International Conference on Music Information Retrieval. 657--662.

[8]

Boyd, S. P. and Vandenberghe, L. 2004. Convex Optimization. Cambridge University Press, Cambridge, UK.

Digital Library

[9]

Cabrera, D. 1999. Psysound: A computer program for psycho-acoustical analysis. In Proceedings of the Australian Acoustic Society Conference. 47--54. http://psysound.wikidot.com/.

[10]

Cai, R., Zhang, C., Wang, C., Zhang, L., and Ma, W.-Y. 2007. MusicSense: Contextual music recommendation using emotional allocation modeling. In Proceedings of the ACM International Conference on Multimedia. 553--556.

Digital Library

[11]

Camacho, A. 2007. SWIPE: A sawtooth waveform inspired pitch estimator for speech and music. Ph.D. dissertation, University of Florida.

Digital Library

[12]

Campbell, W. M., Campbell, J. P., Reynolds, D. A., Singer, E., and Torres-Carrasquillo, P. A. 2006. Support vector machines for speaker and language recognition. Comput. Speech Lang. 20, 2--3, 210--229.

[13]

Cao, C. and Li, M. 2009. Thinkit’s submissions for MIREX2009 audio music classification and similarity tasks. In Proceedings of the International Conference on Music Information Retreival.

[14]

Casey, M. A., Veltkamp, R., Goto, M., Leman, M., Rhodes, C., and Slaney, M. 2008. Content-based music information retrieval: Current directions and future challenges. IEEE 96, 4, 668--696.

[15]

Chang, C.-C. and Lin, C.-J. 2001. LIBSVM: A library for support vector machines. http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Digital Library

[16]

Chen, C.-H., Weng, M.-F., Jeng, S.-K., and Chuang, Y.-Y. 2008. Emotion-based music visualization using photos. In Proceedings of the International Conference on Multimedia Modeling. 358--368.

Digital Library

[17]

Chen, J.-C., Chu, W.-T., Kuo, J.-H., Weng, C.-Y., and Wu, J.-L. 2006. Tiling slideshows. In Proceedings of the ACM International Conference on Multimedia. 25--34.

Digital Library

[18]

Cheng, H.-T., Yang, Y.-H., Lin, Y.-C., and Chen, H.-H. 2009. Multimodal structure segmentation and analysis of music using audio and textual information. In Proceedings of the IEEE International Symposium on Circuits and Systems. 1677--1680.

[19]

Cheng, H.-T., Yang, Y.-H., Lin, Y.-C., Liao, I.-B., and Chen, H.-H. 2008. Automatic chord recognition for music classification and retrieval. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1505--1508.

[20]

Cohen, R. and Swerdlik, M. 2002. Psychological Testing and Assessment: An Introduction to Tests and Measurement. Mayfield Publishing Company, Mountain View, CA.

[21]

Cohen, W. W., Schapire, R. E., and Singer, Y. 1999. Learning to order things. J. Artificial Intell.Res. 10, 243--270.

Digital Library

[22]

Collier, G. 2007. Beyond valence and activity in the emotional connotations of music. Psychol. Music 35, 1, 110--131.

[23]

Cortes, C. and Vapnik, V. 1995. Support vector networks. Machine Learn. 20, 3, 273--297.

Digital Library

[24]

Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., and Schröer, M. 2000. Feeltrace: An instrument for recording perceived emotion in real time. In Proceedings of the ISCA Tutorial and Research Workshop on Speech and Emotion. 19--24.

[25]

Davis, S. and Mermelstein, P. 1980. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoustics, Speech and Signal Processing 28, 4, 357--366.

[26]

Dietz, R. and Lang, A. 1999. Affective agents: Effects of agent affect on arousal, attention, liking and learning. In Proceedings of the International Conference on Cognitive Technology.

[27]

Dornbush, S., Fisher, K., Mckay, K., Prikhodko, A., and Segall, Z. 2005. XPOD: A human activity and emotion aware mobile music player. In Proceedings of the International Conference on Mobile Technology, Applications and Systems. 1--6.

[28]

Duda, R. O., Hart, P. E., and Stork, D. G. 2000. Pattern Classification. John Wiley & Sons, Inc., New York.

Digital Library

[29]

Dunker, P., Nowak, S., Begau, A., and Lanz, C. 2008. Content-based mood classification for photos and music. In Proceedings of the International Conference on Multimedia Information Retrieval. 97--104.

Digital Library

[30]

Eerola, T., Toiviainen, P., and Krumhansl, C. L. 2002. Real-time prediction of melodies: Continuous predictability judgments and dynamic models. In Proceedings of the International Conference on Music Perception and Cognition. 473--476.

[31]

Eerola, T., Lartillot, O., and Toiviainen, P. 2009. Prediction of multidimensional emotional ratings in music from audio using multivariate regression models. In Proceedings of the International Conference on Music Information Retrieval. 621--626.

[32]

Ekman, P. 1992. An argument for basic emotions. Cognition Emotion 6, 3, 169--200.

[33]

Farnsworth, P. R. 1954. A study of the Hevner adjective list. J. Aesthetics Art Criticism 13, 97--103.

[34]

Feng, Y., Zhuang, Y., and Pan, Y. 2003. Popular music retrieval by detecting mood. In Proceedings of the International Conference on Information Retrieval. 375--376.

Digital Library

[35]

Fernandez, R. and Picard, R. W. 2005. Classical and novel discriminant features for affect recognition from speech. In Proceedings of the INTERSPEECH Conference.

[36]

Fornari, J. and Eerola, T. 2008. The pursuit of happiness in music: Retrieving valence with high-level musical descriptors. In Proceedings of the Computer Music Modeling and Retrieval.

[37]

Fornäs, J. 2006. Songs and emotions: Are lyrics and melodies equal partners. Psychol. Music 34, 4, 511--534.

[38]

Fujihara, H. and Goto, M. 2007. A music information retrieval system based on singing voice timbre. In Proceedings of the International Conference on Music Information Retrieval.

[39]

Fujihara, H., Kitahara, T., Goto, M., Komatani, K., Ogata, T., and Okuno, H. G. 2005. Singer identification based on accompaniment sound reduction and reliable frame selection. In Proceedings of the International Conference on Music Information Retrieval.

[40]

Gabrielsson, A. 2002. Emotion perceived and emotion felt: Same or different? Musicae Scientiae (special issue), 123--147.

[41]

Gabrielsson, A. and Lindström, E. 2001. The influence of musical structure on emotional expression. In Music and Emotion: Theory and Research, P. N. Juslin and J. A. Sloboda Eds., Oxford University Press, Oxford, UK.

[42]

Giannakopoulos, T., Pikrakis, A., and Theodoridis, S. 2009. A dimensional approach to emotion recognition of speech from movies. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 65--68.

Digital Library

[43]

Gómez, E. 2006. Tonal description of music audio signal. Ph.D. dissertation, Universitat Pompeu Fabra, Barcelona.

[44]

Goto, M. 2004. A real-time music-scene-description system: Predominant-f0 estimation for detecting melody and bass lines in real-world audio signals. Speech Communication 43, 311--329.

[45]

Goto, M., Hashiguchi, H., Nishimura, T., and Oka, R. 2003. RWC music database: Music genre database and musical instrument sound database. In Proceedings of the International Conference on Music Information Retrieval. 229--230.

[46]

Grubbs, F. E. 1969. Procedures for detecting outlying observations in samples. Technometrics 11, 1, 1--21.

[47]

Han, B.-J., Rho, S., Dannenberg, R. B., and Hwang, E. 2009. SMERS: Music emotion recognition using support vector regression. In Proceedings of the International Conference on Music Information Retrieval. 651--656.

[48]

Hanjalic, A. and Xu, L.-Q. 2005. Affective video content representation and modeling. IEEE Trans. Multimedia 7, 1, 143--154.

Digital Library

[49]

Hargreaves, D. J. 1986. The Developmental Psychology of Music. Cambridge University Press, Cambridge, UK.

[50]

Hevner, K. 1935. Expression in music: A discussion of experimental studies and theories. Psychol. Review 48, 2, 186--204.

[51]

Hofmann, T. 1999. Probabilistic latent semantic indexing. In Proceedings of the ACM International Conference on Information Retrieval. 50--57.

Digital Library

[52]

Hsu, D. C.-W. and Hsu, J. Y.-J. 2006. LyQ: An emotion-aware music player. In Proceedings of the AAAI Workshop on Computational Aesthetics: Artificial Intelligence Approaches to Beauty and Happiness.

[53]

Hu, X. and Downie, J. S. 2007. Exploring mood metadata: Relationships with genre, artist and usage metadata. In Proceedings of the International Conference on Music Information Retrieval.

[54]

Hu, X., Downie, J. S., Laurier, C., Bay, M., and Ehmann, A. F. 2008. The 2007 MIREX audio mood classification task: Lessons learned. In Proceedings of the International Conference on Music Information Retrieval. 462--467.

[55]

Hu, X., Sanghvi, V., Vong, B., On, P. J., Leong, C., and Angelica, J. 2008. Moody: A web-based music mood classification and recommendation system. In Proceedings of the International Conference on Music Information Retrieval.

[56]

Hu, X., Downie, J. S., and Ehmann, A. F. 2009. Lyric text mining in music mood classification. In Proceedings of the International Conference on Music Information Retrieval.

[57]

Hu, Y., Chen, X., and Yang, D. 2009. Lyric-based song emotion detection with affective lexicon and fuzzy clustering method. In Proceedings of the International Conference on Music Information Retrieval.

[58]

Huq, A., Bello, J. P., Sarroff, A., Berger, J., and Rowe, R. 2009. Sourcetone: An automated music emotion recognition system. In Proceedings of the International Conference on Music Information Retrieval.

[59]

Huq, A., Bello, J. P., and Rowe, R. 2010. Automated music emotion recognition: A systematic evaluation. J. New Music Res. 39, 3, 227--244.

[60]

Huron, D. 2000. Perceptual and cognitive applications in music information retrieval. In Proceedings of the International Conference on Music Information Retrieval.

[61]

Huron, D. 2006. Sweet Anticipation: Music and the Psychology of Expectation. MIT Press, Cambridge, MA.

[62]

Jaimes, A. and Sebe, N. 2005. Multimodal human computer interaction: A survey. In Proceedings of the IEEE International Workshop on HCI in Conjuction with the Computer Vision. 1--15.

Digital Library

[63]

Jaimes, A., Sebe, N., and Gatica-Perez, D. 2006. Human-centered computing: A multimedia perspective. In Proceedings of the ACM International Conference on Multimedia. 855--864.

Digital Library

[64]

Jargreaves, D. J. and North, A. C. 1997. The Social Psychology of Music. Oxford University Press, Oxford, UK.

[65]

Jiang, D. N., Lu, L., Zhang, H. J., Tao, J. H., and Cai, L. H. 2002. Music type classification by spectral contrast features. In Proceedings of the IEEE International Conference on Multimedia Expo. 113--116.

[66]

Jonghwa, K. and Ande, E. 2008. Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Machine Intell. 30, 12, 2067--2083.

Digital Library

[67]

Juslin, P. N. 2000. Cue utilization in communication of emotion in music performance: Relating performance to perception. J. Exp. Psychol.: Human Percep. Perform. 16, 6, 1797--1813.

[68]

Juslin, P. N. and Laukka, P. 2004. Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. J. New Music Res. 33, 3, 217--238.

[69]

Juslin, P. N. and Sloboda, J. A. 2001. Music and Emotion: Theory and Research. Oxford University Press, Oxford, UK.

[70]

Karl F. MacDorman, S. O. and Ho, C.-C. 2007. Automatic emotion prediction of song excerpts: Index construction, algorithm design, and empirical comparison. J. New Music Res. 36, 4, 281--299.

[71]

Katayose, H., Imai, M., and Inokuchi, S. 1998. Sentiment extraction in music. In Proceedings of the International Conference on Pattern Recognition. 1083--1087.

[72]

Kim, Y. E., Schmidt, E., and Emelle, L. 2008. Moodswings: A collaborative game for music mood label collection. In Proceedings of the International Conference on Music Information Retrieval. 231--236.

[73]

Kim, Y. E., Schmidt, E. M., Migneco, R., Morton, B. G., Richardson, P., Scott, J., Speck, J. A., and Turnbull, D. 2010. Music emotion recognition: A state of the art review. In Proceedings of the International Conference on Music Information Retrieval.

[74]

Klapuri, A. 1999. Sound onset detection by applying psychoacoustic knowledge. In Proceedings of the International Conference on Acoustics, Speech, and Signal Processing. 3089--3092.

Digital Library

[75]

Korhonen, M. D., Clausi, D. A., and Jernigan, M. E. 2006. Modeling emotional content of music using system identification. IEEE Trans. Syst. Man Cyber. 36, 3, 588--599.

Digital Library

[76]

Krumhansl, C. 2002. Music: A link between cognition and emotion. Current Directions Psychol. Sci. 11, 2, 45--50.

[77]

Laar, B. 2006. Emotion detection in music, a survey. In Proceedings of the Twente Student Conference on IT.

[78]

Lagrange, M., Martins, L., Murdoch, J., and Tzanetakis, G. 2008. Normalized cuts for predominant melodic source separation. IEEE Trans. Audio, Speech, Lang. Process. 16, 2, 278--290.

Digital Library

[79]

Lamere, P. 2008. Social tagging and music information retrieval. J. New Music Res. 37, 2, 101--114.

[80]

Lang, P. J. 1995. The emotion probe. Amer. Psychol. 50, 5, 372--290.

[81]

Lartillot, O. and Toiviainen, P. 2007. MIR in Matlab (II): A toolbox for musical feature extraction from audio. In Proceedings of the International Conference on Music Information Retrieval. 127--130. http://users.jyu.fi/~lartillo/mirtoolbox/.

[82]

Laurier, C. and Herrera, P. 2007. Audio music mood classification using support vector machine. In Proceedings of the International Conference on Music Information Retrieval.

[83]

Laurier, C. and Herrera, P. 2008. Mood cloud: A real-time music mood visualization tool. In Proceedings of the Computer Music Modeling and Retrieval.

[84]

Laurier, C., Sordo, M., Serrà, J., and Herrera, P. 2004. Digital music interaction concepts: A user study. In Proceedings of the International Conference on Music Information Retrieval. 415--420.

[85]

Laurier, C., Grivolla, J., and Herrera, P. 2008. Multimodal music mood classification using audio and lyrics. In Proceedings of the International Conference on Machine Learning and Applications. 105--111.

Digital Library

[86]

Laurier, C., Sordo, M., and Herrera, P. 2009. Mood cloud 2.0: Music mood browsing based on social networks. In Proceedings of the International Conference on Music Information Retrieval.

[87]

Laurier, C., Sordo, M., Serrà, J., and Herrera, P. 2009. Music mood representations from social tags. In Proceedings of the International Conference on Music Information Retrieval. 381--386.

[88]

Law, E. L. M., von Ahn, L., Dannenberg, R. B., and Crawford, M. 2007. TagATune: A game for music and sound annotation. In Proceedings of the International Conference on Music Information Retrieval.

[89]

Lazarus, R. S. 1991. Emotion and Adaptation. Oxford University Press, Oxford, UK.

[90]

Lee, J. H. and Downie, J. S. 2004. Survey of music information needs, uses, and seeking behaviours: Preliminary findings. In Proceedings of the International Conference on Music Information Retrieval. 441--446.

[91]

Lee, C.-M. and Narayanan, S. S. 2005. Toward detecting emotions in spoken dialogs. IEEE Trans. Speech Audio Process. 13, 2, 293--303.

[92]

Leman, M., Vermeulen, V., Voogdt, L. D., Moelants, D., and Lesaffre, M. 2005. Prediction of musical affect using a combination of acoustic structural cues. J. New Music Res. 34, 1, 39--67.

[93]

Levy, M. and Sandler, M. 2007. A semantic space for music derived from social tags. In Proceedings of the International Conference on Music Information Retrieval. 411--416.

[94]

Lew, M., Sebe, N., Djeraba, C., and Jain, R. 2006. Content-based multimedia information retrieval: State-of-the-art and challenges. ACM Trans. Multimedia Comput. Comm. Appl. 2, 1--19.

Digital Library

[95]

Lewis, D. D., Yang, Y., Rose, T. G., and Li, F. 2004. RCV1: A new benchmark collection for text categorization research. J. Mach. Learn. Res. 5, 361--397.

Digital Library

[96]

Li, T. and Ogihara, M. 2003. Detecting emotion in music. In Proceedings of the International Conference on Music Information Retrieval. 239--240.

[97]

Li, T. and Ogihara, M. 2004. Content-based music similarity search and emotion detection. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 17--21.

[98]

Lin, Y.-P., Wang, C.-H., Wu, T.-L., Jeng, S.-K., and Chen, J.-H. 2008. Support vector machine for EEG signal classification during listening to emotional music. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing. 127--130.

[99]

Lin, Y.-C., Yang, Y.-H., and Chen, H.-H. 2009. Exploiting genre for music emotion classification. In Proceedings of the IEEE International Conference on Multimedia and Expo. 618--621.

Digital Library

[100]

Lin, Y.-P., Jung, T.-P., and Chen, J.-H. 2009. EEG dynamics during music appreciation. In Proceedngs of the IEEE International Conference on Engineering in Medicine and Biology Society.

[101]

Lin, Y.-P., Wang, C.-H., Wu, T.-L., Jeng, S.-K., and Chen, J.-H. 2009. EEG-based emotion recognition in music listening: A comparison of schemes for multiclass support vector machine. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing. 489--492.

Digital Library

[102]

Liu, D., Lu, L., and Zhang, H.-J. 2003. Automatic music mood detection from acoustic music data. In Proceedings of the International Conference on Music Information Retrieval. 81--87.

[103]

Liu, C. C., Yang, Y.-H., Wu, P.-H., and Chen, H. H. 2006. Detecting and classifying emotion in popular music. In Proceedings of the Joint International Conference on Information Sciences. 996--999.

[104]

Livingstone, S. R. and Brown, A. R. 2005. Dynamic response: A real-time adaptation for music emotion. In Proceedings of the Australasian Conference on Interactive Entertainment. 105--111.

Digital Library

[105]

Ljung, L. 1999. System Identification: Theory for the User. Prentice-Hall, Upper Saddle River, NJ.

Digital Library

[106]

Lu, L., Liu, D., and Zhang, H. 2006. Automatic mood detection and tracking of music audio signals. IEEE Trans. Audio, Speech Lang. Process. 14, 1, 5--18.

Digital Library

[107]

Lu, Q., Chen, X., Yang, D., and Wang, J. 2010. Boosting for multi-modal music emotion classification. In Proceedings of the International Conference on Music Information Retrieval.

[108]

MacDorman, K. F., Ough, S., and Ho, C.-C. 2007. Automatic emotion prediction of song excerpts: Index construction, algorithm design, and empirical comparison. J. New Music Res. 36, 4, 281--299.

[109]

Maddage, N. C., Xu, C., Kankanhalli, M. S., and Shao, X. 2004. Content-based music structure analysis with applications to music semantics understanding. In Proceedings of the ACM International Conference on Multimedia. 112--119.

Digital Library

[110]

Mandel, M. I. and Ellis, D. P. W. 2007. A web-based game for collecting music metadata. In Proceedings of the International Conference on Music Information Retrieval.

[111]

McKay, C., McEnnis, D., and Fujinaga, I. 2006. A large publicly accessible prototype audio database for music research. In Proceedings of the International Conference on Music Information Retrieval. 160--163.

[112]

Meyers, O. C. 2007. A mood-based music classification and exploration system. M.S. thesis, Massachusetts Institute of Technology.

[113]

Montgomery, D. C., Runger, G. C., and Hubele, N. F. 1998. Engineering Statistics. Wiley, New York, NY.

[114]

Morton, B. G., Speck, J. A., Schmidt, E. M., and Kim, Y. E. 2010. Improving music emotion labeling using human computation. In Proceedings of the ACM SIGKDD Workshop on Human Computation. 45--48.

Digital Library

[115]

Nielsen, F. V. 1986. Musical ‘tension’ and related concepts. In Semiotic Web, 491--513.

[116]

Nwe, T. L. and Li, H. 2007a. Exploring vibrato-motivated acoustic features for singer identification. IEEE Trans. Audio, Speech, Lang. Process. 15, 2, 519--530.

Digital Library

[117]

Nwe, T. L. and Li, H. 2007b. Singing voice detection using perceptually-motivated features. In Proceedings of the ACM International Conference on Multimedia. 309--312.

Digital Library

[118]

Osgood, C. E., Suci, G. J., and Tannenbaum, P. H. 1957. The Measurement of Meaning. University of Illinois Press, Urbana, IL.

[119]

Ovadia, S. 2004. Ratings and rankings: Reconsidering the structure of values and their measurement. Int. J. Social Res. Method. 7, 5, 403--414.

[120]

Pampalk, E. 2004. A Matlab toolbox to compute music similarity from audio. In Proceedings of the International Conference on Music Information Retrieval. http://www.ofai.at/~elias.pampalk/ma/.

[121]

Peeters, G. 2008. A generic training and classification system for MIREX08 classification tasks: Audio music mood, audio genre, audio artist and audio tag. In Proceedings of the International Conference on Music Information Retrieval.

[122]

Picard, R. W., Vyzas, E., and Healey, J. 2001. Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23, 10, 1175--1191.

Digital Library

[123]

Platt, J. C. 1999. Probabilities for Support Vector Machines. MIT Press, Cambridge, MA.

[124]

Plutchik, R. 1980. Emotion: A Psychoevolutionary Synthesis. Harper & Row, New York, NY.

[125]

Rasmussen, C. E. and Williams, C. K. I. 2006. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA. http://www.gaussianprocess.org/gpml/.

Digital Library

[126]

Reddy, S. and Mascia, J. 2006. Lifetrak: Music in tune with your life. In Proceedings of the Human-Centered Multimedia. 25--34.

Digital Library

[127]

Remington, N. A., Fabrigar, L. R., and Visser, P. S. 2000. Reexamining the circumplex model of affect. J. Personality Social Psychol. 79, 286--300.

[128]

Repp, B. H. 1998. A microcosm of musical expression. i. quantitative analysis of pianists’ timing in the initial measures of chopin’s etude in e major. J. Acoustic. Soc. Amer. 104, 1085--1100.

[129]

Rigg, M. G. 1964. The mood effects of music: A comparison of data from four investigators. J. Psychol. 58, 427--438.

[130]

Ross, R. T. 1938. A statistics for circular scales. J. Edu. Psychol. 29, 384--389.

[131]

Russell, J. A. 1980. A circumplex model of affect. J. Personal. Social Psychol. 39, 6, 1161--1178.

[132]

Russell, J. A. 2003. Core affect and the psychological construction of emotion. Psychol. Review 110, 1, 145--172.

[133]

Russell, J. A., Weiss, A., and G. A, M. 1989. Affect grid: A single-item scale of pleasure and arousal. J. Personal. Social Psychol. 57, 3, 493--502.

[134]

Scaringella, N., Zoia, G., and Mlynek, D. 2006. Automatic genre classification of music content: A survey. IEEE Signal Process. Mag. 23, 2, 133--141.

[135]

Schmidt, E. M. and Kim, Y. E. 2009. Projection of acoustic features to continuous valence-arousal mood. In Proceedings of the International Conference on Music Information Retrieval.

[136]

Schmidt, E. M., Turnbull, D., and Kim, Y. E. 2010. Feature selection for content-based, time-varying musical emotion regression. In Proceedings of the ACM International Conference on Multimedia Information Retrieval. 267--274.

Digital Library

[137]

Schölkopf, B., Smola, A. J., Williamson, R. C., and Bartlett, P. L. 2000. New support vector algorithms. Neural Comput. 12, 1207--1245.

Digital Library

[138]

Schubert, E. 1999. Measurement and time series analysis of emotion in music. Ph.D. dissertion, School of Music Education, University of New South Wales, Sydney, Australia.

[139]

Schubert, E. 2001. Correlation analysis of continuous response to music: Correcting for the effects of serial correlation. Musicae Scientiae, 213--236.

[140]

Schubert, E. 2003. Update of the Hevner adjective checklist. Perceptual Motor Skills 96, 1117--1122.

[141]

Schuller, B., Eyben, F., and Rigoll, G. 2008. Tango or waltz? Putting ballroom dance style into tempo detection. EURASIP J. Audio, Speech, Music Process. Article ID 846135.

Digital Library

[142]

Schuller, B., Steidl, S., and Batliner, A. 2009. The INTERSPEECH 2009 Emotion Challenge. In Proceedings of the INTERSPEECH Conference.

[143]

Schuller, B., Dorfner, J., and Rigoll, G. 2010. Determination of nonprototypical valence and arousal in popular music: Features and performances. EURASIP J. Audio, Speech, Music Process. Article ID 735854.

Digital Library

[144]

Schuller, B., Hage, C., Schuller, D., and Rigoll, G. 2010. Mister D. J., cheer me up!: Musical and textual features for automatic mood classification. J. New Music Res. 39, 1, 13--34.

[145]

Sebastiani, F. 2002. Machine learning in automated text categorization. ACM Comput. Surveys 34, 1, 1--47.

Digital Library

[146]

Sen, A. and Srivastava, M. 1990. Regression Analysis: Theory, Methods, and Applications. Springer, New York, NY.

[147]

Shao, B., Li, T., and Ogihara, M. 2008. Quantify music artist similarity based on style and mood. In Proceedings of the ACM Workshop on Web Information and Data Management. 119--124.

Digital Library

[148]

Shen, J., Cui, B., Shepherd, J., and Tan, K.-L. 2006. Towards efficient automated singer identification in large music databases. In Proceedings of the ACM International Conference on Information Retrieval. 59--66.

Digital Library

[149]

Skowronek, J., McKinney, M. F., and van de Par, S. 2006. Ground truth for automatic music mood classification. In Proceedings of the International Conference on Music Information Retrieval. 395--396.

[150]

Skowronek, J., McKinney, M. F., and van de Par, S. 2007. A demonstrator for automatic music mood estimation. In Proceedings of the International Conference on Music Information Retrieval.

[151]

Sloboda, J. A. and Juslin, P. N. 2001. Psychological perspectives on music and emotion. In Music and Emotion: Theory and Research, P. N. Juslin and J. A. Sloboda Eds., Oxford University Press, Oxford, UK.

[152]

Solomatine, D. and Shrestha, D. 2004. AdaBoost.RT: A boosting algorithm for regression problems. In Proceedings of the IEEE International Joint Conference Neural Networks. 1163--1168.

[153]

Thayer, R. E. 1989. The Biopsychology of Mood and Arousal. Oxford University Press, Oxford, UK.

[154]

Toiviainen, P. and Krumhansl, C. L. 2003. Measuring and modeling real-time responses to music: The dynamics of tonality induction. Perception 32, 6, 741--766.

[155]

Tolonen, T. and Karjalainen, M. 2000. A computationally efficient multipitch analysis model. IEEE Trans. Speech Audio Process. 8, 6, 708--716.

[156]

Tolos, M., Tato, R., and Kemp, T. 2005. Mood-based navigation through large collections of musical data. In Proceedings of the IEEE Consumer Communications & Network Conference. 71--75.

[157]

Trohidis, K., Tsoumakas, G., Kalliris, G., and Vlahavas, I. 2008. Multi-label classification of music into emotions. In Proceedings of the International Conference on Music Information Retrieval. 325--330.

[158]

Tsai, W.-H. and Wang, H.-M. 2006. Automatic singer recognition of popular music recordings via estimation and modeling of solo vocal signals. IEEE Trans. Audio, Speech Lang. Process. 14, 1, 330--341.

Digital Library

[159]

Turnbull, D., Liu, R., Barrington, L., and Lanckriet, G. 2007. A game-based approach for collecting semantic annotations of music. In Proceedings of the International Conference on Music Information Retrieval.

[160]

Turnbull, D., Barrington, L., Torres, D., and Lanckriet, G. 2008. Semantic annotation and retrieval of music and sound effects. IEEE Trans. Audio, Speech Lang. Process. 16, 2, 467--476.

Digital Library

[161]

Tzanetakis, G. 2007. MARSYAS submissions to MIREX 2007. In Proceedings of the International Conference on Music Information Retrieval.

[162]

Tzanetakis, G. and Cook, P. 2002. Musical genre classification of audio signals. IEEE Trans. Speech Audio Process. 10, 5, 293--302. http://marsyas.sness.net/.

[163]

van Zaanen, M. and Kanters, P. 2010. Automatic mood classification using tf*idf based on lyrics. In Proceedings of the International Conference on Music Information Retrieval.

[164]

Vercoe, G. S. 2006. Moodtrack: practical methods for assembling emotion-driven music. M.S. thesis, MIT, Cambridge, MA.

[165]

Ververidis, D. and Kotropoulos, C. 2006. Emotional speech recognition: Resources, features, and methods. Speech Comm. 48, 9, 1162--1181.

[166]

Wang, H. L. and Cheong, L. F. 2006. Affective understanding in film. IEEE Trans. Circuits Syst. Video Technol. 16, 6, 689--704.

Digital Library

[167]

Wang, M.-Y., Zhang, N.-Y., and Zhu, H.-C. 2004. User-adaptive music emotion recognition. In Proceedings of the IEEE International Conference on Signal Processing. 1352--1355.

[168]

Whissell, C. M., Fournier, M., Pelland, R., Weir, D., and Makarec, K. 21986. A dictionary of affect in language: IV. reliability, validity, and applications. Perceptual Motor Skills 62, 875--888.

[169]

Wieczorkowska, A. 2004. Towards extracting emotions from music. In Proceedings of the International Workshop on Intelligent Media Technology for Communicative Intelligence. 228--238.

Digital Library

[170]

Wieczorkowska, A., Synak, P., and Raś, Z. W. 2006. Multi-label classification of emotions in music. In Proceedings of the International Conference on Intelligent Information Processing and Web Mining. 307--315.

[171]

Wu, T.-L. and Jeng, S.-K. 2006. Automatic emotion classification of musical segments. In Proceedings of the International Conference on Music Perception and Cognition.

[172]

Wu, T.-L. and Jeng, S.-K. 2007. Regrouping of expressive terms for musical qualia. In Proceedings of the International Workshop on Computer Music Audio Technology.

[173]

Wu, T.-L. and Jeng, S.-K. 2008. Probabilistic estimation of a novel music emotion model. In Proceedings of the International Multimedia Modeling Conference 487--497.

Digital Library

[174]

Wu, T.-L., Wang, H.-K., Ho, C.-C., Lin, Y.-P., Hu, T.-T., Weng, M.-F., Chan, L.-W., Yang, C.-H., Yang, Y.-H., Hung, Y.-P., Chuang, Y.-Y., Chen, H.-H., Chen, H. H., Chen, J.-H., and Jeng, S.-K. 2008. Interactive content presenter based on expressed emotion and physiological feedback. In Proceedings of the ACM International Conference on Multimedia. 1009--1010.

Digital Library

[175]

Xiao, Z., Dellandrea, E., Dou, W., and Chen, L. 2008. What is the best segment duration for music mood analysis. In Proceedings of the IEEE International Workshop on Content-Based Multimedia Indexing. 17--24.

[176]

Yang, D. and Lee, W.-S. 2004. Disambiguating music emotion using software agents. In Proceedings of the International Conference onMusic Information Retrieval.

[177]

Yang, Y.-H. and Chen, H. H. 2011a. Prediction of the distribution of perceived music emotions using discrete samples. IEEE Trans. Audio, Speech Lang. Process 19, 7, 2184--2196.

Digital Library

[178]

Yang, Y.-H. and Chen, H. H. 2011b. Ranking-based emotion recognition for music organization and retrieval. IEEE Trans. Audio, Speech Lang. Process 19, 4, 762--774.

Digital Library

[179]

Yang, Y.-H. and Chen, H. H. 2011c. Music Emotion Recognition. CRC Press, Boca Raton, FL.

Digital Library

[180]

Yang, Y.-H., Liu, C. C., and Chen, H. H. 2006. Music emotion classification: A fuzzy approach. In Proceedings of the ACM International Conference on Multimedia. 81--84.

Digital Library

[181]

Yang, Y.-H., Su, Y.-F., Lin, Y.-C., and Chen, H. H. 2007. Music emotion recognition: The role of individuality. In Proceedings of the ACM International Workshop on Human-Centered Multimedia. 13--21. http://mpac.ee.ntu.edu.tw/~yihsuan/MER/hcm07/.

Digital Library

[182]

Yang, Y.-H., Lin, Y.-C., and Chen, H. H. 2009. Personalized music emotion recognition. In Proceedings of the ACM International Conference on Information Retrieval. 748--749.

Digital Library

[183]

Yang, Y.-H., Lin, Y.-C., Su, Y.-F., and Chen, H. H. 2008. A regression approach to music emotion recognition. IEEE Trans. Audio, Speech Lang. Process. 16, 2, 448--457.

Digital Library

[184]

Yang, Y.-H., Lin, Y.-C., Cheng, H.-T., and Chen, H. H. 2008. Mr. Emo: Music retrieval in the emotion plane. In Proceedings of the ACM International Conference on Multimedia. 1003--1004. http://www.youtube.com/watch?v=ra55xO20UHU.

Digital Library

[185]

Yang, Y.-H., Lin, Y.-C., Cheng, H.-T., Liao, I.-B., Ho, Y.-C., and Chen, H.-H. 2008. Toward multi-modal music emotion classification. In Proceedings of the Pacific-Rim Conference on Multimedia. 70--79.

Digital Library

[186]

Yeh, C.-C., Tseng, S.-S., Tsai, P.-C., and Weng, J.-F. 2006. Building a personalized music emotion prediction system. In Proceedings of the Pacific-Rim Conference on Multimedia. 730--739.

Digital Library

[187]

Zhang, M.-L. and Zhou, Z.-H. 2007. ML-knn: A lazy learning approach to multi-label learning. Pattern Recogn. 40, 7, 2038--2048.

Digital Library

[188]

Zhang, S., Huang, Q., Tian, Q., Jiang, S., and Gao, W. 2008. i.MTV - an integrated system for MTV affective analysis. In Proceedings of the ACM International Conference on Multimedia. 985--986.

Digital Library

[189]

Zhang, S., Tian, Q., Jiang, S., Huang, Q., and Gao, W. 2008. Affective MTV analysis based on arousal and valence features. In Proceedings of the IEEE International Conference on Multimedia and Expo. 1369--1372.

[190]

Zhang, S., Tian, Q., Huang, Q., Gao, W., and Li, S. 2009. Utilizing affective analysis for efficient movie browsing. In Proceedings of the IEEE International Conference on Image Processing. 1853--1856.

Digital Library

Cited By

Xue JWang JLiu XZhang QWu X(2025)Affective Video Content Analysis: Decade Review and New PerspectivesBig Data Mining and Analytics10.26599/BDMA.2024.90200488:1(118-144)Online publication date: Feb-2025
https://doi.org/10.26599/BDMA.2024.9020048
Garcia-Moreno FBadenes-Sastre MExpósito FRodriguez-Fortiz MBermudez-Edo M(2025)EEG headbands vs caps: How many electrodes do I need to detect emotions? The case of the MUSE headbandComputers in Biology and Medicine10.1016/j.compbiomed.2024.109463184(109463)Online publication date: Jan-2025
https://doi.org/10.1016/j.compbiomed.2024.109463
Shelke SPatil M(2024)Exploring Machine Learning Techniques for Music Emotion Classification: A Comprehensive Review2024 11th International Conference on Computing for Sustainable Global Development (INDIACom)10.23919/INDIACom61295.2024.10498480(1188-1195)Online publication date: 28-Feb-2024
https://doi.org/10.23919/INDIACom61295.2024.10498480
Show More Cited By

Index Terms

Machine Recognition of Music Emotion: A Review
1. Applied computing
  1. Arts and humanities
    1. Sound and music computing
2. Information systems
  1. Information retrieval
    1. Retrieval models and ranking

Recommendations

The PMEmo Dataset for Music Emotion Recognition
ICMR '18: Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval

Music Emotion Recognition (MER) has recently received considerable attention. To support the MER research which requires large music content libraries, we present the PMEmo dataset containing emotion annotations of 794 songs as well as the simultaneous ...
Emotion Recognition of Chinese Traditional Folk Music using an Assembling Machine Learning Method
ICMLT '22: Proceedings of the 2022 7th International Conference on Machine Learning Technologies

Various papers published recently about the emotion of western pop music, none have looked into how to describe Chinese traditional folk music. The accuracy of existing algorithms in recognizing emotions in Chinese traditional folk music is just 42%. ...
The Role of Time in Music Emotion Recognition: Modeling Musical Emotions from Time-Varying Music Features
CMMR 2012: Revised Selected Papers of the 9th International Symposium on From Sounds to Music and Emotions - Volume 7900

Music is widely perceived as expressive of emotion. However, there is no consensus on which factors in music contribute to the expression of emotions, making it difficult to find robust objective predictors for music emotion recognition MER. Currently, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Intelligent Systems and Technology

ACM Transactions on Intelligent Systems and Technology Volume 3, Issue 3

May 2012

384 pages

ISSN:2157-6904

EISSN:2157-6912

DOI:10.1145/2168752

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 May 2012

Accepted: 01 October 2010

Revised: 01 August 2010

Received: 01 May 2010

Published in TIST Volume 3, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tag

Music emotion recognition

Qualifiers

Research-article
Survey
Refereed

Funding Sources

National Science Council Taiwan
National Taiwan University

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

238
Total Citations
View Citations
4,940
Total Downloads

Downloads (Last 12 months)259
Downloads (Last 6 weeks)30

Reflects downloads up to 06 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xue JWang JLiu XZhang QWu X(2025)Affective Video Content Analysis: Decade Review and New PerspectivesBig Data Mining and Analytics10.26599/BDMA.2024.90200488:1(118-144)Online publication date: Feb-2025
https://doi.org/10.26599/BDMA.2024.9020048
Garcia-Moreno FBadenes-Sastre MExpósito FRodriguez-Fortiz MBermudez-Edo M(2025)EEG headbands vs caps: How many electrodes do I need to detect emotions? The case of the MUSE headbandComputers in Biology and Medicine10.1016/j.compbiomed.2024.109463184(109463)Online publication date: Jan-2025
https://doi.org/10.1016/j.compbiomed.2024.109463
Shelke SPatil M(2024)Exploring Machine Learning Techniques for Music Emotion Classification: A Comprehensive Review2024 11th International Conference on Computing for Sustainable Global Development (INDIACom)10.23919/INDIACom61295.2024.10498480(1188-1195)Online publication date: 28-Feb-2024
https://doi.org/10.23919/INDIACom61295.2024.10498480
Shang YPeng QWu ZLiu Y(2024)Music-induced emotion flow modeling by ENMI NetworkPLOS ONE10.1371/journal.pone.029771219:10(e0297712)Online publication date: 21-Oct-2024
https://doi.org/10.1371/journal.pone.0297712
Liang QWang Y(2024)Drawlody: Sketch-Based Melody Creation With Enhanced Usability and InterpretabilityIEEE Transactions on Multimedia10.1109/TMM.2024.336069526(7074-7088)Online publication date: 31-Jan-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3360695
Jiang XZhang YLin GYu L(2024)Music Emotion Recognition Based on Deep Learning: A ReviewIEEE Access10.1109/ACCESS.2024.348447012(157716-157745)Online publication date: 2024
https://doi.org/10.1109/ACCESS.2024.3484470
Wen TChuan CAnghelcev GSar SYun JXu Y(2024)Infusing Affective Computing Models into Advertising Research on EmotionsJournal of Advertising10.1080/00913367.2024.240925453:5(710-731)Online publication date: 22-Nov-2024
https://doi.org/10.1080/00913367.2024.2409254
Mancini EGalassi ARuggeri FTorroni P(2024)Disruptive situation detection on public transport through speech emotion recognitionIntelligent Systems with Applications10.1016/j.iswa.2023.20030521(200305)Online publication date: Mar-2024
https://doi.org/10.1016/j.iswa.2023.200305
Deldjoo YSchedl MKnees P(2024)Content-driven music recommendation: Evolution, state of the art, and challengesComputer Science Review10.1016/j.cosrev.2024.10061851(100618)Online publication date: Feb-2024
https://doi.org/10.1016/j.cosrev.2024.100618
Gahlan NSethia D(2024)Federated learning in Emotion Recognition Systems based on physiological signals for privacy preservation: a reviewMultimedia Tools and Applications10.1007/s11042-024-19467-3Online publication date: 3-Jun-2024
https://doi.org/10.1007/s11042-024-19467-3
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents