introduction

Free access

AVEC 2014: 3D Dimensional Affect and Depression Recognition Challenge

Authors:

Michel Valstar,

Björn Schuller,

Jarek Krajewski,

Maja PanticAuthors Info & Claims

AVEC '14: Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge

Pages 3 - 10

https://doi.org/10.1145/2661806.2661807

Published: 07 November 2014 Publication History

Abstract

Mood disorders are inherently related to emotion. In particular, the behaviour of people suffering from mood disorders such as unipolar depression shows a strong temporal correlation with the affective dimensions valence, arousal and dominance. In addition to structured self-report questionnaires, psychologists and psychiatrists use in their evaluation of a patient's level of depression the observation of facial expressions and vocal cues. It is in this context that we present the fourth Audio-Visual Emotion recognition Challenge (AVEC 2014). This edition of the challenge uses a subset of the tasks used in a previous challenge, allowing for more focussed studies. In addition, labels for a third dimension (Dominance) have been added and the number of annotators per clip has been increased to a minimum of three, with most clips annotated by 5. The challenge has two goals logically organised as sub-challenges: the first is to predict the continuous values of the affective dimensions valence, arousal and dominance at each moment in time. The second is to predict the value of a single self-reported severity of depression indicator for each recording in the dataset. This paper presents the challenge guidelines, the common data used, and the performance of the baseline system on the two tasks.

References

[1]

T. Almaev and M. Valstar. Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In Proc. Affective Computing and Intelligent Interaction, 2013.

Digital Library

[2]

T. R. Almaev, A. Yüce, A. Ghitulescu, and M. F. Valstar. Distribution-based iterative pairwise classification of emotions in the wild using lgbp-top. In Proc. Int'l Conf. Multimodal Interaction, ICMI '13, pages 535--542, New York, NY, USA, 2013. ACM.

Digital Library

[3]

M. R. Bagby, A. G. Ryder, D. R. Schuller, and M. B. Marshall. The hamilton depression rating scale: Has the gold standard become a lead weight? American Journal of Psychiatry, 161:2163--2177, 2004.

[4]

A. Beck, R. Steer, R. Ball, and W. Ranieri. Comparison of beck depression inventories -ia and -ii in psychiatric outpatients. Journal of Personality Assessment, 67(3):588--97, December 1996.

[5]

J. F. Cohn, S. Kreuz, I. Matthews, Y. Yang, M. H. Nguyen, M. Tejera Padilla, and et al. Detecting depression from facial actions and vocal prosody. In Proc. Affective Computing and Intelligent Interaction, pages 1--7, 2009.

[6]

R. Cowie, E. Douglas-Cowie, S. Savvidou, E. McMahon, M. Sawey, and M. Schröder. Feeltrace: An instrument for recording perceived emotion in real time. In Proc.\ ISCA Workshop on Speech and Emotion, pages 19--24, Belfast, UK, 2000.

[7]

F. Eyben, F. Weninger, S. Squartini, and B. Schuller. Real-life voice activity detection with lstm recurrent neural networks and an application to hollywood movies. In Proc.\ of ICASSP, Vancouver, Canada. IEEE, 2013. to appear.

[8]

F. Eyben, M. Wöllmer, and B. Schuller. openEAR - Introducing the Munich Open-Source Emotion and Affect Recognition Toolkit. In Proc.\ ACII, pages 576--581, Amsterdam, The Netherlands, 2009.

[9]

F. Eyben, M. Wöllmer, and B. Schuller. openSMILE -- The Munich Versatile and Fast Open-Source Audio Feature Extractor. In Proc.\ ACM Multimedia (MM), pages 1459--1462, Florence, Italy, 2010.

Digital Library

[10]

M. First, R. Spitzer, M. Gibbon, and J. Williams. Structured Clinical Interview for DSM-IV Axis I Disorders SCID-I: Clinician Version, Administration Booklet. SCID-I: Clinician Version. American Psychiatric Press, 1997.

[11]

J. Fontaine, S. K.R., E. Roesch, and P. Ellsworth. The world of emotions is not two-dimensional. Psychological science, 18(2):1050 -- 1057, 2007.

[12]

J. M. Girard, J. F. Cohn, M. H. Mahoor, S. Mavadati, and D. Rosenwald. Social risk and depression: Evidence from manual and automatic facial expression analysis. In IEEE International Conference on Automatic Face and Gesture Recognition, 2013.

[13]

M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I. H. Witten. The weka data mining software: An update. SIGKDD Explor. Newsl., 11(1):10--18, Nov. 2009.

Digital Library

[14]

M. Hamilton. Development of a rating scale for primary depressive illness. British Journal of Social and Clinical Psycology, 8:278--296, 1967.

[15]

J. Hamm, Kohler, C. G., Gur, R. C., and R. Verma. Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders. Journal of Neuroscience Methods, 200(2):237--256, 2011.

[16]

Health & Consumer Protection Directorate General. Improving the mental health of the population: Towards a strategy on mental health for the european union. Technical report, European Union, 2005.

[17]

Health & Consumer Protection Directorate General. Mental health in the eu. Technical report, European Union, 2008.

[18]

M. K\"achele, M. Glodek, D. Zharkov, S. Meudt, and F. Schwenker. Fusion of audio-visual features using hierarchical classifier systems for the recognition of affective states and the state of depression. In M. De Marsico, A. Tabbone, and A. Fred, editors, Proc. Int'l Conf. Pattern Recognition Applications and Methods (ICPRAM), pages 671--678. SciTePress, 2014.

[19]

S. Maji, A. C. Berg, and J. Malik. Classification using intersection kernel support vector machines is efficient. 2013 IEEE Conference on Computer Vision and Pattern Recognition, 0:1--8, 2008.

[20]

H. Meng, D. Huang, H. Wang, H. Yang, M. AI-Shuraifi, and Y. Wang. Depression recognition based on dynamic facial and vocal expression features using partial least square regression. In Proc. Int'l Workshop Audio/Visual Emotion Challenge, AVEC '13, pages 21--30, New York, NY, USA, 2013. ACM.

Digital Library

[21]

M. Nicolaou, V. Pavlovic, and M. Pantic. Dynamic probabilistic cca for analysis of affective behaviour and fusion of continuous annotations. Pattern Analysis and Machine Intelligence, IEEE Transactions on, PP(99):1--1, 2014.

[22]

R. Picard. Affective Computing. MIT Press, 1997.

Digital Library

[23]

S. Scherer, G. Stratou, J. Gratch, J. Boberg, M. Mahmoud, A. S. Rizzo, and L.-P. Morency. Automatic behavior descriptors for psychological disorder analysis. In Proc.\ IEEE Int.\ Conf.\ on Automatic Face and Gesture Recognition, 2013.

[24]

B. Schuller, S. Steidl, and A. Batliner. The INTERSPEECH 2009 Emotion Challenge. In Proc.\ INTERSPEECH 2009, pages 312--315, Brighton, UK, 2009.

[25]

B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. Müller, and S. Narayanan. The INTERSPEECH 2010 Paralinguistic Challenge. In Proc.\ INTERSPEECH 2010, pages 2794--2797, Makuhari, Japan, 2010.

[26]

M. Valstar. Automatic behaviour understanding in medicine. In Proceedings ACM Int'l Conf. Multimodal Interaction, 2014.

Digital Library

[27]

M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic. Avec 2013 - the continuous audio / visual emotion and depression recognition challenge. In Proc. 3rd ACM international workshop on Audio/visual emotion challenge, pages 3--10, 2013.

Digital Library

[28]

A. Vinciarelli, M. Pantic, D. Heylen, C. Pelachaud, I. Poggi, F. D'ericco, and M. Schroeder. Bridging the gap between social animal and unsocial machine: A survey of social signal processing. IEEE Trans. Affective Computing, 3:69--87, April 2012. Issue 1.

Digital Library

[29]

P. Wang, F. Barrett, E. Martin, M. Milonova, R. E. Gur, R. C. Gur, C. Kohler, and et al. Automated video-based facial expression analysis of neuropsychiatric disorders. Journal of Neuroscience Methods, 168(1):224 -- 238, 2008.

[30]

J. R. Williamson, T. F. Quatieri, B. S. Helfer, R. Horwitz, B. Yu, and D. D. Mehta. Vocal biomarkers of depression based on motor incoordination. In Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge, AVEC '13, pages 41--48, New York, NY, USA, 2013. ACM.

Digital Library

[31]

Y. Yang, C. Fairbairn, and J. Cohn. Detecting depression severity from intra- and interpersonal vocal prosody. IEEE Transactions on Affective Computing, 4, 2013.

Digital Library

[32]

M. Zimmerman, I. Chelminski, and M. Posternak. A review of studies of the hamilton depression rating scale in healthy controls: Implications for the definition of remission in treatment studies of depression. Journal of Nervous & Mental Disease, 192(9):595--601, 2004.

Cited By

Fu CQian FSu YSu KSong SNiu MShi JLiu ZLiu CIshi CIshiguro H(2025)Facial action units guided graph representation learning for multimodal depression detectionNeurocomputing10.1016/j.neucom.2024.129106619(129106)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.129106
Han MLi XYi XZheng YXia WLiu YWang Q(2024)Automatic recognition of depression based on audio and video: A reviewWorld Journal of Psychiatry10.5498/wjp.v14.i2.22514:2(225-233)Online publication date: 19-Feb-2024
https://doi.org/10.5498/wjp.v14.i2.225
Mohammed Mahdi Allarakhia Mubashira Shaikh Hussain Sidhpurwala Ayesha Sayyed Dr. Ashfaq Shaikh (2024)Depression Detection through Integrative Multimodal Signals: Exploring Advanced Computational TechniquesInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-19835(194-200)Online publication date: 11-Oct-2024
https://doi.org/10.48175/IJARSCT-19835
Show More Cited By

Index Terms

AVEC 2014: 3D Dimensional Affect and Depression Recognition Challenge
1. Information systems
  1. Information systems applications

Recommendations

AVEC 2016: Depression, Mood, and Emotion Recognition Workshop and Challenge
AVEC '16: Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge

The Audio/Visual Emotion Challenge and Workshop (AVEC 2016) "Depression, Mood and Emotion" will be the sixth competition event aimed at comparison of multimedia processing and machine learning methods for automatic audio, visual and physiological ...
AVEC 2012: the continuous audio/visual emotion challenge
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interaction

We present the second Audio-Visual Emotion recognition Challenge and workshop (AVEC 2012), which aims to bring together researchers from the audio and video analysis communities around the topic of emotion recognition. The goal of the challenge is to ...
AV+EC 2015: The First Affect Recognition Challenge Bridging Across Audio, Video, and Physiological Data
AVEC '15: Proceedings of the 5th International Workshop on Audio/Visual Emotion Challenge

We present the first Audio-Visual+ Emotion recognition Challenge and workshop (AV+EC 2015) aimed at comparison of multimedia processing and machine learning methods for automatic audio, visual and physiological emotion analysis. This is the 5th event in ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

AVEC '14: Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge

November 2014

110 pages

ISBN:9781450331197

DOI:10.1145/2661806

General Chairs:
Michel Valstar
University of Nottingham, UK
,
Björn Schuller
Technische Universität Münich/Imperial College London, DE/UK
,
Jarek Krajewski
University of Wuppertal, Germany
,
Roddy Cowie
Queen's University Belfast, UK
,
Maja Pantic
Imperial College London/Twente University, UK/The Netherlands

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Introduction

Conference

MM '14

Sponsor:

SIGMM

MM '14: 2014 ACM Multimedia Conference

November 7, 2014

Florida, Orlando, USA

Acceptance Rates

AVEC '14 Paper Acceptance Rate 8 of 22 submissions, 36%;

Overall Acceptance Rate 52 of 98 submissions, 53%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

291
Total Citations
View Citations
2,383
Total Downloads

Downloads (Last 12 months)900
Downloads (Last 6 weeks)180

Reflects downloads up to 07 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Fu CQian FSu YSu KSong SNiu MShi JLiu ZLiu CIshi CIshiguro H(2025)Facial action units guided graph representation learning for multimodal depression detectionNeurocomputing10.1016/j.neucom.2024.129106619(129106)Online publication date: Feb-2025
https://doi.org/10.1016/j.neucom.2024.129106
Han MLi XYi XZheng YXia WLiu YWang Q(2024)Automatic recognition of depression based on audio and video: A reviewWorld Journal of Psychiatry10.5498/wjp.v14.i2.22514:2(225-233)Online publication date: 19-Feb-2024
https://doi.org/10.5498/wjp.v14.i2.225
Mohammed Mahdi Allarakhia Mubashira Shaikh Hussain Sidhpurwala Ayesha Sayyed Dr. Ashfaq Shaikh (2024)Depression Detection through Integrative Multimodal Signals: Exploring Advanced Computational TechniquesInternational Journal of Advanced Research in Science, Communication and Technology10.48175/IJARSCT-19835(194-200)Online publication date: 11-Oct-2024
https://doi.org/10.48175/IJARSCT-19835
Li YLi SNaqvi S(2024)A Novel Audio-Visual Information Fusion System for Mental Disorders Detection2024 27th International Conference on Information Fusion (FUSION)10.23919/FUSION59988.2024.10706499(1-7)Online publication date: 8-Jul-2024
https://doi.org/10.23919/FUSION59988.2024.10706499
Xu NHuo HXu JMa LWang J(2024)Automatic diagnosis of depression based on attention mechanism and feature pyramid modelPLOS ONE10.1371/journal.pone.029505119:3(e0295051)Online publication date: 12-Mar-2024
https://doi.org/10.1371/journal.pone.0295051
Islam RBae S(2024)FacePsy: An Open-Source Affective Mobile Sensing System - Analyzing Facial Behavior and Head Gesture for Depression Detection in Naturalistic SettingsProceedings of the ACM on Human-Computer Interaction10.1145/36765058:MHCI(1-32)Online publication date: 24-Sep-2024
https://dl.acm.org/doi/10.1145/3676505
Wang RHuang JZhang JLiu XZhang XLiu ZZhao PChen SSun XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)FacialPulse: An Efficient RNN-based Depression Detection via Temporal Facial LandmarksProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681546(311-320)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681546
Song SLuo YTumer TFu CValstar MGunes H(2024)Loss Relaxation Strategy for Noisy Facial Video-based Automatic Depression RecognitionACM Transactions on Computing for Healthcare10.1145/36486965:2(1-24)Online publication date: 4-Mar-2024
https://dl.acm.org/doi/10.1145/3648696
Li MLu ZCao QGao JHu B(2024)Automatic Assessment Method and Device for Depression Symptom Severity Based on Emotional Facial Expression and Pupil-WaveIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2024.341577873(1-15)Online publication date: 2024
https://doi.org/10.1109/TIM.2024.3415778
Niu MLi YTao JZhou XSchuller B(2024)DepressionMLP: A Multi-Layer Perceptron Architecture for Automatic Depression Level Prediction via Facial Keypoints and Action UnitsIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.338233434:9(8924-8938)Online publication date: Sep-2024
https://doi.org/10.1109/TCSVT.2024.3382334
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents