skip to main content
research-article

metaFERA: a meta-framework for creating emotion recognition frameworks for physiological signals

Published: 26 June 2023 Publication History

Abstract

Recognizing emotions from physiological signals has proven to be important in various scenarios. To assist in developing emotion recognizers, software frameworks and toolboxes have emerged, offering ready-to-use components. However,these have limitations regarding the type of physiological signals supported, the recognition steps covered, or the acquisition of multiple physiological signals. This paper presents metaFERA, an architectural meta-framework for creating software frameworks for end-to-end emotion recognition from physiological signals. The modularity and flexibility of the meta-framework and the resulting frameworks allow the fast prototyping of emotion recognition systems and experiments to test and validate new algorithms. To that end, metaFERA offers: (i) a set of pre-configured blocks to which we can add behavior to create framework components; (ii) an easy way to add behavior to the pre-configured blocks; (iii) a channel-based communication mechanism that transparently and efficiently supports the exchange of information between components; (iv) a simple and easy way to use and link components from a resulting framework to create applications. Additionally, we provide a set of Web services, already configured, to make the resulting recognition systems available as a service. To validate metaFERA, we created a framework for Electrodermal Activity, an emotion recognizer to identify high/low arousal using the aforementioned framework, and a layer to offer the recognizer as a service.

References

[1]
Alves, A.P., Silva, H., Lourenço A. L., Fred, A.L.: BITalino: A Biosignal Acquisition System based on the Arduino. In: Proceeding of the International Conference on Biomedical Electronics and Devices, pp. 261–264 (2013)
[2]
Blechert J, Peyk P, Liedlgruber M, and Wilhelm FH Anslab: Integrated multichannel peripheral biosignal processing in psychophysiological science Behavior Research Methods 2016 48 4 1528-1545
[3]
Brunet, D., Murray, M.M., Michel, C.M.: Spatiotemporal analysis of multichannel eeg: Cartool. Computational intelligence and neuroscience 2011 (2011)
[4]
Delorme A and Makeig S Eeglab: an open source toolbox for analysis of single-trial eeg dynamics including independent component analysis Journal of neuroscience methods 2004 134 1 9-21
[5]
Duin, R.P.W.: Prtools version 3.0: A matlab toolbox for pattern recognition. In: Proceedings of SPIE, p. 1331 (2000)
[6]
Frank E, Hall M, Trigg L, Holmes G, and Witten IH Data mining in bioinformatics using weka Bioinformatics 2004 20 15 2479-2481
[7]
g.BSANALYZE: OFFLINE BIOSIGNAL ANALYSIS FOR MATLAB. https://www.gtec.at/product/gbsanalyze/. [Online; Accessed 02 December 2022]
[8]
Gramfort A, Luessi M, Larson E, Engemann DA, Strohmeier D, Brodbeck C, Parkkonen L, and Hämäläinen MS Mne software for processing meg and eeg data Neuroimage 2014 86 446-460
[9]
Higham, D.J., Higham, N.J.: MATLAB guide. SIAM (2016)
[10]
Hjorth B EEG Analysis Based on Time Domain Properties Electroencephalography and Clinical Neurophysiology 1970 29 3 306-310
[11]
Hofmann, M., Klinkenberg, R.: RapidMiner: Data mining use cases and business analytics applications. CRC Press (2016)
[12]
IMotions: Biometric Research Platform (SW Version). https://imotions.com/ (2001). [Online; accessed 02 December 2022]
[13]
Jayaram, V., Barachant, A.: Moabb: trustworthy algorithm benchmarking for bcis. Journal of neural engineering 15(6) (2018)
[14]
Li, H.: Smile-statistical machine intelligence & learning engine (2016)
[15]
Liapis, A., Katsanos, C., Karousos, N., Xenos, M., Orphanoudakis, T.: User experience evaluation: A validation study of a tool-based approach for automatic stress detection using physiological signals. International Journal of Human-computer Interaction pp. 1–14 (2020)
[16]
Michalska M Openbci: Framework for brain-computer interfaces 2009 Informatics and Mechanics University of Warsaw Faculty of Mathematics
[17]
Miranda Correa, J.A., Abadi, M.K., Sebe, N., Patras, I.: AMIGOS: A Dataset for Affect, Personality and Mood Research on Individuals and Groups. IEEE Transactions on Affective Computing pp. 1–14 (2018). 10.1109/TAFFC.2018.2884461
[18]
Muñoz, J.E., Gouveia, E.R., Cameirão, M.S., i Badia, S.B.: Physiolab-a multivariate physiological computing toolbox for ecg, emg and eda signals: a case of study of cardiorespiratory fitness assessment in the elderly population. Multimedia Tools and Applications 77(9), 11521–11546 (2018)
[19]
Oliphant TE (2007) Python for scientific computing. Computing in Science & Engineering 9(3):10–20
[20]
Oostenveld, R., Fries, P., Maris, E., Schoffelen, J.M.: Fieldtrip: open source software for advanced analysis of meg, eeg, and invasive electrophysiological data. Computational intelligence and neuroscience 2011 (2011)
[21]
Palestra G and Pino O Detecting Emotions During a Memory Training Assisted by a Social Robot for Individuals with Mild Cognitive Impairment (MCI) Multimedia Tools and Applications 2020 79 47 35829-35844
[22]
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Èdouard Duchesnay: Scikit-learn: Machine learning in python (2018)
[23]
Pham MT, Geuens M, and De Pelsmacker P The influence of ad-evoked feelings on brand evaluations: Empirical generalizations from consumer responses to more than 1000 tv commercials International Journal of Research in Marketing 2013 30 4 383-394
[24]
Soleymani M, Villaro-Dixon F, Pun T, and Chanel G Toolbox for emotional feature extraction from physiological signals (teap) Frontiers in ICT 2017 4 1
[25]
Tadel, F., Baillet, S., Mosher, J.C., Pantazis, D., Leahy, R.M.: Brainstorm: a user-friendly application for meg/eeg analysis. Computational intelligence and neuroscience 2011 (2011)
[26]
Tijs, T.J.W., Brokken, D., IJsselsteijn, W.A.: Dynamic game balancing by recognizing affect. In: P. Markopoulos, B. de Ruyter, W. IJsselsteijn, D. Rowland (eds.) Fun and Games, pp. 88–93. Springer Berlin Heidelberg, Berlin, Heidelberg (2008)
[27]
Vidaurre, C., Sander, T.H., Schlögl, A.: Biosig: the free and open source software library for biomedical signal processing. Computational intelligence and neuroscience 2011 (2011)
[28]
Virtanen, P., Gommers, R., Oliphant, T.E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., van der Walt, S.J., Brett, M., Wilson, J., Millman, K.J., Mayorov, N., Nelson, A.R.J., Jones, E., Kern, R., Larson, E., Carey, C.J., Polat, İ., Feng, Y., Moore, E.W., VanderPlas, J., Laxalde, D., Perktold, J., Cimrman, R., Henriksen, I., Quintero, E.A., Harris, C.R., Archibald, A.M., Ribeiro, A.H., Pedregosa, F., van Mulbregt, P., SciPy 1.0 Contributors: SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods 17, 261–272 (2020). 10.1038/s41592-019-0686-2

Cited By

View all
  • (2023)That's AWESOME: Awareness While Experiencing and Surfing On Movies through EmotionsProceedings of the 2023 ACM International Conference on Interactive Media Experiences Workshops10.1145/3604321.3604330(110-117)Online publication date: 12-Jun-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications
Multimedia Tools and Applications  Volume 83, Issue 4
Jan 2024
2884 pages

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 26 June 2023
Accepted: 06 April 2023
Revision received: 09 December 2022
Received: 05 July 2021

Author Tags

  1. Emotion recognition
  2. Physiological signals
  3. Software as a service
  4. Software framework

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)That's AWESOME: Awareness While Experiencing and Surfing On Movies through EmotionsProceedings of the 2023 ACM International Conference on Interactive Media Experiences Workshops10.1145/3604321.3604330(110-117)Online publication date: 12-Jun-2023

View Options

View options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media