article

Assessing the quality of sensory experience for multimedia presentations

Authors:

Christian Timmerer,

Benjamin Rainer,

Hermann HellwagnerAuthors Info & Claims

Image Communication, Volume 27, Issue 8

Pages 909 - 916

https://doi.org/10.1016/j.image.2012.01.016

Published: 01 September 2012 Publication History

Abstract

This paper introduces the concept of sensory experience by utilizing sensory effects such as wind or lighting as another dimension which contributes to the quality of the user experience. In particular, we utilize a representation format for sensory effects that are attached to traditional multimedia resources such as audio, video, and image contents. Sensory effects (e.g., wind, lighting, explosion, heat, cold) are rendered on special devices (e.g., fans, ambient lights, motion chair, air condition) in synchronization with the traditional multimedia resources and shall stimulate other senses than audition and vision (e.g., mechanoreception, equilibrioception, thermoreception), with the intention to increase the users Quality of Experience (QoE). In particular, the paper provides a comprehensive introduction into the concept of sensory experience, its assessment in terms of the QoE, and related standardization and implementation efforts. Finally, we will highlight open issues and research challenges including future work.

References

[1]

Aarts, E. and de Ruyter, B., New research perspectives on ambient intelligence. Journal of Ambient Intelligence and Smart Environments. v1 i1. 5-14.

[2]

Waltl, M., Timmerer, C. and Hellwagner, H., A test-bed for quality of multimedia experience evaluation of sensory effects. In: Ebrahim, T., El-Maleh, K., Dane, G., Karam, L. (Eds.), Proceedings of the First International Workshop on Quality of Multimedia Experience (QoMEX 2009), IEEE, Los Alamitos, CA, USA. pp. 145-150.

[3]

Waltl, M., Timmerer, C. and Hellwagner, H., Increasing the user experience of multimedia presentations with sensory effects. In: Leonardi, R., Migliorati, P., Cavallaro, A. (Eds.), Proceedings of the 11th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS'10), IEEE, Desenzano del Garda, Italy. pp. 1-4.

[4]

Waltl, M., Timmerer, C. and Hellwagner, H., Improving the quality of multimedia experience through sensory effects. In: Perkis, A., Möller, S., Svensson, P., Reibman, A. (Eds.), Proceedings of the Second International Workshop on Quality of Multimedia Experience (QoMEX'10), IEEE, Trondheim, Norway. pp. 124-129.

[5]

Waltl, M., Rainer, B., Timmerer, C. and Hellwagner, H., Sensory experience for videos on the web. In: Böszörmenyi, L., Marques, O., Lux, M., Klamma, R. (Eds.), Proceedings of the Workshop on Multimedia on the Web (MMWeb) 2011, IEEE, Graz, Austria. pp. 1-3.

[6]

Ambient intelligence: visualizing the future. In: AVI'04: Proceedings of the Working Conference on Advanced Visual Interfaces, ACM Press, New York, NY, USA. pp. 203-208.

[7]

Suk, C.B., Hyun, J.S. and Yong, L.H., Sensory effect metadata for SMMD media service. In: Proceedings of the 2009 Fourth International Conference on Internet and Web Applications and Services, IEEE Computer Society, Washington, DC, USA. pp. 649-654.

Digital Library

[8]

Pyo, S., Joo, S., Choi, B., Kim, M. and Kim, J., A metadata schema design on representation of sensory effect information for sensible media and its service framework using UPnP. In: The 10th International Conference on Advanced Communication Technology, 2008, ICACT 2008, vol. 2. pp. 1129-1134.

[9]

Yoon, K., Choi, B., Lee, E.-S. and Lim, T.-B., 4-D broadcasting with MPEG-V. In: IEEE International Workshop on Multimedia Signal Processing (MMSP) 2010, pp. 257-262.

[10]

Chang, A. and O'Sullivan, C., Audio-haptic feedback in mobile phones. In: CHI'05 Extended Abstracts on Human Factors in Computing Systems, CHI EA'05, ACM, New York, NY, USA. pp. 1264-1267.

[11]

Kim, J.-H., Kwon, H.-J. and Hong, K.-S., Location awareness-based intelligent multi-agent technology. Multimedia Systems. v16 i4-5. 275-292.

[12]

Ademoye, O. and Ghinea, G., Synchronization of olfaction-enhanced multimedia. IEEE Transactions on Multimedia. v11 i3. 561-565.

[13]

The International Multisensory Research Forum (IMRF). URL {http://www.imrf.info/} (accessed November 2011).

[14]

Grega, M., Janowski, L., Leszczuk, M., Romaniak, P. and Papir, Z., Quality of experience evaluation for multimedia services-Szacowanie postrzeganej jako***ci usług (QoE) komunikacji multimedialnej. Przegla***d Telekomunikacyjny. v81 i4. 142-153.

[15]

Hands, D., A basic multimedia quality model. IEEE Transactions on Multimedia. v6 i6. 806-816.

[16]

T. Hoí***feld, D. Hock, P. Tran-Gia, K. Tutschku, M. Fiedler, Testing the IQX hypothesis for exponential interdependency between QoS and QoE of voice codecs iLBC and G.711, in: The 18th ITC Specialist Seminar on Quality of Experience, Karlskrona, Sweden, 2008.

[17]

Pereira, F., A triple user characterization model for video adaptation and quality of experience evaluation. In: IEEE Seventh Workshop on Multimedia Signal Processing, 2005, pp. 1-4.

[18]

ISO/IEC 23005-3, Information Technology-Media Context and Control-Sensory Information, ISO/IEC JTC 1/SC 29/WG 11/N11425, Geneva, Switzerland.

[19]

Salembier, P. and Sikora, T., Introduction to MPEG-7: Multimedia Content Description Interface. 2002. John Wiley & Sons, Inc., New York, NY, USA.

[20]

ISO/IEC 21000-7:2007, Information Technology-Multimedia Framework (MPEG-21)-Part 7: Digital Item Adaptation (accessed November 2007).

[21]

amBX. URL {http://www.ambx.com} (accessed November 2011).

[22]

ITU-T Rec. P.910, Subjective Video Quality Assessment Methods for Multimedia Applications, April 2008.

[23]

ITU-T Rec. P.911, Subjective Audiovisual Quality Assessment Methods for Multimedia Applications, December 2008.

[24]

Storms, R.L. and Zyda, M.J., Interactions in perceived quality of auditory-visual displays. Presence: Teleoperators and Virtual Environments. v9 i6. 557-580.

[25]

G. Bjontegaard, Calculation of average PSNR differences between RD curves, in: ITU-T VCEG Meeting VCEG-M33, Austin, USA, 2001.

[26]

Wu, W., Arefin, A., Rivas, R., Nahrstedt, K., Sheppard, R. and Yang, Z., Quality of experience in distributed interactive multimedia environments: toward a theoretical framework. In: Proceedings of the 17th ACM International Conference on Multimedia, MM'09, ACM, New York, NY, USA. pp. 481-490.

Cited By

Abreu RMattos DSantos JMuchaluat-Saade Ddos Santos JMuchaluat Saade Dda Graça C. Pimentel MMacedo A(2019)Semi-automatic synchronization of sensory effects in mulsemedia authoring toolsProceedings of the 25th Brazillian Symposium on Multimedia and the Web10.1145/3323503.3360302(201-208)Online publication date: 29-Oct-2019
https://dl.acm.org/doi/10.1145/3323503.3360302
Rodrigues RJosué MAbreu RAmorim GMuchaluat-Saade DSantos Jdos Santos JMuchaluat Saade Dda Graça C. Pimentel MMacedo A(2019)A proposal for supporting sensory effect rendering in ginga-NCLProceedings of the 25th Brazillian Symposium on Multimedia and the Web10.1145/3323503.3349544(273-280)Online publication date: 29-Oct-2019
https://dl.acm.org/doi/10.1145/3323503.3349544
Amorim MSaleme EAssis Neto FSantos CGhinea G(2019)Crowdsourcing authoring of sensory effects on videosMultimedia Tools and Applications10.1007/s11042-019-7312-278:14(19201-19227)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s11042-019-7312-2
Show More Cited By

Recommendations

A Generic Utility Model Representing the Quality of Sensory Experience
Special Issue on Multiple Sensorial (MulSeMedia) Multimodal Media : Advances and Applications

Current QoE research is mainly focusing on single modalities (audio, visual) or combinations thereof. In our research, we propose annotating traditional multimedia content with additional sensory effects, such as ambient light, vibration, wind, and ...
An end-to-end tool chain for Sensory Experience based on MPEG-V

This paper provides an overview of our research conducted in the area of Sensory Experience including our implementations using MPEG-V Part 3 entitled ''Sensory Information''. MPEG-V Part 3 introduces Sensory Experience as a tool to increase the Quality ...
Sensory Experience for Videos on the Web
MMWEB '11: Proceedings of the 2011 Workshop on Multimedia on the Web

More and more multimedia content is becoming available via the World Wide Web (WWW). These contents stimulate only senses like hearing or vision. Recently, it has been proposed to stimulate also other senses while consuming multimedia content, through ...

Comments

Information & Contributors

Information

Published In

cover image Image Communication

Image Communication Volume 27, Issue 8

September, 2012

150 pages

ISSN:0923-5965

Issue’s Table of Contents

Copyright © Elsevier B.V. © 2012.

Publisher

Elsevier Science Inc.

United States

Publication History

Published: 01 September 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 28 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Abreu RMattos DSantos JMuchaluat-Saade Ddos Santos JMuchaluat Saade Dda Graça C. Pimentel MMacedo A(2019)Semi-automatic synchronization of sensory effects in mulsemedia authoring toolsProceedings of the 25th Brazillian Symposium on Multimedia and the Web10.1145/3323503.3360302(201-208)Online publication date: 29-Oct-2019
https://dl.acm.org/doi/10.1145/3323503.3360302
Rodrigues RJosué MAbreu RAmorim GMuchaluat-Saade DSantos Jdos Santos JMuchaluat Saade Dda Graça C. Pimentel MMacedo A(2019)A proposal for supporting sensory effect rendering in ginga-NCLProceedings of the 25th Brazillian Symposium on Multimedia and the Web10.1145/3323503.3349544(273-280)Online publication date: 29-Oct-2019
https://dl.acm.org/doi/10.1145/3323503.3349544
Amorim MSaleme EAssis Neto FSantos CGhinea G(2019)Crowdsourcing authoring of sensory effects on videosMultimedia Tools and Applications10.1007/s11042-019-7312-278:14(19201-19227)Online publication date: 1-Jul-2019
https://dl.acm.org/doi/10.1007/s11042-019-7312-2
Abreu Rdos Santos JRoesler VValdeni de Lima JSaibel Santos CWillrich R(2017)Using Abstract Anchors for Automatic Authoring of Sensory Effects Based on Ambient Sound RecognitionProceedings of the 23rd Brazillian Symposium on Multimedia and the Web10.1145/3126858.3131604(437-440)Online publication date: 17-Oct-2017
https://dl.acm.org/doi/10.1145/3126858.3131604
Jung TSeo K(2016)A client-driven media synchronization mechanism for RTP packet-based video streamingJournal of Real-Time Image Processing10.1007/s11554-015-0497-312:2(455-464)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11554-015-0497-3
Rainer BTimmerer C(2014)A Generic Utility Model Representing the Quality of Sensory ExperienceACM Transactions on Multimedia Computing, Communications, and Applications10.1145/264842911:1s(1-17)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1145/2648429
Murray NLee BQiao YMuntean G(2014)Multiple-Scent Enhanced Multimedia SynchronizationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/263729311:1s(1-28)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1145/2637293
Ghinea GTimmerer CLin WGulliver S(2014)MulsemediaACM Transactions on Multimedia Computing, Communications, and Applications10.1145/261799411:1s(1-23)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1145/2617994
Luque FGalloso IFeijoo CMartín CCisneros G(2014)Integration of Multisensorial Stimuli and Multimodal Interaction in a Hybrid 3DTV SystemACM Transactions on Multimedia Computing, Communications, and Applications10.1145/261799211:1s(1-22)Online publication date: 1-Oct-2014
https://dl.acm.org/doi/10.1145/2617992
Murray NQiao YLee BMuntean G(2014)User-profile-based perceived olfactory and visual media synchronizationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/254099410:1s(1-24)Online publication date: 5-Jan-2014
https://dl.acm.org/doi/10.1145/2540994
Show More Cited By

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents