Automatic Preparation of Sensory Effects: Managing Synchronization in Mulsemedia Applications
Pages 100 - 108
Abstract
Traditional audiovisual multimedia applications can be synchronized with sensory effects to further enhance the users' quality of experience. These applications are named mulsemedia applications and stimulate other human senses, such as touch, smell, and taste. Some delays may occur when rendering sensory effects due to specific device capabilities and communication between the mulsemedia presentation engine (formatter) and actuators. In this context, this work proposes the automatic preparation of sensory effects to avoid synchronization faults during a mulsemedia application presentation. The automatic preparation of sensory effects depends on the effect type to be rendered. As proof of concept, we implemented the automatic preparation in the Ginga-NCL presentation engine. Furthermore, we present some use cases of the preparation mechanisms applied to sensory effects. Finally, an evaluation of our proposal is presented.
References
[1]
ABNT. 2021. Digital terrestrial television - Data coding and transmission specification for digital broadcasting - Part 2: Ginga-NCL for fixed and mobile receivers -XML application language for application coding (In portuguese).
[2]
Oluwakemi A Ademoye and Gheorghita Ghinea. 2009. Synchronization of olfaction-enhanced multimedia. IEEE Transactions on Multimedia 11, 3 (2009), 561--565.
[3]
B. Choi, E.S. Lee, and K. Yoon. 2011. Streaming media with sensory effect. In Information Science and Applications (ICISA), 2011 International Conference on. IEEE, 1--6.
[4]
R. M. de R Costa, M. F. Moreno, and L. F. G. Soares. 2008. Intermedia synchronization management in DTV systems. In Proceedings of the eighth ACM symposium on Document engineering. ACM, 289--297.
[5]
G. Ghinea, C. Timmerer, W. Lin, and S. R. Gulliver. 2014. Mulsemedia: State of the art, perspectives, and challenges. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) 11 (2014), 17.
[6]
Urs Hunkeler, Hong Linh Truong, and Andy Stanford-Clark. 2008. MQTT-S---A publish/subscribe protocol for Wireless Sensor Networks. In 2008 3rd International Conference on Communication Systems Software and Middleware and Workshops (COMSWARE'08). IEEE, 791--798.
[7]
ISO/IEC. 2013. Information technology --- Media context and control --- Part 2: Control Information. ISO/IEC 23005-2:2019.
[8]
ISO/IEC. 2019. Information technology --- Media context and control --- Part 6: Common types and tools. ISO/IEC 23005-6:2019.
[9]
ISO/IEC. 2019. ISO/IEC 13818-1:2019 Information technology --- Generic coding of moving pictures and associated audio information --- Part 1: Systems.
[10]
ISO/IEC 23005-3 2011. Information technology - Media context and control - Part 3: Sensory information.
[11]
Marina Josué, Marcelo Moreno, and Débora Muchaluat-Saade. 2019. Mulsemedia preparation: a new event type for preparing media object presentation and sensory effect rendering. In Proceedings of the 10th ACM Multimedia Systems Conference. ACM, 110--120.
[12]
Marina Ivanov P Josué, Marcelo F Moreno, and Débora C Muchaluat-Saade. 2022. Supporting Targeted Advertising in Integrated Broadcast-Broadband Systems With Automatic Media Content Preparation. IEEE Access 10 (2022), 126845--126857.
[13]
Niall Murray, Yuansong Qiao, Brian Lee, A Kotegar Karunakar, and Gabriel-Miro Muntean. 2013. Subjective evaluation of olfactory and visual media synchronization. In Proceedings of the 4th ACM Multimedia Systems Conference. 162--171.
[14]
Takamichi Nakamoto and Kenjiro Yoshikawa. 2006. Movie with scents generated by olfactory display using solenoid valves. IEICE transactions on fundamentals of electronics, communications and computer sciences 89, 11 (2006), 3327--3332.
[15]
Alex Rodriguez. 2008. Restful web services: The basics. IBM developerWorks 33 (2008), 18.
[16]
E. B. Saleme. 2019. An Architectural Model and a Framework for Dealing with Variability in Mulsemedia Systems. Ph. D. Dissertation. Programa de Pós-Graduação em Informática da Universidade Federal do Espírito Santo.
[17]
E. B. Saleme, C. A. S. Santos, and G. Ghinea. 2018. Improving response time interval in networked event-based mulsemedia systems. In Proceedings of the 9th ACM Multimedia Systems Conference. 216--224.
[18]
L. F. G. Soares, R. F. Rodrigues, and D. C. M. Saade. 2000. Modeling, authoring and formatting hypermedia documents in the HyperProp system. Multimedia Systems 8, 2 (2000), 118--134.
[19]
K. Yoon. 2013. End-to-end framework for 4-D broadcasting based on MPEG-V standard. Signal Processing: Image Communication 28, 2 (2013), 127--135.
[20]
Zhenhui Yuan, Ting Bi, Gabriel-Miro Muntean, and Gheorghita Ghinea. 2015. Perceived synchronization of mulsemedia services. IEEE Transactions on Multimedia 17, 7 (2015), 957--966.
Index Terms
- Automatic Preparation of Sensory Effects: Managing Synchronization in Mulsemedia Applications
Recommendations
Preparation of Media Object Presentation and Sensory Effect Rendering in Mulsemedia Applications
WebMedia '18: Proceedings of the 24th Brazilian Symposium on Multimedia and the WebNowadays multimedia applications are employed in different fields as entertainment, education, government services, health and e-commerce. Moreover, some applications incorporate sensory effects along with traditional multimedia content, stimulating ...
Mulsemedia preparation: a new event type for preparing media object presentation and sensory effect rendering
MMSys '19: Proceedings of the 10th ACM Multimedia Systems ConferenceIn multimedia applications, a short delay in content delivery may cause synchronization failures among media objects, decreasing the presentation quality and user quality of experience. Moreover, some applications incorporate sensory effects to ...
Sensory, cognitive, and sensorimotor learning effects in recognition memory for music
Recent research suggests that perception and action are strongly interrelated and that motor experience may aid memory recognition. We investigated the role of motor experience in auditory memory recognition processes by musicians using behavioral, ERP, ...
Comments
Information & Contributors
Information
Published In
April 2024
557 pages
ISBN:9798400704123
DOI:10.1145/3625468
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 17 April 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
MMSys '24
Sponsor:
Acceptance Rates
Overall Acceptance Rate 176 of 530 submissions, 33%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 60Total Downloads
- Downloads (Last 12 months)60
- Downloads (Last 6 weeks)10
Reflects downloads up to 25 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in