short-paper

Using Abstract Anchors for Automatic Authoring of Sensory Effects Based on Ambient Sound Recognition

Authors:

Raphael Abreu,

Joel dos SantosAuthors Info & Claims

WebMedia '17: Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web

Pages 437 - 440

https://doi.org/10.1145/3126858.3131604

Published: 17 October 2017 Publication History

Get Access

Abstract

Mulsemedia applications are shown to improve user experience by synchronizing sensory effects along with audiovisual content. An usual approach for specifying such synchronization is to mark the moments when a given effect has to be executed in the main media object. Multimedia authoring languages provide a way to mark moments of a media object through anchors. An anchor indicates a temporal portion (video frames or audio samples) in relation to its parent media object. However, to synchronize such portions with sensory content can be costly. When, for example, a scene component appear multiple times on the media content.

This problem was tackled in previous work by providing an approach for creating abstract anchors in declarative multimedia documents. An abstract anchor represents (possibly) several media anchors. By using a scene recognition software, the anchors where duplicated whenever the same component was identified on the main media. Such approach, was proven useful for creating automatic sensory effects based on video recognition.

This paper extends the proposal of abstract anchors in order to enable the automatic generation of sensory effects using sound recognition. This paper also extends the Abstract Anchor Processor to implement media pre-processing and to adapt the sound recognition software response.

References

[1]

Raphael Abreu and Joel A. F. dos Santos. 2017. Using Abstract Anchors to Aid The Development of MultimediaApplications With Sensory Effects. In 2017 ACM Symposium on Document Engineering (DocEng 2017).

Digital Library

Google Scholar

[2]

Joel André Ferreira dos Santos and Débora Christina Muchaluat Saade. 2010. XTemplate 3.0: Adding Semantics to Hypermedia Compositions and Providing Document Structure Reuse. In Proceedings of the 2010 ACM Symposium on Applied Computing (SAC '10). ACM, New York, NY, USA, 1892--1897.

Digital Library

Google Scholar

[3]

Gheorghita Ghinea, Christian Timmerer,Weisi Lin, and Stephen R. Gulliver. 2014. Mulsemedia : State of the Art, Perspectives, and Challenges. ACM Transactions on Multimedia Computing, Communications, and Applications 11, 1s (2014), 1--23.

Digital Library

Google Scholar

[4]

ITU. 2014. Nested Context Language (NCL) and Ginga-NCL for IPTV services. http://handle.itu.int/11.1002/1000/12237. (2014).

Google Scholar

[5]

Carlos de Salles Soares Neto, Luiz Fernando Gomes Soares, and Clarisse Sieckenius de Souza. 2012. TAL-Template Authoring Language. Journal of the Brazilian Computer Society 18, 3 (2012), 185--199.

Crossref

Google Scholar

[6]

K. J. Piczak. 2015. Environmental sound classification with convolutional neural networks. In 2015 IEEE 25th International Workshop on Machine Learning for Signal Processing (MLSP). 1--6.

Crossref

Google Scholar

[7]

Justin Salamon, Christopher Jacoby, and Juan Pablo Bello. 2014. A Dataset and Taxonomy for Urban Sound Research. In Proceedings of the 22Nd ACM International Conference on Multimedia (MM '14). ACM, New York, NY, USA, 4.

Digital Library

Google Scholar

[8]

Estêvão Bissoli Saleme and Celso Alberto Saibel Santos. 2015. PlaySEM: A Platform for Rendering MulSeMedia Compatible with MPEG-V. Proceedings of the 21st Brazilian Symposium on Multimedia and the Web (2015), 145--148.

Digital Library

Google Scholar

[9]

Sang Ho Shin, Keum Sook Ha, Han O. Yun, and Yoon Seok Nam. 2016. Realistic media authoring tool based on MPEG-V international standard. International Conference on Ubiquitous and Future Networks, ICUFN (2016), 730--732.

Google Scholar

[10]

Y. Sulema. 2016. Mulsemedia vs. Multimedia: State of the art and future trends. In 2016 International Conference on Systems, Signals and Image Processing. 1--5.

Crossref

Google Scholar

[11]

Christian Timmerer, Markus Waltl, Benjamin Rainer, and Hermann Hellwagner. 2012. Assessing the quality of sensory experience for multimedia presentations. Signal Processing: Image Communication 27, 8 (2012), 909--916.

Digital Library

Google Scholar

[12]

W3C. 2008. Synchronized Multimedia Integration Language - SMIL 3.0 Specification. http://www.w3c.org/TR/SMIL3. (2008).

Google Scholar

[13]

Markus Waltl, Benjamin Rainer, Christian Timmerer, and Hermann Hellwagner. 2013. An end-to-end tool chain for Sensory Experience based on MPEG-V. Signal Processing: Image Communication 28, 2 (2013), 136--150.

Digital Library

Google Scholar

Cited By

View all

Abreu RMattos DSantos JMuchaluat-Saade Ddos Santos JMuchaluat Saade Dda Graça C. Pimentel MMacedo A(2019)Semi-automatic synchronization of sensory effects in mulsemedia authoring toolsProceedings of the 25th Brazillian Symposium on Multimedia and the Web10.1145/3323503.3360302(201-208)Online publication date: 29-Oct-2019
https://dl.acm.org/doi/10.1145/3323503.3360302

Index Terms

Using Abstract Anchors for Automatic Authoring of Sensory Effects Based on Ambient Sound Recognition

Recommendations

Using Abstract Anchors to Aid The Development of Multimedia Applications With Sensory Effects
DocEng '17: Proceedings of the 2017 ACM Symposium on Document Engineering

Declarative multimedia authoring languages allows authors to combine multiple media objects, generating a range of multimedia presentations. Novel multimedia applications, focusing at improving user experience, extend multimedia applications with ...
Modeling sensory effects as first-class entities in multimedia applications
MMSys '18: Proceedings of the 9th ACM Multimedia Systems Conference

Multimedia applications are usually composed by audiovisual content. Traditional multimedia conceptual models, and consequently declarative multimedia authoring languages, do not support the definition of multiple sensory effects. Multiple sensorial ...
Crowdsourcing authoring of sensory effects on videos

Human perception is inherently multi-sensorial involving five traditional senses: sight, hearing, touch, taste, and smell. In contrast to traditional multimedia, based on audio and visual stimuli, mulsemedia seek to stimulate all the human senses. One ...

Comments

Information & Contributors

Information

Published In

WebMedia '17: Proceedings of the 23rd Brazillian Symposium on Multimedia and the Web

October 2017

522 pages

ISBN:9781450350969

DOI:10.1145/3126858

General Chairs:
Valter Roesler
UFRGS, Brazil
,
José Valdeni de Lima
UFRGS, Brazil
,
Program Chairs:
Celso Alberto Saibel Santos
UFES, Brazil
,
Roberto Willrich
UFSC, Brazil

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 October 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Conference

Webmedia '17

Sponsor:

SBC
CNPq
CGIBR
CAPES

Webmedia '17: Brazilian Symposium on Multimedia and the Web

October 17 - 20, 2017

RS, Gramado, Brazil

Acceptance Rates

WebMedia '17 Paper Acceptance Rate 38 of 138 submissions, 28%;

Overall Acceptance Rate 270 of 873 submissions, 31%

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
50
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

View all

Abreu RMattos DSantos JMuchaluat-Saade Ddos Santos JMuchaluat Saade Dda Graça C. Pimentel MMacedo A(2019)Semi-automatic synchronization of sensory effects in mulsemedia authoring toolsProceedings of the 25th Brazillian Symposium on Multimedia and the Web10.1145/3323503.3360302(201-208)Online publication date: 29-Oct-2019
https://dl.acm.org/doi/10.1145/3323503.3360302

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Cited By

Index Terms

Recommendations

Using Abstract Anchors to Aid The Development of Multimedia Applications With Sensory Effects

Modeling sensory effects as first-class entities in multimedia applications

Crowdsourcing authoring of sensory effects on videos

Comments

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Other Metrics

Article Metrics

Other Metrics

Cited By

Login options

Full Access

PDF

eReader

Abstract

References

Cited By

Index Terms

Recommendations

Using Abstract Anchors to Aid The Development of Multimedia Applications With Sensory Effects

Modeling sensory effects as first-class entities in multimedia applications

Crowdsourcing authoring of sensory effects on videos

Comments

Information

Published In

Sponsors

In-Cooperation

Publisher

Publication History

Permissions

Check for updates

Author Tags

Qualifiers

Conference

Acceptance Rates

Contributors

Other Metrics

Bibliometrics

Article Metrics

Other Metrics

Citations

Cited By

Login options

Full Access

View options

PDF

eReader

Figures

Other

Share

Share this Publication link

Share on social media

Affiliations