Work in Progress

Disruptive Approaches for Subtitling in Immersive Environments

Authors:

Mario Montagud Climent,

Peter tho PeschAuthors Info & Claims

TVX '19: Proceedings of the 2019 ACM International Conference on Interactive Experiences for TV and Online Video

Pages 216 - 229

https://doi.org/10.1145/3317697.3325123

Published: 04 June 2019 Publication History

Abstract

The Immersive Accessibility Project (ImAc) explores how accessibility services can be integrated with 360o video as well as new methods for enabling universal access to immersive content. ImAc is focused on inclusivity and addresses the needs of all users, including those with sensory or learning disabilities, of all ages and considers language and user preferences. The project focuses on moving away from the constraints of existing technologies and explores new methods for creating a personal experience for each consumer. It is not good enough to simply retrofit subtitles into immersive content: this paper attempts to disrupt the industry with new and often controversial methods.

This paper provides an overview of the ImAc project and proposes guiding methods for subtitling in immersive environments. We discuss the current state-of-the-art for subtitling in immersive environments and the rendering of subtitles in the user interface within the ImAc project. We then discuss new experimental rendering modes that have been implemented including a responsive subtitle approach, which dynamically re-blocks subtitles to fit the available space and explore alternative rendering techniques where the subtitles are attached to the scene.

References

[1]

European Commission, The 2018 Ageing Report: Economic and Budgetary Projections for the EU Member States (2016-2070), Institutional Paper 079. May 2018. Brussels. ISBN 978-92-79-77460-7

[2]

UN General Assembly, Convention on the Rights of Persons with Disabilities : resolution / adopted by the General Assembly, 24 January 2007, A/RES/61/106, available at: https://www.refworld.org/docid/45f973632.html {accessed 20 March 2019}

[3]

Sahil Patel, 85 percent of Facebook video is watched without sound, Dididay UK Article, May 2016, https://digiday.com/media/silent-world-facebook-video/ {accessed 20 March 2019}

[4]

Latifi, Mehdi, Mobalegh, Ali, Mohammadi, Elham, Movie subtitles and the improvement of listening comprehension ability: Does it help?,The Journal of Language Learning and Teaching 1 / 2 (July 2016): 18-29.

[5]

Hayati, Abdolmajid & Mohmedi, Firooz. (2009). The effect of films with and without subtitles on listening comprehension of EFL learners. British Journal of Educational Technology. 42. 181 - 192.

[6]

6aM. Armstrong, A. Brown, M. Crabb, C. J. Hughes, R. Jones and J. Sandford, "Understanding the Diverse Needs of Subtitle Users in a Rapidly Evolving Media Landscape," in SMPTE Motion Imaging Journal, vol. 125, no. 9, pp. 33-41, Nov.-Dec. 2016.

[7]

M. Armstrong, A. Brown, M. Crabb, C. J. Hughes, R. Jones and J. Sandford, "Understanding the Diverse Needs of Subtitle Users in a Rapidly Evolving Media Landscape," in SMPTE Motion Imaging Journal, vol. 125, no. 9, pp. 33-41, Nov.-Dec. 2016.

[8]

[9]

Crabb, M, Jones, R, Armstrong, M and Hughes, CJ 2015, Online news videos : the UX of subtitle position, in: 17th International ACM SIGACCESS Conference on Computers & Accessibility, 26-28 October 2015, Lisbon, Portugal.

Digital Library

[10]

P. Orero, C. A. Martín, M. Zorrilla, “HBB4ALL: Deployment of HbbTV services for all”, IEEE BMSB’15, Ghent (Belgium), June 2015.

[11]

Hybrid Broadcast Broadband TV (HbbTV) 2.0.2 Specification, HbbTV Association Resource Library, https://www.hbbtv.org/resource-library, February 2018.

[12]

J. A. Núñez, M. Montagud, I. Fraile, D. Gómez, S. Fernández, “ImmersiaTV: an end-to-end toolset to enable customizable and immersive multi-screen TV experiences”, Workshop on Virtual Reality, co-located with ACM TVX 2018, Seoul (South Korea), June 2018.

[13]

ISO/IEC 23009-1: 2012. Information Technology. Dynamic Adaptive Streaming over HTTP (DASH). Part 1: Media Presentation Description and Segment Formats. April 2012.

[14]

Andy Brown, Jayson Turner, Jake Patterson, Anastasia Schmitz, Mike Armstrong, and Maxine Glancy. 2017. Subtitles in 360-degree Video. In Adjunct Publication of the 2017 ACM International Conference on Interactive Experiences for TV and Online Video (TVX '17 Adjunct). ACM, New York, NY, USA, 3-8.

Digital Library

[15]

S. Rothe, K.Tran, H. Hussmann, “Positioning of Subtitles in Cinematic Virtual Reality”, ICAT-EGVE 2018 - International Conference on Artificial Reality and Telexistence and Eurographics Symposium on Virtual Environments, November 2018

[16]

M. Schmierl, S.Rothe, “Methods of Speaker Identification for Subtitles in Cinematic Virtual Reality“ (Bachelor Thesis), November 2018

[17]

WD on ISO/IEC 23000-20 Omnidirectional Media Application Format

[18]

L. Men, N. Bryan-Kinns, A.S. Hassard, Z. Ma, “The impact of transitions on user experience in virtual reality”, IEEE Virtual Reality (VR) 2017, Los Angeles (USA), March 2017.

[19]

Y. Lin, Y. Chang, H. Hu, H. Cheng, C. Huang, M. Sun, “Tell Me Where to Look: Investigating Ways for Assisting Focus in 360° Video”, ACM CHI '17, Denver (USA), March 2017.

Digital Library

[20]

Hughes, CJ, Armstrong, M, Jones, R and Crabb, M 2015, Responsive design for personalised subtitles, in: The 12th Web for All Conference, 18-20 May 2015

Digital Library

Cited By

Nevsky ABircanin FCruice MWilson SSimperl ENeate T(2024)"I Wish You Could Make the Camera Stand Still": Envisioning Media Accessibility Interventions with People with AphasiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675598(1-17)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675598
Nevsky ANeate TSimperl ECruice M(2024)Lights, Camera, Access: A Closeup on Audiovisual Media Accessibility and AphasiaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641893(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641893
Sami HHammoud AArafeh MWazzeh MArisdakessian SChahoud MWehbi OAjaj MMourad AOtrok HAbdel Wahab OMizouni RBentahar JTalhi CDziong ZDamiani EGuizani M(2024)The Metaverse: Survey, Trends, Novel Pipeline Ecosystem & Future DirectionsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.339264226:4(2914-2960)Online publication date: Dec-2025
https://doi.org/10.1109/COMST.2024.3392642
Show More Cited By

Recommendations

Accessibility in 360° video players
Abstract
Accessibility is a key requirement for any multimedia tool and application. With the current trend towards immersive experiences, such as Virtual Reality (VR) and 360^o video, it becomes key that these environments are adapted to be fully ...
Audiovisual translation models for visually impaired users of Interactive Television (iTV)
Abstract
Audiovisual translation techniques, such as audio description (AD), subtitling and dubbing, have a significant role in the overall television experience of people with visual or hearing impairments, illiterate or persons unfamiliar with foreign ...
Quality is in the eye of the stakeholders: what do professional subtitlers and viewers think about subtitling?
Abstract
Quality is a rather slippery concept, and its assessment in subtitling can be a challenging task, as its appreciation can easily vary depending on the different stakeholders involved in the production and reception of subtitles. In this paper, we ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

TVX '19: Proceedings of the 2019 ACM International Conference on Interactive Experiences for TV and Online Video

June 2019

299 pages

ISBN:9781450360173

DOI:10.1145/3317697

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 June 2019

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Funding Sources

Horizon 2020

Conference

TVX '19

Sponsor:

SIGCHI

TVX '19: ACM International Conference on Interactive Experiences for TV and Online Video

June 5 - 7, 2019

Salford (Manchester), United Kingdom

Acceptance Rates

Overall Acceptance Rate 69 of 245 submissions, 28%

Upcoming Conference

IMX '25

Sponsor:
sigchi

ACM International Conference on Interactive Media Experiences

June 3 - 6, 2025

Niter?i , Brazil

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
215
Total Downloads

Downloads (Last 12 months)32
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Nevsky ABircanin FCruice MWilson SSimperl ENeate T(2024)"I Wish You Could Make the Camera Stand Still": Envisioning Media Accessibility Interventions with People with AphasiaProceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3663548.3675598(1-17)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3663548.3675598
Nevsky ANeate TSimperl ECruice M(2024)Lights, Camera, Access: A Closeup on Audiovisual Media Accessibility and AphasiaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641893(1-17)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641893
Sami HHammoud AArafeh MWazzeh MArisdakessian SChahoud MWehbi OAjaj MMourad AOtrok HAbdel Wahab OMizouni RBentahar JTalhi CDziong ZDamiani EGuizani M(2024)The Metaverse: Survey, Trends, Novel Pipeline Ecosystem & Future DirectionsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.339264226:4(2914-2960)Online publication date: Dec-2025
https://doi.org/10.1109/COMST.2024.3392642
Nevsky ANeate TSimperl EVatavu R(2023)Accessibility Research in Digital Audiovisual Media: What Has Been Achieved and What Should Be Done Next?Proceedings of the 2023 ACM International Conference on Interactive Media Experiences10.1145/3573381.3596159(94-114)Online publication date: 12-Jun-2023
https://dl.acm.org/doi/10.1145/3573381.3596159
Anderton C(2022)Investigating Sign Language Interpreter Rendering and Guiding Methods in Virtual Reality 360-Degree ContentProceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3517428.3563373(1-6)Online publication date: 23-Oct-2022
https://dl.acm.org/doi/10.1145/3517428.3563373
Hughes C(2022)Universal access: user needs for immersive captioningUniversal Access in the Information Society10.1007/s10209-021-00828-w21:2(393-403)Online publication date: 1-Jun-2022
https://dl.acm.org/doi/10.1007/s10209-021-00828-w
Montagud MHurtado CDe Rus JFernández S(2021)Subtitling 3D VR Content with Limited 6DoF: Presentation Modes and Guiding MethodsApplied Sciences10.3390/app1116747211:16(7472)Online publication date: 14-Aug-2021
https://doi.org/10.3390/app11167472
Vatavu R(2021)Accessibility of Interactive Television and Media Experiences: Users with Disabilities Have Been Little Voiced at IMX and TVXProceedings of the 2021 ACM International Conference on Interactive Media Experiences10.1145/3452918.3465485(218-222)Online publication date: 21-Jun-2021
https://dl.acm.org/doi/10.1145/3452918.3465485
Gorman BCrabb MArmstrong MKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)Adaptive Subtitles: Preferences and Trade-Offs in Real-Time Media AdaptionProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445509(1-11)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445509
Climent MSoler-Vilageliu OVila ILanga S(2021)VR360 Subtitling: Requirements, Technology and User ExperienceIEEE Access10.1109/ACCESS.2020.30473779(2819-2838)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2020.3047377
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten