research-article

Generating story variants with constrained video recombination

Authors:

Alberto Piacenza,

Fabrizio Guerrini,

Riccardo Leonardi,

Julie Porteous,

Jonathan Teutenberg,

Marc CavazzaAuthors Info & Claims

MM '11: Proceedings of the 19th ACM international conference on Multimedia

Pages 223 - 232

https://doi.org/10.1145/2072298.2072329

Published: 28 November 2011 Publication History

Abstract

We present a novel approach to the automatic generation of filmic variants within an implemented Video-Based Storytelling (VBS) system that successfully integrates video segmentation with stochastically controlled re-ordering techniques and narrative generation via AI planning. We have introduced flexibility into the video recombination process by sequencing video shots in a way that maintains local video consistency and this is combined with exploitation of shot polysemy to enable shot reuse in a range of valid semantic contexts. Results of evaluations on output narratives using a shared set of video data show consistency in terms of local video sequences and global causality with no loss of generative power.

References

[1]

N. Adami and R. Leonardi. Identification of editing effect in image sequences by statistical modeling. In Proc. Picture Coding Symposium (PCS), pages 157--160, Portland, OR, USA, 1999.

[2]

S. Benini, P. Migliorati, and R. Leonardi. Hierarchical structuring of video previews by leading cluster analysis. Signal, Image and Video Processing, 4(4):435--450, 2010.

[3]

S. Bocconi, F. Nack, and L. Hardman. Using Rhetorical Annotations for Generating Video Documentaries. In IEEE Int. Conf. on Multimedia and Expo (ICME), pages 1070--1073, Netherlands, 2005.

[4]

S. Bocconi, F. Nack, and L. Hardman. Automatic generation of matter-of-opinion video documentaries. Journal of Web Semantics, 6(2):139--150, 2008.

Digital Library

[5]

J. Boreczky and L. Rowe. Comparison of video shot boundary detection techniques. Electronic Imaging, 5(2):122--128, 1996.

[6]

K. M. Brooks. Do Story Agents Use Rocking Chairs? The Theory and Implementation of One Model for Computational Narrative. In Proc. ACM Multimedia Conf., pages 317--328, New York, NY, USA, 1996.

Digital Library

[7]

M. Cavazza, F. Charles, and S. J. Mead. Character-based interactive storytelling. IEEE Intelligent Systems, 17(4):17--24, 2002.

Digital Library

[8]

P. Cesar, D. Bulterman, and L. Soares. Human- centered television: directions in interactive television research. ACM Trans. on Multimedia Computing, Communication and Applications, 4(4), 2008.

Digital Library

[9]

Y.-G. Cheong and R. M. Young. Narrative generation for suspense: Modelling and evaluation. In Proc. 1st Joint Int. Conf. on Interactive Digital Storytelling (ICIDS), pages 144--155, 2008.

Digital Library

[10]

D. Christian and M. Young. Comparing cognitive and computational models of narrative structure. In Proc. of the National Conf. of the American Association for AI (AAAI), pages 385--390, San Jose, CA, USA, 2004.

Digital Library

[11]

J. Cohen. Statistical power analysis for the behavioral sciences. Academic Press, New York, 1988.

[12]

C. Cotsaces, N. Nikolaidis, and I. Pitas. Video shot detection and condensed representation. IEEE Signal Processing Magazine, 23(2):28--37, 2006.

[13]

C. Crawford. Chris Crawford on Game Design. New Riders Publishing, 2003.

Digital Library

[14]

G. Davenport, T. Smith, and N. Pincever. Cinematic Primitives for Multimedia. IEEE Computer Graphics and Applications, 11(4):67--74, 1991.

Digital Library

[15]

A. Gerevini and D. Long. Technical report, 2005. http://www.cs.yale.edu/homes/dvm/papers/pddl-bnf.pdf.

[16]

A. Gersho and R. M. Gray. Vector Quantization and Signal Compression. Kluwer Academic, 1992.

Digital Library

[17]

A. C. Graesser and D. Hemphill. Question answering in the context of scientific mechanisms. Journal of Memory and Language, 30(2):186--209, 1991.

[18]

B. Jung, J. Song, and Y. Lee. A narrative-based abstraction framework for story-oriented video. ACM Trans. on Multimedia Computing, Communication and Applications, 3(2), 2007.

Digital Library

[19]

C. Lanz, S. Nowak, and U. Kühhirt. Determination of categories for tagging and automated classification of film scenes. In Proc. 8th European Conf. on Interactive TV (EUROITV), pages 297--300, Finland, 2010.

Digital Library

[20]

G. Lupatini, C. Saraceno, and R. Leonardi. Scene break detection: A comparison. In Proc. Workshop on Research Issues In Data Engineering (RIDE), pages 34--41, Orlando, FL, USA, 1998.

Digital Library

[21]

M. Mateas and A. Stern. Structuring Content in the Façade Interactive Drama Architecture. In Proc. 1st Conf on AI and Interactive Digital Entertainment (AIIDE), Marina Del Rey, USA, 2005.

[22]

F. Nack. AUTEUR: The Application of Video Semantics and Theme Representation for Automated Film Editing. PhD thesis, 1996.

[23]

J. Porteous, S. Benini, L. Canini, F. Charles, M. Cavazza, and R. Leonardi. Interactive Storytelling via Video Content Recombination. In Proc. ACM Multimedia Conf., pages 1715--1718, Italy, 2010.

Digital Library

[24]

J. Porteous, M. Cavazza, and F. Charles. Applying Planning to Interactive Storytelling: Narrative Control using State Constraints. ACM TIST, 1(2):1--21, 2010.

Digital Library

[25]

M. Radford. MGM Home Ent. (Europe) Ltd., 2004. The Merchant of Venice (film adaptation).

[26]

W. Sack and M. Davis. IDIC: Assembling Video Sequences from Story Plans and Content Annotations. In Proc. ICMCS, pages 14--19, Boston, USA, 1994.

[27]

E. Shen, H. Lieberman, and G. Davenport. What's Next?: Emergent Storytelling from Video Collection. In Proc. 27th Int. Conf. on Human Factors in Computing Systems (CHI), pages 809--818, USA, 2009.

Digital Library

[28]

T. Smith and G. Davenport. The Stratification System. A Design Environment for Random Access Video. In 3rd Int. Workshop on Network and Operating System Support for Digital Audio and Video, (NOSSDAV), pages 250--261, California, USA, 1992.

Digital Library

[29]

T. Smith and N. Pincever. Parsing Movies In Context. In Proc. Summer Usenix Conf., Nashville, USA, 1991.

[30]

M. M. Yeung and B.-L. Yeo. Time-constrained clustering for segmentation of video into story units. In Proc. Int. Conf. on Pattern Recognition (ICPR), volume 3, pages 375--380, Wien, Austria, 1996.

Digital Library

[31]

R. Young. Creating Interactive Narrative Structures: The Potential for AI Approaches. In Notes of the AAAI Spring Symposia on AI and Interactive Entertainment, pages 81--82, Stanford, CA, 2000.

[32]

V. Zsombori, M. Ursu, J. Wyver, D. Williams, and I. Kegel. ShapeShifting Documentary: A Golden Age. In Proc. 6th European Conf. on Interactive TV (EUROITV), pages 40--50, 2008.

Digital Library

Cited By

Marcelino GSemedo DMourão ABlasi SMagalhães JMrak MShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Assisting News Media Editors with Cohesive Visual StorylinesProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475476(3257-3265)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475476
Kuze NKominami DKashima KHashimoto TMurata M(2018)Self-Organizing Control Mechanism Based on Collective Decision-Making for Information UncertaintyACM Transactions on Autonomous and Adaptive Systems10.1145/318334013:1(1-21)Online publication date: 16-Apr-2018
https://dl.acm.org/doi/10.1145/3183340
Wang CUrgaonkar BKesidis GGupta AChen LBirke R(2018)Effective Capacity Modulation as an Explicit Control Knob for Public Cloud ProfitabilityACM Transactions on Autonomous and Adaptive Systems10.1145/313929113:1(1-25)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.1145/3139291
Show More Cited By

Index Terms

Generating story variants with constrained video recombination
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Video summarization

Recommendations

Interactive Film Recombination

In this article, we discuss an innovative media entertainment application called Interactive Movietelling. As an offspring of Interactive Storytelling applied to movies, we propose to integrate narrative generation through artificial intelligence (AI) ...
Using Virtual Narratives to Explore Children's Story Understanding
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Interactive Narratives are systems that use automated narrative generation techniques to create multiple story variants which can be shown to an audience, as virtual narratives, using cinematic staging techniques. Previous research in this area has ...
An Interactive Narrative Platform for Story Understanding Experiments
AAMAS '17: Proceedings of the 16th Conference on Autonomous Agents and MultiAgent Systems

Interactive Narratives are systems that use automated narrative generation techniques to create multiple story variants which can be shown to an audience, as virtual narratives, using cinematic staging techniques. The focus of previous research has ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '11: Proceedings of the 19th ACM international conference on Multimedia

November 2011

944 pages

ISBN:9781450306164

DOI:10.1145/2072298

General Chairs:
K. Selçuk Candan
Arizona State University, USA
,
Sethuraman Panchanathan
Arizona State University, USA
,
Balakrishnan Prabhakaran
University of Texas at Dallas, USA
,
Program Chairs:
Hari Sundaram
Arizona State University, USA
,
Wu-Chi Feng
Portland State University, USA
,
Nicu Sebe
University of Trento, Italy

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 November 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '11

Sponsor:

SIGMM

MM '11: ACM Multimedia Conference

November 28 - December 1, 2011

Arizona, Scottsdale, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

27
Total Citations
View Citations
320
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 04 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Marcelino GSemedo DMourão ABlasi SMagalhães JMrak MShen HZhuang YSmith JYang YCesar PMetze FPrabhakaran B(2021)Assisting News Media Editors with Cohesive Visual StorylinesProceedings of the 29th ACM International Conference on Multimedia10.1145/3474085.3475476(3257-3265)Online publication date: 17-Oct-2021
https://dl.acm.org/doi/10.1145/3474085.3475476
Kuze NKominami DKashima KHashimoto TMurata M(2018)Self-Organizing Control Mechanism Based on Collective Decision-Making for Information UncertaintyACM Transactions on Autonomous and Adaptive Systems10.1145/318334013:1(1-21)Online publication date: 16-Apr-2018
https://dl.acm.org/doi/10.1145/3183340
Wang CUrgaonkar BKesidis GGupta AChen LBirke R(2018)Effective Capacity Modulation as an Explicit Control Knob for Public Cloud ProfitabilityACM Transactions on Autonomous and Adaptive Systems10.1145/313929113:1(1-25)Online publication date: 21-May-2018
https://dl.acm.org/doi/10.1145/3139291
Angelopoulos KPapadopoulos ASouza VMylopoulos J(2018)Engineering Self-Adaptive Software SystemsACM Transactions on Autonomous and Adaptive Systems10.1145/310574813:1(1-27)Online publication date: 16-Apr-2018
https://dl.acm.org/doi/10.1145/3105748
Lima EFeijó BFurtado A(2018)Video-based interactive storytelling using real-time video compositing techniquesMultimedia Tools and Applications10.1007/s11042-017-4423-577:2(2333-2357)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1007/s11042-017-4423-5
Zünd FPoulakos SKapadia MSumner R(2017)Story Version Control and Graphical Visualization for Collaborative Story AuthoringProceedings of the 14th European Conference on Visual Media Production (CVMP 2017)10.1145/3150165.3150175(1-10)Online publication date: 11-Dec-2017
https://dl.acm.org/doi/10.1145/3150165.3150175
Guerrini FAdami NBenini SPiacenza APorteous JCavazza MLeonardi R(2017)Interactive Film RecombinationACM Transactions on Multimedia Computing, Communications, and Applications10.1145/310324113:4(1-22)Online publication date: 12-Aug-2017
https://dl.acm.org/doi/10.1145/3103241
Bozgeyikli LBozgeyikli ERaij AAlqasemi RKatkoori SDubey R(2017)Vocational Rehabilitation of Individuals with Autism Spectrum Disorder with Virtual RealityACM Transactions on Accessible Computing10.1145/304678610:2(1-25)Online publication date: 6-Apr-2017
https://dl.acm.org/doi/10.1145/3046786
Poddar SGhoshal SChakrabarty KBhattacharya B(2016)Error-Correcting Sample Preparation with Cyberphysical Digital Microfluidic Lab-on-ChipACM Transactions on Design Automation of Electronic Systems10.1145/289899922:1(1-29)Online publication date: 10-Aug-2016
https://dl.acm.org/doi/10.1145/2898999
Cavazza MYoung R(2016)Introduction to Interactive StorytellingHandbook of Digital Games and Entertainment Technologies10.1007/978-981-4560-52-8_55-1(1-16)Online publication date: 18-Feb-2016
https://doi.org/10.1007/978-981-4560-52-8_55-1
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten