Article

Synchronizing multimodal data streams acquired using commodity hardware

Authors:

Martial Michel,

Vincent StanfordAuthors Info & Claims

VSSN '06: Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks

Pages 3 - 8

https://doi.org/10.1145/1178782.1178785

Published: 27 October 2006 Publication History

Abstract

We have developed tools and techniques that allow video frame level synchronization of multiple free-running commodity video cameras,microphones,and computer nodes using non-realtime operating systems.The techniques rely on physical audiovisual synchronization pulses,statistical procedures to correlate and interpolate the multiple timestamp streams,and software tools for review to produce smoothed and drift-corrected timestamp streams in our multimodal corpora. In this article we present those techniques and tools. Our project is open source and we are seeking collaborative developers for future work.

References

[1]

National Institute of Standards and Technology. http://www.nist.gov/

[2]

Smart Space Project. http://www.nist.gov/smartspace/

[3]

Automatic Meeting Recognition Project. http://www.nist.gov/speech/test beds/mr proj/

[4]

M. Michel, V. Stanford and O. Galibert (2005). Network Transfer of Control Data: An Application of the NIST Smart Data Flow. Proceedings of CCCT 2003. Extended version published in 2005 in the Journal of Systemics, Cybernetics and Informatics (Volume 2, Number 6).

[5]

V. Stanford, J. Garofolo, O. Galibert, M. Michel and Christophe Laprun (2003). The NIST Smart Space and Meeting Room projects: Signals, Acquisition, Annotation, and Metrics. Proceedings of ICASSP 2003.

[6]

R. Xu, G. Mei, Z. Ren, C. Kwan, J. Aube and C. Rochet and V. Stanford (2006). Towards User Sensitive Interfaces: Autodirective Speech Acquisition for Speaker Identification and Speech Recognition Using Phased Arrays. Springer Lecture Notes On Computer Science AI Subseries, 2006 Yang Cai, Ph.D.Ed.

Digital Library

[7]

J. Garofolo, C. Laprun, M. Michel, V. Stanford and Elham Tabassi (2004).The NIST Meeting Room Pilot Corpus. International Conference on Language Resources and Evaluation (LREC '04)'s Speech Corpora and Annotation/Processing Tools.

[8]

J. Garofolo, M. Michel, V. Stanford, E. Tabassi, J. Fiscus,C. Laprun, N. Pratz and J. Lard (2004). NIST Meeting Pilot Corpus Speech (ISBN 1-58563-302-x). http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2004S09

[9]

J. Garofolo, M. Michel, V. Stanford, E. Tabassi, J. Fiscus, C. Laprun, N. Pratz, J. Lard and S. Strassel (2004). NIST Meeting Pilot Corpus Transcripts and Metadata (ISBN 1-58563-303-8). http://www.ldc.upenn.edu/Catalog/CatalogEntry.jsp?catalogId=LDC2004T13

[10]

Network Time Protocol.http://www.ntp.org/

[11]

B. Widrow and E. Walach (1996. Adaptive Inverse Control. Englewood Cliffs, NJ: Prentice-Hall.

Digital Library

[12]

Rich Transcription 2002 Meeting Recognition Evaluation,documentation. http://www.nist.gov/speech/tests/rt/rt2002/

[13]

Rich Transcription 2002 STT and Metadata Extraction results,presentations,RT-02 Workshop. http://www.nist.gov/speech/tests/rt/rt2002/presentations/index.htm

[14]

Rich Transcription 2004 Spring Meeting Recognition Evaluation, documentation. http://www.nist.gov/speech/tests/rt/rt2004/spring/

[15]

Systems Plus,Inc.http://www.sysplus.com/

Cited By

Malawski FGałka J(2018)System for multimodal data acquisition for human action recognitionMultimedia Tools and Applications10.1007/s11042-018-5696-z77:18(23825-23850)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1007/s11042-018-5696-z
Shrestha PBarbieri MWeda HSekulovski D(2010)Synchronization of Multiple Camera Videos Using Audio-Visual FeaturesIEEE Transactions on Multimedia10.1109/TMM.2009.203628512:1(79-92)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1109/TMM.2009.2036285
Michel MAjot JFiscus J(2006)The NIST meeting room corpus 2 phase 1Proceedings of the Third international conference on Machine Learning for Multimodal Interaction10.1007/11965152_2(13-23)Online publication date: 1-May-2006
https://dl.acm.org/doi/10.1007/11965152_2

Index Terms

Recommendations

Summarizing spatial data streams using ClusterHulls

We consider the following problem: given an on-line, possibly unbounded stream of two-dimensional (2D) points, how can we summarize its spatial distribution or shape using a small, bounded amount of memory? We propose a novel scheme, called ClusterHull, ...
Data Streams with Bounded Deletions
PODS '18: Proceedings of the 37th ACM SIGMOD-SIGACT-SIGAI Symposium on Principles of Database Systems

Two prevalent models in the data stream literature are the insertion-only and turnstile models. Unfortunately, many important streaming problems require a Θ(log(n)) multiplicative factor more space for turnstile streams than for insertion-only streams. ...
Performance comparison of hardware virtualization platforms
NETWORKING'11: Proceedings of the 10th international IFIP TC 6 conference on Networking - Volume Part I

Hosting virtual servers on a shared physical hardware by means of hardware virtualization is common use at data centers, web hosters, and research facilities. All platforms include isolation techniques that restrict resource consumption of the virtual ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

VSSN '06: Proceedings of the 4th ACM international workshop on Video surveillance and sensor networks

October 2006

230 pages

ISBN:1595934960

DOI:10.1145/1178782

General Chairs:
Jake K. Aggarwal
University of Texas
,
Rita Cucchiara
University of Modena and Reggio Emilia, Italy
,
Program Chair:
Andrea Prati
University of Modena and Reggio Emilia, Italy

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM06

Sponsor:

MM06: The 14th ACM International Conference on Multimedia 2006

October 27, 2006

California, Santa Barbara, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
286
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Malawski FGałka J(2018)System for multimodal data acquisition for human action recognitionMultimedia Tools and Applications10.1007/s11042-018-5696-z77:18(23825-23850)Online publication date: 1-Sep-2018
https://dl.acm.org/doi/10.1007/s11042-018-5696-z
Shrestha PBarbieri MWeda HSekulovski D(2010)Synchronization of Multiple Camera Videos Using Audio-Visual FeaturesIEEE Transactions on Multimedia10.1109/TMM.2009.203628512:1(79-92)Online publication date: 1-Jan-2010
https://dl.acm.org/doi/10.1109/TMM.2009.2036285
Michel MAjot JFiscus J(2006)The NIST meeting room corpus 2 phase 1Proceedings of the Third international conference on Machine Learning for Multimodal Interaction10.1007/11965152_2(13-23)Online publication date: 1-May-2006
https://dl.acm.org/doi/10.1007/11965152_2

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents