research-article

Bundled camera paths for video stabilization

Authors:

Shuaicheng Liu,

Jian SunAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 32, Issue 4

Article No.: 78, Pages 1 - 10

https://doi.org/10.1145/2461912.2461995

Published: 21 July 2013 Publication History

Abstract

We present a novel video stabilization method which models camera motion with a bundle of (multiple) camera paths. The proposed model is based on a mesh-based, spatially-variant motion representation and an adaptive, space-time path optimization. Our motion representation allows us to fundamentally handle parallax and rolling shutter effects while it does not require long feature trajectories or sparse 3D reconstruction. We introduce the 'as-similar-as-possible' idea to make motion estimation more robust. Our space-time path smoothing adaptively adjusts smoothness strength by considering discontinuities, cropping size and geometrical distortion in a unified optimization framework. The evaluation on a large variety of consumer videos demonstrates the merits of our method.

Supplementary Material

ZIP File (a78-liu.zip)

Supplemental material.

Download
164.66 MB

MP4 File (tp176.mp4)

Download
34.47 MB

References

[1]

Baker, S., Bennett, E. P., Kang, S. B., and Szeliski, R. 2010. Removing rolling shutter wobble. In Proc. CVPR.

[2]

Bay, H., Ess, A., Tuytelaars, T., and Van Gool, L. 2008. Speeded-up robust features (surf). Comput. Vis. Image Underst. 110, 3, 346--359.

Digital Library

[3]

Bronshtein, I. N., and Semendyayev, K. A. 1997. Handbook of Mathematics. Springer-Verlag, New York, NY, USA.

Digital Library

[4]

Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. 2004. High accuracy optical flow estimation based on a theory for warping. In Proc. ECCV.

[5]

Buehler, C., Bosse, M., and McMillan, L. 2001. Non-metric image-based rendering for video stabilization. In Proc. CVPR.

[6]

Chen, B.-Y., Lee, K.-Y., Huang, W.-T., and Lin, J.-S. 2008. Capturing intention-based full-frame video stabilization. Computer Graphics Forum 27, 7, 1805--1814.

[7]

Cho, S., Wang, J., and Lee, S. 2012. Video deblurring for hand-held cameras using patch-based synthesis. ACM Trans. Graph. (Proc. of SIGGRAPH) 31, 4.

Digital Library

[8]

Fischler, M. A., and Bolles, R. C. 1981. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6, 381--395.

Digital Library

[9]

Forssén, P.-E., and Ringaby, E. 2010. Rectifying rolling shutter video from hand-held devices. In CVPR.

[10]

Gao, J., Kim, S. J., and Brown, M. S. 2011. Constructing image panoramas using dual-homography warping. In Proc. CVPR.

Digital Library

[11]

Gleicher, M. L., and Liu, F. 2007. Re-cinematography: Improving the camera dynamics of casual video. In Proc. of ACM Multimedia.

Digital Library

[12]

Goldstein, A., and Fattal, R. 2012. Video stabilization using epipolar geometry. ACM Trans. Graph. (TOG) 31, 5, 126:1--126:10.

Digital Library

[13]

Grundmann, M., Kwatra, V., and Essa, I. 2011. Auto-directed video stabilization with robust l1 optimal camera paths. In Proc. CVPR.

Digital Library

[14]

Grundmann, M., Kwatra, V., Castro, D., and Essa, I. 2012. Calibration-free rolling shutter removal. In Proc. ICCP.

[15]

Hartley, R., and Zisserman, A. 2003. Multiple View Geometry in Computer Vision, 2 ed. Cambridge University Press, New York, NY, USA.

Digital Library

[16]

Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Trans. Graph. (Proc. of SIGGRAPH) 24, 3, 1134--1141.

Digital Library

[17]

Karpenko, A., Jacobs, D., Baek, J., and Levoy, M. 2011. Digital video stabilization and rolling shutter correction using gyroscopes. In Stanford CS Tech Report.

[18]

Lee, K.-Y., Chuang, Y.-Y., Chen, B.-Y., and Ouhyoung, M. 2009. Video stabilization using robust feature trajectories. In Proc. ICCV.

[19]

Liang, C.-K., Chang, L.-W., and Chen, H. H. 2008. Analysis and compensation of rolling shutter effect. In IEEE Trans. on Image Processing.

Digital Library

[20]

Lin, W.-Y., Liu, S., Matsushita, Y., Ng, T.-T., and Cheong, L.-F. 2011. Smoothly varying affine stitching. In Proc. CVPR.

Digital Library

[21]

Liu, F., Gleicher, M., Jin, H., and Agarwala, A. 2009. Content-preserving warps for 3d video stabilization. ACM Trans. Graph. (Proc. of SIGGRAPH) 28.

Digital Library

[22]

Liu, F., Gleicher, M., Wang, J., Jin, H., and Agarwala, A. 2011. Subspace video stabilization. ACM Trans. Graph. 30.

Digital Library

[23]

Liu, S., Wang, Y., Yuan, L., Bu, J., Tan, P., and Sun, J. 2012. Video stabilization with a depth camera. In Proc. CVPR.

Digital Library

[24]

Lucas, B. D., and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. of the International Joint Conference on Artificial Intelligence (IJCAI), 674--679.

Digital Library

[25]

Matsushita, Y., Ofek, E., Ge, W., Tang, X., and Shum, H.-Y. 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell. 28, 1150--1163.

Digital Library

[26]

Morimoto, C., and Chellappa, R. 1998. Evaluation of image stabilization algorithms. In Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing, 2789--2792.

[27]

Nakamura, J. 2005. Image Sensors and Signal Processing for Digital Still Cameras. CRC Press, Inc.

Digital Library

[28]

Nir, T., Bruckstein, A. M., and Kimmel, R. 2008. Over-parameterized variational optical flow. Int. J. Comput. Vision (IJCV) 76, 2, 205--216.

Digital Library

[29]

Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Trans. Graph. (Proc. of SIGGRAPH) 25, 3, 533--540.

Digital Library

[30]

Shum, H.-Y., and Szeliski, R. 2000. Construction of panoramic image mosaics with global and local alignment. Int. J. Comput. Vision (IJCV) 36, 2, 101--130.

Digital Library

[31]

Smith, B. M., Zhang, L., Jin, H., and Agarwala, A. 2009. Light field video stabilization. In Proc. ICCV.

[32]

Szeliski, R. 1996. Motion estimation with quadtree splines. IEEE Trans. Pattern Anal. Mach. Intell. 18, 12, 1199--1210.

Digital Library

[33]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. ICCV, 839--846.

Digital Library

Cited By

刘少(2024)Research on Electronic Image Stabilization Algorithm Based on Smooth Camera PathOperations Research and Fuzziology10.12677/orf.2024.14325714:03(188-199)Online publication date: 2024
https://doi.org/10.12677/orf.2024.143257
Yuan CShi HLan LQiu YYuksel CWang HJiang CWu KYang Y(2024)Volumetric Homogenization for Knitwear SimulationACM Transactions on Graphics10.1145/368791143:6(1-19)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687911
Li HWang WWang XYuan XXu X(2024)Blind 3D Video Stabilization with Spatio-Temporally Varying Motion BlurACM Transactions on Multimedia Computing, Communications, and Applications10.1145/368615920:11(1-23)Online publication date: 8-Aug-2024
https://dl.acm.org/doi/10.1145/3686159
Show More Cited By

Index Terms

Bundled camera paths for video stabilization
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Image manipulation

Recommendations

Video stabilization using epipolar geometry

We present a new video stabilization technique that uses projective scene reconstruction to treat jittered video sequences. Unlike methods that recover the full three-dimensional geometry of the scene, this model accounts for simple geometric relations ...
Stabilization of panoramic videos from mobile multi-camera platforms

Wide field of view panoramic videos have recently become popular due to the availability of high resolution displays. These panoramic videos are generated by stitching video frames captured from a panoramic video acquisition system, typically comprising ...
Rethinking two-dimensional camera motion estimation assessment for digital video stabilization: A camera motion field-based metric
Abstract
Digital video stabilization aims to remove camera motion jitters through software implementation. The first step of the classical video stabilization methodology is called camera motion estimation, which is usually performed using only RGB frames ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 32, Issue 4

July 2013

1215 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2461912

Issue’s Table of Contents

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 July 2013

Published in TOG Volume 32, Issue 4

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Singapore project

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

282
Total Citations
View Citations
2,774
Total Downloads

Downloads (Last 12 months)120
Downloads (Last 6 weeks)22

Reflects downloads up to 26 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

刘少(2024)Research on Electronic Image Stabilization Algorithm Based on Smooth Camera PathOperations Research and Fuzziology10.12677/orf.2024.14325714:03(188-199)Online publication date: 2024
https://doi.org/10.12677/orf.2024.143257
Yuan CShi HLan LQiu YYuksel CWang HJiang CWu KYang Y(2024)Volumetric Homogenization for Knitwear SimulationACM Transactions on Graphics10.1145/368791143:6(1-19)Online publication date: 19-Dec-2024
https://dl.acm.org/doi/10.1145/3687911
Li HWang WWang XYuan XXu X(2024)Blind 3D Video Stabilization with Spatio-Temporally Varying Motion BlurACM Transactions on Multimedia Computing, Communications, and Applications10.1145/368615920:11(1-23)Online publication date: 8-Aug-2024
https://dl.acm.org/doi/10.1145/3686159
Park HGrama Srinivasan SCong MKim DKim BSwartz JMuseth KSifakis E(2024)Near-realtime Facial Animation by Deep 3D Simulation Super-ResolutionACM Transactions on Graphics10.1145/367068743:5(1-20)Online publication date: 9-Aug-2024
https://dl.acm.org/doi/10.1145/3670687
Li HJiang HLuo ATan PFan HZeng BLiu S(2024)DMHomo: Learning Homography with Diffusion ModelsACM Transactions on Graphics10.1145/365220743:3(1-16)Online publication date: 11-Mar-2024
https://dl.acm.org/doi/10.1145/3652207
Shao MTasdizen TJoshi S(2024)Analyzing the Domain Shift Immunity of Deep Homography Estimation2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00473(4788-4796)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00473
Liu TWan GBai HKong XTang BWang F(2024)Real-Time Video Stabilization Algorithm Based on SuperPointIEEE Transactions on Instrumentation and Measurement10.1109/TIM.2023.334284973(1-13)Online publication date: 2024
https://doi.org/10.1109/TIM.2023.3342849
Liu YLi HLiu SZeng B(2024)CodingHomo: Bootstrapping Deep Homography With Video CodingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.341877134:11_Part_1(11214-11228)Online publication date: 24-Jun-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3418771
Peng YYang WWang ZXu Y(2024)Research on infrared video stabilization based on SLIC and path optimization2024 4th International Conference on Neural Networks, Information and Communication (NNICE)10.1109/NNICE61279.2024.10498672(725-729)Online publication date: 19-Jan-2024
https://doi.org/10.1109/NNICE61279.2024.10498672
Cheng SChen ZGuo LTao W(2024)Deep Homography Estimation via Dense Scene MatchingIEEE Robotics and Automation Letters10.1109/LRA.2024.34263789:9(7493-7500)Online publication date: Sep-2024
https://doi.org/10.1109/LRA.2024.3426378
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents