Article

Predictive perceptual compression for real time video communication

Authors:

Oleg Komogortsev,

Javed KhanAuthors Info & Claims

MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia

Pages 220 - 227

https://doi.org/10.1145/1027527.1027577

Published: 10 October 2004 Publication History

Abstract

Approximately 2 degrees in our 140 degree vision span has sharp vision. Many researchers have been fascinated by the idea of eye-tracking integrated perceptual compression of an image or video, yet any practical system has yet to emerge. The unique challenge presented by real time perceptual video streaming is how to handle the fast nature of the human eye and provide its integration with computationally intensive video transcoding scheme. The delay introduced by video transmission in the network presents a difficulty. This delay creates a problem when we try to use information about eye movements for perceptual encoding. In this paper we discuss a new approach to the eye-tracker based video compression. Rather than relying on the point of gaze, this novel scheme tracks a vicinity of interest and offers a prediction mechanism for eye movements. The described system compensates the interim eye movements between the sampling and actual coding. The proposed scheme can be applied to a large variety of today's video compression standards. We have developed an eye gaze-aware MPEG-2 transcoder that can perceptually re-encode a live video stream in real time. The experiments we have conducted illustrate the substantial impact this integrated prediction method has on perceptual video compression and bit-rate reduction.

References

[1]

Daly, S. J., Matthews, K., Ribas-Corbera, J. Visual eccentricity models in face-based video compression. In Human Vision and Electronic Imaging IV, May 1995, SPIE.

[2]

Duchowski, A. T. Acuity-Matching Resolution Degradation Through Wavelet Coefficient Scaling. IEEE Transactions on Image Processing 9, 8. August 2000.

Digital Library

[3]

Geisler, W. S., Perry, J. S. Real-time Foveated Multiresolution System for Low-bandwidth Video Communication. In Human Vision and Electronic Imaging III, July 1998, SPIE.

[4]

Irwin, D. E. Visual Memory Within and Across Fixations. In Eye movements and Visual Cognition: Scene Preparation And Reading, K. Rayner, Ed. Springer-Verlag, New-York, NY,1992, pp. 146--165. Springer Series in Neuropsychology.

[5]

Khan, J., Gu, Q., and Zaghal, R. Symbiotic Video Streaming by Transport Feedback based Quality rate Selection In Proceedings of the 12th IEEE International Packet Video Workshop 2002, Pittsburg, PA, April 2002,

[6]

Khan.J., Yun D. Multi Resolution Perceptual Encoding for Interactive Image Sharing in Remote Tele-Diagnostics, Manufacturing Agility and Hybrid Automation -I. In Proceedings of the International Conference on Human Aspects of Advanced Manufacturing: Agility & Hybrid Automation, HAAMAHA'96, Hawaii, Aug. 1996, pp183--187.

[7]

Komogortsev, O., Khan, J., Video Set for Predictive Perceptual Compression Test. At www.cs.kent.edu/~okomogor/ACM04VideoSet.htm.

[8]

Kuyel, T., Geisler, W. S., Ghosh, J. Retinally reconstructed images (RRIs): digital images having a resolution match with the human eye. In Human Vision and Electronic Imaging III, July 1998, SPIE .

[9]

Lee, S., Pattichis, M., Bovok, A. Foveated Video Compression with Optimal Rate Control. In IEEE Transaction of Image Processing, V. 10, n.7, July 2001, pp-977--992.

Digital Library

[10]

Loschky, L., McConkie, G. User performance with gaze contingent multiresolutional displays. In Proceedings of the symposium on Eye tracking research & applications. November 2000.

Digital Library

[11]

Niu, E. L. Gaze-based video compression using wavelets. M.S. Thesis. University of Illinois at Urbana-Champaign. The Graduate College. August 1995.

[12]

Wang, Z., Lu, L., and Bovik, A. Rate scalable video coding using a foveation-based human visual system model. IEEE International Conference on Acoustics, Speech, & Signal Processing, May 2001.

Digital Library

[13]

Westen, S. J., Lagendijk, R., Biemond, J. Spatio-temporal model of human vision for digital video compression. In Human Vision and Electronic Imaging II, June 1997, SPIE.

[14]

Yoon, S., Ratakonda, K., Ahuja, N. Region-Based Video Coding Using A Multiscale Image Segmentation. In Proceedings of 1997 International Conference on Image Processing (ICIP '97).

[15]

Kortum, P., Geisler, W. S., Implementation of a Foveated Image Coding System for Image Bandwidth Reduction. In Human Vision and Electronic Imaging, April 1996, SPIE.

Cited By

Li DDu RBabu ABrumar CVarshney A(2021)A Log-Rectilinear Transformation for Foveated 360-degree Video StreamingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.306776227:5(2638-2647)Online publication date: May-2021
https://doi.org/10.1109/TVCG.2021.3067762
Ryoo JYun KSamaras DDas SZelinsky GTimmerer C(2016)Design and evaluation of a foveated video streaming service for commodity client devicesProceedings of the 7th International Conference on Multimedia Systems10.1145/2910017.2910592(1-11)Online publication date: 10-May-2016
https://dl.acm.org/doi/10.1145/2910017.2910592
Kim GYi KKyung C(2016)A Content-Aware Video Encoding Scheme Based on Single-Pass Consistent Quality ControlIEEE Transactions on Broadcasting10.1109/TBC.2016.256999962:4(800-816)Online publication date: Dec-2016
https://doi.org/10.1109/TBC.2016.2569999
Show More Cited By

Index Terms

Predictive perceptual compression for real time video communication

Recommendations

Content-aware error-resilient transcoding using prioritized intra-refresh for video streaming

Transmitting video data over wireless networks can be very unreliable due to packet-loss, leading to serious video quality degradation which is annoying to human perception. The lost packets not only affect the quality of current frame, but also lead to ...
Fast video transcoding from H.263 to H.264/MPEG-4 AVC

In the past 10 years detailed works on different video transcoders have been published. However, the new ITU-T Recommendation H.264--also adapted as ISO/IEC MPEG-4 Part 10 (AVC)--provides many new encoding options for the prediction processes that lead ...
Real-time lossless compression of mosaic video sequences
Special issue on multi-dimensional image processing

This paper presents a simple, fast coding technique for lossless compression of mosaic video data. The design of a video codec needs to strike a balance between the compression performance and the codec throughput. Aiming to make the encoding throughput ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MULTIMEDIA '04: Proceedings of the 12th annual ACM international conference on Multimedia

October 2004

1028 pages

ISBN:1581138938

DOI:10.1145/1027527

General Chairs:
Henning Schulzrinne
Columbia University
,
Nevenka Dimitrova
Philips Research
,
Program Chairs:
Angela Sasse
UCL
,
Sue Moon
KAIST
,
Rainer Lienhart
U Augsburg

Copyright © 2004 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2004

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM04

Sponsor:

MM04: 2004 12th Annual ACM International Conference on Multimedia

October 10 - 16, 2004

NY, New York, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
627
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li DDu RBabu ABrumar CVarshney A(2021)A Log-Rectilinear Transformation for Foveated 360-degree Video StreamingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.306776227:5(2638-2647)Online publication date: May-2021
https://doi.org/10.1109/TVCG.2021.3067762
Ryoo JYun KSamaras DDas SZelinsky GTimmerer C(2016)Design and evaluation of a foveated video streaming service for commodity client devicesProceedings of the 7th International Conference on Multimedia Systems10.1145/2910017.2910592(1-11)Online publication date: 10-May-2016
https://dl.acm.org/doi/10.1145/2910017.2910592
Kim GYi KKyung C(2016)A Content-Aware Video Encoding Scheme Based on Single-Pass Consistent Quality ControlIEEE Transactions on Broadcasting10.1109/TBC.2016.256999962:4(800-816)Online publication date: Dec-2016
https://doi.org/10.1109/TBC.2016.2569999
Bai XWang JSimons DSapiro GFunkhouser T(2009)Video SnapCutACM SIGGRAPH 2009 papers10.1145/1576246.1531376(1-11)Online publication date: 27-Jul-2009
https://dl.acm.org/doi/10.1145/1576246.1531376
Bai XWang JSimons DSapiro G(2009)Video SnapCutACM Transactions on Graphics10.1145/1531326.153137628:3(1-11)Online publication date: 27-Jul-2009
https://dl.acm.org/doi/10.1145/1531326.1531376
Duchowski ABate DStringfellow PThakur KMelloy BGramopadhye A(2009)On spatiochromatic visual sensitivity and peripheral color LOD managementACM Transactions on Applied Perception10.1145/1498700.14987036:2(1-18)Online publication date: 10-Mar-2009
https://dl.acm.org/doi/10.1145/1498700.1498703
Komogortsev O(2009)Gaze-contingent video compression with targeted gaze containment performanceJournal of Electronic Imaging10.1117/1.315860918:3(033001)Online publication date: 1-Jul-2009
https://doi.org/10.1117/1.3158609
Komogortsev OKhan J(2008)Predictive real-time perceptual compression based on eye-gaze-position analysisACM Transactions on Multimedia Computing, Communications, and Applications10.1145/1386109.13861164:3(1-16)Online publication date: 18-Sep-2008
https://dl.acm.org/doi/10.1145/1386109.1386116
Khan JGuo Z(2008)Fast perceptual region tracking with coding-depth sensitive access for stream transcodingJournal of Visual Communication and Image Representation10.1016/j.jvcir.2008.01.00519:6(355-371)Online publication date: 1-Aug-2008
https://dl.acm.org/doi/10.1016/j.jvcir.2008.01.005
Loschky LWolverton G(2007)How late can you update gaze-contingent multiresolutional displays without detection?ACM Transactions on Multimedia Computing, Communications, and Applications10.1145/1314303.13143103:4(1-10)Online publication date: 12-Dec-2007
https://dl.acm.org/doi/10.1145/1314303.1314310
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten