research-article

Developing a predictive model of quality of experience for internet video

Authors:

Athula Balachandran,

Srinivasan Seshan,

Hui ZhangAuthors Info & Claims

ACM SIGCOMM Computer Communication Review, Volume 43, Issue 4

Pages 339 - 350

https://doi.org/10.1145/2534169.2486025

Published: 27 August 2013 Publication History

Abstract

Improving users' quality of experience (QoE) is crucial for sustaining the advertisement and subscription based revenue models that enable the growth of Internet video. Despite the rich literature on video and QoE measurement, our understanding of Internet video QoE is limited because of the shift from traditional methods of measuring video quality (e.g., Peak Signal-to-Noise Ratio) and user experience (e.g., opinion scores). These have been replaced by new quality metrics (e.g., rate of buffering, bitrate) and new engagement centric measures of user experience (e.g., viewing time and number of visits). The goal of this paper is to develop a predictive model of Internet video QoE. To this end, we identify two key requirements for the QoE model: (1) it has to be tied in to observable user engagement and (2) it should be actionable to guide practical system design decisions. Achieving this goal is challenging because the quality metrics are interdependent, they have complex and counter-intuitive relationships to engagement measures, and there are many external factors that confound the relationship between quality and engagement (e.g., type of video, user connectivity). To address these challenges, we present a data-driven approach to model the metric interdependencies and their complex relationships to engagement, and propose a systematic framework to identify and account for the confounding factors. We show that a delivery infrastructure that uses our proposed model to choose CDN and bitrates can achieve more than 20\% improvement in overall user engagement compared to strawman approaches.

References

[1]

Buyer's Guide: Content Delivery Networks. http://goo.gl/B6gMK.

[2]

Cisco study. http://goo.gl/tMRwM.

[3]

Driving Engagement for Online Video. http://goo.gl/pO5Cj.

[4]

Microsoft Smooth Streaming. http://goo.gl/6JOXh.

[5]

P.800 : Methods for subjective determination of transmission quality. http://www.itu.int/rec/T-REC-P.800-199608-I/en.

[6]

P.910: Subjective video quality assessment methods for multimedia applications. http://goo.gl/QjFhZ.

[7]

Peak signal to noise ratio. http://en.wikipedia.org/wiki/Peak_signal-to-noise_ratio.

[8]

SPEC philosophy. http://www.spec.org/spec/#philosophy.

[9]

Turbobytes: How it works. http://www.turbobytes.com/products/optimizer/.

[10]

Video quality metrics. http://goo.gl/Ga9Xz.

[11]

Vqeg. http://www.its.bldrdoc.gov/vqeg/vqeg-home.aspx.

[12]

S. Akhshabi, L. Anantakrishnan, C. Dovrolis, and A. C. Begen. What Happens when HTTP Adaptive Streaming Players Compete for Bandwidth? In Proc. NOSSDAV, 2012.

Digital Library

[13]

S. Akhshabi, A. Begen, and C. Dovrolis. An Experimental Evaluation of Rate Adaptation Algorithms in Adaptive Streaming over HTTP. In MMSys, 2011.

Digital Library

[14]

R. H. Allen and R. D. Sriram. The Role of Standards in Innovation. Elsevier: Technology Forecasting and Social Change, 2000.

[15]

A. Balachandran, V. Sekar, A. Akella, S. Seshan, I. Stoica, and H. Zhang. A quest for an internet video quality-of-experience metric. In Hotnets, 2012.

Digital Library

[16]

K. Chen, C. Huang, P. Huang, and C. Lei. Quantifying skype user satisfaction. In SIGCOMM, 2006.

Digital Library

[17]

S. Chikkerur, V. Sundaram, M. Reisslein, and L. J. Karam. Objective video quality assessment methods: A classification, review, and performance comparison. In IEEE Transactions on Broadcasting, 2011.

[18]

N. Cranley, P. Perry, and L. Murphy. User perception of adapting video quality. International Journal of Human-Computer Studies, 2006.

Digital Library

[19]

H. Deng, G. Runger, and E. Tuv. Bias of importance measures for multi-valued attributes and solutions. In ICANN, 2011.

Digital Library

[20]

F. Dobrian, V. Sekar, A. Awan, I. Stoica, D. A. Joseph, A. Ganjam, J. Zhan, and H. Zhang. Understanding the impact of video quality on user engagement. In Proc. SIGCOMM, 2011.

Digital Library

[21]

J. Esteban, S. Benno, A. Beck, Y. Guo, V. Hilt, and I. Rimac. Interactions Between HTTP Adaptive Streaming and TCP. In Proc. NOSSDAV, 2012.

Digital Library

[22]

A. Finamore, M. Mellia, M. Munafo, R. Torres, and S. G. Rao. Youtube everywhere: Impact of device and infrastructure synergies on user experience. In Proc. IMC, 2011.

Digital Library

[23]

A. Halevy, P. Norvig, and F. Pereira. The unreasonable effectiveness of data. In IEEE Intelligent Systems, 2009.

Digital Library

[24]

L. Huang, J. Jia, B. Yu, B. G. Chun, P. Maniatis, and M. Naik. Predicting Execution Time of Computer Programs Using Sparse Polynomial Regression. In Proc. NIPS, 2010.

[25]

A. Khan, L. Sun, and E. Ifeachor. Qoe prediction model and its applications in video quality adaptation over umts networks. In IEEE Transactions on Multimedia, 2012.

[26]

S. S. Krishnan and R. K. Sitaraman. Video stream quality impacts viewer behavior: inferring causality using quasi-experimental designs. In IMC, 2012.

Digital Library

[27]

B. Liu, M. Hu, and W. Hsu. Intuitive Representation of Decision Trees Using General Rules and Exceptions. In Proc. AAAI, 2000.

Digital Library

[28]

X. Liu, F. Dobrian, H. Milner, J. Jiang, V. Sekar, I. Stoica, and H. Zhang. A case for a coordinated internet video control plane. In SIGCOMM, 2012.

Digital Library

[29]

A. Mahimkar, Z. Ge, A. Shaikh, J. Wang, J. Yates, Y. Zhang, and Q. Zhao. Towards Automated Performance Diagnosis in a Large IPTV Network. In Proc. SIGCOMM, 2009.

Digital Library

[30]

V. Menkvoski, A. Oredope, A. Liotta, and A. C. Sanchez. Optimized online learning for qoe prediction. In BNAIC, 2009.

[31]

T. Mitchell. Machine Learning. McGraw-Hill.

[32]

R. K. P. Mok, E. W. W. Chan, X. Luo, and R. K. C. Chang. Inferring the QoE of HTTP Video Streaming from User-Viewing Activities . In SIGCOMM W-MUST, 2011.

Digital Library

[33]

L. Plissonneau and E. Biersack. A Longitudinal View of HTTP Video Streaming Performance. In MMSys, 2012.

Digital Library

[34]

I. Sodagar. The MPEG-DASH Standard for Multimedia Streaming Over the Internet. IEEE Multimedia, 2011.

Digital Library

[35]

H. H. Song, Z. Ge, A. Mahimkar, J. Wang, J. Yates, Y. Zhang, A. Basso, and M. Chen. Q-score: Proactive Service Quality Assessment in a Large IPTV System. In Proc. IMC, 2011.

Digital Library

[36]

M. Watson. Http adaptive streaming in practice. In MMSys - Keynote, 2011.

[37]

I. H. Witten and E. Frank. Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, 2000.

Digital Library

[38]

C. Wu, B. Li, and S. Zhao. Diagnosing Network-wide P2P Live Streaming Inefficiencies. In Proc. INFOCOM, 2009.

[39]

W. Xu, L. Huang, A. Fox, D. Patterson, and M. Jordan. Detecting large-scale system problems by mining console logs. In Proc. SOSP, 2009.

Digital Library

[40]

H. Yin et al. Inside the Bird's Nest: Measurements of Large-Scale Live VoD from the 2008 Olympics.

[41]

H. Yu, D. Z. B. Y. Zhao, and W. Zheng. Understanding User Behavior in Large-Scale Video-on-Demand Systems. In Proc. Eurosys, 2006.

Digital Library

Cited By

Yu EZhou JLi ZTyson GLi WZhang XXu ZXie G(2024)Mustang: Improving QoE for Real-Time Video in Cellular Networks by Masking JitterACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367239920:9(1-23)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3672399
Zhang ZZhu XZhang AQian FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)An In-depth Study of Bandwidth Allocation across Media Sources in Video ConferencingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681007(7696-7704)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681007
Chen TLin YChristianson NAkhtar ZDharmaji SHajiesmaili MWierman ASitaraman RSekar VYu MSeneviratne AVeitch D(2024)SODA: An Adaptive Bitrate Controller for Consistent High-Quality Video StreamingProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672260(613-644)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672260
Show More Cited By

Index Terms

Developing a predictive model of quality of experience for internet video
1. General and reference
  1. Cross-computing tools and techniques

Recommendations

Developing a predictive model of quality of experience for internet video
SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM

Improving users' quality of experience (QoE) is crucial for sustaining the advertisement and subscription based revenue models that enable the growth of Internet video. Despite the rich literature on video and QoE measurement, our understanding of ...
Understanding the impact of video quality on user engagement
SIGCOMM '11

As the distribution of the video over the Internet becomes main- stream and its consumption moves from the computer to the TV screen, user expectation for high quality is constantly increasing. In this context, it is crucial for content providers to ...
Profiting by Experience

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review

ACM SIGCOMM Computer Communication Review Volume 43, Issue 4

October 2013

595 pages

ISSN:0146-4833

DOI:10.1145/2534169

Issue’s Table of Contents

SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM
August 2013
580 pages
ISBN:9781450320566
DOI:10.1145/2486001
General Chairs:
Dah Ming Chiu
Chinese University of Hong Kong, China
,
Jia Wang
AT&T Labs - Research, USA
,
Program Chairs:
Paul Barford
University of Wisconsin, USA
,
Srinivasan Seshan
Carnegie Mellon University, USA

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2013

Published in SIGCOMM-CCR Volume 43, Issue 4

Check for updates

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

382
Total Citations
View Citations
2,996
Total Downloads

Downloads (Last 12 months)346
Downloads (Last 6 weeks)42

Reflects downloads up to 06 Nov 2024

Other Metrics

View Author Metrics

Citations

Cited By

Yu EZhou JLi ZTyson GLi WZhang XXu ZXie G(2024)Mustang: Improving QoE for Real-Time Video in Cellular Networks by Masking JitterACM Transactions on Multimedia Computing, Communications, and Applications10.1145/367239920:9(1-23)Online publication date: 10-Jun-2024
https://dl.acm.org/doi/10.1145/3672399
Zhang ZZhu XZhang AQian FCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)An In-depth Study of Bandwidth Allocation across Media Sources in Video ConferencingProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681007(7696-7704)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681007
Chen TLin YChristianson NAkhtar ZDharmaji SHajiesmaili MWierman ASitaraman RSekar VYu MSeneviratne AVeitch D(2024)SODA: An Adaptive Bitrate Controller for Consistent High-Quality Video StreamingProceedings of the ACM SIGCOMM 2024 Conference10.1145/3651890.3672260(613-644)Online publication date: 4-Aug-2024
https://dl.acm.org/doi/10.1145/3651890.3672260
Wang YLyu MSivaraman VVallina-Rodríguez NSuarez-Tángil GLevin DPelsser C(2024)Characterizing User Platforms for Video Streaming in Broadband NetworksProceedings of the 2024 ACM on Internet Measurement Conference10.1145/3646547.3688435(563-579)Online publication date: 4-Nov-2024
https://dl.acm.org/doi/10.1145/3646547.3688435
Artioli ETashtarian FTimmerer C(2024)DIGITWISEProceedings of the 15th ACM Multimedia Systems Conference10.1145/3625468.3647613(78-88)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1145/3625468.3647613
Jeong SKim BCha SSeo KChang HLee JKim YNoh J(2024)Real-Time CNN Training and Compression for Neural-Enhanced Adaptive Live StreamingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.337737246:9(6023-6039)Online publication date: 1-Sep-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3377372
Zhang YCheng SGuo ZZhang X(2024)Inferring Video Streaming Quality of Real-Time Communication Inside NetworkIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.337560434:8(7756-7770)Online publication date: 1-Aug-2024
https://dl.acm.org/doi/10.1109/TCSVT.2024.3375604
Feng TZhang ZWong WSun SSikdar B(2024)A Framework for Tradeoff Between Location Privacy Preservation and Quality of Experience in Location Based ServicesIEEE Open Journal of Vehicular Technology10.1109/OJVT.2024.33641845(428-439)Online publication date: 2024
https://doi.org/10.1109/OJVT.2024.3364184
Tao XDuan YQin ZHuang DWang LTao XDuan YQin ZHuang DWang L(2024)QoE Evaluation Model Based on EEGWireless Multimedia Computational Communications10.1007/978-3-031-64155-8_2(5-39)Online publication date: 20-Jun-2024
https://doi.org/10.1007/978-3-031-64155-8_2
Li YDeng GBai CYang JWang GZhang HBai JYuan HXu MWang S(2023)Demystifying the QoS and QoE of Edge-hosted Video Streaming Applications in the Wild with SNESetProceedings of the ACM on Management of Data10.1145/36267231:4(1-29)Online publication date: 12-Dec-2023
https://dl.acm.org/doi/10.1145/3626723
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents