Poster: Device-Cloud Collaborative Video Storage for Real-time Ultra HD Video Playback
Authors: 
Zhui Zhu, Xu Wang, Fan DangAuthors Info & Claims
Zhui Zhu, Xu Wang, Fan DangAuthors Info & ClaimsPage 142
Abstract
With the rapid advancement of mobile hardware, smart-phones are now capable of capturing and playing videos in 4K and even 8K resolution, improving immersive and enjoyable viewing experience for users. However, storing these ultra-high-resolution videos poses a significant burden on the local storage of mobile devices. An alternative method is storing video on cloud with the benefits of scalable storage space, cross-device data access, data sharing and backup. Recently, the policy has been integrated into mainstream mobile operation systems. Nevertheless, downloading video from cloud may face high latency under poor network conditions, significantly diminishing the user experience.
Recent advances in client-side computation create a new opportunity for utilizing network-based super-resolution techniques to improve the quality of the displayed low-resolution videos during playback[1]. However, super-resolution is an ill-posed problem, indicating that even the same low-resolution video may correspond to multiple high-resolution videos. This implies that relying solely on low-resolution information to recover high-definition details is a challenging task.
Studies have revealed that the utilization of reference high-definition information can significantly enhance the effectiveness of super-resolution algorithms[2]. We observe that a device-cloud collaborative paradigm offers new opportunities to address the challenges of storing and playing high-resolution videos. By integrating cloud storage with on-device video enhancement and using high-resolution patches from the cloud as reference information, the quality of displayed local video can be significantly improved, thereby enhancing the playback experience.
We propose the DuoSR system, which encompasses two phases: storage phase and playback phase.
In the storage phase (Figure 1a), We have designed a down-sampling neural network and joint training it with the super-resolution network to optimize video compression. Concurrently, we design a neural-enhanced quality prediction module to predict the video quality under different reference patch and different key frame. The module generates a quality prediction table, facilitating cloud quality planner during the playback phase.
In the playback phase (Figure 1b), initially, when the user accesses a video, real-time device and network information are uploaded to the cloud. Based on these data, the cloud, aiming to maximize the Quality of Experience (QoE), determines the key frames for super-resolution and the reference information to be transmitted. Upon receiving the information about the key frames and the reference information on the device, RefVSR is performed on the key frames to reconstruct high resolution key frames. For non-key frames, we reuse the results from key frames, employing the video codec's motion vectors and residual information to upsample them.
References
[1]
Y. Huang, Y. Chen, and et al. 2021. Video rescaling networks with joint optimization strategies for downscaling and upscaling. In CVPR. 3527--3536.
[2]
Y. Jiang, K. Chan, and et al. 2022. Reference-based Image and Video Super-Resolution via C2-Matching. TPAMI (2022).
Index Terms
- Poster: Device-Cloud Collaborative Video Storage for Real-time Ultra HD Video Playback
Index terms have been assigned to the content through auto-classification.
Recommendations
Self-content super-resolution for ultra-HD up-sampling
CVMP '12: Proceedings of the 9th European Conference on Visual Media ProductionWe describe a self-content single image super-resolution algorithm based on multi-scale neighbor embeddings of image patches. We make use of the recurrence property of similar patches across different scales of an image. Inspired by manifold learning ...
Real-time Super Resolution Equipment for 8K Video
ICETE 2014: Proceedings of the 11th International Joint Conference on e-Business and Telecommunications - Volume 5Resolution is one of the most important things when assessing image quality, and image quality improves in proportion to resolution. Super Resolution (SR) is a resolution improving technology that is different from conventional image enhancement ...
PatchSR: Achieving HD Video Streaming with Pre-downloaded Universal SR Models
ICCAI '23: Proceedings of the 2023 9th International Conference on Computing and Artificial IntelligenceThe high-definition (HD) video streaming has gained tremendous popularity with the proliferation of smartphones and mobile networks. However, it is quite challenging to deliver HD online video streams directly to devices with very low bandwidth in ...
Comments
Information & Contributors
Information
Published In
February 2024
167 pages
Copyright © 2024 Owner/Author(s).
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s).
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 28 February 2024
Check for updates
Qualifiers
- Poster
Funding Sources
- NSFC 62202263
Conference
HOTMOBILE '24
Sponsor:
HOTMOBILE '24: 25th International Workshop on Mobile Computing Systems and Applications
February 28 - 29, 2024
CA, San Diego, USA
Acceptance Rates
Overall Acceptance Rate 96 of 345 submissions, 28%
Upcoming Conference
HOTMOBILE '25
- Sponsor:
- sigmobile
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 39Total Downloads
- Downloads (Last 12 months)39
- Downloads (Last 6 weeks)4
Reflects downloads up to 27 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in