research-article

Deep Super-Resolution Network for Single Image Super-Resolution with Realistic Degradations

Authors:

Rao Muhammad Umer,

Gian Luca Foresti,

Christian MicheloniAuthors Info & Claims

ICDSC 2019: Proceedings of the 13th International Conference on Distributed Smart Cameras

Article No.: 21, Pages 1 - 7

https://doi.org/10.1145/3349801.3349823

Published: 09 September 2019 Publication History

Abstract

Single Image Super-Resolution (SISR) aims to generate a high-resolution (HR) image of a given low-resolution (LR) image. The most of existing convolutional neural network (CNN) based SISR methods usually take an assumption that a LR image is only bicubicly down sampled version of an HR image. However, the true degradation (i.e. the LR image is a bicubicly downsampled, blurred and noisy version of an HR image) of a LR image goes beyond the widely used bicubic assumption, which makes the SISR problem highly ill-posed nature of inverse problems. To address this issue, we propose a deep SISR network that works for blur kernels of different sizes, and different noise levels in an unified residual CNN-based denoiser network, which significantly improves a practical CNN-based super-resolver for real applications. Extensive experimental results on synthetic LR datasets and real images demonstrate that our proposed method not only can produce better results on more realistic degradation but also computational efficient to practical SISR applications.

References

[1]

Pablo Andres Arbelaez, Michael Maire, Charless C. Fowlkes, and Jitendra Malik. 2011. Contour Detection and Hierarchical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33 (2011), 898--916.

Digital Library

[2]

Siavash Arjomand Bigdeli, Matthias Zwicker, Paolo Favaro, and Meiguang Jin. 2017. Deep mean-shift priors for image restoration. In Advances in Neural Information Processing Systems. 763--772.

[3]

Giacomo Boracchi and Alessandro Foi. 2012. Modeling the Performance of Image Restoration From Motion Blur. IEEE Transactions on Image Processing 21 (2012), 3502--3517.

Digital Library

[4]

Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. 2011. Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers. Found. Trends Mach. Learn. 3, 1 (Jan. 2011), 1--122.

Digital Library

[5]

Antonin Chambolle and Thomas Pock. 2011. A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging. Journal of Mathematical Imaging and Vision 40, 1 (01 May 2011), 120--145.

Digital Library

[6]

Yunjin Chen and Thomas Pock. 2017. Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (2017), 1256--1272.

Digital Library

[7]

Bernhard Dieber, Christian Micheloni, and Bernhard Rinner. 2011. Resource-Aware Coverage and Task Assignment in Visual Sensor Networks. IEEE Trans. Circuits Syst. Video Techn. 21 (10 2011), 1424--1437. https://doi.org/10.1109/TCSVT.2011.2162770

[8]

Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2014. Learning a Deep Convolutional Network for Image Super-Resolution. In ECCV.

[9]

Chao Dong, Chen Change Loy, Kaiming He, and Xiaoou Tang. 2016. Image Super-Resolution Using Deep Convolutional Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38 (2016), 295--307.

Digital Library

[10]

Weisheng Dong, Lei Zhang, Guangming Shi, and Xin Li. 2013. Nonlocally Centralized Sparse Representation for Image Restoration. IEEE Transactions on Image Processing 22 (2013), 1620--1630.

Digital Library

[11]

Netalee Efrat, Daniel Glasner, Alexander Apartsin, Boaz Nadler, and Anat Levin. 2013. Accurate Blur Models vs. Image Priors in Single Image Super-resolution. 2013 IEEE International Conference on Computer Vision (2013), 2832--2839.

[12]

Karen O. Egiazarian and Vladimir Katkovnik. 2015. Single image super-resolution via BM3D sparse coding. 2015 23rd European Signal Processing Conference (EUSIPCO) (2015), 2849--2853.

[13]

G. L. Foresti, C. Micheloni, L. Snidaro, and C. Marchiol. 2003. Face detection for visual surveillance. In 12th International Conference on Image Analysis and Processing, 2003.Proceedings. 115--120. https://doi.org/10.1109/ICIAP.2003.1234036

[14]

Jorge García, Niki Martinel, Alfredo Gardel, Ignacio Bravo, Gian Luca Foresti, and Christian Micheloni. 2016. Modeling Feature Distances by Orientation Driven Classifiers for Person Re-identification. J. Vis. Comun. Image Represent. 38, C (July 2016), 115--129. https://doi.org/10.1016/j.jvcir.2016.02.009

Digital Library

[15]

D. Geman and Chengda Yang. 1995. Nonlinear image recovery with half-quadratic regularization. IEEE Transactions on Image Processing 4, 7 (July 1995), 932--946.

Digital Library

[16]

T. Goldstein and S. Osher. 2009. The Split Bregman Method for L1-Regularized Problems. SIAM Journal on Imaging Sciences 2, 2 (2009), 323--343. https://doi.org/10.1137/080725891

Digital Library

[17]

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. 2015 IEEE International Conference on Computer Vision (ICCV) (2015), 1026--1034.

Digital Library

[18]

Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. 2015. Single image superresolution from transformed self-exemplars. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015), 5197--5206.

[19]

Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1646--1654.

[20]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980 (2015).

[21]

Stamatios Lefkimmiatis. 2017. Non-local Color Image Denoising with Convolutional Neural Networks. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 5882--5891.

[22]

Stamatios Lefkimmiatis. 2018. Universal Denoising Networks: A Novel CNN Architecture for Image Denoising. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 3204--3213.

[23]

Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. 2017. Enhanced Deep Residual Networks for Single Image Super-Resolution. 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (2017), 1132--1140.

[24]

Julien Mairal, Francis R. Bach, Jean Ponce, Guillermo Sapiro, and Andrew Zisserman. 2009. Non-local sparse models for image restoration. 2009 IEEE 12th International Conference on Computer Vision (2009), 2272--2279.

[25]

Neal Parikh and Stephen Boyd. 2014. Proximal Algorithms. Found. Trends Optim. 1, 3 (Jan. 2014), 127--239.

Digital Library

[26]

Asha Rani, Gian Luca Foresti, and Christian Micheloni. 2015. A Neural Tree for Classification Using Convex Objective Function. Pattern Recogn. Lett. 68, P1 (Dec. 2015), 41--47. https://doi.org/10.1016/j.patrec.2015.08.017

Digital Library

[27]

J. C. SanMiguel, C. Micheloni, K. Shoop, G. Foresti, and A. Cavallaro. 2014. Self-Reconfigurable Smart Camera Networks. Computer 47, 05 (May 2014), 67--73. https://doi.org/10.1109/MC.2014.133

[28]

Uwe Schmidt and Stefan Roth. 2014. Shrinkage Fields for Effective Image Restoration. 2014 IEEE Conference on Computer Vision and Pattern Recognition (2014), 2774--2781.

[29]

Wenzhe Shi, Jose Caballero, Ferenc Huszar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1874--1883.

[30]

Radu Timofte, Vincent De Smet, and Luc Van Gool. 2014. A+: Adjusted Anchored Neighborhood Regression for Fast Super-Resolution. In ACCV.

[31]

Norbert Wiener. 1964. Extrapolation, Interpolation, and Smoothing of Stationary Time Series. The MIT Press.

[32]

Jianchao Yang, John N. Wright, Thomas S. Huang, and Yi Ma. 2010. Image Super-Resolution Via Sparse Representation. IEEE Transactions on Image Processing 19 (2010), 2861--2873.

Digital Library

[33]

Linwei Yue, Huanfeng Shen, Jie Li, Qiangqiang Yuan, Hongyan Zhang, and Liangpei Zhang. 2016. Image super-resolution: The techniques, applications, and future. Signal Processing 128 (2016), 389--408.

Digital Library

[34]

Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing 26 (2017), 3142--3155.

Digital Library

[35]

Kai Zhang, Wangmeng Zuo, Shuhang Gu, and Lei Zhang. 2017. Learning Deep CNN Denoiser Prior for Image Restoration. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 2808--2817.

[36]

Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2018. Learning a Single Convolutional Super-Resolution Network for Multiple Degradations. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 3262--3271.

Cited By

Khan AMicheloni CMartinel N(2024)Lightweight Prompt Learning Implicit Degradation Estimation Network for Blind Super ResolutionIEEE Transactions on Image Processing10.1109/TIP.2024.344261333(4556-4567)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3442613
Khan AUmer RDunnhofer MMicheloni CMartinel N(2023)LBKENet:Lightweight Blur Kernel Estimation Network for Blind Image Super-ResolutionImage Analysis and Processing – ICIAP 202310.1007/978-3-031-43153-1_18(209-222)Online publication date: 5-Sep-2023
https://doi.org/10.1007/978-3-031-43153-1_18
Chen ZTang JZhang XWu QWang YSo DJin SWong K(2022)Offset Learning Based Channel Estimation for Intelligent Reflecting Surface-Assisted Indoor CommunicationIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2021.312935016:1(41-55)Online publication date: Jan-2022
https://doi.org/10.1109/JSTSP.2021.3129350
Show More Cited By

Deep Super-Resolution Network for Single Image Super-Resolution with Realistic Degradations
1. Computing methodologies

Recommendations

Deep Residual Attention Network for Spectral Image Super-Resolution
Computer Vision – ECCV 2018 Workshops
Abstract
Spectral imaging sensors often suffer from low spatial resolution, as there exists an essential tradeoff between the spectral and spatial resolutions that can be simultaneously achieved, especially when the temporal resolution needs to be ...
Spatial-Spectral Deep Residual Network for Hyperspectral Image Super-Resolution
Abstract
Recently, single hyperspectral image super-resolution (SR) methods based on deep learning have been extensively studied. However, there has been limited technical development focusing on single hyperspectral image super-resolution due to the high-...
Image Super-Resolution Using Deep Belief Networks
ICIMCS '14: Proceedings of International Conference on Internet Multimedia Computing and Service

In this paper, we aim at using Deep Belief Networks (DBNs) to solve the problem of image super-resolution (SR). We exploit the hierarchical structure of the DBNs to capture the non-linear mapping from low-resolution (LR) patches to their high-resolution ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICDSC 2019: Proceedings of the 13th International Conference on Distributed Smart Cameras

September 2019

172 pages

ISBN:9781450371896

DOI:10.1145/3349801

General Chairs:
Nicola Conci,
Caifeng Shan,
Program Chairs:
Lucio Marcenaro,
Jungong Han

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

University of Trento

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 09 September 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICDSC 2019

ICDSC 2019: 13th International Conference on Distributed Smart Cameras

September 9 - 11, 2019

Trento, Italy

Acceptance Rates

Overall Acceptance Rate 92 of 117 submissions, 79%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
226
Total Downloads

Downloads (Last 12 months)29
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Khan AMicheloni CMartinel N(2024)Lightweight Prompt Learning Implicit Degradation Estimation Network for Blind Super ResolutionIEEE Transactions on Image Processing10.1109/TIP.2024.344261333(4556-4567)Online publication date: 2024
https://doi.org/10.1109/TIP.2024.3442613
Khan AUmer RDunnhofer MMicheloni CMartinel N(2023)LBKENet:Lightweight Blur Kernel Estimation Network for Blind Image Super-ResolutionImage Analysis and Processing – ICIAP 202310.1007/978-3-031-43153-1_18(209-222)Online publication date: 5-Sep-2023
https://doi.org/10.1007/978-3-031-43153-1_18
Chen ZTang JZhang XWu QWang YSo DJin SWong K(2022)Offset Learning Based Channel Estimation for Intelligent Reflecting Surface-Assisted Indoor CommunicationIEEE Journal of Selected Topics in Signal Processing10.1109/JSTSP.2021.312935016:1(41-55)Online publication date: Jan-2022
https://doi.org/10.1109/JSTSP.2021.3129350
Ayyoob MIlyas P(2022)Efficient Binarization of Ancient Handwritten Vattezhuthu Documents2022 3rd International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT)10.1109/ICICT55121.2022.10064503(1-3)Online publication date: 11-Nov-2022
https://doi.org/10.1109/ICICT55121.2022.10064503
Umer RMicheloni C(2022)Real Image Super-Resolution using GAN through modeling of LR and HR process2022 18th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS)10.1109/AVSS56176.2022.9959415(1-8)Online publication date: 29-Nov-2022
https://doi.org/10.1109/AVSS56176.2022.9959415
Umer RMunir AMicheloni C(2021)A Deep Residual Star Generative Adversarial Network for multi-domain Image Super-Resolution2021 6th International Conference on Smart and Sustainable Technologies (SpliTech)10.23919/SpliTech52315.2021.9566406(01-05)Online publication date: 8-Sep-2021
https://doi.org/10.23919/SpliTech52315.2021.9566406
Qin ZTang YWang WXu DPan Y(2021)Subspace Capsule Network for Blurred Image Super-Resolution2021 4th International Conference on Pattern Recognition and Artificial Intelligence (PRAI)10.1109/PRAI53619.2021.9551063(416-421)Online publication date: 20-Aug-2021
https://doi.org/10.1109/PRAI53619.2021.9551063
Umer RForesti GMicheloni C(2021)Deep Iterative Residual Convolutional Network for Single Image Super-Resolution2020 25th International Conference on Pattern Recognition (ICPR)10.1109/ICPR48806.2021.9412159(1852-1858)Online publication date: 10-Jan-2021
https://doi.org/10.1109/ICPR48806.2021.9412159
Umer RMicheloni C(2021)Deep Cyclic Generative Adversarial Residual Convolutional Networks for Real Image Super-ResolutionComputer Vision – ECCV 2020 Workshops10.1007/978-3-030-67070-2_29(484-498)Online publication date: 30-Jan-2021
https://doi.org/10.1007/978-3-030-67070-2_29

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents