research-article

Deep network based stereoscopic image quality assessment via binocular summing and differencing

Authors:

Qiuping JiangAuthors Info & Claims

Volume 82, Issue C

https://doi.org/10.1016/j.jvcir.2021.103420

Published: 01 January 2022 Publication History

Abstract

With the development of deep networks in dealing with various visual tasks, the deep network based on binocular vision is expected to tackle the issue of stereoscopic image quality assessment. Here, we present a stereoscopic image quality assessment method using the deep network with four channels together, which takes the left view, right view, binocular summing view, and binocular differencing view as the inputs of the network. The visual features are enhanced through the concatenation in a weighted way, so that the binocular vision can be adequately included in the binocular addition and subtraction information. Compared with the state-of-the-art metrics, the proposed method exhibits relatively high performances on four benchmark databases.

References

[1]

S. Hong, A. Dorado, G. Saavedra, J. Sola-Pikabea, M. Martinez-Corral, Full-parallax 3D display from the hole-filtered depth information, in Proc. of 3DTV-Conference: The True Vision - Capture, Transmission and Display of 3D Video, 2015, pp. 1-4.

[2]

S. Hong, G. Saavedra, M. Martínez-Corral, Full-parallax immersive 3D display from depth-map cameras, in Proc. of 15th Workshop on Information Optics, 2016, pp. 1–3.

[3]

Z. Chong, L. Yee, A. Causo, I. Chen, Autonomous robot driving decision strategy following road signs and traffic rules: Simulation validation, in Proc. of 16th International Conference on Control, Automation and Systems, 2016, pp. 377-381.

[4]

Y. Xiong, F. Shao, X. Meng, B. Zhou, Y. Ho, Sparse representation for no-reference quality assessment of satellite stereo images, IEEE Access 7 (2019) 106295–106306.

[5]

R. Szabó, A. Gontean, Industrial robotic automation with Raspberry PI using image processing, in Proc. of International Conference on Applied Electronics, 2016, pp. 265–268.

[6]

S. Khan Md, B. Appina, S.S. Channappayya, Full-reference stereo image quality assessment using natural stereo scene statistics, IEEE Sign. Process. Lett. 22 (11) (2015) 1985–1989.

[7]

H. Ko, R. Song, C.-C.-J. Kuo, A paraboost stereoscopic image quality assessment (PBSIQA) system, J. Vis. Commun. Image Represent. 45 (2017) 156–169.

[8]

X. Wang, Q. Liu, R. Wang, Z. Chen, Natural image statistics based 3D reduced reference image quality assessment in contourlet domain, Neurocomputing 151 (2015) 683–691.

[9]

Q. Li, W. Lin, Y. Fang, No-reference quality assessment for multiply-distorted images in gradient domain, IEEE Sign. Process. Lett. 23 (4) (2016) 541–545.

[10]

M.-J. Chen, C.-C. Su, D.-L. Kwon, L.K. Cormack, A.C. Bovik, Full-reference quality assessment of stereopairs accounting for binocular rivalry, Signal Process. Image Commun. 28 (9) (2013) 1143–1155.

[11]

F. Shao, W. Lin, S. Gu, G. Jiang, T. Srikanthan, Perceptual full-reference quality assessment of stereoscopic images by considering binocular visual characteristics, IEEE Trans. Image Process. 22 (5) (2013) 1940–1953.

[12]

X. Wang, M. Qi, F. Shao, Q. Jiang, X. Meng, Blind quality assessment for multiply distorted stereoscopic images towards IoT-based 3D capture systems, J. Vis. Commun. Image Represent. 71 (2020).

[13]

J.D. Pettigrew, The neurophysiology of binocular vision, Sci. Am. 227 (2) (1972) 84–95.

[14]

D Stidwill, R Fletcher, Normal binocular vision: Theory, investigation and practical aspects, Optometry Today (2010).

[15]

Z. Li, J.J. Atick, Efficient stereo coding in the multiscale representation, Netw. Comput. Neural Syst. 5 (1994) 157–174.

[16]

G. Zhai, X. Min, Perceptual image quality assessment: a survey, Sci. China Inf. Sci., 63(11) (2020) 211301:1-211301:52.

[17]

F.A.A. Kingdom, Binocular Vision: The Eyes Add and Subtract, Curr. Biol. 22 (1) (2012) 22–24.

[18]

J. Yang, Y. Liu, Z. Gao, R. Chu, Z. Song, A perceptual stereoscopic image quality assessment model accounting for binocular combination behavior, J. Vis. Commun. Image Represent. 31 (2015) 138–145.

[19]

J. Yang, K. Sim, X. Gao, W. Lu, Q. Meng, B. Li, A blind stereoscopic image quality evaluator with segmented stacked autoencoders considering the whole visual perception route, IEEE Trans. Image Process. 28 (3) (2019) 1314–1328.

Digital Library

[20]

C. Lin, Z. Chen, N. Liao, Full-reference quality assessment for stereoscopic images based on binocular vision model, in Proc. of Visual Communications and Image Processing (VCIP), Chengdu, China, 2016, pp. 1–4.

[21]

J. Yang, Z. Yang, Y. Zhu, H. Xu, Q. Meng, Blind assessment for stereo images considering binocular characteristics and deep perception map based on deep belief network, Inf. Sci. 474 (2019) 1–17.

[22]

M. Chen, L.K. Cormack, A.C. Bovik, No-reference quality assessment of natural stereopairs, IEEE Trans. Image Process. 22 (9) (2013) 3379–3391.

[23]

X. Wang, Q. Liu, R. Wang, Z. Chen, Natural image statistics based 3D reduced reference image quality assessment in contourlet domain, Neurocomputing 151 (2) (2015) 683–691.

[24]

X. Gao, W. Lu, D. Tao, X. Li, Image quality assessment based on multiscale geometric analysis, IEEE Trans. Image Process. 18 (7) (2009) 1409–1423.

[25]

F. Shao, W. Lin, S. Wang, G. Jiang, M. Yu, Blind image quality assessment for stereoscopic images using binocular guided quality lookup and visual codebook, IEEE Trans. Broadcast. 61 (2) (2015) 154–165.

[26]

F. Shao, K. Li, W. Lin, G. Jiang, M. Yu, Q. Dai, Full-reference quality assessment of stereoscopic images by learning binocular receptive field properties, IEEE Trans. Image Process. 24 (10) (2015) 2971–2983.

[27]

L. Kang, P. Ye, Y. Li, D. Doermann, Convolutional neural networks for no-reference image quality assessment, in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2014, pp. 1733–1740.

[28]

S. Bosse, D. Maniry, K. Müller, T. Wiegand, W. Samek, Deep neural networks for no-reference and full-reference image quality assessment, IEEE Trans. Image Process. 27 (1) (2017) 206–219.

[29]

H. Oh, S. Ahn, J. Kim, S. Lee, Blind deep S3D image quality evaluation via local to global feature aggregation, IEEE Trans. Image Process. 26 (10) (2017) 4923–4936.

[30]

Y. Fang, J. Yan, X. Liu, J. Wang, Stereoscopic image quality assessment by deep convolutional neural network, J. Vis. Commun. Image Represent. 58 (2019) 400–406.

[31]

W. Zhou, J. Lei, Q. Jiang, L. Yu, T. Luo, Blind binocular visual quality predictor using deep fusion network, IEEE Trans. Comput. Imaging 6 (2020) 883–893.

[32]

W. Zhang, K. Ma, G. Zhai, X. Yang, Uncertainty-aware blind image quality assessment in the laboratory and wild, IEEE Trans. Image Process. 30 (2021) 3474–3486.

[33]

H. Zhu, L. Li, J. Wu, W. Dong, G. Shi, MetaIQA: Deep meta-learning for no-reference image quality assessment, in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, 2020, pp. 14131–14140.

[34]

X. Yang, F. Li, H. Liu, TTL-IQA: Transitive transfer learning based no-reference image quality assessment, IEEE Trans. Multimedia (2020).

[35]

L. Shen, R. Fang, Y. Yao, X. Geng, D. Wu, No-reference stereoscopic image quality assessment based on image distortion and stereo perceptual information, IEEE Trans. Emerg. Topics Comput. Intell. 3 (1) (2018) 59–72.

[36]

X. Jiang, L. Shen, L. Yu, M. Jiang, G. Feng, No-reference screen content image quality assessment based on multi-region features, Neurocomputing 386 (2020) 30–41.

[37]

X. Jiang, L. Shen, Q. Ding, L. Zheng, P. An, Screen content image quality assessment based on convolutional neural networks, J. Vis. Commun. Image Represent. 67 (2020).

[38]

J. Kim, S. Lee, Fully deep blind image quality predictor, IEEE J. Sel. Top. Signal Process. 11 (1) (2017) 206–220.

[39]

J.C.A. Read, B.G. Cumming, The psychophysics of stereopsis can be explained without invoking independent ON and OFF channels, J. Vis. 19 (6) (2019) 1–14.

[40]

F.A.A. Kingdom, N.M. Seulami, B.J. Jennings, M.A. Georgeson, Interocular difference thresholds are mediated by binocular differencing, not summing, channels, J. Vis. 19 (14):18 (2019) 1–15.

[41]

A. Mittal, A.K. Moorthy, A.C. Bovik, No-reference image quality assessment in the spatial domain, IEEE Trans. Image Process. 21 (12) (2012) 4695–4708.

Digital Library

[42]

D. P. Kingma, J. Ba, Adam: a method for stochastic optimization, CoRR (2014). abs/1412.6980.

[43]

A.K. Moorthy, C.C. Su, A. Mittal, A.C. Bovik, Subjective evaluation of stereoscopic image quality, Signal Process. Image Commun. 28 (8) (2013) 870–883.

[44]

M.J. Chen, C.C. Su, D.K. Kwon, L.K. Cormack, Full-reference quality assessment of stereopairs accounting for rivalry, Signal Process. Image Commun. 28 (9) (2013) 1143–1155.

[45]

F. Shao, W. Tian, W. Lin, G. Jiang, Q. Dai, Learning sparse representation for no-reference quality assessment of multiply distorted stereoscopic images, IEEE Trans. Multimedia 19 (8) (2017) 1821–1836.

[46]

F. Shao, Y. Gao, Q. Jiang, G. Jiang, Y. Ho, Multistage pooling for blind quality prediction of asymmetric multiply-distorted stereoscopic images, IEEE Trans. Multimedia 20 (10) (2018) 2605–2619.

[47]

Q. Jiang, F. Shao, W. Lin, G. Jiang, BLIQUE-TMI: Blind quality evaluator for tone-mapped images based on local and global feature analyses, IEEE Trans. Circuits Syst. Video Technol. 29 (2) (2019) 323–335.

Digital Library

[48]

B. Appina, S. Khan, S.S. Channappayya, No-reference stereoscopic image quality assessment using natural scene statistics, Signal Process. Image Commun. 43 (2016) 1–14.

[49]

W. Zhou, L. Yu, Binocular responses for no-reference 3D image quality assessment, IEEE Trans. Multimedia 18 (6) (2016) 1077–1084.

[50]

M.A. Saad, A.C. Bovik, C. Charrier, Blind image quality assessment: a natural scene statistics approach in the DCT domain, IEEE Trans. Image Process. 21 (8) (2012) 3339–3352.

Digital Library

[51]

A. Mittal, R. Soundararajan, A. Bovik, Making a ‘completely blind’ image quality analyzer, IEEE Sign. Process. Lett. 20 (3) (2013) 209–212.

Cited By

Zhou YCao RLi P(2024)A target spatial location method for fuze detonation point based on deep learning and sensor fusionExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122176238:PFOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122176

Index Terms

Deep network based stereoscopic image quality assessment via binocular summing and differencing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
  2. Computer graphics
    1. Image manipulation

Index terms have been assigned to the content through auto-classification.

Recommendations

No-reference Stereoscopic Image Quality Assessment Using Binocular Self-similarity and Deep Neural Network

Quality assessment of three-dimensional (3D) images is more challenging than that of 2D images. The quality of 3D visual experience is one of the most challenging areas of human binocular perception and is affected by multiple factors such as asymmetric ...
No-Reference Stereoscopic Image Quality Assessment Based on Binocular Visual Saliency Weighting
ICIGP '23: Proceedings of the 2023 6th International Conference on Image and Graphics Processing

Deep learning is widely used in the field of image quality assessment. The complex binocular vision mechanism and the multidimensional characteristics make the quality assessment of stereoscopic images more challenging. In this paper, a new no-reference ...
Binocular vision based objective quality assessment method for stereoscopic images

Human visual system (HVS) can perceive the difference between two retinal images to create a mental image with depth perception, which is the result of two binocular interactions, i.e., binocular fusion and suppression. According to perceptual ...

Comments

Information & Contributors

Information

Published In

cover image Journal of Visual Communication and Image Representation

Journal of Visual Communication and Image Representation Volume 82, Issue C

Jan 2022

395 pages

ISSN:1047-3203

Issue’s Table of Contents

Copyright © 2021.

Publisher

Academic Press, Inc.

United States

Publication History

Published: 01 January 2022

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhou YCao RLi P(2024)A target spatial location method for fuze detonation point based on deep learning and sensor fusionExpert Systems with Applications: An International Journal10.1016/j.eswa.2023.122176238:PFOnline publication date: 27-Feb-2024
https://dl.acm.org/doi/10.1016/j.eswa.2023.122176

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents