research-article

Unsupervised Joint Salient Region Detection and Object Segmentation

Authors:

Yong Zhao, Nikos KomodakisAuthors Info & Claims

IEEE Transactions on Image Processing, Volume 24, Issue 11

Pages 3858 - 3873

https://doi.org/10.1109/TIP.2015.2456497

Published: 01 November 2015 Publication History

Abstract

This paper presents a novel unsupervised algorithm to detect salient regions and to segment out foreground objects from background. In contrast to previous unidirectional saliency-based object segmentation methods, in which only the detected saliency map is used to guide the object segmentation, our algorithm mutually exploits detection/segmentation cues from each other. To achieve this goal, an initial saliency map is generated by the proposed segmentation driven low-rank matrix recovery model. Such a saliency map is exploited to initialize object segmentation model, which is formulated as energy minimization of Markov random field. Mutually, the quality of saliency map is further improved by the segmentation result, and serves as a new guidance for the object segmentation. The optimal saliency map and the final segmentation are achieved by jointly optimizing the defined objective functions. Extensive evaluations on MSRA-B and PASCAL-1500 datasets demonstrate that the proposed algorithm achieves the state-of-the-art performance for both the salient region detection and the object segmentation.

References

[1]

W. Zou, K. Kpalma, Z. Liu, and J. Ronsin, “Segmentation driven low-rank matrix recovery for saliency detection,” in Proc. Brit. Mach. Vis. Conf. (BMVC), 2013, pp. 1–13.

[2]

C. Koch and S. Ullman, “Shifts in selective visual attention: Towards the underlying neural circuitry,” in Matters of Intelligence. New York, NY, USA: Springer-Verlag, 1987, pp. 115–141.

[3]

L. Itti, C. Koch, and E. Niebur, “A model of saliency-based visual attention for rapid scene analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 20, no. 11, pp. 1254–1259, Nov. 1998.

Digital Library

[4]

D. Walther and C. Koch, “Modeling attention to salient proto-objects,” Neural Netw., vol. 19, no. 9, pp. 1395–1407, Nov. 2006.

Digital Library

[5]

D. Gao, V. Mahadevan, and N. Vasconcelos, “The discriminant center-surround hypothesis for bottom-up saliency,” in Proc. Adv. Neural Inf. Process. Syst., vol. 20. 2007, pp. 1–8.

[6]

R. Achanta, S. Hemami, F. Estrada, and S. Susstrunk, “Frequency-tuned salient region detection,” in Proc. IEEE Comput. Vis. Pattern Recognit. (CVPR), Jun. 2009, pp. 1597–1604.

[7]

M.-M. Cheng, G.-X. Zhang, N. J. Mitra, X. Huang, and S.-M. Hu, “Global contrast based salient region detection,” in Proc. IEEE Comput. Vis. Pattern Recognit. (CVPR), Jun. 2011, pp. 409–416.

[8]

M.-M. Cheng, N. J. Mitra, X. Huang, P. H. S. Torr, and S.-M. Hu, “Global contrast based salient region detection,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 3, pp. 569–582, Mar. 2015.

Digital Library

[9]

S. Goferman, L. Zelnik-Manor, and A. Tal, “Context-aware saliency detection,” in Proc. IEEE Comput. Vis. Pattern Recognit. (CVPR), Jun. 2010, pp. 2376–2383.

[10]

H. Jiang, J. Wang, Z. Yuan, T. Liu, N. Zheng, and S. Li, “Automatic salient object segmentation based on context and shape prior,” in Proc. Brit. Mach. Vis. Conf. (BMVC), 2011, pp. 1–9.

[11]

Z. Liu, O. L. Meur, S. Luo, and L. Shen, “Saliency detection using regional histograms,” Opt. Lett., vol. 38, no. 5, pp. 700–702, 2013.

[12]

N. Bruce and J. Tsotsos, “Saliency based on information maximization,” in Proc. Adv. Neural Inf. Process. Syst., vol. 18. 2006, pp. 155–162.

[13]

L. Zhang, M. H. Tong, T. K. Marks, H. Shan, and G. W. Cottrell, “SUN: A Bayesian framework for saliency using natural statistics,” J. Vis., vol. 8, no. 7, p. 32, Dec. 2008.

[14]

J. Harel, C. Koch, and P. Perona, “Graph-based visual saliency,” in Advances in Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2007, pp. 545–552.

Digital Library

[15]

T. Avraham and M. Lindenbaum, “Esaliency (extended saliency): Meaningful attention using stochastic image modeling,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 32, no. 4, pp. 693–708, Apr. 2010.

Digital Library

[16]

T. Liu et al., “Learning to detect a salient object,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 2, pp. 353–367, Feb. 2011.

Digital Library

[17]

H. Jiang, J. Wang, Z. Yuan, Y. Wu, N. Zheng, and S. Li, “Salient object detection: A discriminative regional feature integration approach,” in Proc. IEEE Comput. Vis. Pattern Recognit. (CVPR), Jun. 2013, pp. 2083–2090.

[18]

Z. Liu, R. Shi, L. Shen, Y. Xue, K. N. Ngan, and Z. Zhang, “Unsupervised salient object segmentation based on kernel density estimation and two-phase graph cut,” IEEE Trans. Multimedia, vol. 14, no. 4, pp. 1275–1289, Aug. 2012.

Digital Library

[19]

V. Gopalakrishnan, Y. Hu, and D. Rajan, “Salient region detection by modeling distributions of color and orientation,” IEEE Trans. Multimedia, vol. 11, no. 5, pp. 892–905, Aug. 2009.

Digital Library

[20]

W. Zhang, Q. M. J. Wu, G. Wang, and H. Yin, “An adaptive computational model for salient object detection,” IEEE Trans. Multimedia, vol. 12, no. 4, pp. 300–316, Jun. 2010.

Digital Library

[21]

Y. Xie, H. Lu, and M. Yang, “Bayesian saliency via low and mid level cues,” IEEE Trans. Image Process., vol. 22, no. 5, pp. 1689–1698, May 2013.

Digital Library

[22]

Z. Liu, W. Zou, and O. Le Meur, “Saliency tree: A novel saliency detection framework,” IEEE Trans. Image Process., vol. 23, no. 5, pp. 1937–1952, May 2014.

[23]

X. Hou and L. Zhang, “Saliency detection: A spectral residual approach,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2007, pp. 1–8.

[24]

C. Guo, Q. Ma, and L. Zhang, “Spatio-temporal saliency detection using phase spectrum of quaternion Fourier transform,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2008, pp. 1–8.

[25]

C. Jung and C. Kim, “A unified spectral-domain approach for saliency detection and its application to automatic object segmentation,” IEEE Trans. Image Process., vol. 21, no. 3, pp. 1272–1283, Mar. 2012.

Digital Library

[26]

F. Perazzi, P. Krahenbuhl, Y. Pritch, and A. Hornung, “Saliency filters: Contrast based filtering for salient region detection,” in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 733–740.

[27]

Q. Yan, L. Xu, J. Shi, and J. Jia, “Hierarchical saliency detection,” in Proc. IEEE Int. Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2013, pp. 1155–1162.

[28]

M.-M. Cheng, J. Warrell, W.-Y. Lin, S. Zheng, V. Vineet, and N. Crook, “Efficient salient region detection with soft image abstraction,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Dec. 2013, pp. 1529–1536.

[29]

Y. Wei, F. Wen, W. Zhu, and J. Sun, “Geodesic saliency using background priors,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2012, pp. 29–42.

[30]

K.-Y. Chang, T.-L. Liu, H.-T. Chen, and S.-H. Lai, “Fusing generic objectness and visual saliency for salient object detection,” in Proc. IEEE Int. Conf. Comput. Vis. (ICCV), Nov. 2011, pp. 914–921.

[31]

A. Borji, D. N. Sihite, and L. Itti, “Salient object detection: A benchmark,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2012, pp. 414–429.

[32]

J. Yan, M. Zhu, H. Liu, and Y. Liu, “Visual saliency detection via sparsity pursuit,” IEEE Signal Process. Lett., vol. 17, no. 8, pp. 739–742, Aug. 2010.

[33]

X. Shen and Y. Wu, “A unified approach to salient object detection via low rank matrix recovery,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2012, pp. 853–860.

[34]

B. C. Ko and J.-Y. Nam, “Object-of-interest image segmentation based on human attention and semantic region clustering,” J. Opt. Soc. Amer. A, vol. 23, no. 10, pp. 2462–2470, 2006.

[35]

J. Han, K. N. Ngan, M. Li, and H.-J. Zhang, “Unsupervised extraction of visual attention objects in color images,” IEEE Trans. Circuits Syst. Video Technol., vol. 16, no. 1, pp. 141–145, Jan. 2006.

Digital Library

[36]

Y. Boykov, O. Veksler, and R. Zabih, “Fast approximate energy minimization via graph cuts,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 23, no. 11, pp. 1222–1239, Nov. 2001.

Digital Library

[37]

E. Rahtu, J. Kannala, M. Salo, and J. Heikkilä, “Segmenting salient objects from images and videos,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2010, pp. 366–379.

[38]

C. Rother, V. Kolmogorov, and A. Blake, “‘GrabCut’: Interactive foreground extraction using iterated graph cuts,” ACM Trans. Graph., vol. 23, no. 3, pp. 309–314, 2004.

Digital Library

[39]

E. J. Candes, X. Li, Y. Ma, and J. Wright. (2009). “Robust principal component analysis?” [Online]. Available: http://arxiv.org/abs/0912.3599

[40]

C. Lang, G. Liu, J. Yu, and S. Yan, “Saliency detection by multitask sparsity pursuit,” IEEE Trans. Image Process., vol. 21, no. 3, pp. 1327–1338, Mar. 2012.

Digital Library

[41]

V. Lempitsky, P. Kohli, C. Rother, and T. Sharp, “Image segmentation with a bounding box prior,” in Proc. IEEE 12th Int. Conf. Comput. Vis. (ICCV), Sep./Oct. 2009, pp. 277–284.

[42]

M. Everingham, L. Van Gool, C. K. I. Williams, J. Winn, and A. Zisserman, “The PASCAL visual object classes (VOC) challenge,” Int. J. Comput. Vis., vol. 88, no. 2, pp. 303–338, Jun. 2010.

Digital Library

[43]

H. G. Feichtinger, Gabor Analysis and Algorithms: Theory and Applications. Boston, MA, USA: Birkhäuser, 1997.

Digital Library

[44]

E. P. Simoncelli and W. T. Freeman, “The steerable pyramid: A flexible architecture for multi-scale derivative computation,” in Proc. IEEE Int. Conf. Image Process., Oct. 1995, pp. 444–447.

[45]

P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898–916, May 2011.

Digital Library

[46]

D. Comaniciu and P. Meer, “Mean shift: A robust approach toward feature space analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603–619, May 2002.

Digital Library

[47]

Z. Lin, M. Chen, and Y. Ma. (2010). “The augmented Lagrange multiplier method for exact recovery of corrupted low-rank matrices.” [Online]. Available: http://arxiv.org/abs/1009.5055

[48]

B. Catanzaro, B.-Y. Su, N. Sundaram, Y. Lee, M. Murphy, and K. Keutzer, “Efficient, high-quality image contour detection,” in Proc. IEEE 12th Int. Conf. Comput. Vis. (ICCV), Sep./Oct. 2009, pp. 2381–2388.

Cited By

Cai SHuang JChen JHuang YDing XZeng D(2019)Prominent edge detection with deep metric expression and multi-scale featuresMultimedia Tools and Applications10.1007/s11042-018-6581-578:20(29121-29135)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s11042-018-6581-5
Tang YZou WJin ZLi XAizawa KLew MSatoh S(2018)Multi-Scale Spatiotemporal Conv-LSTM Network for Video Saliency DetectionProceedings of the 2018 ACM on International Conference on Multimedia Retrieval10.1145/3206025.3206052(362-369)Online publication date: 5-Jun-2018
https://dl.acm.org/doi/10.1145/3206025.3206052
Ye LLiu ZLi LShen LBai CWang Y(2017)Salient Object Segmentation via Effective Integration of Saliency and ObjectnessIEEE Transactions on Multimedia10.1109/TMM.2017.269302219:8(1742-1756)Online publication date: 17-Jul-2017
https://dl.acm.org/doi/10.1109/TMM.2017.2693022
Show More Cited By

Index Terms

Unsupervised Joint Salient Region Detection and Object Segmentation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
        Interest point and salient region detections
        Video segmentation
      2. Computer vision tasks
        Scene understanding
  2. Machine learning

Index terms have been assigned to the content through auto-classification.

Recommendations

An efficient segmentation method using saliency object detection

Automatic co-segmentation is a challenging task because it lacks of prior cues. In this paper, an efficient region contrast based method is proposed for salient object detection and segmentation. The coarse location information of the salient object and ...
Salient region detection: an integration approach based on image pyramid and region property

Salient region detection is important for many computer vision tasks. The saliency detection results may serve as the basis for further high‐level vision tasks like object segmentation and tracking. In this study, the authors propose an integration ...
Salient Object Detection through Over-Segmentation
ICME '12: Proceedings of the 2012 IEEE International Conference on Multimedia and Expo

In this paper we present a salient object detection model from an over-segmented image. The input image is initially segmented by the mean-shift segmentation algorithm and then over-segmented by a quad mesh to even smaller segments. Such segmented ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Image Processing

IEEE Transactions on Image Processing Volume 24, Issue 11

Nov. 2015

1407 pages

ISSN:1057-7149

Issue’s Table of Contents

Copyright © 2015.

Publisher

IEEE Press

Publication History

Published: 01 November 2015

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cai SHuang JChen JHuang YDing XZeng D(2019)Prominent edge detection with deep metric expression and multi-scale featuresMultimedia Tools and Applications10.1007/s11042-018-6581-578:20(29121-29135)Online publication date: 1-Oct-2019
https://dl.acm.org/doi/10.1007/s11042-018-6581-5
Tang YZou WJin ZLi XAizawa KLew MSatoh S(2018)Multi-Scale Spatiotemporal Conv-LSTM Network for Video Saliency DetectionProceedings of the 2018 ACM on International Conference on Multimedia Retrieval10.1145/3206025.3206052(362-369)Online publication date: 5-Jun-2018
https://dl.acm.org/doi/10.1145/3206025.3206052
Ye LLiu ZLi LShen LBai CWang Y(2017)Salient Object Segmentation via Effective Integration of Saliency and ObjectnessIEEE Transactions on Multimedia10.1109/TMM.2017.269302219:8(1742-1756)Online publication date: 17-Jul-2017
https://dl.acm.org/doi/10.1109/TMM.2017.2693022
Li AChen Z(2017)Individual trait oriented scanpath prediction for visual attention analysis2017 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP.2017.8296982(3745-3749)Online publication date: 17-Sep-2017
https://dl.acm.org/doi/10.1109/ICIP.2017.8296982

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents