research-article

FCPN: : Pruning redundant part-whole relations for more streamlined pattern parsing

Authors:

Zengwei ZhengAuthors Info & Claims

Volume 174, Issue C

https://doi.org/10.1016/j.neunet.2024.106258

Published: 09 July 2024 Publication History

Abstract

Cropping-and-segmenting pattern parsers often combine diverse inner correlations into a single metric/scheme, resulting in over-generalizations and redundant representations. It is proposed to streamline pattern parsing by using presenting a redundant association elimination network (RAEN) with capsule attention twisters (CATs) and capsule-attention routing agreement (CARA). CATs trim delicate relationships between parts and wholes that are weak and interchangeable. Senior entities can only be updated by primary entities that meet the requirements of inter-part diversity and intra-object cohesiveness. In order to enhance results, CARA is designed to protect against the unnecessary voting signals of traditional routing protocols. Experiments involving facial and human segmentation show that RAEN is better than current remarkable methods, particularly for defining detailed semantic boundaries.

References

[1]

D. Shao, Y. Zhao, B. Dai, D. Lin, Intra-and inter-action understanding via temporal action parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Seattle, WA, USA, 2020, pp. 727–736,.

[2]

Z. Lin, X. Jiang, Z. Zheng, A Coarse-to-fine pattern parser for mitigating the issue of drastic imbalance in pixel distribution, Pattern Recognition 148 (2024),.

Digital Library

[3]

P. Huang, J. Han, D. Zhang, M. Xu, Clrnet: Component-level refinement network for deep face parsing, IEEE Transactions on Pattern Analysis and Machine Intelligence 34 (3) (2021) 1439–1453,.

[4]

Z. Lin, J. Jia, F. Huang, W. Gao, Feature correlation-steered capsule network for object detection, Neural Networks : The Official Journal Of The International Neural Network Society 147 (2022) 25–41,.

Digital Library

[5]

W. Wang, H. Zhu, J. Dai, Y. Pang, J. Shen, L. Shao, Hierarchical human parsing with typed part-relation reasoning, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Seattle, WA, USA, 2020, pp. 8926–8936,.

[6]

G.E. Hinton, S. Sabour, N. Frosst, Matrix capsules with em routing, in: in Proc. Int. Conf. Learn. Represent. (ICLR), Feb, 2018.

[7]

Z. Lin, Y. Wang, Z. Zheng, IOP-CapsNet with ISEMRA: Fetching part-to-whole topology for improving detection performance of articulated instances, Expert Systems with Applications 226 (2023),.

Digital Library

[8]

X. Chen, R. Mottaghi, X. Liu, S. Fidler, R. Urtasun, A. Yuille, Detect what you can: Detecting and representing objects using holistic models and body parts, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), Columbus, OH, USA, 2014, pp. 1979–1986,.

Digital Library

[9]

W. Yang, P. Luo, L. Lin, Clothing co-parsing by joint image segmentation and labeling, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Columbus, OH, USA, 2014, pp. 3182–3189,.

Digital Library

[10]

S. Liu, J. Feng, C. Domokos, Fashion parsing with weak color-category labels, IEEE Transactions On Multimedia 16 (1) (2013) 253–265,.

[11]

K. Yamaguchi, M.H. Kiapour, L.E. Ortiz, T.L. Berg, Parsing clothing in fashion photographs, in: in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Providence, RI, USA, 2012, pp. 3570–3577,.

[12]

N. Wang, H. Ai, Who blocks who: Simultaneous clothing segmentation for grouping images, in: in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , Barcelona, Spain, 2011, pp. 1535–1542,.

Digital Library

[13]

Y. Bo, C.C. Fowlkes, Shape-based pedestrian parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Colorado Springs, CO, USA, 2011, pp. 2265–2272,.

Digital Library

[14]

J. Dong, Q. Chen, W. Xia, Z. Huang, S. Yan, A deformable mixture parsing model with parselets, in: in Proc. IEEE Int. Conf. Comput. Vis. (ICCV) , Sydney, NSW, Australia, 2013, pp. 3408–3415,.

Digital Library

[15]

J. Dong, Q. Chen, X. Shen, J. Yang, S. Yan, Towards unified human parsing and pose estimation, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Columbus, OH, USA, 2014, pp. 843–850,.

Digital Library

[16]

K. Yamaguchi, M.H. Kiapour, T.L. Berg, Paper doll parsing: Retrieving similar styles to parse clothing items, in: in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , Sydney, NSW, Australia, 2013, pp. 3519–3526,.

Digital Library

[17]

H. Chen, Z. Xu, Z. Liu, S. Zhu, Composite templates for cloth modeling and sketching, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , New York, NY, USA, 2006, pp. 943–950,.

Digital Library

[18]

L. Zhu, Y. Chen, Y. Lu, C. Lin, A. Yuille, Max margin and/or graph learning for parsing the human body, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Anchorage, AK, 2008, pp. 1–8,.

[19]

S. Eslami, C. Williams, A generative model for parts-based object segmentation, in: in Proc. Neural Inf. Process. Syst. (NIPS) , 2013, pp. 100–107b.

[20]

I. Rauschert, R.T. Collins, A generative model for simultaneous estimation of human body shape and pixel-level segmentation, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2012,.

Digital Library

[21]

L. Zhou, Z. Liu, and X. He, “Face parsing via a fully-convolutional continuous CRF neural network,” 2017, arXiv:1708.03736. [Online], 10.48550/arXiv.1708.03736.

[22]

X. Liang, et al., Deep human parsing with active template regression, IEEE Transactions on Pattern Analysis and Machine Intelligence 37 (12) (2015) 2402–2414,.

Digital Library

[23]

S. Liu, et al., Matching-cnn meets knn: Quasi-parametric human parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Boston, MA, USA, 2015, pp. 1419–1427,.

[24]

X. Liang, et al., Human parsing with contextualized convolutional neural network, IEEE Transactions on Pattern Analysis and Machine Intelligence 39 (1) (1 Jan. 2017) 115–127,.

Digital Library

[25]

X. Liang, X. Shen, J. Feng, L. Lin, S. Yan, Semantic object parsing with graph lstm, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , Amsterdam, The Netherlands, 2016,. October 11–14Proceedings, Part I 14. Springer International Publishing, 2016: 125-143.

[26]

X. Liang, X. Shen, D. Xiang, J. Feng, L. Lin, S. Yan, Semantic object parsing with local-global long short-term memory, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Las Vegas, NV, USA, 2016, pp. 3185–3193,.

[27]

P. Luo, X. Wang, X. Tang, Pedestrian Parsing via Deep Decompositional Network, in: in Proc. IEEE Int. Conf. Comput. Vis. (ICCV) , Sydney, NSW, Australia, 2013, pp. 2648–2655,.

Digital Library

[28]

L. Chen, Y. Yang, J. Wang, W. Xu, A.L. Yuille, Attention to scale: Scale-aware semantic image segmentation, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Las Vegas, NV, USA, 2016, pp. 3640–3649,.

[29]

F. Xia, P. Wang, L. Chen, A.L. Yuille, Zoom better to see clearer: Human and object parsing with hierarchical auto-zoom net, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , Amsterdam, The Netherlands, Springer International Publishing, 2016, pp. 648–663,. October 11-14, 2016Proceedings, Part V 14.

[30]

X. Luo, Z. Su, J. Guo, G. Zhang, X. He, Trusted guidance pyramid network for human parsing, in: in Proc. ACM Int. Conf. Multimedia (ACMMM) , 2018, pp. 654–662,.

Digital Library

[31]

S. Liu, C. Wang, R. Qian, H. Yu, R. Bao, Y. Sun, Surveillance video parsing with single frame supervision, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Honolulu, HI, USA, 2017, pp. 1013–1021,.

[32]

Z. Zheng, W. Wang, S. Qi, S. Zhu, Reasoning visual dialogs with structural and partial observations, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Long Beach, CA, USA, 2019, pp. 6662–6671,.

[33]

J. Zhao, J. Li, Y. Cheng, T. Sim, S. Yan, J. Feng, Understanding humans in crowded scenes: Deep nested adversarial learning and a new benchmark for multi-human parsing, in: in Proc. ACM Int. Conf. Multimedia (ACMMM) , 2018, pp. 792–800,.

[34]

Y. Luo, Z. Zheng, L. Zheng, T. Guan, J. Yu, Y. Yang, Macro-micro adversarial network for human parsing, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2018, pp. 418–434,.

Digital Library

[35]

S. Liu, et al., Cross-domain human parsing via adversarial feature and label adaptation, in: in Proc. AAAI Conf. Artif. Intell , 32, 2018,.

[36]

W. Xu, Y. Li, C. Lu, Srda: Generating instance segmentation annotation via scanning, reasoning and domain adaptation, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2018, pp. 120–136,.

Digital Library

[37]

H. Fang, Y. Xu, W. Wang, X. Liu, S. Zhu, Learning pose grammar to encode human body configuration for 3d pose estimation, in: in Proc. AAAI Conf. Artif. Intell , 32, 2018,.

[38]

K. Gong, Y. Gao, X. Liang, X. Shen, M. Wang, L. Lin, Graphonomy: Universal human parsing via graph transfer learning, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Long Beach, CA, USA, 2019, pp. 7442–7451,.

[39]

F. Xia, J. Zhu, P. Wang, A.L. Yuille, Pose-guided human parsing by an and/or graph using pose-context features, in: in Proc. AAAI Conf. Artif. Intell , 30, 2016, pp. 3632–3640,.

[40]

J. Zhao, et al., Self-supervised neural aggregation networks for human parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Honolulu, HI, USA, 2017, pp. 1595–1603,.

[41]

K. Gong, X. Liang, D. Zhang, X. Shen, L. Lin, Look into person: Self-supervised structure-sensitive learning and a new benchmark for human parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Honolulu, HI, USA, 2017, pp. 6757–6765,.

[42]

X. Nie, J. Feng, S. Yan, Mutual learning to adapt for joint human parsing and pose estimation, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , 2018, pp. 502–517.

[43]

H. Fang, S. Xie, Y. Tai, C. Lu, Rmpe: Regional multi-person pose estimation, in: in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV) , Venice, Italy, 2017, pp. 2353–2362,.

[44]

L. Li, T. Zhou, W. Wang, J. Li, Y. Yang, Deep Hierarchical Semantic Segmentation, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , New Orleans, LA, USA, 2022, pp. 1236–1247,.

[45]

T. Zhou, Y. Yang, W. Wang, Differentiable multi-granularity human parsing, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (7) (1 July 2023) 8296–8310,.

Digital Library

[46]

W. Wang, T. Zhou, S. Qi, J. Shen, S.-C. Zhu, Hierarchical human semantic parsing with comprehensive part-relation modeling, IEEE Trans. Pattern Anal. Mach. Intell. 44 (7) (1 July 2022) 3508–3522,.

[47]

S. Liu, J. Shi, J. Liang, and M. Yang, “Face parsing via recurrent propagation,” 2017, arXiv:1708.01936. [Online], 10.48550/arXiv.1708.01936.

[48]

D. Eigen, R. Fergus, Predicting depth surface normals and semantic labels with a common multi-scale convolutional architecture, in: in Proc. IEEE Int. Conf. Comput. Vis. (ICCV) , Santiago, Chile, 2015, pp. 2650–2658,.

Digital Library

[49]

B.M. Smith, L. Zhang, J. Brandt, Z. Lin, J. Yang, Exemplar-based face parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Portland, OR, USA, 2013, pp. 3484–3491,.

Digital Library

[50]

A. Kae, K. Sohn, H. Lee, E. Learned-Miller, Augmenting CRFs with Boltzmann machine shape priors for image labeling, in: in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Portland, OR, USA, 2013, pp. 2019–2026,.

Digital Library

[51]

F. Xia, P. Wang, X. Chen, Alan L. Yuille, Joint multi-person pose estimation and semantic part segmentation, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Honolulu, HI, USA, 2017, pp. 6080–6089,.

[52]

Z. Lin, Z. Zheng, J. Jia, et al., DR-CapsNet with CAEMRA: Looking deep inside instance for boosting object detection effect, Engineering Applications Of Artificial Intelligence 123 (2023),.

Digital Library

[53]

Z. Lin, Y. Wang, Z. Zheng, CtFPPN: A coarse-to-fine pattern parser for dealing with distribution imbalance of pixels, Knowledge-Based System 280 (2023),.

Digital Library

[54]

N. Liu, J. Han, M. Yang, PiCANet: Pixel-Wise Contextual Attention Learning for Accurate Saliency Detection, IEEE Transactions On Image Processing : A Publication Of The IEEE Signal Processing Society 29 (2020) 6438–6451,.

Digital Library

[55]

Y. Liu, H. Shi, H. Shen, Y. Si, X. Wang, T. Mei, A new dataset and boundary-attention semantic segmentation for face parsing, in: in Proc. AAAI Conf. Artif. Intell. (AAAI) , 34, 2020, pp. 11637–11644,.

[56]

G. Te, Y. Liu, W. Hu, H. Shi, T. Mei, Edge-aware graph representation learning and reasoning for face parsing, in: in Proc. Eur. Conf. Comput. Vis. (ECCV) , Springer, Cham, Switzerland, 2020, pp. 258–274,.

[57]

J. Lin, H. Yang, D. Chen, M. Zeng, F. Wen, L. Yuan, Face parsing with RoI tanh-warping, in: in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , Long Beach, CA, USA, 2019, pp. 5647–5656,.

[58]

Y. Lin, J. Shen, Y. Wang, M. Pantic, Roi tanh-polar transformer network for face parsing in the wild, Image and vision computing 112 (2021),.

[59]

I. Masi, J. Mathai, W. AbdAlmageed, Towards learning structure via consensus for face segmentation and parsing, in: in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) , Seattle, WA, USA, 2020, pp. 5507–5517,.

[60]

Z. Wei, S. Liu, Y. Sun, H. Ling, Accurate facial image parsing at real-time speed, IEEE Transactions On Image Processing : A Publication Of The IEEE Signal Processing Society 28 (9) (Sep. 2019) 4659–4670,.

[61]

L. Chen, Y. Zhu, G. Papandreou, F. Schroff, H. Adam, Encoder-decoder with atrous separable convolution for semantic image segmentation, in: in Proc. Eur. conf. comput. vis. (ECCV) , 2018, pp. 801–818,.

[62]

L. Yang, Q. Song, Z. Wang, Z. Liu, S. Xu, Z. Li, Quality-aware network for human parsing, IEEE Transactions On Multimedia (2024),.

Digital Library

[63]

K. Liu, O. Choi, J. Wang, W. Hwang, CDGNet: Class distribution guided network for human parsing, in: in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR) , New Orleans, LA, USA, 2022, pp. 4463–4472,.

[64]

X. Zhang, Y. Chen, M. Tang, J. Wang, X. Zhu, Z. Lei, Human parsing with part-aware relation modeling, IEEE Transactions On Multimedia 25 (2023) 2601–2612,.

Digital Library

[65]

S. Zhang, X. Cao, G.-J. Qi, Z. Song, J. Zhou, AIParsing: Anchor-free instance-level human parsing, IEEE Transactions On Image Processing : A Publication Of The IEEE Signal Processing Society 31 (2022) 5599–5612,.

Recommendations

FCPN: Pruning redundant part-whole relations for more streamlined pattern parsing
Abstract
Most cropping-and-segmenting pattern parsers typically establish a single metric/scheme to reason diverse inner correlations, resulting in over-general and redundant representations. To make pattern parsing more streamlined and efficient, a ...
Reducing vulnerable internal feature correlations to enhance efficient topological structure parsing
Abstract
Most cropping-and-segmenting pattern parsers typically establish a single metric/scheme to reason diverse inner correlations, resulting in over-general and redundant representations. To make pattern parsing procedure more streamlined and concise, ...
Incorporating rich syntax information in Grammatical Error Correction
Abstract
Syntax parse trees are a method of representing sentence structure and are often used to provide models with syntax information and enhance downstream task performance. Because grammar and syntax are inherently linked, the ...

Comments

Information & Contributors

Information

Published In

cover image Neural Networks

Neural Networks Volume 174, Issue C

Jun 2024

632 pages

Issue’s Table of Contents

Elsevier Ltd.

Publisher

Elsevier Science Ltd.

United Kingdom

Publication History

Published: 09 July 2024

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

0
Total Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Nov 2024

Other Metrics

View Author Metrics

Citations

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents