Semantics-Adding Flaw-Erasing Network for Semantic Human Matting
Jiayu Sun (Dalian University of Technology),* Zhanghan Ke (City University of Hong Kong), Ke Xu (City University of Hong Kong), Fan Shao ( Wonxing Technology), Lihe Zhang (Dalian University of Technology), Huchuan Lu (Dalian University of Technology), Rynson W.H. Lau (City University of Hong Kong)
The 33rd British Machine Vision Conference
Abstract
Trimap-free semantic human matting is a very challenging task. While using a single model to directly predict an alpha matte can be extremely unreliable, some latest works propose to predict a segmentation mask or a pseudo trimap as a priori for the matting task. However, errors in the priori can significantly affect the subsequent prediction. Motivated by recent flaw-based correction approaches, we propose a different approach to the problem here. We first predict an initial alpha matte with a single model. Guided by a flaw detector to check for potential prediction errors, we then correct the errors in the initial alpha matte with a refinement process to produce the output matte. To this end, we propose a semantics-adding flaw-erasing network (SAFE-Net) with two novel modules, a semantic addition module (SAM) to enhance the human semantics using an attention mechanism and a flaw elimination module (FEM) to refine the details of the predicted defective regions of the initial matte. Experimental results show that SAFE-Net outperforms existing trimap-free human matting methods notably. Finally, we have created a large human matting dataset containing 4,729 unique foregrounds with fine annotations, which will be made available to the public.Video
Citation
@inproceedings{Sun_2022_BMVC,
author = {Jiayu Sun and Zhanghan Ke and Ke Xu and Fan Shao and Lihe Zhang and Huchuan Lu and Rynson W.H. Lau},
title = {Semantics-Adding Flaw-Erasing Network for Semantic Human Matting},
booktitle = {33rd British Machine Vision Conference 2022, {BMVC} 2022, London, UK, November 21-24, 2022},
publisher = {{BMVA} Press},
year = {2022},
url = {https://bmvc2022.mpi-inf.mpg.de/0592.pdf}
}
Copyright © 2022
The British Machine Vision Conference is organised by