README.md

MS R-CNN

Mask Scoring R-CNN

Abstract

Letting a deep network be aware of the quality of its own predictions is an interesting yet important problem. In the task of instance segmentation, the confidence of instance classification is used as mask quality score in most instance segmentation frameworks. However, the mask quality, quantified as the IoU between the instance mask and its ground truth, is usually not well correlated with classification score. In this paper, we study this problem and propose Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks. The proposed network block takes the instance feature and the corresponding predicted mask together to regress the mask IoU. The mask scoring strategy calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation. By extensive evaluations on the COCO dataset, Mask Scoring R-CNN brings consistent and noticeable gain with different models, and outperforms the state-of-the-art Mask R-CNN. We hope our simple and effective approach will provide a new direction for improving instance segmentation.

Results and Models

Backbone	style	Lr schd	Mem (GB)	Inf time (fps)	box AP	mask AP	Config	Download
R-50-FPN	caffe	1x	4.5		38.2	36.0	config	model | log
R-50-FPN	caffe	2x	-	-	38.8	36.3	config	model | log
R-101-FPN	caffe	1x	6.5		40.4	37.6	config	model | log
R-101-FPN	caffe	2x	-	-	41.1	38.1	config	model | log
R-X101-32x4d	pytorch	2x	7.9	11.0	41.8	38.7	config	model | log
R-X101-64x4d	pytorch	1x	11.0	8.0	43.0	39.5	config	model | log
R-X101-64x4d	pytorch	2x	11.0	8.0	42.6	39.5	config	model | log

Citation

@inproceedings{huang2019msrcnn,
    title={Mask Scoring R-CNN},
    author={Zhaojin Huang and Lichao Huang and Yongchao Gong and Chang Huang and Xinggang Wang},
    booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
    year={2019},
}