Skip to content

Latest commit

 

History

History
77 lines (55 loc) · 7.74 KB

README.md

File metadata and controls

77 lines (55 loc) · 7.74 KB
Raw

A novel Region of Interest Extraction Layer for Instance Segmentation

Abstract

Given the wide diffusion of deep neural network architectures for computer vision tasks, several new applications are nowadays more and more feasible. Among them, a particular attention has been recently given to instance segmentation, by exploiting the results achievable by two-stage networks (such as Mask R-CNN or Faster R-CNN), derived from R-CNN. In these complex architectures, a crucial role is played by the Region of Interest (RoI) extraction layer, devoted to extracting a coherent subset of features from a single Feature Pyramid Network (FPN) layer attached on top of a backbone. This paper is motivated by the need to overcome the limitations of existing RoI extractors which select only one (the best) layer from FPN. Our intuition is that all the layers of FPN retain useful information. Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance. A comprehensive ablation study at component level is conducted to find the best set of algorithms and parameters for the GRoIE layer. Moreover, GRoIE can be integrated seamlessly with every two-stage architecture for both object detection and instance segmentation tasks. Therefore, the improvements brought about by the use of GRoIE in different state-of-the-art architectures are also evaluated. The proposed layer leads up to gain a 1.1% AP improvement on bounding box detection and 1.7% AP improvement on instance segmentation.

Introduction

By Leonardo Rossi, Akbar Karimi and Andrea Prati from IMPLab.

We provide configs to reproduce the results in the paper for "A novel Region of Interest Extraction Layer for Instance Segmentation" on COCO object detection.

This paper is motivated by the need to overcome to the limitations of existing RoI extractors which select only one (the best) layer from FPN.

Our intuition is that all the layers of FPN retain useful information.

Therefore, the proposed layer (called Generic RoI Extractor - GRoIE) introduces non-local building blocks and attention mechanisms to boost the performance.

Results and models

The results on COCO 2017 minival (5k images) are shown in the below table.

Application of GRoIE to different architectures

Backbone Method Lr schd box AP mask AP Config Download
R-50-FPN Faster Original 1x 37.4 config model | log
R-50-FPN + GRoIE 1x 38.3 config model | log
R-50-FPN Grid R-CNN 1x 39.1 config model | log
R-50-FPN + GRoIE 1x config
R-50-FPN Mask R-CNN 1x 38.2 34.7 config model | log
R-50-FPN + GRoIE 1x 39.0 36.0 config model | log
R-50-FPN GC-Net 1x 40.7 36.5 config model | log
R-50-FPN + GRoIE 1x 41.0 37.8 config model | log
R-101-FPN GC-Net 1x 42.2 37.8 config model | log
R-101-FPN + GRoIE 1x 42.6 38.7 config model | log

Citation

If you use this work or benchmark in your research, please cite this project.

@inproceedings{rossi2021novel,
  title={A novel region of interest extraction layer for instance segmentation},
  author={Rossi, Leonardo and Karimi, Akbar and Prati, Andrea},
  booktitle={2020 25th International Conference on Pattern Recognition (ICPR)},
  pages={2203--2209},
  year={2021},
  organization={IEEE}
}

Contact

The implementation of GRoIE is currently maintained by Leonardo Rossi.