adeelh / pytorch-fpn

PyTorch implementations of some FPN-based semantic segmentation architectures: vanilla FPN, Panoptic FPN, PANet FPN; with ResNet and EfficientNet backbones.

pytorch-fpn

PyTorch implementation of the following semantic segmentation architectures (see fpn.py):

• FPN, Feature Pyramid Networks for Object Detection by Lin et al., https://arxiv.com/abs/1612.03144.
• Panoptic FPN, Panoptic Feature Pyramid Networks by Kirilov et al., https://arxiv.com/abs/1901.02446.
• PANet FPN, Path Aggregation Network for Instance Segmentation by Liu et al., https://arxiv.com/abs/1803.01534

The implementations are all based on nn.Sequential with no fancy forward methods, meaning that they can be easily modified and combined together or with other modules.

Backbones

The repo provides factory functions (make_fpn_resnet() and make_fpn_efficientnet()) for creating FPN's with 2 kinds of backbones:

• ResNet backbones (via Torch Vision)
• EfficientNet backbones (via https://github.com/lukemelas/EfficientNet-PyTorch)

Multiband images

The factory methods make_fpn_resnet() and make_fpn_efficientnet() support in_channels != 3.

make_fpn_resnet(), in particular, makes use of the fusion technique described in the paper, FuseNet, by Hazirbas et al. (https://vision.in.tum.de/_media/spezial/bib/hazirbasma2016fusenet.pdf) if in_channels > 3 that adds a parallel resnet backbone for the new channels. All the pretrained weights are retained.

Usage

from factory import make_fpn_resnet

model = make_fpn_resnet(
	name='resnet18',
	fpn_type='fpn',
	pretrained=True,
	num_classes=2,
	fpn_channels=256,
	in_channels=3,
	out_size=(224, 224))

Loading through torch.hub

This repo supports importing modules through torch.hub. The models can be easily imported into your code via the factory functions in factory.py.

import torch

model = torch.hub.load(
	'AdeelH/pytorch-fpn',
	'make_fpn_resnet',
	name='resnet18',
	fpn_type='panoptic',
	num_classes=2,
	fpn_channels=256,
	in_channels=3,
	out_size=(224, 224)
)

Working example on Colab: https://colab.research.google.com/drive/1pjkw-QoqiXgKDrEYZ47EwfXcW7vgO4KX?usp=sharing

pytorch-fpn's Issues

Tuple Unpacking Error

I am getting an error on this line. It seems that sometimes inps is a tuple of four rather than the expect two. I am using this with the last pre-nochip version of raster vision.

Fusion Process Assumes Location of RGB Channels

The fusion process here seems to assume that the RGB channels are the first three channels of the input (that is, the 3 channels from the pretrained input layer are always matched with the first three channels of the input).