li-js / fcn.berkeleyvision.org

Fully Convolutional Networks for Semantic Segmentation by Jonathan Long*, Evan Shelhamer*, and Trevor Darrell. CVPR 2015.

These are models and scripts for the paper:

Fully Convolutional Models for Semantic Segmentation
Jonathan Long*, Evan Shelhamer*, Trevor Darrell
CVPR 2015
arXiv:1411.4038

and its journal edition in PAMI (to appear).

Note that this is a work in progress and the final, reference version is coming soon. Please ask Caffe and FCN usage questions on the caffe-users mailing list.

These models are compatible with BVLC/caffe:master @ 8c66fa5 with the merge of PRs BVLC/caffe#3613 and BVLC/caffe#3570. The code and models here are available under the same license as Caffe (BSD-2) and the Caffe-bundled models (that is, unrestricted use; see the BVLC model license).

PASCAL VOC models: trained online with high momentum for a ~5 point boost in mean intersection-over-union over the original models. These models are trained using extra data from Hariharan et al., but excluding SBD val. FCN-32s is fine-tuned from the ILSVRC-trained VGG-16 model, and the finer striders are then fine-tuned in turn.

• FCN-32s PASCAL: single stream, 32 pixel prediction stride version, scoring 63.6 mIU on seg11valid
• FCN-16s PASCAL: two stream, 16 pixel prediction stride version, scoring 65.0 mIU on seg11valid
• FCN-8s PASCAL: three stream, 8 pixel prediction stride version, scoring 65.5 mIU on seg11valid and 67.2 mIU on seg12test

To reproduce the validation scores, use the seg11valid split defined by the paper in footnote 7. Since SBD train and PASCAL VOC 11 segval intersect, we only evaluate on the non-intersecting set for validation purposes.

The following models have not yet been ported to master and trained with the latest settings. Check back soon.

PASCAL VOC:

• FCN-AlexNet PASCAL: AlexNet (CaffeNet) single stream, 32 pixel prediction stride version

SIFT Flow model (also fine-tuned from VGG-16):

• FCN-16s SIFT Flow: two stream, 16 pixel prediction stride version

NYUDv2 models (also fine-tuned from VGG-16, and using HHA features from Gupta et al. https://github.com/s-gupta/rcnn-depth):

• FCN-32s NYUDv2: single stream, 32 pixel prediction stride version
• FCN-16s NYUDv2: two stream, 16 pixel prediction stride version

PASCAL-Context models including architecture definition, solver configuration, and bare-bones solving script (fine-tuned from the ILSVRC-trained VGG-16 model):

• FCN-32s PASCAL-Context: single stream, 32 pixel prediction stride version
• FCN-16s PASCAL-Context: two stream, 16 pixel prediction stride version
• FCN-8s PASCAL-Context: three stream, 8 pixel prediction stride version