yangxue0827 / r2cnn_head_fpn_tensorflow

Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multiscale Rotation Region Convolutional Neural Network

R2CNN_HEAD (The paper is under review.): Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multiscale Rotation Region Convolutional Neural Network

Recommend improved code： https://github.com/DetectionTeamUCAS

A Tensorflow implementation of FPN or R2CNN detection framework based on FPN.
You can refer to the papers R2CNN Rotational Region CNN for Orientation Robust Scene Text Detection or Feature Pyramid Networks for Object Detection
Other rotation detection method reference R-DFPN, RRPN and R2CNN

If useful to you, please star to support my work. Thanks.

Citation

Some relevant achievements based on this code.

@article{[yang2018position](https://ieeexplore.ieee.org/document/8464244),
	title={Position Detection and Direction Prediction for Arbitrary-Oriented Ships via Multitask Rotation Region Convolutional Neural Network},
	author={Yang, Xue and Sun, Hao and Sun, Xian and  Yan, Menglong and Guo, Zhi and Fu, Kun},
	journal={IEEE Access},
	volume={6},
	pages={50839-50849},
	year={2018},
	publisher={IEEE}
}

@article{[yang2018r-dfpn](http://www.mdpi.com/2072-4292/10/1/132),
	title={Automatic ship detection in remote sensing images from google earth of complex scenes based on multiscale rotation dense feature pyramid networks},
	author={Yang, Xue and Sun, Hao and Fu, Kun and Yang, Jirui and Sun, Xian and Yan, Menglong and Guo, Zhi},
	journal={Remote Sensing},
	volume={10},
	number={1},
	pages={132},
	year={2018},
	publisher={Multidisciplinary Digital Publishing Institute}
}

Configuration Environment

ubuntu(Encoding problems may occur on windows) + python2 + tensorflow1.2 + cv2 + cuda8.0 + GeForce GTX 1080
If you want to use cpu, you need to modify the parameters of NMS and IOU functions use_gpu = False in cfgs.py
You can also use docker environment, command: docker pull yangxue2docker/tensorflow3_gpu_cv2_sshd:v1.0

Installation

Clone the repository

git clone https://github.com/yangxue0827/R2CNN_HEAD_FPN_Tensorflow.git    

Make tfrecord

The data is VOC format, reference here
data path format ($R2CNN_HEAD_ROOT/data/io/divide_data.py)

├── VOCdevkit
│   ├── VOCdevkit_train
│       ├── Annotation
│       ├── JPEGImages
│    ├── VOCdevkit_test
│       ├── Annotation
│       ├── JPEGImages

Clone the repository

cd $R2CNN_HEAD_ROOT/data/io/  
python convert_data_to_tfrecord.py --VOC_dir='***/VOCdevkit/VOCdevkit_train/' --save_name='train' --img_format='.jpg' --dataset='ship'
     

Compile

cd $PATH_ROOT/libs/box_utils/
python setup.py build_ext --inplace

Demo

1、Unzip the weight $R2CNN_HEAD_ROOT/output/res101_trained_weights/*.rar
2、put images in $R2CNN_HEAD_ROOT/tools/inference_image
3、Configure parameters in $R2CNN_HEAD_ROOT/libs/configs/cfgs.py and modify the project's root directory
4、

cd $R2CNN_HEAD_ROOT/tools      

5、image slice

python inference.py   

6、big image

cd $FPN_ROOT/tools
python demo.py --src_folder=.\demo_src --des_folder=.\demo_des         

Train

1、Modify $R2CNN_HEAD_ROOT/libs/lable_name_dict/***_dict.py, corresponding to the number of categories in the configuration file
2、download pretrain weight(resnet_v1_101_2016_08_28.tar.gz or resnet_v1_50_2016_08_28.tar.gz) from here, then extract to folder $R2CNN_HEAD_ROOT/data/pretrained_weights
3、

cd $R2CNN_HEAD_ROOT/tools  
python train.py      

Test tfrecord

cd $R2CNN_HEAD_ROOT/tools   
python test.py      

eval(Not recommended, Please refer here)

cd $R2CNN_HEAD_ROOT/tools   
python eval.py    

Summary

tensorboard --logdir=$R2CNN_HEAD_ROOT/output/res101_summary/ 

Graph

Test results

r2cnn_head_fpn_tensorflow's Issues

vif2xml.py 怎么使用？

我想问一下R2CNN和RRPN这两个网络的参数量和FLOPs在Res101和输入图像大小在800×800下是多少呢

where is io folder? where is inference.py?

@yangJirui @yangxue0827 Hi,
I can not find io folder and inference.py

training and inference procedure cannot be done without them.
there is no sample of annotations as well.

Can not find the Adaptive ROI Align?

Adaptive ROI Align was seen in the paper, but I can not find it in the code, Can you share your code of Adaptive ROI Align?

Batched inference

Thank you for you great work and ideas. I would like to try this repo on my own satellite images. However since I don't get good utilization by passing images one by one in inference.py I decided to inference by batches. Initially I added one dimension to the image placeholder:

img_plac = tf.placeholder(shape=[None, None, 3], dtype=tf.uint8

Can you please guide me through this process because I am not familiar with tensorflow semantics

Batched inference

Thank you for you great work and ideas. I would like to try this repo on my own satellite images. However since I don't get good utilization by passing images one by one in inference.py I decided to inference by batches. Initially I added one dimension to the image placeholder:

img_plac = tf.placeholder(shape=[None, None, None, 3], dtype=tf.uint8)

And created a batch of images before feeding session.run

feed_dict={img_plac: IMG_BATCH}

I also changed the batch size in the config, and the preprocesding code that resizes the image to take.batch into account.

The error message i get with batch_size=8 is:
InvalidArgumentError: Incompatible shapes: [1964160] vs. [245520]
[[Node: rpn_proposals/decode/mul = Mul[T=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"](rpn_proposals/decode/truediv, rpn_proposals/decode/sub)]]

Can you please guide me through this process because I am not familiar with tensorflow semantics? Thank you very much once more

Dataset SRSS

Could you also share SRSS dataset?

dota数据集中的船只head_x, head_y文件没有找到

你好，感谢你的代码，让我学到了许多。 有个问题就是关于图像的head_x,head_y数据的标签，没有找到，需要我们自己标吗？

head_x, head_y ？？

head_x = (x_0 +x_1)/2
head_y = (y_0 + y_1) / 2
是这样计算的吗？？？

ImportError: dynamic module does not define module export function (PyInit_rotate_polygon_nms)

1. clone the repo
2. run docker

docker run --rm --name tfgpu -p 5552:5552 --gpus all -it -v /Data/share:/Data/share yangxue2docker/tensorflow3_gpu_cv2_sshd:v1.0 /bin/bash

3. In the clonned repo
   cd tools
python3 inference.py

4. Get this error:

root@27260403fc95:/scratch/MDA000_BOAT00/DEVELOPMENT/USER_WORKSPACE/pgalles/R2CNN_HEAD_FPN_Tensorflow/tools# python inference.py
Traceback (most recent call last):
  File "inference.py", line 17, in <module>
    from libs.fast_rcnn import build_fast_rcnn
  File "../libs/fast_rcnn/build_fast_rcnn.py", line 13, in <module>
    from libs.box_utils import nms_rotate
  File "../libs/box_utils/nms_rotate.py", line 12, in <module>
    from libs.box_utils.rotate_polygon_nms import rotate_gpu_nms
ImportError: dynamic module does not define module export function (PyInit_rotate_polygon_nms)

I am not sure. Am I missing some steps?