haozhengli / east_icpr

Forked from argman/EAST for the ICPR MTWI 2018 CHALLENGE

EAST_ICPR: EAST for ICPR MTWI 2018 CHALLENGE

Introduction

This is a repository forked from argman/EAST for the ICPR MTWI 2018 CHALLENGE.
Origin Repository: argman/EAST - A tensorflow implementation of EAST text detector
Origin Author: argman

Author: Haozheng Li
Email: [email protected] or [email protected]

Contents

1. Transform
2. Models
3. Demo
4. Train
5. Test
6. Results

Transform

Some data in the dataset is abnormal, just like ICPR MTWI 2018[BaiduYun]. Abnormal means that the ground true labels are anticlockwise, or the images are not in 3 channels. Then errors like 'poly in wrong direction' will occur while using argman/EAST.

So I wrote a matlab program to check and transform the dataset. The program named <transform.m> is in the folder 'data_transform/' and its parameters are descripted as bellow:

icpr_img_folder = 'image_9000\';                   %origin images
icpr_txt_folder = 'txt_9000\';                     %origin ground true labels
icdar_img_folder = 'ICPR2018\';                    %transformed images
icdar_gt_folder = 'ICPR2018\';                     %transformed ground true labels
icdar_img_abnormal_folder = 'ICPR2018_abnormal\';  %images not in 3 channels, which give errors in argman/EAST
icdar_gt_abnormal_folder = 'ICPR2018_abnormal\';   %transformed ground true labels

%images and ground true labels files must be renamed as <img_1>, <img_2>, ..., <img_xxx> while using argman/EAST
first_index =  0;                                  %first index of the dataset
transform_list_name = 'transform_list.txt';        %file name of the rename list

Note: For abnormal images not in 3 channels, please transform them to normal ones through other tools like Format Factory. Then add the right data to the <icdar_img_folder> and <icdar_gt_folder>, so finally you get a whole normal dataset which has been checked and transformed.

Models

1. Resnet_V1_50 Models trained on ICPR MTWI 2018 (train): [100k steps] [500k steps] [1035k steps]
2. Resnet_V1_101 Models trained on ICPR MTWI 2018 (train) + ICDAR 2017 MLT (train + val) + RCTW-17 (train): [100k steps]
3. Resnet_V1_101 Models pre-trained on Models-2, then trained on just ICPR MTWI 2018 (train): [987k steps]
4. (In argman/EAST) Resnet_V1_50 Models trained on ICDAR 2013 (train) + ICDAR 2015 (train): [50k steps]
5. (In argman/EAST) Resnet_V1_50 Models provided by tensorflow slim: [slim_resnet_v1_50]

Demo

Download the pre-trained models and run:

python run_demo_server.py --checkpoint-path models/east_icpr2018_resnet_v1_50_rbox_1035k/

Then Open http://localhost:8769 for the web demo server, or get the results in 'static/results/'.
Note: See argman/EAST#demo for more details.

Train

Prepare the training set and run:

python multigpu_train.py --gpu_list=0 --input_size=512 --batch_size_per_gpu=8 \
--checkpoint_path=models/east_icpr2018_resnet_v1_50_rbox/ \
--text_scale=512 --training_data_path=data/ICPR2018/ --geometry=RBOX \
--learning_rate=0.0001 --num_readers=18 --max_steps=50000

Note 1: Images and ground true labels files must be renamed as <img_1>, <img_2>, ..., <img_xxx> while using argman/EAST. Please see the examples in the folder 'training_samples/'.
Note 2: If --restore=True, training will restore from checkpoint and ignore the --pretrained_model_path. If --restore=False, training will delete checkpoint and initialize with the --pretrained_model_path (if exists).
Note 3: See argman/EAST#train for more details.

Test

Names of the images in ICPR MTWI 2018 are abnormal. Like <LB1gXi2JVXXXXXUXFXXXXXXXXXX.jpg> but not <img_10001.jpg>. Then errors will occur while using argman/EAST#test.

So I wrote two matlab programs to rename and inversely rename the dataset. Before evaluating, run the program named <rename.m> to make names of the images normal. This program is in the folder 'data_transform/' and its parameters are descripted as bellow:

icpr_img_folder = 'image_10000\';                      %origin images
icdar_img_folder = 'ICPR2018_test\';                   %transformed images
icdar_img_abnormal_folder = 'ICPR2018_test_abnormal\'; %images not in 3 channels, which give errors in argman/EAST

icpr_count =  10000;                                   %first index of the dataset
rename_list_name = 'rename_list.txt';                  %file name of the rename list

Note: Just like <transform.m>, please transform abnormal images through other tools like Format Factory.

After you have prepared the test set, run:

python eval.py --test_data_path=data/ICPR2018/ --gpu_list=0 \
--checkpoint_path=models/east_icpr2018_resnet_v1_50_rbox_1035k/ --output_dir=results/1035k/

Then get the results in 'results/'.

Finally inversely rename the result labels files from <img_10001.txt> to <LB1gXi2JVXXXXXUXFXXXXXXXXXX.txt> according to the rename list generated by <rename.m>. Run the program named <rename_inverse.m> which is in the folder 'data_transform/' and its parameters are descripted as bellow:

rename_list_name = 'rename_list.txt';  %file name of the rename list
icpr_img_folder = 'image_10000\';      %origin images
icpr_txt_folder = 'results\';          %result labels files generated by argman/EAST
icdar_gt_folder = 'txt_10000\';        %inversely renamed result labels files

Then zip the results in 'txt_10000/' and submit it to the ICPR MTWI 2018 CHALLENGE.

Results

Finally our model <east_icpr2018_resnet_v1_50_rbox_1035k> rank 31 in the ICPR MTWI 2018 CHALLENGE:

Here are some results on ICPR MTWI 2018:

Have fun!! :)

east_icpr's Issues

OSError: [WinError 126] The specified module could not be found

I think this error is because of the shapely package but I have installed this package still I am getting an error :

Traceback (most recent call last):
  File "eval.py", line 8, in <module>
    import locality_aware_nms as nms_locality
  File "C:\Users\HPO2KOR\Desktop\Work\EAST-virtualenv\EAST_ICPR-master\locality_aware_nms.py", line 2, in <module>
    from shapely.geometry import Polygon
  File "C:\Users\HPO2KOR\Desktop\Work\EAST-virtualenv\eastvenv\lib\site-packages\shapely\geometry\__init__.py", line 4, in <module>
    from .base import CAP_STYLE, JOIN_STYLE
  File "C:\Users\HPO2KOR\Desktop\Work\EAST-virtualenv\eastvenv\lib\site-packages\shapely\geometry\base.py", line 18, in <module>
    from shapely.coords import CoordinateSequence
  File "C:\Users\HPO2KOR\Desktop\Work\EAST-virtualenv\eastvenv\lib\site-packages\shapely\coords.py", line 8, in <module>
    from shapely.geos import lgeos
  File "C:\Users\HPO2KOR\Desktop\Work\EAST-virtualenv\eastvenv\lib\site-packages\shapely\geos.py", line 145, in <module>
    _lgeos = CDLL(os.path.join(sys.prefix, 'Library', 'bin', 'geos_c.dll'))
  File "C:\Users\HPO2KOR\Desktop\Work\EAST-virtualenv\eastvenv\lib\ctypes\__init__.py", line 348, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] The specified module could not be found

"invalid poly" and "poly in wrong detection" in training

Hello
I have got a normal trainingset by using transform.m ,however there was some "invalid poly" and "poly in wrong detection" in training.Is this normal？How should i correct this error？ @HaozhengLi

Is there any improvement compare to origin east?

Is there any improvement compare to origin east?

Windows

Hi
I am trying to start training on Windows 10.
Has anyone tried this code in the windows environment?
Has anyone tried training in windows?

Can pre-trained models be uploaded to Google Drive? BaiduYun does not work in some country.

I guess competition is completed on 2018/05/23

Cannot download ICPR data

Hi
I am unable to download the ICPR data.
Can you share it elsewhere?

ImportError: dynamic module does not define init function (initadaptor)

when i run eval.py, there is an error:ImportError: dynamic module does not define init function (initadaptor)

How can i test Resnet101 backbone based model?

Thanks for ur work on this project, when i use resnet101 backbone based pre-trained model by replace the checkpoint path, it just didn't work, so what can i do to test resnet101 pre-trained model? Thx!

ICPR数据集

作者你好，由于ICPR比赛结束了，我希望能下载训练和测试数据集，能否提供下ICPR的测试数据集的下载链接或者云盘链接，谢谢

请问训练自己的数据，input_size和text_size分别表示什么？

问题1：样本图片是否必须是正方形的，比如512512的？我的样本图片是34564608大小的可以嘛？
问题2：input_size表示的是什么意思，正方形图片的一个边的像素吗？
问题3：text_size表示的是什么意思，为什么和input_size一样都是512?

请问res101混合数据集的那个模型为什么不行运行呢？

Pretrained model

The link for shared pretrained model isnt working. Can you please share the pretrained model?