fyangneil / pavement-crack-detection

Pavement crack detection: dataset and model

The project is used to share our recent work on pavement crack detection. For the details of the work, the readers are refer to the paper "Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection" (FPHB), T-ITS 2019. You can find the paper in https://www.researchgate.net/publication/330244656_Feature_Pyramid_and_Hierarchical_Boosting_Network_for_Pavement_Crack_Detection or https://arxiv.org/abs/1901.06340.

The pavement crack datasets used in paper, crack detection results on each datasets, trained model, and crack annotation tool are stored in Google Drive, One Drive, and Daidu Yunpan extract code: jviq. If you think this project is useful for you, feel free to leave a star. (^^)

Installing

1. Install prerequisites for Caffe
2. Clone the repository

git clone https://github.com/fyangneil/pavement-crack-detection.git

3. Build Caffe

cd $ROOT_DIR/pavement-crack-detection
make -j8&make pycaffe

Training

The training and test steps are same with HED, please read the instruction in https://github.com/s9xie/hed. Here we use CRACK500 dataset as an example to demonstrate how to set experiment (assume you have successfully train and test HED on BSD500 dataset).

1. Create a "crack" folder in "pavement-crack-detection/data' folder.

cd $ROOT_DIR/pavement-crack-detection/data
mkdir crack

2. Download CRACK500 and extract it to crack folder and put "train.txt" in "crack" folder.
3. Download the fully convolutional VGG model (248MB) from here and put it in "pavement-crack-detection/examples/fphb" folder.
4. Train fpn and fphn model on CRACK500 data

cd $ROOT_DIR/pavement-crack-detection/examples/fphb
python solve_fphb_crack.py or python solve_fpn_crack.py

Test

We use CRACk500 as an example to demonstrate the process of testing the trained model.

1. Download CRACK500 and extract it to "crack" folder and put "test.txt" in "crack" folder.
2. Test the model

cd $ROOT_DIR/pavement-crack-detection/examples/fphb
python test.py

When testing on one image, you are referred to "pavement-crack-detection/examples/fphb/FPHB-tutorial.ipynb" file.

Evaluate result

We provide the evaluation tool "eval_tool" to assess the result. Before using the tool, please make sure that in predicted crack map the bright regions are crack, background is black. To get ODS and OIS, run 'crack_nms.m' first then 'crack_eval.m'. To get AIU, run 'crack_AIU.m'.

If you encounter any issue when using our code or model, feel free to contact me [email protected].

Note: please cite the corresponding papers when using these datasets.

CRACK500:

@inproceedings{zhang2016road, title={Road crack detection using deep convolutional neural network}, author={Zhang, Lei and Yang, Fan and Zhang, Yimin Daniel and Zhu, Ying Julie}, booktitle={Image Processing (ICIP), 2016 IEEE International Conference on}, pages={3708--3712}, year={2016}, organization={IEEE} }' .

@article{yang2019feature, title={Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection}, author={Yang, Fan and Zhang, Lei and Yu, Sijia and Prokhorov, Danil and Mei, Xue and Ling, Haibin}, journal={IEEE Transactions on Intelligent Transportation Systems}, year={2019}, publisher={IEEE} }

GAPs384:

@article{yang2019feature, title={Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection}, author={Yang, Fan and Zhang, Lei and Yu, Sijia and Prokhorov, Danil and Mei, Xue and Ling, Haibin}, journal={IEEE Transactions on Intelligent Transportation Systems}, year={2019}, publisher={IEEE} }

@inproceedings{eisenbach2017how, title={How to Get Pavement Distress Detection Ready for Deep Learning? A Systematic Approach.}, author={Eisenbach, Markus and Stricker, Ronny and Seichter, Daniel and Amende, Karl and Debes, Klaus and Sesselmann, Maximilian and Ebersbach, Dirk and Stoeckert, Ulrike and Gross, Horst-Michael}, booktitle={International Joint Conference on Neural Networks (IJCNN)}, pages={2039--2047}, year={2017} }

CFD:

@article{shi2016automatic, title={Automatic road crack detection using random structured forests}, author={Shi, Yong and Cui, Limeng and Qi, Zhiquan and Meng, Fan and Chen, Zhensong}, journal={IEEE Transactions on Intelligent Transportation Systems}, volume={17}, number={12}, pages={3434--3445}, year={2016}, publisher={IEEE} }

AEL:

@article{amhaz2016automatic, title={Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm Based on Minimal Path Selection.}, author={Amhaz, Rabih and Chambon, Sylvie and Idier, J{'e}r{^o}me and Baltazart, Vincent} }

cracktree200:

@article{zou2012cracktree, title={CrackTree: Automatic crack detection from pavement images}, author={Zou, Qin and Cao, Yu and Li, Qingquan and Mao, Qingzhou and Wang, Song}, journal={Pattern Recognition Letters}, volume={33}, number={3}, pages={227--238}, year={2012}, publisher={Elsevier} }

pavement-crack-detection's Issues

Welcome to our crack program

Nice project. We have a related work this year and it's available at DeepCrack.

Unable to generate exact output

Hello @fyangneil ,

with the same test code provided i am not able to generate same output for images.

Your output

Generated output

Please help me to figure what i missed.

Question

When I evaluate the result on my dataset, I need to process nms and thinned to one pixel wide on GT and crack detection . I run eval/nms_process.m to process nms,but how to thinned to one pixel wide, to run EvalEdge.m? Can you give some advice
Thank you very much

runipy FPHB-tutorial.ipynb

please help

I'm sorry to disturb you. I'd like to ask if it's convenient for you to upload the HED, RCF and FCN models you used in the experiment to Baidu cloud disk.In your paper , these models are trained by crack500 dataset and change some hyperparameter. If you can give me these models , I will thank you very much.

error == cudaSuccess (29 vs. 0)

HI, I got the following error at then end of the test.py,

F0806 20:08:39.098415 28262 syncedmem.cpp:19] Check failed: error == cudaSuccess (29 vs. 0) driver shutting down

Can you tell me how to solve with this problem, thank you!

I'm sorry to disturb you

I want to reproduce the FCN model in your paper, but the training has been unsuccessful.Can I ask how you train the model, such as the pre training model and the changes to the protoxt file.I would appreciate it if you could remember and tell me the details.Tank you very much.

How to use eval_tool?

Hello @fyangneil could you explain the step how to do evaluation?,

1. should we convert all the predicted .png file into .mat?
2. How to setup your toolbox? I always got this error.

Error in crack_nms (line 11)
    load(fullfile(path_to_input, iids(i).name));

3. What should we do first? I tried to compile the toolbox successfully and modify the code into this.

mkdir(path_to_output);
iids = dir(fullfile(path_to_input, '*.png'));
for i = 1:length(iids)
    %edge = imread(fullfile(path_to_input, iids(i).name));\
    %load(fullfile(path_to_input, iids(i).name));
    %predmap=predmap;
    %disp(fullfile(path_to_input, iids(i).name))
    filename = fullfile(path_to_input, iids(i).name);
    predmap = imread(filename);
    predmap = rgb2gray(predmap);
    [Ox, Oy] = gradient2(convTri(predmap, 4));
    [Oxx, ~] = gradient2(Ox);
    [Oxy, Oyy] = gradient2(Oy);
    O = mod(atan(Oyy .* sign(-Oxy) ./ (Oxx + 1e-5)), pi);
    predmap = edgesNmsMex(predmap, O, 2, 5, 1.01, 8);
    imwrite(predmap, fullfile(path_to_output, [iids(i).name(1:end-4) '.png']));
    save(fullfile(path_to_output, [iids(i).name(1:end-4) '.mat']),'predmap');
end

However I got this error. Suppose that I have the predicted crack in .png file in the input_folder and I want to get the evaluation result only.

Error using convConst
A must be a 4x4 or bigger 2D or 3D float array.

Error in convTri (line 87)
    J = convConst('convTri',I,r,s);

Error in crack_nms (line 17)
    [Ox, Oy] = gradient2(convTri(predmap, 4));

Testing result issue

I tried the project you provided, but I got some errors.
I used model trainedmodel/fphb_crack.caffemodel which downloaded from Google Drive, dataset CRACK500 test.txt also downloaded from Google Drive and test.py with test_fphb_crack.prototxt, but I got error messages.
F0613 18:57:03.865557 146139 net.cpp:861] Cannot copy param 0 weights from layer 'conv4_fuse'; shape mismatch. Source param shape is 512 1024 1 1 (524288); target param shape is 512 2048 3 3 (9437184). To learn this layer's parameters from scratch rather than copying from a saved net, rename the layer.

I replaced layers name conv1_fuse, conv2_fuse, conv3_fuse, conv4_fuse, conv5_fuse, score-dsn1, score-dsn2, score-dsn3, score-dsn4, score-dsn5 to conv1_fuse_1, conv2_fuse_1, conv3_fuse_1, conv4_fuse_1, conv5_fuse_1, score-dsn1_1, score-dsn2_1, score-dsn3_1, score-dsn4_1, score-dsn5_1. It works, but I got all black result images.

Did I do wrong steps or miss some steps? How can I get normal results? Thanks!

AIU

Anther question；
Can you release the evaluation code of AIU?

test image size is (2048, 2048, 3),python test.py Traceback (most recent call last): File "test.py", line 40, in <module> im = im[:, :, ::-1] IndexError: too many indices for array

test image size is (2048, 2048, 3)
python test.py
Traceback (most recent call last):
File "test.py", line 40, in
im = im[:, :, ::-1]
IndexError: too many indices for array

If I make my own data set with the same format as crack500, what code do I need to modify to work normally? Is there any requirement for the size of the picture?

solver_fuse_fpn.prototxt

where is the solver_fuse_fpn.prototxt?

Details of the labels

Hi looking for Punchout type distress data , Do you have included this ?

标签分类分为几类

你好！我没有在你的代码中看到标签分类的类别数目以及类别的名称

数据集获取

您好，我目前是学习裂缝识别任务的研究生，在GitHub中没有找到经过处理后的数据集（Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection，640*360的数据集），想用您的数据完成自己的部分工作

CUDA_VISIBLE_DEVICES=0,1 python test.py F0223 08:15:30.722712 2135 syncedmem.cpp:19] Check failed: error == cudaSuccess (29 vs. 0) driver shutting down *** Check failure stack trace: *** Aborted (core dumped)

CUDA_VISIBLE_DEVICES=0,1 python test.py

F0223 08:15:30.722712 2135 syncedmem.cpp:19] Check failed: error == cudaSuccess (29 vs. 0) driver shutting down
*** Check failure stack trace: ***
Aborted (core dumped)

dataset

Hello, I want to download your data set, but your data set is shared in Google Drive. I can't download it. Is there any other way to get the data set you share, such as Baidu Yunpan? Thank you.

vgg link not working

it seems the link to VGG is not working. could you share a new one? thanks.

How to get exact crack500 dataset

I found exact dataset from google drive. thank you.

The version of protobuf

Plotting Precision-Recall Curve

Hello @fyangneil I have a question how to plot the precision and recall curve?, Can we use the mTpIUPR.txt that available in evalIu folder where the column indicates [threshold, IU, precision, recall]?

How to make ground truth data

We want to make a crack detection dataset and want to know how to make ground truth data (methods ， softwares or tools). Crack ground truth is different from ordinary segmentation ground truth. The shape of crack is so slim to draw contours, while ordinary object, such as human and cat, is easy to draw contours. Thankyou very much!

help

Hello, when I use your code to reproduce, there are always errors as shown in the picture. Please help to solve them when you see them. Thank you

Do U resize with the crack500 dataset?

Do U resize with the crack500 dataset?

val.txt test.txt

Where are the "val.txt" and "test.txt" filse?
Do I need edit them myselt?

I did not successfully compile your code

I am using Ubuntu16.04+cuda8.0, make all -j8 and make pycaffe have no problem, but when I ran solve_fphb_crack.py, I encountered an error, "Check failed: error == cudaSuccess (10 vs 0) invalid device ordinal", so please tell me the environment information you used at that time and some details of the compilation. Thank you very much for your help.

Data links (Google Drive and OneDrive) are not working

Hi! I would like to use your data, but it seems that links are not available. Could you please check and upload them? Thank you in thank you in advance

problem

Hello, I used the original FCN to experiment on your dataset and found that the results obtained by FCN are better than the results of your model under the MIOU indicator. After reading your article, I found that you used The indicator is the indicator of edge detection. You have come to the conclusion that your model is better than FCN. Have you compared the test results of the two models under the MIOU indicator? Thank you.

python test.py CUDA_VISIBLE_DEVICES=0,1,2,3

LICENSE

Hello there,

What's the license of this repository?

Data problem

Thank you for sharing your data set, but the original truth map of the test results you got at fphb_testresult is not shared. Can you share this piece? Thank you.