dvornikita / blitznet Goto Github PK

Hi,

I have encountered a problem during running demo with pretrained models. Console output:

NotFoundError (see above for traceback): Key ssd/confidence/ssd_back/block_rev1/biases not found in checkpoint
[[Node: save/RestoreV2 = RestoreV2[dtypes=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, ..., DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2/tensor_names, save/RestoreV2/shape_and_slices)]]

Have anyone had similar issue?

hello
I'v runned the script demo.py (I turned off draw option when call detector.feed_forward) using your pre-trained model BlitzNet300_COCO+VOC07+12 on our GPU Tesla P100. but I found that only 12 fps when feed 300x200 pictures to the model. Your article "BlitzNet: A Real-Time Deep Network for Scene Understanding", However, shows that It can reach 24 fps on GPU Titan X. Actually, the performance of Tesla P100(4.7 ~ 5.3 TFLOPS) is better than Titan X(up to 0.355 TFLOPS), Why?

Running the COCO+VOC model on demo.py gives me a dataloss error.

DataLossError (see above for traceback): Checksum does not match: stored 2903795978 vs. calculated on the restored bytes 3205988616
[[Node: save/RestoreV2_299 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save/Const_0_0, save/RestoreV2_299/tensor_names, save/RestoreV2_299/shape_and_slices)]]

Has anybody experienced this ? I am just trying to run inference using demo.py

Hi, would you like to share the results which do not adopt jointly training on coco dataset.
Best

how to creat it?

I understand that Blitznet resizes images to 512x512 or 300x300 in its current implementation. This is being done during training in training.py in function extract_batch in function data_augmentation (

However if I would like to keep the aspect ratio of my dataset (all images are hxw, e.g. 256x512) that line can be removed. However then there is a mismatch between the number of anchor boxes and the net output (i.e. in function detection_loss size of tensors location and pos_mask do no longer match).

I have healed that kind of ad hoc by removing the last layer with fm_sizes=1 (

After I download the trained checkpoint on COCO dataset, after test, the AP are all 0, the command is
python test.py --run_name=BlitzeNet300_x4_coco_det --dateset=coco --split=val2017 --x4 --image_size=300 --detect --ckpt=1
so I don't know what is the problem, looking forward for your help~~~

No such file or directory: '/home/cbl/PycharmProjects/blitznet-master/Datasets/VOCdevkit/VOC2012/ImageSets/SegmentationAug/train.txt'

SegmentationAug is not found in the voc12

I started a training on my GPU machine (GeForce GTX 1080), but nothing happens after the message "Starting training..." is shown. No checkpoints are written and the log file is no longer updated. After 24 hours I stopped training.

Call:
python3 training.py --run_name=BlitzNet300_x4_VOC12 --dataset=voc12-train --trunk=resnet50 --x4 --batch_size=32 --optimizer=adam --detect --segment --max_iterations=65000 --lr_decay 40000 50000

Hi! Thanks for your kind share! I have met some problems when I used your code to train the models to reproduce the performance as you list in your paper.
With the same configuration (batch size=32, step=40000 for Pascal voc2012), I trained the network for several times. Then I found that the values of mIOU and mAP are unstable. The amplitude of variations is about 1%. Have you ever met this problem? Moreover, I can’t trained a better model to get the detection performance as you list in Table 3 in your paper. My best is about 79.1 (80.0 in Table 3). Can you help me for a better performance reproduction? Thanks a lot !

Hi,
I am planning to run this on cityscapes dataset to check the results(Detection got from instance).However It seems extremely complicated to train a simple network. Just changing the image paths is not sufficient.
Are you planning to add any read me on how to train on other datasets?
If not could you let me know what all I files I would need to change to simply train ?

In my experiments on various datasets I am getting fairly coarse semantic segmentation results (with 512), even if running on 512x512 images. They are often similar to what can be seen for the Cityscapes segmentation example in #15.

I have run trainings with and without --x4, results do not seem to be dramatically better for segmentation with stride 4 in the last layer.

Is that something others can confirm? If so what could be a good way to improve on it (possibly at the expense of somewhat reduced FPS)?

Using the code below in the demo.py, I converted the graph proto to a file so I can see all the nodes using tensorboard.

tf.train.write_graph(sess.graph.as_graph_def(),'./tmp', 'output_inference_graph.pb', as_text=False)
writer = tf.summary.FileWriter("./tmp/demo", sess.graph)

However, I tried to freeze the model using the latest checkpoints, but I got graph_def invalid when I try to use it. Maybe it is because the detection outputs come from a list of tensors and not a single output, and also because without export function, the file become big (250mb).

     nodes1='Gather_2,Gather_6,Gather_10,Gather_14,Gather_18,Gather_22,Gather_26,Gather_30,Gather_34,Gather_38,Gather_42,Gather_46,Gather_50,Gather_54,Gather_58,Gather_62,Gather_66,Gather_70,Gather_74,Gather_78,'          

nodes2='Gather_3,Gather_7,Gather_11,Gather_15,Gather_19,Gather_23,Gather_27,Gather_31,Gather_35,Gather_39,Gather_43,Gather_47,Gather_51,Gather_55,Gather_59,Gather_63,Gather_67,Gather_71,Gather_75,Gather_79,' 
 
nodes3='Reshape,mean_iou/mean_iou,mean_iou/AssignAdd'
            output_node_names=nodes1+nodes2+nodes3

            print(output_node_names)
            output_graph_def = graph_util.convert_variables_to_constants(sess,input_graph_def,output_node_names.split(","))
        
            output_graph="./tmp/blitznet_frozen_model.pb"
            with tf.gfile.GFile(output_graph, "wb") as f:
                f.write(output_graph_def.SerializeToString())

In the object detection api, they use tf.identity to create dummy nodes for the final 3 outputs[class, score, num_det]. According to what do I understand from Blitznet graph, the boolean_mask_1 subgraph handles the object detection task and the mean_iou subgraph handles the segmentation task. However, in the blitznet graph, the outputs [detection boxes, detection scores, detection categories] are not visible. Mean_iou ('mean_iou/mean_iou') is the only one that I can find. Is it necessary that the outputs for detection and score have to be gathered in a list (net_out) with 20 tensors for detection and 20 tensors for score? Could it be possible to create a dummy identity operation to allocate these results? Do they already exist in the graph but I didn't notice them?

@dvornikita I will try my best to do something about it and post any result. Let me know if you have any suggestion, correct me if I'm wrong and comment about my observations

PD: I know there is nothing wrong with the code and we dont need to freeze the model to get the results but I would like to try this model in C# and till now, the only way that I know to get results is to input a frozen graph and also to separate the outputs in independent nodes so I can run the operations (tf.identity of each output) and get the results. Something like:
Results = Run(Output[] inputs, Tensor[] inputValues, Output[] outputs, Operation[] targetOperations)

Hi
I want to freeze the ckpt/meta to pb file.Is anyone can help me?
Thanks a lot!

I have a new paper which intend to submit to ICME 2018 (the DDL is tomorrow, 12.15), when I cited your paper in my paper, I got one question about the speed in your paper: when the input is 300x300, the speed of yours is much lower than the corresponding SSD, however when the input is 512x512, yours is better than the corresponding SSD which uses VGGNet as base net, it is amazing, can you explain some details? Or since the BlitzNet is realized with TensorFlow, which will be faster than official SSD ? Or else...
I will be very grateful for your reply to [email protected]. (I could not find your email.... I am so sorry for such hurry)

On running training using:

python demo.py --run_name=BlitzNet300_COCO+VOC07+12 --x4 --detect --segment --eval_min_conf=0.5 --ckpt=1

I keep getting the error:

INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.NotFoundError'>, /home/anarayanan/TenserflowPlayground/blitznet/Datasets/voc12-train-segmentation
	 [[Node: parallel_read/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](parallel_read/TFRecordReaderV2, parallel_read/filenames)]]
[INFO]: Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.NotFoundError'>, /home/anarayanan/TenserflowPlayground/blitznet/Datasets/voc12-train-segmentation
	 [[Node: parallel_read/ReaderReadV2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/cpu:0"](parallel_read/TFRecordReaderV2, parallel_read/filenames)] 

this is clearly because my VOC folder paths are not correct..Could you let me know how the folder structure should look like for training and val datasets?When I download Pascal VOC and extract I only get a folder called VOCDEVKIT in Datasets folder.

Do you think that we could add to the detector a list class indexs to filter?
I think like something that could be used in https://github.com/dvornikita/blitznet/blob/master/detector.py#L122 and put in Detector class constructor.

What do you think?

@dvornikita
Thank your for share your code. Based on your papers, this code will become very important in this field.

I have been trying to test the code using the models already trained by you for VOC07+12.

$ python test.py --run_name=BlitzNet300_VOC07+12 --dataset=voc07 --split=test --image_size=300 --detect --segment --ckpt=1 --x4

However, I always get the error:

2017-09-18 09:55:31.514242: W C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\framework\op_kernel.cc:1192] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [1,1,512,64] rhs shape= [3,3,512,64]
         [[Node: save/Assign_266 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv/weights, save/RestoreV2_266/_31)]]
2017-09-18 09:55:31.515242: W C:\tf_jenkins\home\workspace\rel-win\M\windows-gpu\PY\35\tensorflow\core\framework\op_kernel.cc:1192] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [1,1,512,64] rhs shape= [3,3,512,64]

The input tensor has different shape, could it be a bug or I am doing something wrong?

Solution:
Since VOC don't have segmentation annotations, the flag --segment has to be removed.

Can you explain the calculation of the true_number_of_negatives ?
true_number_of_negatives = tf.minimum(3 * number_of_positives, tf.shape(pos_mask)[1] - number_of_positives)

tensorflow.python.framework.errors_impl.NotFoundError: /home/cbl/PycharmProjects/blitznet-master/Datasets/voc07-trainval-segmentation; No such file or directory

how to solve it?,thanks

the different between the tf.nn.conv2d_transpose() and the tf.image.resize_nearest_neighbor(), thx.

I read through your paper and found no result on COCO right?

I am attempting to run the demo.py without any change to the code, by downloading the COCO model and I get a shape error as following:

InvalidArgumentError : Assign requires shapes of both tensors to match. lhs shape= [1,1,512,64] rhs shape= [3,3,512,64]
[[Node: save/Assign_266 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/device:CPU:0"](ssd/Conv/weights, save/RestoreV2_266)]]

Any help please ?

hi，
when run the command:
python test.py --run_name=BlitzNet512_COCO --dataset=voc07 --split=test --image_size=512 --detect --segment --ckpt=1

it show errors:

shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.694924: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.695087: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.695199: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.695386: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.695567: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.695809: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.695970: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.696142: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.696327: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.696403: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.696473: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.696734: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
2017-08-23 18:41:49.696875: W tensorflow/core/framework/op_kernel.cc:1158] Invalid argument: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]
Traceback (most recent call last):
File "test.py", line 113, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "test.py", line 62, in main
detector.restore_from_ckpt(args.ckpt)
File "/media/media_share/linkfile/blitznet/blitznet/detector.py", line 107, in restore_from_ckpt
saver.restore(self.sess, ckpt_path)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1548, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]

Caused by op u'save/Assign_279', defined at:
File "test.py", line 113, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "test.py", line 62, in main
detector.restore_from_ckpt(args.ckpt)
File "/media/media_share/linkfile/blitznet/blitznet/detector.py", line 106, in restore_from_ckpt
saver = tf.train.Saver(tf.global_variables())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1139, in init
self.build()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1170, in build
restore_sequentially=self._restore_sequentially)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 691, in build
restore_sequentially, reshape)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 419, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 155, in restore
self.op.get_shape().is_fully_defined())
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/state_ops.py", line 271, in assign
validate_shape=validate_shape)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_state_ops.py", line 45, in assign
use_locking=use_locking, name=name)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [21] rhs shape= [81]
[[Node: save/Assign_279 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_7/biases"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_7/biases, save/RestoreV2_279/_1)]]

Could you give me some adivce?

thank you very much

Is there any link to download the pretrained model?

How do I test the code in my own image?

Thanks for your kind share!
Because there is no segmentation mask for voc07 dataset, as you said , zeros are padded for these samples as segmentation masks. What confused me is whether to compute the segmentation loss for these samples or not. As known, '0' represents background in segmentation labels. when I using
--run_name=BlitzNet300_x4_VOC0712_detseg
--dataset=voc07+12-segmentation
--trunk=resnet50
--x4
--batch_size=32
--optimizer=adam
--detect
--segment
--max_iterations=65000
--lr_decay
40000
50000
for training the model, and validate the trained model on voc12 validation dataset, the detection performance is 83% map, but only 47%mIoU for segmentation. Can you help me?
Thanks!

Hey,
Thanks for releasing the awesome work on instance segmentation.
I am inspired from your work and just want to check out the results.

I started training on pascal Voc 2012 data-set with extra annotations. Algorithm combined loss starts from 14 and after 50k iterations it drop to 2-3 and after that it is just fluctuating between these values, even i tried to train it for more iterations but no drop in loss.

Can you please suggest what happening.
I am using Geforce GTX 1080 with 8 gb GPU Ram and training the algorithm with batch_size 16.

If possible can you provide the Pre-trained model on any dataset either pascal or coco, so that i can check out the performance.

Thanks

How do you calculate the GFLOPS of a model? The less the parameter quantity of the model, the shorter the inference time of the model?

@dvornikita Many thanks for making this publicly available, very nice results!! My questions relate to current version a0403d3.

I have been wondering why in https://github.com/dvornikita/blitznet/blob/master/datasets.py row 210 segmentation=True, I was expecting segmentation=segmentation?

Also, in https://github.com/dvornikita/blitznet/blob/master/voc_loader.py row row 39
assert not self.segmentation or self.segmentation and self.year == '12'
it seems impossible to get a segmentation dataset from VOC2007 (although it should be possible in my understanding), i.e. python datasets.py with create_voc_dataset('07', 'trainval', True) uncommented is running into this assert. What is the reason for this assert?

Last but not least a question on splits_to_sizes in https://github.com/dvornikita/blitznet/blob/master/datasets.py (lines 71-87):

1. voc12-train-segmentation: 10582 - relates to the number of training files in the augmented segmentation dataset, i.e. number of lines in VOCdevkit/VOC2012/ImageSets/SegmentationAug/train.txt.
2. voc12-train-segmentation-original: 1464 - corresponds to the number of training files in original VOC2012 segmentation, i.e. VOCdevkit/VOC2012/ImageSets/Segmentation/train.txt
3. voc12-val-segmentation: 1449 - corresponds to the number of training files in original VOC 2012 segmentation, i.e. VOCdevkit/VOC2012/ImageSets/Segmentation/val.txt

So far so good, however, I have hard time to confirm numbers for VOC2007. I am finding that VOCdevkit/VOC2007/ImageSets/Segmentation/train.txt is containing 209 rows, val.txt 213 rows, trainval.txt (not surprisingly) 422 rows and test.txt 210 rows. This is very different from the numbers I am seeing in splits_to_sizes for voc2007. What am I doing wrong?

Hi there,

I have tried to execute the example training command:

python training.py --run_name=BlitzNet300_x4_VOC0712_detseg --dataset=voc07+12-segmentation --trunk=resnet50 --x4 --batch_size=32 --optimizer=adam --max_iterations=65000 --lr_decay 40000 50000

but have encountered an assertion error:

Traceback (most recent call last):
  File "training.py", line 327, in <module>
    tf.app.run()
  File "/home/drsleep/.local/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "training.py", line 280, in main
    assert args.detect or args.segment, "Either detect or segment should be True"
AssertionError: Either detect or segment should be True

Does it mean I need to provide detect and segment explicitly?

Hi dvornikita,

First of all thanks for this amazing work! :)
I have some problems when I trying to test the pretrain BlitzNet512 model.

The following message appears and I don't know how to deal with it.. maybe you have some solution for me :)

"InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [3,3,512,64] rhs shape= [1,1,512,64]"

when I run
python test.py --run_name=BlitzNet300_x4_VOC0712_detseg --dataset=voc07 --split=test --image_size=300 --x4 --detect --segment --ckpt=65 --save_first_n=100
I meet :ModuleNotFoundError: No module named 'progressbar'
I have install progressbar and with python3.6
could you help me?

Hello,@dvornikita,I'm interested in your work,and after read your paper,I decide to run your demo,but I encounter the problem below,could you please help me?
My tensorflow is '1.5.0-dev20171114'
my opencv version is 3.3.0,
python version is anaconda2
I download the voc2007 and voc2012 dataset,and put them at /home/datasets after run and I comment the datasets.py #from coco_loader import COCOLoader, COCO_CATS and uncomment the main function about voc dataset,after run datasets.py,I got the traceback:

5000 files are processed
Done
Traceback (most recent call last):
File "datasets.py", line 239, in
create_voc_dataset('12', 'train', True, True)
File "datasets.py", line 210, in create_voc_dataset
loader = VOCLoader(year, split, segmentation=segmentation, augmented_seg=augmented_seg)
File "/home/winter/blitznet/voc_loader.py", line 46, in init
with open(os.path.join(self.root, filelist % self.split), 'r') as f:
IOError: [Errno 2] No such file or directory: '/home/winter/Datasets/VOCdevkit/VOC2012/ImageSets/SegmentationAug/train.txt'

I guess this is because I didn't use extra annotations for segmentation provided by B. Hariharan et al,
then I go on to run the demo,I have downloaded BlitzNet300* and BlitzNet500* that for voc07+12 datasets,after run
python demo.py --run_name=BlitzNet300_VOC07+12 --x4 --detect --segment --eval_min_conf=0.5 --ckpt=1
I got the error:

Traceback (most recent call last):
File "demo.py", line 69, in
tf.app.run()
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "demo.py", line 58, in main
detector.restore_from_ckpt(args.ckpt)
File "/home/winter/blitznet/detector.py", line 111, in restore_from_ckpt
saver.restore(self.sess, ckpt_path)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1548, in restore
{self.saver_def.filename_tensor_name: save_path})
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 789, in run
run_metadata_ptr)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 997, in _run
feed_dict_string, options, run_metadata)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1132, in _do_run
target_list, options, run_metadata)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1152, in _do_call
raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: Assign requires shapes of both tensors to match. lhs shape= [1,1,512,64] rhs shape= [3,3,512,64]
[[Node: save/Assign_270 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_2/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_2/weights, save/RestoreV2_270/_23)]]

Caused by op u'save/Assign_270', defined at:
File "demo.py", line 69, in
tf.app.run()
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "demo.py", line 58, in main
detector.restore_from_ckpt(args.ckpt)
File "/home/winter/blitznet/detector.py", line 110, in restore_from_ckpt
saver = tf.train.Saver(tf.global_variables())
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1139, in init
self.build()
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1170, in build
restore_sequentially=self._restore_sequentially)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 691, in build
restore_sequentially, reshape)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 419, in _AddRestoreOps
assign_ops.append(saveable.restore(tensors, shapes))
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 155, in restore
self.op.get_shape().is_fully_defined())
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 271, in assign
validate_shape=validate_shape)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 45, in assign
use_locking=use_locking, name=name)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op
op_def=op_def)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2506, in create_op
original_op=self._default_original_op, op_def=op_def)
File "/home/winter/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1269, in init
self._traceback = _extract_stack()

InvalidArgumentError (see above for traceback): Assign requires shapes of both tensors to match. lhs shape= [1,1,512,64] rhs shape= [3,3,512,64]
[[Node: save/Assign_270 = Assign[T=DT_FLOAT, _class=["loc:@ssd/Conv_2/weights"], use_locking=true, validate_shape=true, _device="/job:localhost/replica:0/task:0/gpu:0"](ssd/Conv_2/weights, save/RestoreV2_270/_23)]]

I'm confusing about this ,could you help me? Thank you

There are no results stored after the demo file has been executed.
input:
python demo.py --run_name=BlitzNet300_COCO+VOC07+12 --x4 --detect --segment --eval_min_conf=0.5 --ckpt=1

output:
2019-02-12 16:50:43.588949: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
2019-02-12 16:50:43.798149: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1432] Found device 0 with properties:
name: GeForce GTX 1070 major: 6 minor: 1 memoryClockRate(GHz): 1.645
pciBusID: 0000:01:00.0
totalMemory: 8.00GiB freeMemory: 6.63GiB
2019-02-12 16:50:43.807290: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1511] Adding visible gpu devices: 0
2019-02-12 16:50:44.757899: I tensorflow/core/common_runtime/gpu/gpu_device.cc:982] Device interconnect StreamExecutor with strength 1 edge matrix:
2019-02-12 16:50:44.762746: I tensorflow/core/common_runtime/gpu/gpu_device.cc:988] 0
2019-02-12 16:50:44.766075: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1001] 0: N
2019-02-12 16:50:44.770290: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6387 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)

Processing C:\Users\madhu\Desktop\working\blitznet-master\demodemo\000456.jpg
Traceback (most recent call last):
File "demo.py", line 68, in
tf.app.run()
File "D:\Users\madhu\Anaconda3\envs\blitznet\lib\site-packages\tensorflow\python\platform\app.py", line 125, in run
_sys.exit(main(argv))
File "demo.py", line 63, in main
seg_gt=None, gt_bboxes=None, gt_cats=None)
File "C:\Users\madhu\Desktop\working\blitznet-master\detector.py", line 248, in feed_forward
name, draw=draw)
File "C:\Users\madhu\Desktop\working\blitznet-master\detector.py", line 213, in process_detection
self.draw(img, dets, cats, scores, name, gt_bboxes, gt_cats)
File "C:\Users\madhu\Desktop\working\blitznet-master\detector.py", line 85, in draw
args.eval_min_conf)), 'JPEG')
File "D:\Users\madhu\Anaconda3\envs\blitznet\lib\site-packages\PIL\Image.py", line 1991, in save
fp = builtins.open(filename, "w+b")
OSError: [Errno 22] Invalid argument: 'C:\Users\madhu\Desktop\working\blitznet-master\demodemo\output/C:\Users\madhu\Desktop\working\blitznet-master\demodemo\000456_det_50.jpg'

what is the problem here?

Seems that all the API requires always the GT cause also the test code works with datasets.
Do you have a minimal demo code to just visually evaluate results on new images from internet?

This network is perfect.
While using this network, i have some questions about some arguments in demo.py. In line 62 and 63 in demo.py
detector.feed_forward(img=image, name=name, w=w, h=h, draw=True,
seg_gt=None, gt_bboxes=None, gt_cats=None)
i don't get what seg_gt, gt_bboxes, gt_cats means. Could you explicitly explain to me about what these argument means and how it works? Thankyou

@dvornikita I have been measuring inference times by using python test.py --detect --segment ... For that I have enclosed row 238 in detector.py (outputs = self.sess.run(net_out, feed_dict=feed_dict)) in t0 = time.clock() / t_inf = time.clock() -t0 (I have seen your benchmark.py, however I am still getting errors when using it), for BlitzNet300 (Resnet50) on 980ti I am getting:

• python test.py --detect ... : ~45ms
• python test.py --segment ... : ~92ms
• python test.py --segment --detect ... : ~140ms

My questions:

1. Does that way of measuring inference time look ok? If I understand your code correctly the inference time measured that way does include NMS, right?
2. Inference time for object detection seems to be in line with your observation in https://arxiv.org/pdf/1708.02813.pdf (Table 7, 24fps on Titan X). Is the above inference time for segmentation and combined segmentation and detection in line with your observations, too?

Hi,
if you have the pre-trained model of ResNet101, can you please share a copy?
Best,

I train the network on Pascal VOC dataset.
The training progresses well at the beginning. But an error occurs after thousands of iterations.
If I restart training, the same error will occur again after thousands of iterations.
It seems that it is caused by the gradient or loss.
Could you please take a look and help me fix it up?
Thanks.

2017-10-10 11:53:49.895995: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.896694: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.897436: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.901151: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.901662: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.915369: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.915371: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.915423: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.915459: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] 2017-10-10 11:53:49.916251: W tensorflow/core/kernels/queue_base.cc:295] _3_shuffle_batch/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed 2017-10-10 11:53:49.916323: W tensorflow/core/kernels/queue_base.cc:295] _3_shuffle_batch/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed 2017-10-10 11:53:49.916366: W tensorflow/core/kernels/queue_base.cc:295] _0_parallel_read/filenames: Skipping cancelled enqueue attempt with queue not closed 2017-10-10 11:53:49.916420: W tensorflow/core/kernels/queue_base.cc:295] _2_parallel_read/common_queue: Skipping cancelled enqueue attempt with queue not closed 2017-10-10 11:53:49.916468: W tensorflow/core/kernels/queue_base.cc:295] _3_shuffle_batch/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed 2017-10-10 11:53:49.916490: W tensorflow/core/kernels/queue_base.cc:295] _3_shuffle_batch/random_shuffle_queue: Skipping cancelled enqueue attempt with queue not closed 2017-10-10 11:53:49.916524: W tensorflow/core/kernels/queue_base.cc:295] _2_parallel_read/common_queue: Skipping cancelled enqueue attempt with queue not closed Traceback (most recent call last): INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.CancelledError'>, Enqueue operation was cancelled [[Node: parallel_read/filenames/filenames_EnqueueMany = QueueEnqueueManyV2[Tcomponents=[DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](parallel_read/filenames, parallel_read/filenames/RandomShuffle)]] File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1327, in _do_call

[INFO]: Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.CancelledError'>, Enqueue operation was cancelled [[Node: parallel_read/filenames/filenames_EnqueueMany = QueueEnqueueManyV2[Tcomponents=[DT_STRING], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/cpu:0"](parallel_read/filenames, parallel_read/filenames/RandomShuffle)]] return fn(*args) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1306, in _run_fn status, run_metadata) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/contextlib.py", line 66, in __exit__ next(self.gen) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/errors_impl.py", line 466, in raise_exception_on_not_ok_status pywrap_tensorflow.TF_GetCode(status)) tensorflow.python.framework.errors_impl.InvalidArgumentError: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] [[Node: train_op/control_dependency/_879 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_18944 _train_op/control_dependency", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

During handling of the above exception, another exception occurred: Traceback (most recent call last): File "training_person_edge.py", line 364, in <module> tf.app.run() File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run _sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "training_person_edge.py", line 348, in main train(dataset, net, net_config) File "training_person_edge.py", line 292, in train train_loss, acc, iou, _, lr = sess.run([train_op, train_acc, mean_iou, update_mean_iou, learning_rate]) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 895, in run run_metadata_ptr) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1124, in _run feed_dict_tensor, options, run_metadata) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1321, in _do_run options, run_metadata) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/client/session.py", line 1340, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] [[Node: train_op/control_dependency/_879 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_18944 _train_op/control_dependency", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

Caused by op 'gradients/TopKV2_grad/Reshape', defined at: File "training_person_edge.py", line 364, in <module> tf.app.run() File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run _sys.exit(main(_sys.argv[:1] + flags_passthrough)) File "training_person_edge.py", line 348, in main train(dataset, net, net_config) File "training_person_edge.py", line 253, in train summarize_gradients=True) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 440, in create_train_op check_numerics=check_numerics) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/contrib/training/python/training/training.py", line 439, in create_train_op colocate_gradients_with_ops=colocate_gradients_with_ops) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/training/optimizer.py", line 386, in compute_gradients colocate_gradients_with_ops=colocate_gradients_with_ops) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py", line 542, in gradients grad_scope, op, func_call, lambda: grad_fn(op, *out_grads)) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py", line 348, in _MaybeCompile return grad_fn() # Exit early File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/gradients_impl.py", line 542, in <lambda> grad_scope, op, func_call, lambda: grad_fn(op, *out_grads)) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/nn_grad.py", line 707, in _TopKGrad ind_2d = array_ops.reshape(op.outputs[1], array_ops.stack([-1, ind_lastdim])) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2619, in reshape name=name) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op op_def=op_def) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op original_op=self._default_original_op, op_def=op_def) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__ self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

...which was originally created as op 'TopKV2', defined at: File "training_person_edge.py", line 364, in <module> tf.app.run() [elided 1 identical lines from previous traceback] File "training_person_edge.py", line 348, in main train(dataset, net, net_config) File "training_person_edge.py", line 219, in train seg_logits, seg_gt, edge_logits, edge_gt, dataset, config) File "training_person_edge.py", line 153, in objective detection_loss(location, confidence, refine_ph, classes_ph, pos_mask) File "training_person_edge.py", line 106, in detection_loss number_of_negatives) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/nn_ops.py", line 1949, in top_k return gen_nn_ops._top_kv2(input, k=k, sorted=sorted, name=name) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 2577, in _top_kv2 name=name) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op op_def=op_def) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 2630, in create_op original_op=self._default_original_op, op_def=op_def) File "/home/gongke/anaconda2/envs/py35/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 1204, in __init__ self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): Reshape cannot infer the missing input size for an empty tensor unless all specified input sizes are non-zero [[Node: gradients/TopKV2_grad/Reshape = Reshape[T=DT_INT32, Tshape=DT_INT32, _device="/job:localhost/replica:0/task:0/gpu:0"](TopKV2/_835, gradients/TopKV2_grad/stack)]] [[Node: train_op/control_dependency/_879 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/cpu:0", send_device="/job:localhost/replica:0/task:0/gpu:0", send_device_incarnation=1, tensor_name="edge_18944 _train_op/control_dependency", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/cpu:0"]()]]

I get [] , could you please help me ?

python demo.py --run_name=BlitzNet300_COCO+VOC07+12 --x4 --detect --segment --eval_min_conf=0.5 --ckpt=1

Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 AVX512F FMA
WARNING:tensorflow:From /home/cbl/PycharmProjects/blitznet-master/resnet.py:136: calling reduce_mean (from tensorflow.python.ops.math_ops) with keep_dims is deprecated and will be removed in a future version.
Instructions for updating:
keep_dims is deprecated, use keepdims instead

however,found nothing change in the demodemo/output

I am trying to retrain a model with dataset argument --dataset=voc07+12-segmentation

I keep getting this error

INFO]: Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.NotFoundError'>, /home/safy/Downloads/Datasets/voc07-trainval-segmentation; No such file or directory

Using the datasets.py script, I have try to create voc07-trainval-segmentation
by create_voc_dataset('07','trainval',segmentation=True)

but I get the assertion error

assert not self.segmentation or self.segmentation and self.year == '12'

As shown on the image the width and height dimension sizes are exchanged for segmentation.
We used VOC12 train and segmentation split dataset for training and VOC12 val and segmentation split dataset for test. Image size is default(300).

I have python 2.7 and got the following error when I ran:
python test.py --run_name=BlitzNet300_VOC07+12 --dataset=voc07 --split=test --image_size=300 --detect --segment --ckpt=1

Traceback (most recent call last):
File "test.py", line 111, in
tf.app.run()
File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 48, in run
_sys.exit(main(_sys.argv[:1] + flags_passthrough))
File "test.py", line 46, in main
loader = VOCLoader('07', 'test')
File "/mnt/2TB/src/blitznet/voc_loader.py", line 22, in init
super().init()
TypeError: super() takes at least 1 argument (0 given)

Thanks,

Hello.
When I trained the blitznet using resnet50, I met the followed error information:

Traceback (most recent call last):
  File "training.py", line 323, in <module>
    tf.app.run()
  File "/home/xum/anaconda/envs/py35/lib/python3.5/site-packages/tensorflow/python/platform/app.py", line 48, in run
    _sys.exit(main(_sys.argv[:1] + flags_passthrough))
  File "training.py", line 293, in main
    dataset = get_dataset('voc12-' + args.split)
  File "/home/xum/github/blitznet/datasets.py", line 144, in get_dataset
    num_samples=sum(splits_to_sizes[s] for s in files),
  File "/home/xum/github/blitznet/datasets.py", line 144, in <genexpr>
    num_samples=sum(splits_to_sizes[s] for s in files),
KeyError: 'voc12-train'

Then I checked the dataset file folder.

$ ls -l
total 3185444
-rw-r--r-- 1 xum g-xum        375 Aug 29 14:44 go.sh
-rw------- 1 xum g-xum    5387494 Aug 29 15:32 nohup.out
-rw-r--r-- 1 xum g-xum  166310709 Aug 30 11:31 voc12-train-segmentation
-rw-r--r-- 1 xum g-xum  166660879 Aug 30 11:32 voc12-val-segmentation
drwxr-xr-x 4 xum g-xum       4096 Nov  6  2007 VOCdevkit
-rw-r--r-- 1 xum g-xum  451020800 Nov  6  2007 VOCtest_06-Nov-2007.tar
-rw-r--r-- 1 xum g-xum  460032000 Nov  6  2007 VOCtrainval_06-Nov-2007.tar
-rw-r--r-- 1 xum g-xum 1999639040 May 11  2012 VOCtrainval_11-May-2012.tar

Both voc12-train-segmentation and voc12-val-segmentation should be produced by datasets.py, but why errors?

Here is my training command

python training.py --run_name=BlitzNet300_x4_VOC0712_detseg --dataset=voc12 --trunk=resnet50 --x4 --detect --segment

I ran the following command: python test.py --run_name=BlitzNet300_COCO --datasets=coco --split=val2014 --x4 --detect --ckpt=1 --save_first_n=100
The above command is run on the model you posted. But I found that all the final detection indicators are 0. Then I open and view the json file generated during the detection process. I found that the width and height of many boxes are less than 0 which is logically wrong.

I really hope to get your help.