hellochick / icnet-tensorflow

TensorFlow-based implementation of "ICNet for Real-Time Semantic Segmentation on High-Resolution Images".

Hi there 👋,

• 🔭 I am a PhD Student in Taiwan.
• 🚀 I am interested in computer vision (CV) and reinforcement learning (RL).
• ⚡ I work at Woven Planet, Japan as a research scientist!

Get in touch ☕

• Linkedin - Hsuan-Kung Yang
• Email - [email protected]

icnet-tensorflow's Issues

batch_size idea

Hi,

As of now, The time complexity for UHD images and 1080p images are as the following:

On UHD images, (e.g. images in a folder) the first one is slow (understandable) and the average execution time on the rest is around 0.71 sec. I have GeForce GTX 1080 and GeForce GTX 980. I disabled the memory growth flag though due to the 980 one.

The same experiment on the 1080p gives me 0.18 sec.
Do you expect these numbers?

Is there anyway to send the images not one by one to the network (as the current code status). Something like batch_size of 5-8 for each call in the loop. It might speed up the network in case is doable.
Let me know your thoughts.
Thanks

How to convert layer BN and interp?

There should be two layers, BN and Interp, which cannot be directly converted with the tool. Could you please provide some details for translating model from caffe to tensorflow?

Error arises from the size of own training data

Hello,
I tried to use own dataset to train the model, but it has some problems. The following are the steps I've done:

note: The dataset has 32 classes, and the image size is 360*480.

1. changed the "INPUT_SIZE" to 360*480 in train.py
2. changed the "NUM_CLASSES" to 32 in train.py
3. To use your pretrained model, I modified the name of "conv6_cls", "sub4_out" and "sub24_out" in class ICNet_BN, since they all related to num_classes. Then I set the "ignore_missing" to True in line 186 in train.py.
4. run the command "python train.py --update-mean-var --train-beta-gamma", but it show error "ValueError: Dimension 1 in both shapes must be equal, but are 44 and 45
   From merging shape 0 with other shapes. for 'sub12_sum' (op: 'AddN') with input shapes: [32,44,60,128], [32,45,60,128].
   "
5. change the "INPUT_SIZE" back to 480*480 (though it does not make sense)
6. again run the command "python train.py --update-mean-var --train-beta-gamma" and it successfully start to train.
7. try to use the checkpoint just trained to do inference, so I run the command "python inference.py --img-path=./input/test.png --model=others", but it has error (as I expected).

It seems that your model can train different size of image, and you also have updated the code enabling to inference in different size. I am still figuring out the code. I try to clearly explain what I thought. Any comments will be much appreciated. Really thanks your help in advance.

What's your output node of your model file ?help.

Hi, friends, I am a postgraduate at school. I am trying to transplant this model file to my android examples, I added the following code in your inference.py to save your ckpt model file
saver=tf.train.Saver() saver.save(sess,'checkpoint/ICNet.ckpt')
Then I adopt the 'CkptToPb.py' written by myself to transform the ckpt model to pb file.
But I need know your output node for your model, and I can not find something obviously about your output node, I need your help .Can you update your 'model.ckpt' to your git ? I really want to check it with mine to find what is wrong.
I am so appreciated your warm heart, thank you very much.
Thanks again sincerely.

Code licensing

Thanks a lot for this wonderful software. Just wondering what is the license for this code? Can it be used for commercial projects?

training model with other dataset

I used voc2012(21 categories including background) dataset for training without pretrained model,only change the 'NUM_CLASSES' to 21 in your training code,but after about 200 steps,the total loss can not be droped(about 0.5),the result is completely wrong. can you give some suggestion on training with other dataset ? tks

a bug in the network structure for ICNet_BN

Hi,

Very nice work.

Seems that there's a small bug in the network structure for ICNet_BN. In line 320 in your model.py, it should be 'conv3_2_1x1_increase_bn' instead of 'conv3_2_1x1_increase'
and the same for line 385

numpy.load problem

when i try to use your code,it reports that a byte is not in range 128 when using numpy.load to decode model file.I think it may be the python version accounting for this.could you tell me your py and tensorflow version?

Optimizing training

Hey,
during training of the network the loss alternates a lot and is not converging to a limit. How can I improve this? Do you have any suggestion on how to change learning rate and the optimizer or some other things I can try?

Thanks in advance

TypeError: __init__() missing 2 required positional arguments: 'img_mean' and 'coord'

I was trying to run evaluate.py with cityscapes dataset in the path as below in Windows 10 (Git Bash)
using Python 3.5.2 and tensorflow-gpu 1.3.0

I have modified data directory path with my local path as below path

DATA_DIRECTORY = './data/cityscapes_dataset/cityscape'
(where I have copied cityscapes contents, leftImg8bit and gtFine folders)
changed the path of SAVE_DIR = './output_new/'

But I could not excecute the evaluation script due to following error

npv0@LAPTOP-8FO36PI9 MINGW64 /e/Projects/DeepLearning/tf/ICNet-tensorflow (master)
$ python evaluate.py --model=train --measure-time
Namespace(flipped_eval=False, measure_time=True, model='train', save_dir='./output_new/')
Traceback (most recent call last):
File "evaluate.py", line 153, in
main()
File "evaluate.py", line 82, in main
coord)
TypeError: init() missing 2 required positional arguments: 'img_mean' and 'coord'

I searched about the issue and it could be related to missing number of arguments to ImageReader

Even if at the line 72 has coord = tf.train.Coordinator() , I was getting the above error.

The same is observed in Python 2.7 in windows and Python 3.5 in Ubuntu
Please have a look at this issue. Thanks

picture path in ade20k dataset seems not right

when I put this command
python evaluate.py --dataset=cityscapes --filter-scale=2 --model=others
ValueError: Failed to find file: /home/media/data/ade20k/images/validation/ADE_val_00000001.jpg
but in fact the path is
(in folder "validation", there are several folders to arrange pictures in alphabetical order )

[media@localhost validation]$ find . -name 'ADE_val_00000001.jpg'
./a/abbey/ADE_val_00000001.jpg

I download it on http://groups.csail.mit.edu/vision/datasets/ADE20K/ ,the full dataset(ADE20K_2016_07_26.zip)

After I copy all pictures in one folder , it shows

ValueError: Failed to find file: /home/media/data/ade20k/annotations/validation/ADE_val_00000001.png

but this png file doesn't exist

Is this a bug or I did something wrong?

about the pretrain model "90k" and "90k_bn"

I have evalute the miou of these two pre train model.
The results are as follows:
90k: 78.56%
90k_bn: 70.34%
Where does the pretrain model "90k_bn" come from?
Can I fune-tuning the "90k" model?
Thanks a lot for your implementation, and looking forword for your reply~~:)

image size for training

Hey,
what image size do I need for training the model to get the best inference results on the cityscapes test images? When I use INPUT_SIZE = '1024,2048' I can only do a BATCH_SIZE = 8 to get the data into GPU memory.

Is it better to train with smaller images and than just upscale?

thanks in advance

about THE NUM_CLASSES

The num_classes in the train.py equals 27.
But the cityscapes dataset has 19 classes in the trainIDs?
I feel a little confused about this...?

Did you train the model on cityscapes? or you load the pretrained model provided by the author?
Can I change the num_class to 19 and still ues the parameter you offered？

Running interface.py "No checkpoint file."

I downloaded from the google drive link ade20K includes 5 files one of them is checkpoint.txt
I dropped them on ./model

Then:
python inference.py --img-path=./input/outdoor_1.png --dataset=ade20k --filter-scale=2 --model=others

It says:
030] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 980, pci bus id: 0000:82:00.0)
No checkpoint file found.

do we have to change anything on interface.py script?

Also, I installed tensorflow-gpu 1.2.0 due to my CUDA library 8.0

Training on cityscapes dataset

Hi,

I tried to run train.py to train the model from stretch on cityscapes dataset, and only commented the line that restores from pre-trained model and modified the path of dataset. My program managed to reach loss of 1 in the first 100 steps, however the loss continued to fluctuate around 1 until 5000 steps. I am wondering if there's anything that I should further modify, any ideas?

Thanks!

JPG image size error

I try to inference my own .jpg images with different sizes (e.g. 312492 264385 1435*832), i got invalidargument error like this:
(inputs to operation sub12_sum of type AddN must have the same size and shape. input 0: [1，128，256，128]!=input 1: [1，49，33，128])

it seems that the code did have preprocess to resize the input image size to 1024*2048，so i got confused.

image size problems

1. image size problems in training：
   When the image size in my own dataset is small (e.g. h=200, w=80), when I train the model, how should I set the INPUT_SIZE?
   INPUT_SIZE='720,720'
   or
   INPUT_SIZE='480,480'
   or it should be:
   INPUT_SIZE='200, 80' ?

2. image size problems for inference：
   When I run inference.py with a smaller size input image (e.g. h=212, w=87),
   the image is padding to 224,96, then some error came up:
   something like "stride must be >0 got 0 for conv5_3_pool6 ..."
   However, if I try inference image with bigger size (e.g.360,480), everything works well.
   So, is there still any limits for the input image size? can not support any arbitrary image size?

About the ignore_label

In the default hyper-parameters in train.py, the ignore_label is set to 0.
Wondering if it should be 255 when training on cityscape-dataset?

names of segmented objects in the image not coming up

How to retrieve the names of segmented objects and their area occupied in the images?

what's the label image format in training？

Failed to find any matching files for ./snapshots/model.ckpt-0

hi @hellochick awesome work! thanks for sharing it.
when i trying to train the model with

python train.py --update-mean-var --train-beta-gamma

i got following error

...
2018-02-09 14:43:25.867146: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./snapshots/model.ckpt-0
2018-02-09 14:43:25.867175: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./snapshots/model.ckpt-0
2018-02-09 14:43:25.867233: W tensorflow/core/framework/op_kernel.cc:1192] Not found: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./snapshots/model.ckpt-0
Traceback (most recent call last):
  File "train.py", line 213, in <module>
    main()
  File "train.py", line 188, in main
    load(loader, sess, ckpt.model_checkpoint_path)
  File "train.py", line 95, in load
    saver.restore(sess, ckpt_path)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1666, in restore
    {self.saver_def.filename_tensor_name: save_path})
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 889, in run
    run_metadata_ptr)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1120, in _run
    feed_dict_tensor, options, run_metadata)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1317, in _do_run
    options, run_metadata)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1336, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.NotFoundError: Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./snapshots/model.ckpt-0
	 [[Node: save_1/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2_9/tensor_names, save_1/RestoreV2_9/shape_and_slices)]]
	 [[Node: save_1/RestoreV2_97/_69 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_714_save_1/RestoreV2_97", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

Caused by op u'save_1/RestoreV2_9', defined at:
  File "train.py", line 213, in <module>
    main()
  File "train.py", line 186, in main
    loader = tf.train.Saver(var_list=restore_var)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1218, in __init__
    self.build()
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1227, in build
    self._build(self._filename, build_save=True, build_restore=True)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 1263, in _build
    build_save=build_save, build_restore=build_restore)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 751, in _build_internal
    restore_sequentially, reshape)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 427, in _AddRestoreOps
    tensors = self.restore_op(filename_tensor, saveable, preferred_shard)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/training/saver.py", line 267, in restore_op
    [spec.tensor.dtype])[0])
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_io_ops.py", line 1021, in restore_v2
    shape_and_slices=shape_and_slices, dtypes=dtypes, name=name)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)
  File "/home/k/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

NotFoundError (see above for traceback): Unsuccessful TensorSliceReader constructor: Failed to find any matching files for ./snapshots/model.ckpt-0
	 [[Node: save_1/RestoreV2_9 = RestoreV2[dtypes=[DT_FLOAT], _device="/job:localhost/replica:0/task:0/device:CPU:0"](_arg_save_1/Const_0_0, save_1/RestoreV2_9/tensor_names, save_1/RestoreV2_9/shape_and_slices)]]
	 [[Node: save_1/RestoreV2_97/_69 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device_incarnation=1, tensor_name="edge_714_save_1/RestoreV2_97", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]

is there any where that i can download the checkpoint file? thanks~!

indoor model

I want to test indoor scenes semantic segmentation,and do some research about merging.I tried to train on sun rgb-d but can't converge,would you be willing to provide your own data model?I would be very thankful.

a ValueError raised when training

Hi，I'm a newcomer on TensorFlow and Machine Learning. When I run the file "train.py", I encountered a problem, and I have no idea how to solve it. Could you help me on this problem ?Thanks

ValueError: Dimension 3 in both shapes must be equal, but are 27 and 19 for 'conv6_cls_1/Assign' (op: 'Assign') with input shapes: [1,1,128,27], [1,1,128,19].

train.py: error: unrecognized arguments: --model=others

why i can't find the parameters about the '--model'?

Different eval metrics for model with/without bn

Hi,

When I evaluate on cityscapes using train_30k and train_30k_bnnomerge models, I am getting different mIOU of 65.6% and 59.3% respectively. As per my understanding, they should ideally give the same results. Am I missing something?

Thanks

question about the pretrained model

The dataset has 19 classes, plus the background, the output of last layer should be 20 classes. But the pre-trained model produces a 19 channels output. Do the background just be ignored? But in the train code, the ignored-label is set to 0.
I am a little confused, please help!

train the model

Hello,

I am interested in your work and would like to try using your code to train my dataset. Would you provide the training code and give me some suggestions about this?

Thank you so much.

mIoU improves over validation set

Hey, I run your evaluation script and plotted the mIoU of every image over the whole validation dataset. The metric result is improving after every image. Why is it like this? In my opinion the result should alternate around some value. I saw that this mean IoU metric is updating the conf_matrix (sum up all pixel values) every step but this shouldn't affect the mean value?

Thanks for your help.

about evaluate.py

Hello,
Thank you for your great job .I meet a question when I use the evaluate code.Please give me a hand.
At first,I use your train code trained my own dataset with size 512X512X3 and save a model,all things seems to be right so far.But when I use the evaluate code with the saved model,I found that the mIOU result is about 0.004, all the images are predit to label[0].(I set n_classes =5 and label[0] is blackground in my code)
In evaluate ,When I set net = ICNet_BN({'data': image_batch}, is_training=True, num_classes=args.num_classes),the result will be a little bettle.

Train with different NUM_CLASSES

Hi , I want train ICNet with different NUM_CLASSES (eg NUM_CLASSES = 12).So I changed all the num_classes=19 to 12 in *.py and num_output in *.prototxt. But when I run python train.py I get error:
Dimension 3 in both shapes must be equal, but are 17 and 19. Shapes are [1,1,128,12] and [1,1,128,19]. for 'conv6_cls_1/Assign'
So , can you give me some advice on how to traing ICNet with different class num? And can I use icnet_cityscapes_trainval_90k.npy as pretrained model on this situation?

ValueError: Dimension 3 in both shapes must be equal, but are 27 and 19 for 'conv6_cls_1/Assign' (op: 'Assign') with input shapes: [1,1,128,27], [1,1,128,19].

I was trying to run inference.py with example images

I am trying to run inference.py with example images, I got tensorflow ValueError like this

Example run 1, model=train

user@instance-2-cudnnv5:~/exp/tf_codes_new/ICNet-tensorflow$ python inference.py --img-path=input/IMG_0416_640x480.png --model=train
successful load img: input/IMG_0416_640x480.png
input image shape: (480, 640, 3)
WARNING:tensorflow:From inference.py:119: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.
Instructions for updating:
Use the axis argument instead
2017-11-18 18:19:04.969994: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Restore from train30k model...
Traceback (most recent call last):
File "inference.py", line 160, in
main()
File "inference.py", line 134, in main
net.load(model_train30k, sess)
File "/home/user/exp/tf_codes_new/ICNet-tensorflow/network.py", line 77, in load
session.run(var.assign(data))
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/ops/variables.py", line 573, in assign
return state_ops.assign(self._variable, value, use_locking=use_locking)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/ops/state_ops.py", line 276, in assign
validate_shape=validate_shape)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/ops/gen_state_ops.py", line 57, in assign
use_locking=use_locking, name=name)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
op_def=op_def)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
set_shapes_for_outputs(ret)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
shapes = shape_func(op)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
return call_cpp_shape_fn(op, require_shape_fn=True)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
require_shape_fn)
File "/home/user/.local/lib/python2.7/site-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
raise ValueError(err.message)
ValueError: Dimension 3 in both shapes must be equal, but are 27 and 19 for 'conv6_cls_1/Assign' (op: 'Assign') with input shapes: [1,1,128,27], [1,1,128,19].

Example run 2, model=trainval_bn

user@instance-2-cudnnv5:~/exp/tf_codes_new/ICNet-tensorflow$ python inference.py --img-path=input/indoor3.jpg --model=trainval_bn
successful load img: input/indoor3.jpg
input image shape: (360, 640, 3)
Image shape cannot divided by 32, padding to (384, 672)
WARNING:tensorflow:From inference.py:119: calling argmax (from tensorflow.python.ops.math_ops) with dimension is deprecated and will be removed in a future version.
Instructions for updating:
Use the axis argument instead
2017-11-18 18:06:18.996643: I tensorflow/core/platform/cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
Restore from trainval90k bnnomerge model...
Traceback (most recent call last):
File "inference.py", line 160, in
main()
File "inference.py", line 143, in main
net.load(model_trainval90k_bn, sess)
File "/home/user/exp/tf_codes_new/ICNet-tensorflow/network.py", line 68, in load
data_dict = np.load(data_path).item()
File "/home/user/.local/lib/python2.7/site-packages/numpy/lib/npyio.py", line 419, in load
pickle_kwargs=pickle_kwargs)
File "/home/user/.local/lib/python2.7/site-packages/numpy/lib/format.py", line 640, in read_array
array = pickle.load(fp, **pickle_kwargs)
cPickle.UnpicklingError: invalid load key, '4'.

Tensorflow version 1.4.0
Python 2.7.12

I'm hoping you will help me to resolve this issue. Thanks

README instructions not working for training on my own dataset

Hi,
I tried to follow README instructions for training on my own dataset but it didn't work. Here is what I did:

• Update DATA_DIR to point to dataset dir
• Update DATA_LIST_PATH to point to train dataset list file.
• Update INPUT_SIZE to '1280, 720'
• Update NUM_CLASSES to 1
• Update LAMBDA1 and LAMBDA2 to 0.4 and 0.6 respectively.

Then ran cmd

python train.py --update-mean-var --train-beta-gamma

Then got this error (shortened)

ValueError: Dimension 3 in both shapes must be equal, but are 1 and 19. Shapes are [1,1,128,1] and [1,1,128,19]. for 'conv6_cls_1/Assign' (op: 'Assign') with input shapes: [1,1,128,1], [1,1,128,19].

Troubleshooting (none of that worked):

• Tried to follow advise from #20 by doing the following
• Updating icnet_cityscapes_bnnomerge.prototxt by changing conv6_cls num_output from 19 to 1
• Then replaced this line in train.py

 restore_var = tf.global_variables()

with

restore_var = [v for v in tf.global_variables() if 'conv6_cls' not in v.name]

Then I got same exact error mentioned above.

If anyone was able to train on their own dataset (either using pretrained model or from scratch) please provide steps of changes you did.

Thanks

inference times too large

hi

I am getting an inference time of around .8s on GTX 1070 (8GB). The image sizes are 1080x1920. Cuda version is 9.0. What may I be doing wrong?

About the evaluation result

Hello,
First I want to thank you for your great job. I have some problems when using evaluate.py. As you mentioned on README, I downloaded the Cityscape dataset. However, the annotation file name is end with "labelIds.png" instead of "labelTrainIds.png". Thus I just generated a new eval_list.txt based on yours. The final evaluation mIoU is only about 0.006. I am not quite sure where is the mistake. Do I miss something? Any comments will be much appreciated.

TypeError: interp() got an unexpected keyword argument 'factor'

Hi, when trying to run train.py from scratch i have gotten the error in the title.
as i'm unfamiliar with the model I have no idea why the code tries to put factor into a function that only has z_factor and s_factor as possible arguments, Im not sure if the issue is with my understanding or some mistake in an update of the code?

also, im trying to re-train the model to infer face vs background on the LFW segmentation dataset. I have re-created the masks from the data set to have 1 where faces are located and zeros for background. any Tips on how to train the model on said dataset?

About the pad at the max_pool layer?

The original ICNet has a pad at the max_pool layer, but this code does not have, why? @hellochick

Have issue how to train my own dataset

Hi, Happy new year and thanks for you share.

Recently I do some research work on real-time semantic segmentation and I have to train the model on my own indoor dataset . I am very appreciated if you can share code that can train my own dataset.

Inference Time is too high

Hi, I have directly taken your code and just modified the path to the datasets, and then ran evaluate.py to measure the inference time. I am getting an inference time of 4sec(as opposed to ~0.04sec) on Tesla X GPU. Could you please point out on what could be the reason on why this could be happening.

Thanks,
Sudhir

Results of training with cityscapes

As far as I understood the inference results of the ICNet tensorflow model are from the already trained weights of the original paper. Did someone tried to reach the results of the original model with the provided trainingsscript? I tried different configurations but my scores are never higher than 60% on the cityscape val set (compared to the original model it should be 67.67%). I test different parameters, filter scales and tried it with and without batch normalization. I just wonder if this score of 67,67% is reachable?

Own dataset training produces low loss, but unsatisfying results

Hi.

I got training working and used icnet_cityscapes_trainval_90k.npy with --filter-scale=1
The loss reduces quite quickly to 0.08 in ~2000 epochs which I presumed was quite good. But when doing evaluate or inference the output is garbage as seen on the example.

I had to change nr of classes, perhaps that could be the error or I have labaled them incorrectly?

Currently my image is shape (480, 870, 3) and gt shape is (480, 870) and gt values are in [2,5,8,10, etc] belonging to corresponding cityscapes classes.

My own one theory would be that the pretrained model is slowly transitionig to my classes and that is why the output is mixed, but that would not explain the constant low loss.

Do you have some other ideas perhaps?

Regards,
Tamme

How to generate information like color150.mat?

Hi,
I want generate all names of 150 classes, object names of each class, and number of pictures which contain this class. How can I do that? or how to generate information like color150.mat? (I didnt find these information in the official provided tools of ADE20K.)
Thanks in advance!

label indices mismatch between evaluation and gt data

Hello~,

I recently downloaded the cityscape dataset from the official website.

and I tried to run evaluation.py code with cityscape eval & gt data.

Fortunately, I produce the the prediction result from the evaluation code using your trained model provided on GoogleDrive.

But Badly, I found the label indices generated from evaluation and those from ground-truth are mismatched. For example, the index for class person is 11 in your dataset, but it is 24 in cityscape dataset. so the final mIOU is extremely low like 0.00XX...

I guess, you trained the model with old version of cityscape dataset or something different from mine.

Am I right?? could you confirm my opinion??

Grayscale images

I want to use a grayscale images for semantic segmentation, but get very bad results. I think this is due to lack of such images in train datasets. What can you advise? Thanks.

how is the accuracy calculated?

Hi,

Thanks so much for sharing the code. I just have a question about the accuracy (65.56/67.7) in the readme. Is the accuracy calculated as mIOU? Did you get this number from the model weights provided by the author or from the weights trained by yourself on the cityscapes data?

AttributeError: 'dict' object has no attribute 'iteritems'

When I try to run python inference.py --model=train --img-path=./input/IMG_0416_640x480.png, I get the following output:

Restore from train30k model...
Traceback (most recent call last):
  File "inference.py", line 158, in <module>
    main()
  File "inference.py", line 134, in main
    net.load(model_train30k, sess)
  File "/Users/user/workspace/cv/ICNet-tensorflow/network.py", line 71, in load
    for param_name, data in data_dict[op_name].iteritems():
AttributeError: 'dict' object has no attribute 'iteritems'

Any ideas why this might happen?

I'm running this on OSX 10.13, w/ TF 1.4.1, python 3.6

Meaning behind --update-mean-var --train-beta-gamma

Hi.

I havent encountered in other projects this kind of updating values. Is it something originating from PSPNet or this really is the way to use momentum or is it something third?

Thanks,
Tamme

Increase in accuracy after fixing #41 ?

Hi @hellochick. After fixing the bug referenced in #41 , did you get a bump in accuracy too? The Trainval_bn model gives me around 80% class IoU. However I am not sure if there is some mistake.

ImportError: cannot import name '_ellipsoid'

I get this error. What is the version of scipy?

Traceback (most recent call last):
  File "inference.py", line 10, in <module>
    from scipy import misc
  File "/home/media/ruodai/code/anaconda/lib/python3.6/site-packages/scipy/misc/__init__.py", line 67, in <module>
    from scipy.interpolate._pade import pade as _pade
  File "/home/media/ruodai/code/anaconda/lib/python3.6/site-packages/scipy/interpolate/__init__.py", line 175, in <module>
    from .interpolate import *
  File "/home/media/ruodai/code/anaconda/lib/python3.6/site-packages/scipy/interpolate/interpolate.py", line 21, in <module>
    import scipy.special as spec
  File "/home/media/ruodai/code/anaconda/lib/python3.6/site-packages/scipy/special/__init__.py", line 648, in <module>
    from ._ellip_harm import ellip_harm, ellip_harm_2, ellip_normal
  File "/home/media/ruodai/code/anaconda/lib/python3.6/site-packages/scipy/special/_ellip_harm.py", line 7, in <module>
    from ._ellip_harm_2 import _ellipsoid, _ellipsoid_norm
ImportError: cannot import name '_ellipsoid'

class number issues

in train.py, for CITYSCAPES_DATASET, the NUM_CLASSES is set 19 (in line 29).
from line 57 we can see
parser.add_argument("--num-classes", type=int, default=NUM_CLASSES,
help="Number of classes to predict (including background).")
So it seems that the --num-classes should "including background".

However, from tools.py we can see that the label_colours (in line 6) includes 19 classes, without background class. it means if we including background class, the NUM_CLASSES should be 20.

So, any explanations about that? or I miss something?