How to use your own dataset: ValueError: Node 'generator_1/latent_vector/Pad' has an _output_shapes attribute inconsistent with the GraphDef for output #0: about twingan

[jerryli27]

I'm not completely sure, but I believe the problem in the image size you used for training is different from what you used for inference. The error was complaining that the padding size is different from the network created in training.

Notice the --train_image_size=4 flag you used for training and the default tf.flags.DEFINE_integer('image_hw', 256, 'height and width of the input image.') during inference. I think that might be the cause. Try changing the two to be the same and let me know if it works!

If that doesn't work please copy/paste the exact command you used. Thanks.

[itsss]

@jerryli27
I changed --train_image_size to 256, it doesn't work...
python image_generation.py --dataset_name="image_only" --dataset_dir="./test/train-B" --dataset_use_target=True --train_dir="./checkpoints/test/" --dataset_split_name=train --learning_rate=0.0001 --learning_rate_decay_type=fixed --is_training=True --train_image_size=256 --preprocessing_name="danbooru" --generator_network="pggan" --ignore_missing_vars=True --max_number_of_steps=1000

(P.S. --train_image_size=4 is working well)

2018-07-25 14:43:02.680615: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
INFO:tensorflow:Running local_init_op.
INFO:tensorflow:Done running local_init_op.
INFO:tensorflow:Starting Session.
INFO:tensorflow:Saving checkpoint to path ./checkpoints/test/model.ckpt
INFO:tensorflow:Starting Queues.
INFO:tensorflow:global_step/sec: 0
INFO:tensorflow:Error reported to Coordinator: <type 'exceptions.AttributeError'>, 'NoneType' object has no attribute 'endswith'
INFO:tensorflow:Recording summary at step 0.
Traceback (most recent call last):
  File "image_generation.py", line 1063, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "image_generation.py", line 1059, in main
    model.main()
  File "image_generation.py", line 1054, in main
    super(GanModel, self).main()
  File "/home/itsc/Desktop/TwinGAN/model/model_inheritor.py", line 1127, in main
    session_config=session_config)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/slim/python/slim/learning.py", line 770, in train
    sess, train_op, global_step, train_step_kwargs)
  File "/home/itsc/Desktop/TwinGAN/model/model_inheritor.py", line 1107, in train_step_fn
    self.do_extra_train_step(session, end_points, global_step)
  File "image_generation.py", line 765, in do_extra_train_step
    GanModel._calc_swd(session, end_points, current_step=current_step)
  File "image_generation.py", line 878, in _calc_swd
    util_io.touch_folder(FLAGS.eval_dir)
  File "/home/itsc/Desktop/TwinGAN/util_io.py", line 36, in touch_folder
    if not file_path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

[itsss]

@jerryli27 image_translation_infer.py also not working.
when i use my current checkpoints, (--train_image_size=4 checkpoints)

tf.flags.DEFINE_string("model_path", "checkpoints/test", "Path containing a checkpoint.")
tf.flags.DEFINE_integer('image_hw', 4, 'height and width of the input image.')
tf.flags.DEFINE_string('input_tensor_name', "custom_input_ph", 'Optional input tensor name. e.g. sources_ph.')
tf.flags.DEFINE_string('output_tensor_name', "custom_generated_targets:0", 'e.g. custom_generated_t_style_source:0')

Model directory: checkpoints/test
Metagraph file: model.ckpt-50000.meta
Checkpoint file: model.ckpt-50000
Traceback (most recent call last):
  File "image_translation_infer.py", line 95, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "image_translation_infer.py", line 81, in main
    inferer = ImageInferer()
  File "image_translation_infer.py", line 33, in __init__
    util_io.load_model(FLAGS.model_path, input_map=input_map)
  File "/home/itsc/Desktop/TwinGAN-master/util_io.py", line 168, in load_model
    saver = tf.train.import_meta_graph(os.path.join(model_exp, meta_file), input_map=input_map)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/saver.py", line 1960, in import_meta_graph
    **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/meta_graph.py", line 744, in import_scoped_meta_graph
    producer_op_list=producer_op_list)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/util/deprecation.py", line 432, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/importer.py", line 422, in import_graph_def
    raise ValueError(str(e))
ValueError: Node 'generator_1/latent_vector/Pad' has an _output_shapes attribute inconsistent with the GraphDef for output #0: Dimension 1 in both shapes must be equal, but are 10 and 7. Shapes are [1,10,10,?] and [?,7,7,256].

ValueError: Node 'generator_1/latent_vector/Pad' has an _output_shapes attribute inconsistent with the GraphDef for output #0: Dimension 1 in both shapes must be equal, but are 10 and 7. Shapes are [1,10,10,?] and [?,7,7,256].

[jerryli27]

You have a misunderstanding of what is PGGAN vs TwinGAN. PGGAN is not an image translation model so it can't translate human faces. Try to use TwinGAN following the training guide.

Also on a side note, I tried to use Danbooru datasets as well. The quality was quite poor because there are much more variation in that dataset. Not all faces are forward facing etc.

[itsss]

@jerryli27
Yes. I think I'm misunderstood about PGGAN and TwinGAN.
Now, I need to know how to training TwinGAN, so I visited the training guide page, but it makes error like this.

WARNING:tensorflow:IOError [Errno 2] No such file or directory: './data/celeba/sample_source_validation_100.npy'. Not outputting custom generated images.
INFO:tensorflow:global_step/sec: 0

so I understand i need to convert the celeba dataset. I used this command to run convert_celeba.py

python datasets/convert_celeba.py --partition_fn /home/itsc/Desktop/list_eval_partition.txt --fn_root /home/itsc/Desktop/img_align_celeba/

• list_eval_partition.txt is a text file that i downloaded in CelebA official site.
• img_align_celeba is a jpg file that i downloaded in CelebA official site.

/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
2018-07-26 10:48:14.370525: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Saving results to None
Traceback (most recent call last):
  File "datasets/convert_celeba.py", line 99, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "datasets/convert_celeba.py", line 95, in main
    converter.main()
  File "datasets/convert_celeba.py", line 85, in main
    util_io.touch_folder(FLAGS.output_directory)
  File "/home/itsc/Desktop/TwinGAN/util_io.py", line 18, in touch_folder
    if not file_path.endswith('/'):
AttributeError: 'NoneType' object has no attribute 'endswith'

How to solve AttributeError: 'NoneType' object has no attribute 'endswith'?
please answer to my question!

[itsss]

@jerryli27 I want to make Celeba dataset like Getchu preprocessed dataset.

CelebA dataset formating.
Adapted from https://github.com/tensorflow/models/blob/master/research/real_nvp/celeba_formatting.py .
Download img_align_celeba.zip from
http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html under the
link "Align&Cropped Images" in the "Img" directory and list_eval_partition.txt
under the link "Train/Val/Test Partitions" in the "Eval" directory. Then do:
unzip img_align_celeba.zip
Use the script as follow:
python celeba_formatting.py \
    --partition_fn [PARTITION_FILE_PATH] \
    --output_directory [OUTPUT_FILE_PATH_PREFIX] \
    --fn_root [CELEBA_FOLDER]

I proceeded through this process and i get three .tfrecord file. (train / test / validation)
So, what should I do next? how can i make celebA dataset???

[jerryli27]

WARNING:tensorflow:IOError [Errno 2] No such file or directory: './data/celeba/sample_source_validation_100.npy'. Not outputting custom generated images. INFO:tensorflow:global_step/sec: 0

The first one you pointed out was a warning, not an error. It does not affect the code. It simply does not output the image translation of a fixed set of images. (Which is useful to see how training progresses, but is totally optional).

The second one I think you figured out already. When you train, just point your dataset_dir flag at the directory containing the tfrecord files. The tfrecord files contains the celeba dataset. You can google what is tfrecord and why it is used.

--dataset_name="celeba" --dataset_dir="./data/celeba/or/wherever/your/dataset/is"

[jerryli27]

If that gives you an error, try to set --dataset_name=image_only, which only reads the encoded image in the dataset and nothing else.

[itsss]

@jerryli27 I executed the command like this.

python pggan_runner.py --program_name=twingan --dataset_name="celeba" --dataset_dir="./test/" --unpaired_target_dataset_name="anime_faces" --unpaired_target_dataset_dir="./test/" --train_dir="./checkpoints/pggan/" --dataset_split_name=train --preprocessing_name="danbooru" --resize_mode=RANDOM_CROP --do_random_cropping=True --learning_rate=0.0001 --learning_rate_decay_type=fixed --is_training=True --generator_network="pggan" --num_images_per_resolution=50000 --loss_architecture=dragan --pggan_max_num_channels=256 --generator_norm_type=batch_renorm --use_ttur=True

python pggan_runner.py --program_name=twingan --dataset_name=image_only --dataset_dir="./test/" --unpaired_target_dataset_name="anime_faces" --unpaired_target_dataset_dir="./test/" --train_dir="./checkpoints/pggan/" --dataset_split_name=train --preprocessing_name="danbooru" --resize_mode=RANDOM_CROP --do_random_cropping=True --learning_rate=0.0001 --learning_rate_decay_type=fixed --is_training=True --generator_network="pggan" --num_images_per_resolution=50000 --loss_architecture=dragan --pggan_max_num_channels=256 --generator_norm_type=batch_renorm --use_ttur=True

python pggan_runner.py --program_name=twingan --dataset_name="image_only" --dataset_dir="./test/" --unpaired_target_dataset_name="anime_faces" --unpaired_target_dataset_dir="./test/" --train_dir="./checkpoints/pggan/" --dataset_split_name=train --preprocessing_name="danbooru" --resize_mode=RANDOM_CROP --do_random_cropping=True --learning_rate=0.0001 --learning_rate_decay_type=fixed --is_training=True --generator_network="pggan" --num_images_per_resolution=50000 --loss_architecture=dragan --pggan_max_num_channels=256 --generator_norm_type=batch_renorm --use_ttur=True

(Three Command Makes same error...)

INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.FailedPreconditionError'>, test/train-B; Is a directory

OutOfRangeError (see above for traceback): FIFOQueue '_5_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
[[Node: fifo_queue_Dequeue = QueueDequeueV2component_types=[DT_STRING, DT_STRING, DT_STRING, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]

[Folder Structure]
./test/train-A : result.tflearn, result1.tflearn, result2.tflearn
./test/train-B : Getchu Preprocessed Dataset (train-00000-of-00008, vaildation-00001-of-00002, etc...)

INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.FailedPreconditionError'>, test/train-B; Is a directory
         [[Node: parallel_read/ReaderReadV2_2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/device:CPU:0"](parallel_read/TFRecordReaderV2_2, parallel_read/filenames)]]
WARNING:tensorflow:IOError [Errno 2] No such file or directory: './test/sample_source_validation_100.npy'. Not outputting custom generated images.
2018-07-26 15:46:47.800863: W tensorflow/core/kernels/queue_base.cc:285] _4_parallel_read/common_queue: Skipping cancelled dequeue attempt with queue not closed
2018-07-26 15:46:47.801005: W tensorflow/core/kernels/queue_base.cc:285] _5_prefetch_queue/fifo_queue: Skipping cancelled dequeue attempt with queue not closed
2018-07-26 15:46:47.801261: W tensorflow/core/kernels/queue_base.cc:285] _4_parallel_read/common_queue: Skipping cancelled dequeue attempt with queue not closed
2018-07-26 15:46:47.801409: W tensorflow/core/kernels/queue_base.cc:285] _4_parallel_read/common_queue: Skipping cancelled dequeue attempt with queue not closed
2018-07-26 15:46:47.801436: W tensorflow/core/kernels/queue_base.cc:285] _4_parallel_read/common_queue: Skipping cancelled dequeue attempt with queue not closed
2018-07-26 15:46:47.801506: W tensorflow/core/kernels/queue_base.cc:285] _4_parallel_read/common_queue: Skipping cancelled dequeue attempt with queue not closed
INFO:tensorflow:Caught OutOfRangeError. Stopping Training. FIFOQueue '_5_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_STRING, DT_STRING, DT_STRING, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prefetch_queue/fifo_queue)]]

Caused by op u'fifo_queue_Dequeue', defined at:
  File "pggan_runner.py", line 164, in <module>
    tf.app.run()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "pggan_runner.py", line 154, in main
    model.main()
  File "/data/TwinGAN/twingan.py", line 894, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/image_generation.py", line 1054, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/model/model_inheritor.py", line 1044, in main
    'is_training': FLAGS.is_training})
  File "/data/TwinGAN/deployment/model_deploy.py", line 237, in create_clones
    outputs = model_fn(*args, **kwargs)
  File "/data/TwinGAN/twingan.py", line 150, in _clone_fn
    data_batched = super(GanModel, GanModel)._get_data_batched(batch_queue, batch_names, data_batched)
  File "/data/TwinGAN/model/model_inheritor.py", line 433, in _get_data_batched
    dequeued = batch_queue.dequeue()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 440, in dequeue
    self._queue_ref, self._dtypes, name=name)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3734, in queue_dequeue_v2
    timeout_ms=timeout_ms, name=name)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

OutOfRangeError (see above for traceback): FIFOQueue '_5_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_STRING, DT_STRING, DT_STRING, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prefetch_queue/fifo_queue)]]

INFO:tensorflow:Finished training! Saving model to disk.
Traceback (most recent call last):
  File "pggan_runner.py", line 164, in <module>
    tf.app.run()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "pggan_runner.py", line 154, in main
    model.main()
  File "/data/TwinGAN/twingan.py", line 894, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/image_generation.py", line 1054, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/model/model_inheritor.py", line 1127, in main
    session_config=session_config)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 784, in train
    ignore_live_threads=ignore_live_threads)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 828, in stop
    ignore_live_threads=ignore_live_threads)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
    six.reraise(*self._exc_info_to_raise)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/queue_runner_impl.py", line 252, in _run
    enqueue_callable()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1244, in _single_operation_run
    self._call_tf_sessionrun(None, {}, [], target_list, None)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.FailedPreconditionError: test/train-B; Is a directory
         [[Node: parallel_read/ReaderReadV2_2 = ReaderReadV2[_device="/job:localhost/replica:0/task:0/device:CPU:0"](parallel_read/TFRecordReaderV2_2, parallel_read/filenames)]]
root@bc5b90e77883:/data/TwinGAN#

How can i solve this error? Please answer to my question.
Thank You!

[jerryli27]

--dataset_dir="./test/" is incorrect. It should be where your tfrecord files are "./test/train-A". Similarly for --unpaired_target_dataset_dir

[itsss]

@jerryli27
[Folder Structure]
./test/train-A : result.tflearn, result1.tflearn, result2.tflearn
./test/train-B : Getchu Preprocessed Dataset (train-00000-of-00008, vaildation-00001-of-00002, etc...)

What should i do? i executed the command like this, but it makes error.
python pggan_runner.py --program_name=twingan --dataset_name="image_only" --dataset_dir="./test/train-A" --unpaired_target_dataset_name="anime_faces" --unpaired_target_dataset_dir="./test/train-B" --train_dir="./checkpoints/pggan/" --dataset_split_name=train --preprocessing_name="danbooru" --resize_mode=RANDOM_CROP --do_random_cropping=True --learning_rate=0.0001 --learning_rate_decay_type=fixed --is_training=True --generator_network="pggan" --num_images_per_resolution=50000 --loss_architecture=dragan --pggan_max_num_channels=256 --generator_norm_type=batch_renorm --use_ttur=True

WARNING:tensorflow:IOError [Errno 2] No such file or directory: './test/train-A/sample_source_validation_100.npy'. Not outputting custom generated images.
2018-07-28 02:47:33.716050: W tensorflow/core/kernels/queue_base.cc:285] _1_prefetch_queue/fifo_queue: Skipping cancelled dequeue attempt with queue not closed
INFO:tensorflow:Error reported to Coordinator: <class 'tensorflow.python.framework.errors_impl.InvalidArgumentError'>, assertion failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]
         [[Node: case/cond/cond_jpeg/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert = Assert[T=[DT_STRING], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](case/cond/cond_jpeg/decode_image/cond_jpeg/cond_png/cond_gif/is_bmp, case/cond/cond_jpeg/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert/data_0)]]
INFO:tensorflow:Caught OutOfRangeError. Stopping Training. FIFOQueue '_1_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_STRING, DT_STRING, DT_STRING, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prefetch_queue/fifo_queue)]]

Caused by op u'fifo_queue_Dequeue', defined at:
  File "pggan_runner.py", line 164, in <module>
    tf.app.run()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "pggan_runner.py", line 154, in main
    model.main()
  File "/data/TwinGAN/twingan.py", line 894, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/image_generation.py", line 1054, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/model/model_inheritor.py", line 1044, in main
    'is_training': FLAGS.is_training})
  File "/data/TwinGAN/deployment/model_deploy.py", line 237, in create_clones
    outputs = model_fn(*args, **kwargs)
  File "/data/TwinGAN/twingan.py", line 150, in _clone_fn
    data_batched = super(GanModel, GanModel)._get_data_batched(batch_queue, batch_names, data_batched)
  File "/data/TwinGAN/model/model_inheritor.py", line 433, in _get_data_batched
    dequeued = batch_queue.dequeue()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/data_flow_ops.py", line 440, in dequeue
    self._queue_ref, self._dtypes, name=name)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_data_flow_ops.py", line 3734, in queue_dequeue_v2
    timeout_ms=timeout_ms, name=name)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 3392, in create_op
    op_def=op_def)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1718, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

OutOfRangeError (see above for traceback): FIFOQueue '_1_prefetch_queue/fifo_queue' is closed and has insufficient elements (requested 1, current size 0)
         [[Node: fifo_queue_Dequeue = QueueDequeueV2[component_types=[DT_STRING, DT_STRING, DT_STRING, DT_FLOAT, DT_FLOAT, DT_FLOAT], timeout_ms=-1, _device="/job:localhost/replica:0/task:0/device:CPU:0"](prefetch_queue/fifo_queue)]]

INFO:tensorflow:Finished training! Saving model to disk.
Traceback (most recent call last):
  File "pggan_runner.py", line 164, in <module>
    tf.app.run()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "pggan_runner.py", line 154, in main
    model.main()
  File "/data/TwinGAN/twingan.py", line 894, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/image_generation.py", line 1054, in main
    super(GanModel, self).main()
  File "/data/TwinGAN/model/model_inheritor.py", line 1127, in main
    session_config=session_config)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/contrib/slim/python/slim/learning.py", line 784, in train
    ignore_live_threads=ignore_live_threads)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/supervisor.py", line 828, in stop
    ignore_live_threads=ignore_live_threads)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/coordinator.py", line 389, in join
    six.reraise(*self._exc_info_to_raise)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/training/queue_runner_impl.py", line 252, in _run
    enqueue_callable()
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1244, in _single_operation_run
    self._call_tf_sessionrun(None, {}, [], target_list, None)
  File "/data/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1409, in _call_tf_sessionrun
    run_metadata)
tensorflow.python.framework.errors_impl.InvalidArgumentError: assertion failed: [Unable to decode bytes as JPEG, PNG, GIF, or BMP]
         [[Node: case/cond/cond_jpeg/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert = Assert[T=[DT_STRING], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](case/cond/cond_jpeg/decode_image/cond_jpeg/cond_png/cond_gif/is_bmp, case/cond/cond_jpeg/decode_image/cond_jpeg/cond_png/cond_gif/Assert_1/Assert/data_0)]]

[jerryli27]

The error message was " [Unable to decode bytes as JPEG, PNG, GIF, or BMP]" as displayed. Probably your image encoding was somehow wrong or corrupted.

I will double check that everything provided on this repo works out of the box. Sorry for the inconvenience, but it will take some time. In the mean time, please try to debug yourself as it is probably the fastest way of getting things done and I can't provide 24/7 debugging support.