Speeding up a pretrained classifier about dataset

[gkrasin]

Hi @okasanasan!

tools/classify.py is a simple example-demonstrator. It does not use GPU at all, does not run inference in batches and it also has to load the checkpoint and initialize the graph before it has a chance to classify an image.

So, the answer is yes, it's possible to speed it up.

Hi @gkrasin!

Thanks for your answer!

Can you tip me how to force using GPU?

[gkrasin]

@okasanasan this is very well covered in Using GPUs docs.

[gkrasin]

I also suspect that currently the majority of the time goes into loading the checkpoint. If you're doing a bulk classification, getting rid of this overhead might be even more important than running on GPU.

Thank you @gkrasin.

I changed the script a little bit - added a possibility to put multiple paths at once and process them in a loop.

[AdityaChaganti]

Hi @okasanasan, I'm working on this right now, and was curious about the speed improvement you achieved.

[AdityaChaganti]

@gkrasin In order to avoid the I/O latency resulting from tensorflow slim loading the architecture on each run, I first dumped the architecture initialized by slim into a protobuf file, say 'init_graph.pb'. I then used freeze_graph.py to combine the architecture from 'init_graph.pb' with the trained weights from data/model.ckpt and dumped them into a second .pb file, 'graph.pb'. I then loaded graph.pb in a session, and ran the session feeding in a single pre-processed image. The resulting output however, seems binary.

Any pointers?

[gkrasin]

@AdityaChaganti how do I reproduce your issue? Any code I can patch and run? Also, based on the description, I am unsure what do you mean by the resulting output being binary.

[AdityaChaganti]

@gkrasin I can email you my code with steps to reproduce it. Would that work?

And sorry for the ambiguity! I meant that label classifications coming out of the graph (The output of the final sigmoid layer) are made with a confidence of either 0, or 1.

[gkrasin]

I can email you my code with steps to reproduce it. Would that work?

@AdityaChaganti probably not, as that will move the whole discussion into a private area, and therefore will not help others with the same issue in the future.

And sorry for the ambiguity! I meant that label classifications coming out of the graph (The output of the final sigmoid layer) are made with a confidence of either 0, or 1.

That's an interesting glitch! I don't have any guesses at the moment.

[AdityaChaganti]

@gkrasin Ah, of course. I'll post the code here in a while. Thank you!

[AdityaChaganti]

Hi @gkrasin,

Run the attached script from the root of the dataset directory, specifying the path to the image you're using. Any JPEG image should work, but I've also attached the specific image I was using for testing. My output is as follows:

freeze.zip

[gkrasin]

@AdityaChaganti I have taken a quick look, and I find the following snippet very confusing. Can you please clarify why is it implemented like that?

image = tf.gfile.FastGFile(FLAGS.image_path, 'rb').read()
image = PreprocessImage(image)
image = image.eval(session=sess)  # Returns a numpy array
image = image[0]  # Extracts Row*Column*Channel
print(image.shape)
image = Image.fromarray(image, 'RGB')
image.save('./tmp/image.jpg')
with tf.gfile.FastGFile('./tmp/image.jpg', 'r') as f:
  image = f.read()

predictions = sess.graph.get_tensor_by_name('multi_predictions:0')
predictions = np.squeeze(sess.run(predictions, {'input:0': image}))

I am also confused by the fact that 'input:0' is feeded with an already preprocessed JPEG-encoded image. Would not it be preprocessed by the graph again?

[AdityaChaganti]

@gkrasin You're right, it would. My bad. I missed the fact that the preprocessing is already fed into the graph while saving it. Accounting for that change, my modified script is attached. The output is still similar:
freeze.zip

[gkrasin]

My recommendation would be to compare the layer activations on freezed vs normal graph and find the first layer with a material difference.