Cannot convert to tflite, problem with lambda about yolov3-tf2

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

[peace195]

Here is my solution: https://github.com/peace195/tensorflow-lite-yolo-v3

The .pb should be in right format (SavedModel).

Please try it. I would appreciate if you give me a star for this project 👍

[1165048017]

For "NameError: name 'tf' is not defined"
you can add the 'custom_objects' parameter in load_model function:
tf.keras.models.load_model('xxx.h5',custom_objects={"tf":tf})

I also met the problem 'keras is not defined' while load model, the same method to solve it:
tf.keras.models.load_model('xxx.h5',custom_objects={"keras":tf.keras})

[zzh8829]

I tried running tflite_convert in tensorflow 2.0 it says
tflite_convert is currently unsupported in 2.0

Following the instruction on TF2.0 API
export to tfserving saved_model first

python export_tfserving.py --output serving/yolov3/1/

run these in python

import tensorflow as tf
model = tf.saved_model.load("serving/yolov3/1")
converter = tf.lite.TFLiteConverter.from_concrete_function(model.signatures['serving_default'])
tflite_model = converter.convert() 
open("converted_model.tflite", "wb").write(tflite_model)

This should work but i'm getting weird errors

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/lite/python/lite.py", line 261, in convert
    shape_list = tensor.get_shape().as_list()
  File "/usr/local/anaconda3/envs/tf2/lib/python3.7/site-packages/tensorflow/python/framework/tensor_shape.py", line 1128, in as_list
    raise ValueError("as_list() is not defined on an unknown TensorShape.")
ValueError: as_list() is not defined on an unknown TensorShape.

¯_(ツ)_/¯

Sorry I can't really help right now, i guess we will have to wait for new updates from tflite 2.0.

[gwestner94]

Is there any update for this?
I would love to try using the model in tflite.

[zzh8829]

I followed the new instruction with model support
https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter

model = YoloV3()
model.load_weights('weights.tf')
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()

But unfortunately i got this error

terminate called after throwing an instance of 'std::bad_alloc'
  what():  std::bad_alloc
Aborted

You can try it too and see if it works

[Lap1n]

Did you guys managed to convert it to TFlite? I'm getting this error :

2019-08-28 11:02:00.428562: F tensorflow/contrib/lite/toco/tflite/export.cc:374] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.TocoConverter(). Here is a list of operators for which  you will need custom implementations: AddV2, FusedBatchNormV3, LeakyRelu, ResizeNearestNeighbor.

Did you guys managed to convert it to TFlite? I'm getting this error :

2019-08-28 11:02:00.428562: F tensorflow/contrib/lite/toco/tflite/export.cc:374] Some of the operators in the model are not supported by the standard TensorFlow Lite runtime. If you have a custom implementation for them you can disable this error with --allow_custom_ops, or by setting allow_custom_ops=True when calling tf.contrib.lite.TocoConverter(). Here is a list of operators for which  you will need custom implementations: AddV2, FusedBatchNormV3, LeakyRelu, ResizeNearestNeighbor.

Hey, Tensorflow 2.0 RC1 is released and all my error messages have now disappeared.
Can anyone confirm that?

[JJ306]

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,

You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,

You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version of Tensorflow do you use?

[JJ306]

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,
You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version o

yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)

For me, this works, but I am still trying to implement it to android. Maybe one of you can help?

Hi,
You did nice work. but When I set Training= False. I got the error " No module named '_tensorflow_wrap_toco' ". Do you know why is it?

Which Version of Tensorflow do you use?

I am using TF2.0 & Anaconda on windows 10.

[dao-kun]

those code is also works for me , but the the tfliite output is totally different, did anyone of you meet this before?
`yolo_size = 416
yolo_weights = './checkpoints/yolov3_train_116.tf'
tflite_model_path = './weights/yolo.tflite'
NUM_CLASSES = 80

yolo = YoloV3(yolo_size, training=True, classes=NUM_CLASSES)
yolo.load_weights(yolo_weights)

converter = tf.lite.TFLiteConverter.from_keras_model(yolo)
tflite_model = converter.convert()
open(tflite_model_path, 'wb').write(tflite_model)`

my tflite input :
[{'name': 'input_1', 'index': 3, 'shape': array([ 1, 416, 416, 3], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}]

output :

[{'name': 'Identity', 'index': 0, 'shape': array([ 1, 13, 13, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}, {'name': 'Identity_1', 'index': 1, 'shape': array([ 1, 26, 26, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}, {'name': 'Identity_2', 'index': 2, 'shape': array([ 1, 52, 52, 3, 85], dtype=int32), 'dtype': <class 'numpy.float32'>, 'quantization': (0.0, 0), 'quantization_parameters': {'scales': array([], dtype=float32), 'zero_points': array([], dtype=int32), 'quantized_dimension': 0}}]

[zzh8829]

Looks like TFLite converter has problem with CombinedNonMaxSupression operator, we will have to wait for official support for now

[zzh8829]

tensorflow/tensorflow#34693

[DanKlarman]

Hi, amazing work here!
any update on tflite issue?

[lijiong16]

I tried to convert my model to .tflite but got the following error.

RuntimeError: Encountered unresolved custom op: ResizeNearestNeighbor.Node number 26 (ResizeNearestNeighbor) failed to prepare.

[edurenye]

CombinedNonMaxSupression is not whitelisted: tensorflow/tensorflow#37301

[edurenye]

But as it is suggested here: tensorflow/tensorflow#33059 (comment)

We could replace it for tf.image.non_max_suppression_with_scores

Then everything would just work, tflite, onnx...