Cannot convert YoloV5s to EdgeTPU about openvino2tensorflow

[luiscastro1995]

Indeed tf-nightly solved most of the problems, however, RESIZE_NEAREST_NEIGHBOR and LEAKY_RELU could not be compiled. To achieve a full TPU compilation, from the .h5 and .json model here, I did:

• In the the json file, change every LeakyReLU to ReLU and delete the parameter "alpha" from the "config" key.

• Perform the full integer quantization use code "04_full_integer_quantization.py" from here, using tf-nightly. Note: In that code I added the line "converter.experimental_new_converter = False"

• Go to the code "gen_image_ops.py" in my case was in the path "/home/luis/anaconda3/envs/pinto_yolov4/lib/python3.6/site-packages/tensorflow/python/ops/" and on the funtions: resize_nearest_neighbor_grad_eager_fallback, resize_nearest_neighbor_grad, resize_nearest_neighbor_eager_fallback, resize_nearest_neighbor; add the following line after the header definition (half_pixel_centers = False). This solution comes from here. It is definitly not an elegant solution but it seems to work.

• edgetpu_compile -sa edgetpu_compiler -sa yolov4_tiny_voc_416x416_full_integer_quant.tflite

The result is the following:

Now, I have to train the Yolov4-tiny implementation that you suggested and see if I can come up with the same compilation.

As always thanks for the support @PINTO0309

[PINTO0309]

The error message can be ignored. The saved_model should be output.

Switch to the output of an optimized protocol buffer file (.pb).

Use --output_edgetpu instead of --output_integer_quant_tflite.

[luiscastro1995]

According to your suggestion, I ran the command:

openvino2tensorflow --model_path best_opt.xml --model_output_path Models --output_saved_model True --output_h5 True --output_pb True --output_edgetpu True

So, without any --output_integer_quant_tflite. These were the generated files,

The output from the command was the following:

.
.
.
__________________________________________________________________________________________________
tf.identity_2 (TFOpLambda)      (1, 80, 3, 6, 80)    0           tf.compat.v1.transpose_2[0][0]   
==================================================================================================
Total params: 7,233,672
Trainable params: 7,233,672
Non-trainable params: 0
__________________________________________________________________________________________________
TensorFlow/Keras model building process complete!
saved_model output started ==========================================================
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: model/tf.math.multiply_49/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: model/tf.math.multiply_56/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: tf.math.multiply_49/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: tf.math.multiply_56/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: tf.math.multiply_49/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: tf.math.multiply_56/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: inputs:0
ERROR: can't pickle module objects
Traceback (most recent call last):
  File "/home/luis/anaconda3/envs/yolo2lite/bin/openvino2tensorflow", line 1754, in convert
    tf.saved_model.save(model, model_output_path)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 1033, in save
    obj, signatures, options, meta_graph_def)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 1198, in _build_meta_graph
    return _build_meta_graph_impl(obj, signatures, options, meta_graph_def)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 1163, in _build_meta_graph_impl
    asset_info.asset_index)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 755, in _serialize_object_graph
    saveable_view.function_name_map)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/saved_model/save.py", line 800, in _write_object_proto
    metadata=obj._tracking_metadata)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 3079, in _tracking_metadata
    return self._trackable_saved_model_saver.tracking_metadata
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/base_serialization.py", line 55, in tracking_metadata
    return json_utils.Encoder().encode(self.python_properties)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 41, in python_properties
    return self._python_properties_internal()
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/model_serialization.py", line 35, in _python_properties_internal
    metadata = super(ModelSavedModelSaver, self)._python_properties_internal()
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 59, in _python_properties_internal
    metadata.update(get_config(self.obj))
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/saving/saved_model/layer_serialization.py", line 118, in get_config
    config = generic_utils.serialize_keras_object(obj)['config']
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/utils/generic_utils.py", line 245, in serialize_keras_object
    config = instance.get_config()
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/keras/engine/functional.py", line 650, in get_config
    return copy.deepcopy(get_network_config(self))
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 215, in _deepcopy_list
    append(deepcopy(a, memo))
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 240, in _deepcopy_dict
    y[deepcopy(key, memo)] = deepcopy(value, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 150, in deepcopy
    y = copier(x, memo)
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 220, in _deepcopy_tuple
    y = [deepcopy(a, memo) for a in x]
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 220, in <listcomp>
    y = [deepcopy(a, memo) for a in x]
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/copy.py", line 169, in deepcopy
    rv = reductor(4)
TypeError: can't pickle module objects
Switch to the output of an optimized protocol buffer file (.pb).
.pb output started ==================================================================
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: model/tf.math.multiply_49/Mul:0
WARNING: The weights after Upsampling (tf.compat.v1.image.resize_nearest_neighbor) are shifted to the upper left. If you do not need to generate EdgeTPU models, set --output_edgetpu False and run again. OP: model/tf.math.multiply_56/Mul:0
.pb output complete! - Models/model_float32.pb
WARNING:tensorflow:From /home/luis/anaconda3/envs/yolo2lite/bin/openvino2tensorflow:1837: simple_save (from tensorflow.python.saved_model.simple_save) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.simple_save.
WARNING:tensorflow:From /home/luis/anaconda3/envs/yolo2lite/lib/python3.6/site-packages/tensorflow/python/saved_model/signature_def_utils_impl.py:201: build_tensor_info (from tensorflow.python.saved_model.utils_impl) is deprecated and will be removed in a future version.
Instructions for updating:
This function will only be available through the v1 compatibility library as tf.compat.v1.saved_model.utils.build_tensor_info or tf.compat.v1.saved_model.build_tensor_info.
Optimized graph converted to SavedModel! - Models
TFDS download started ===============================================================
TFDS download complete!
Full Integer Quantization started ===================================================
Full Integer Quantization complete! - Models/model_full_integer_quant.tflite
EdgeTPU convertion started ==========================================================
ERROR: 
Traceback (most recent call last):
  File "/home/luis/anaconda3/envs/yolo2lite/bin/openvino2tensorflow", line 2053, in convert
    stderr=subprocess.PIPE).decode('utf-8')
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/subprocess.py", line 356, in check_output
    **kwargs).stdout
  File "/home/luis/anaconda3/envs/yolo2lite/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['edgetpu_compiler', '-o', 'Models', '-s', 'Models/model_full_integer_quant.tflite']' returned non-zero exit status 1.
--------------------------------------------------------------------------------
Please install edgetpu_compiler according to the following website.
https://coral.ai/docs/edgetpu/compiler/#system-requirements
All the conversion process is finished! =============================================

As you can see the TPU compilation failed even though and I have installed the edgetpu_compiler cmd tool and did not write any --output_integer_quant_tflite. I checked the generated "model_full_integer_quant.tflite" in Netron and again it seems to be a valid model. Any clues? Do you know if is possible to convert the yolov5s model using your strategy?

One other question is ot possible to suppress the "TFDS download" phase? I had to install 25 Gigas of stuff and it took quite some time. I tried some argument to the openvino2tensorflow call but could not skip the download phase.

[PINTO0309]

I have committed the conversion script and the converted model here. However, to avoid the EdgeTPU compiler error, you need to manually eliminate the OP that causes the EdgeTPU compiler error. For example, the Post-Process part is very complex, so I often remove it. Also, the EdgeTPU compiler seems to abort models with large input resolutions.

• My converted model
  https://github.com/PINTO0309/PINTO_model_zoo/tree/main/059_yolov5

• Issue.1
  [Future Request] 5D Transpose support in the edgetpu compiler
  google-coral/edgetpu#274

• Issue.2
  [Future Request] EfficientDet D0 conversion support by EdgeTPU Compiler
  google-coral/edgetpu#279

One other question is ot possible to suppress the "TFDS download" phase? I had to install 25 Gigas of stuff and it took quite some time. I tried some argument to the openvino2tensorflow call but could not skip the download phase.

I am considering making it possible to read Numpy format datasets as an additional feature in the future. This is not supported at the moment. If you don't want to download a large dataset, there is a way to generate your own Numpy file. The number of images required for calibration during quantization is about 100, so it is by far the lightest. For example, you can create your own script that modifies the following range to read image data from a Numpy(.npy) file and perform full INT8 quantization.

https://github.com/PINTO0309/PINTO_model_zoo/blob/316bddd6009f93958e288a95208d7d02fd9b31b8/059_yolov5/01_float32/05_full_integer_quantization.py#L10-L17

def representative_dataset_gen():
    for image in raw_test_data.take(10):
        image = image['image'].numpy()
        image = tf.image.resize(image, (height, width))
        image = image[np.newaxis,:,:,:]
        image = image - 127.5
        image = image * 0.007843
        yield [image]

The following is a good reference on how to create a .npy file. Converts image data stored in a specific folder to a .npy file.
https://github.com/PINTO0309/PINTO_model_zoo/blob/main/029_human-pose-estimation-3d-0001/01_float32/00_create_calibration_image.py

import os
import sys
from glob import glob
from PIL import Image
import numpy as np

def resize(all_file):

  tmp = None
  for f in all_file:
    img = Image.open(f)
    nparray = np.array(img)
    nparray = nparray[np.newaxis, :, :, :]

    if tmp is not None:
        tmp = np.vstack((tmp, nparray))
        print("tmp.shape=", tmp.shape)
        np.save('calibration_data_img', tmp)
    else:
        tmp = nparray.copy()
        print("tmp.shape=", tmp.shape)
        np.save('calibration_data_img', tmp)

path = "./images/"
all_file = os.listdir(path)
all_file = [os.path.join(path, f) for f in all_file]
resize(all_file)
print("Finish!!")

Alternatively, the default dataset is MS-COCO, which is large, but you can choose a lighter dataset from the URL below.
https://www.tensorflow.org/datasets/catalog/overview

  --ds_name_for_tfds_for_calibration DS_NAME_FOR_TFDS_FOR_CALIBRATION
                        Dataset name for TensorFlow Datasets for calibration.
                        https://www.tensorflow.org/datasets/catalog/overview

[luiscastro1995]

Indeed it was the resolution. My model accepted an images of size (3640640). I repeated all steps using now a resolution of (3320320) and was able to compile it to the TPU.

Command run:

openvino2tensorflow --model_path best_opt.xml --model_output_path Models --output_saved_model True --output_h5 True --output_pb True --output_edgetpu True

Result log:

.
.
.

Full Integer Quantization started ===================================================
Full Integer Quantization complete! - Models/model_full_integer_quant.tflite
EdgeTPU convertion started ==========================================================
Edge TPU Compiler version 15.0.340273435

Model compiled successfully in 4144 ms.

Input model: Models/model_full_integer_quant.tflite
Input size: 7.33MiB
Output model: Models/model_full_integer_quant_edgetpu.tflite
Output size: 7.47MiB
On-chip memory used for caching model parameters: 6.74MiB
On-chip memory remaining for caching model parameters: 1.75KiB
Off-chip memory used for streaming uncached model parameters: 369.88KiB
Number of Edge TPU subgraphs: 1
Total number of operations: 264
Operation log: Models/model_full_integer_quant_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 255
Number of operations that will run on CPU: 9

Operator                       Count      Status

QUANTIZE                       13         Mapped to Edge TPU
MUL                            76         Mapped to Edge TPU
CONV_2D                        70         Mapped to Edge TPU
RESIZE_NEAREST_NEIGHBOR        2          Mapped to Edge TPU
RESHAPE                        3          Mapped to Edge TPU
PRELU                          8          Mapped to Edge TPU
STRIDED_SLICE                  6          Only Strided-Slice with unitary strides supported
CONCATENATION                  14         Mapped to Edge TPU
TRANSPOSE                      3          Operation not supported
ADD                            66         Mapped to Edge TPU
MAX_POOL_2D                    3          Mapped to Edge TPU

EdgeTPU convert complete! - Models/model_full_integer_quant_edgetpu.tflite
All the conversion process is finished! =============================================

The resulting network after the TPU compilation is as follows:

One question though. According to the command log, the operations "STRIDED_SLICE" and "TRANSPOSE" are run on the CPU. I have been told that, per inference stage, if the TPU caught an incompatible operation (CPU compatible) the rest of the operations would be run on the CPU even if compatible with the TPU. If that is true, my whole model will run on the CPU since the "STRIDED_SLICE" operation is the first one. Is this true? I find it very strange, but since I have no experience in this filed I just accepted it.

Regarding the manual removal of OP that you mentioned, just for curiosity, how can you do it while keeping the model functional?

Let me just thank you for help so far and the thorough guide for the Pythorch-Tensorflow conversion. amazing job!

[PINTO0309]

if the TPU caught an incompatible operation (CPU compatible) the rest of the operations would be run on the CPU even if compatible with the TPU. If that is true, my whole model will run on the CPU since the "STRIDED_SLICE" operation is the first one. Is this true?

Presumably all OPs run on the CPU, but I haven't tried that either. For now, I am more interested in YoloV4 than YoloV5.

Regarding the manual removal of OP that you mentioned, just for curiosity, how can you do it while keeping the model functional?

This is just an example.

1. When exporting a model from PyTorch to ONNX, the input/output layer is adjusted before exporting.
2. It is possible to reverse convert tflite files to JSON. Since the JSON file can be freely edited, you can regenerate tflite from JSON with a simple command after manually deleting unnecessary layers. However, I think you need to implement this yourself by replacing the deleted OP with Numpy's Transpose function, Python's Slice process, and the element replacement process.https://github.com/PINTO0309/tflite2tensorflow
   

• My articles (Japanese)
  How to reverse convert a tflite with Custom Operation to JSON, replace it with standard OP, and then regenerate the tflite

[luiscastro1995]

I have tried the YoloV4-tiny-relu from here which is implemented in Tensorflow. After quantizing the model and convert it to TPU it gives 31 operation in CPU

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 101
Number of operations that will run on CPU: 31

Operator                       Count      Status

PAD                            2          Mapped to Edge TPU
SPLIT                          7          Mapped to Edge TPU
ADD                            6          Mapped to Edge TPU
SUB                            6          Mapped to Edge TPU
QUANTIZE                       27         Mapped to Edge TPU
QUANTIZE                       6          Operation is otherwise supported, but not mapped due to some unspecified limitation
DEQUANTIZE                     6          Operation is working on an unsupported data type
EXP                            6          Operation is working on an unsupported data type
CONCATENATION                  9          Mapped to Edge TPU
MAX_POOL_2D                    3          Mapped to Edge TPU
CONV_2D                        21         Mapped to Edge TPU
LOGISTIC                       2          Mapped to Edge TPU
MUL                            18         Mapped to Edge TPU
RESIZE_BILINEAR                1          Operation version not supported
SPLIT_V                        12         Operation not supported

Do you know of any YoloV4-tiny implementation fully compatible to TPU?

[PINTO0309]

I have removed Post-Process, but the conversion is clean.
https://github.com/PINTO0309/PINTO_model_zoo/tree/main/046_yolov4-tiny

[luiscastro1995]

If I want to train my custom data set on a yolov4-tiny and arrive to the same edgetpu model as you presented, what should I do?

Is the following a possible solution?

• I downloaded the files "yolov4_tiny_voc.h5" and "yolov4_tiny_voc.json" from here. Could I use this ".h5" to train on my custom dataset, then, after having the final trained model, convert it to tflite and finally compile it to the TPU? Do I need to make the Post-Processing removal, or the h5 already has this process in it?

Do you have a better alternative?

[PINTO0309]

Use the following repository to train, adjust the output layer, and output to saved_model.
https://github.com/bubbliiiing?tab=repositories&q=yolov4&type=&language=

For example, the following is easy to handle.
https://github.com/bubbliiiing/yolov4-tiny-tf2

Then you quantize saved_model to Full INT8.

[luiscastro1995]

Thank you, I will definitely give it a try.

Meanwhile, I tried to edgetpu_compile your model yolov4_tiny_voc_416x416_full_integer_quant.tflite from here but got the error "Model not quantized".
To compile it I had to modify your code "04_full_integer_quantization.py" by adding the line "converter.experimental_new_converter = False". After this, I was able to quantize the model "yolov4_tiny_voc.h5" and compile the resulting .tflite to the TPU, however, the compiled model is no where near the one you presented 3 comments ago.

$ edgetpu_compiler -s yolov4_tiny_voc_416x416_full_integer_quant.tflite 
Edge TPU Compiler version 15.0.340273435

Model compiled successfully in 161 ms.

Input model: yolov4_tiny_voc_416x416_full_integer_quant.tflite
Input size: 5.77MiB
Output model: yolov4_tiny_voc_416x416_full_integer_quant_edgetpu.tflite
Output size: 5.78MiB
On-chip memory used for caching model parameters: 3.00KiB
On-chip memory remaining for caching model parameters: 8.03MiB
Off-chip memory used for streaming uncached model parameters: 0.00B
Number of Edge TPU subgraphs: 1
Total number of operations: 65
Operation log: yolov4_tiny_voc_416x416_full_integer_quant_edgetpu.log

Model successfully compiled but not all operations are supported by the Edge TPU. A percentage of the model will instead run on the CPU, which is slower. If possible, consider updating your model to use only operations supported by the Edge TPU. For details, visit g.co/coral/model-reqs.
Number of operations that will run on Edge TPU: 2
Number of operations that will run on CPU: 63

Operator                       Count      Status

PAD                            1          More than one subgraph is not supported
PAD                            1          Mapped to Edge TPU
CONCATENATION                  7          More than one subgraph is not supported
LEAKY_RELU                     19         Operation not supported
SPLIT                          3          More than one subgraph is not supported
MAX_POOL_2D                    3          More than one subgraph is not supported
CONV_2D                        1          Mapped to Edge TPU
CONV_2D                        20         More than one subgraph is not supported
QUANTIZE                       9          More than one subgraph is not supported
RESIZE_NEAREST_NEIGHBOR        1          Operation version not supported

Maybe I got it wrong, but I thought that by using your yolov4-tiny model I could get a fully TPU compilation like the one you presented. I am missing something?

If it is relevant I used:
Edge TPU Compiler version 15.0.340273435 and tensorflow==2.3.0-rc1

[PINTO0309]

Try quantization using tf-nightly and convert it to the TPU model again. There is a bug in TensorFlow v2.3.x and v2.4.x.

$ sudo pip3 uninstall tensorboard-plugin-wit tb-nightly tensorboard \
                      tf-estimator-nightly tensorflow-gpu \
                      tensorflow tf-nightly tensorflow_estimator -y

$ sudo pip3 install tf-nightly

[PINTO0309]

This tool of mine has options available from the beginning to help you solve the problems you are having. It will automatically adjust half_pixel_centers = False without modifying the logic. Besides that, it also has the ability to automatically convert layers that are not supported by the EdgeTPU compiler. I know a trick that solves the LeakyReLU problem if you convert to OpenVINO IR and then reconvert again with this tool. Incidentally, LeakyReLU can be replaced by PReLU, and EdgeTPU Compiler supports compilation of PReLU.
--output_edgetpu True
--optimizing_hardswish_for_edgetpu True

The current latest version is v1.7.0, so it is recommended to update to the latest package.
$ sudo pip3 install openvino2tensorflow --upgrade

It is recommended that you also update the EdgeTPU Compiler to the latest version.
Edge TPU Compiler version 15.0.340273435

[luiscastro1995]

Thanks @PINTO0309 guess I was reinventing the wheel xD

I finally was able to train a Yolov4-tiny model from your recommended link (https://github.com/bubbliiiing/yolov4-tiny-tf2). I had to change some part of the code though, like removing some output and input layers that were unnecessary. Then I was able to convert the saved_model to openvino and then use your conversion tool to make the compilation to the TPU. From Netron the output is this:

I was comparing this model from the one you posted, and I noticed that my two right most output layers do not have the same dimentions as your model. I have not removed any Post-processing as far as I know. Could it be that?

I have removed Post-Process, but the conversion is clean.
https://github.com/PINTO0309/PINTO_model_zoo/tree/main/046_yolov4-tiny

One other thing, what is the meaning of each output layer, I will need to know that when running inference with this model right? I I am expecting the outputs to be the bounding boxes, confidence scores and class names but each one is each? I am new to this world of Deep Learning specially to deployment in TPUs so, sorry for the stupid question probably.

[PINTO0309]

I would recommend using YOLOv5-Lite.
#82