[Consolidate all YOLOv5 issues] YOLOv5 export error `ValueError: axes don't match array` about openvino2tensorflow

[glenn-jocher]

@PINTO0309 super interesting, thanks for the detailed response!

The 3 transpose ops are occurring 1 in each of the output grids P3/8, P4/16, P5/32. The permute op is undesirable as it leaves the output not contiguous, so it requires a .contiguous() method afterward.

I'll revisit the idea of removing the permutation completely from YOLOv5. It's a legacy from YOLOv3 onward, but maybe there's a better way I can restructure this module.

[PINTO0309]

0. convert - first try

openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite

1. Here. Shape Error occurred.

ERROR: axes don't match array
ERROR: model_path  : yolov5n.xml
ERROR: weights_path: yolov5n.bin
ERROR: layer_id    : 271 <<<<<<<<<<<--------------------- Shape Error occured
ERROR: input_layer0 layer_id=269: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 3, 85, 2), dtype=tf.float32, name=None), name='tf.math.multiply/Mul:0', description="created by layer 'tf.math.multiply'")
ERROR: input_layer1 layer_id=270: Const(ndarray).shape  (1, 3, 80, 80, 2)
array([[[[[ -4.,  -4.],
          [  4.,  -4.],
          [ 12.,  -4.],
:
:
ERROR: Please refer to 6-7 in the README first. https://github.com/PINTO0309/openvino2tensorflow

2. The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point. Mechanically unpredictable transposition operations require the tool's behavior to be modified using JSON files. The cause of this error is that the transpose before the OP in which the error occurs failed to correct the transposition order.

layer_type: Reshape
layer_id: 261
input_layer0: layer_id=259: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 80, 255), dtype=tf.float32, name=None), name='tf.math.add_52/Add:0', description="created by layer 'tf.math.add_52'")
input_layer1_shape: layer_id=260: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 80, 3, 85), dtype=tf.float32, name=None), name='tf.reshape/Reshape:0', description="created by layer 'tf.reshape'")
====================================================================================
layer_type: Const
layer_id: 262
tf_layers_dict_shape: (5,)
====================================================================================
layer_type: Transpose
layer_id: 263 <<<<<<<<<<<--------------------- The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
input_layer0: layer_id=261: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 80, 3, 85), dtype=tf.float32, name=None), name='tf.reshape/Reshape:0', description="created by layer 'tf.reshape'")
input_layer1_shape: layer_id=262: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 80, 3, 85, 80), dtype=tf.float32, name=None), name='tf.compat.v1.transpose/transpose:0', description="created by layer 'tf.compat.v1.transpose'")

3. replace_n.json - first step
   Rewrite the transposition order of Transpose so that it behaves as expected. The inverted order constant corresponds to the layer number of the name of output minus 1. output name: 263:2 -> layer_id:262 = custom name: Constant_3067

{
    "format_version": 2,
    "layers": [
        {
          "layer_id": "262",
          "type": "Const",
          "replace_mode": "direct",
          "values": [
              0,
              3,
              1,
              2,
              4
          ]
        }
    ]
}

5. re-convert

openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite \
--weight_replacement_config replace_n.json

6. Shape Error occurred.

ERROR: axes don't match array
ERROR: model_path  : yolov5n.xml
ERROR: weights_path: yolov5n.bin
ERROR: layer_id    : 328 <<<<<<<<<<<--------------------- Shape Error occured
ERROR: input_layer0 layer_id=326: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 3, 85, 2), dtype=tf.float32, name=None), name='tf.math.multiply_3/Mul:0', description="created by layer 'tf.math.multiply_3'")
ERROR: input_layer1 layer_id=327: Const(ndarray).shape  (1, 3, 40, 40, 2)
array([[[[[ -8.,  -8.],
          [  8.,  -8.],
          [ 24.,  -8.],

7. The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
   

====================================================================================
layer_type: Reshape
layer_id: 318
input_layer0: layer_id=316: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 40, 255), dtype=tf.float32, name=None), name='tf.math.add_60/Add:0', description="created by layer 'tf.math.add_60'")
input_layer1_shape: layer_id=317: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 40, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_2/Reshape:0', description="created by layer 'tf.reshape_2'")
====================================================================================
layer_type: Const
layer_id: 319
tf_layers_dict_shape: (5,)
====================================================================================
layer_type: Transpose
layer_id: 320 <<<<<<<<<<<--------------------- The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
input_layer0: layer_id=318: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 40, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_2/Reshape:0', description="created by layer 'tf.reshape_2'")
input_layer1_shape: layer_id=319: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 40, 3, 85, 40), dtype=tf.float32, name=None), name='tf.compat.v1.transpose_1/transpose:0', description="created by layer 'tf.compat.v1.transpose_1'")
====================================================================================

8. replace_n.json - second step

{
    "format_version": 2,
    "layers": [
        {
          "layer_id": "262",
          "type": "Const",
          "replace_mode": "direct",
          "values": [
              0,
              3,
              1,
              2,
              4
          ]
        },
        {
          "layer_id": "319",
          "type": "Const",
          "replace_mode": "direct",
          "values": [
              0,
              3,
              1,
              2,
              4
          ]
        }
    ]
}

9. re-convert

openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite \
--weight_replacement_config replace_n.json

10. Shape Error occurred.

ERROR: axes don't match array
ERROR: model_path  : yolov5n.xml
ERROR: weights_path: yolov5n.bin
ERROR: layer_id    : 384
ERROR: input_layer0 layer_id=382: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 3, 85, 2), dtype=tf.float32, name=None), name='tf.math.multiply_6/Mul:0', description="created by layer 'tf.math.multiply_6'")
ERROR: input_layer1 layer_id=383: Const(ndarray).shape  (1, 3, 20, 20, 2)
array([[[[[-16., -16.],
          [ 16., -16.],

11. The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.

====================================================================================
layer_type: Reshape
layer_id: 374
input_layer0: layer_id=372: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 20, 255), dtype=tf.float32, name=None), name='tf.math.add_68/Add:0', description="created by layer 'tf.math.add_68'")
input_layer1_shape: layer_id=373: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 20, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_4/Reshape:0', description="created by layer 'tf.reshape_4'")
====================================================================================
layer_type: Const
layer_id: 375
tf_layers_dict_shape: (5,)
====================================================================================
layer_type: Transpose
layer_id: 376 <<<<<<<<<<<--------------------- The conversion of the unpredictable transposition operation from NCDHW to NCHWD fails at this point.
input_layer0: layer_id=374: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 20, 3, 85), dtype=tf.float32, name=None), name='tf.reshape_4/Reshape:0', description="created by layer 'tf.reshape_4'")
input_layer1_shape: layer_id=375: (5,)
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 20, 3, 85, 20), dtype=tf.float32, name=None), name='tf.compat.v1.transpose_2/transpose:0', description="created by layer 'tf.compat.v1.transpose_2'")
====================================================================================

12. replace_n.json - second step

{
    "format_version": 2,
    "layers": [
        {
          "layer_id": "262",
          "type": "Const",
          "replace_mode": "direct",
          "values": [
              0,
              3,
              1,
              2,
              4
          ]
        },
        {
          "layer_id": "319",
          "type": "Const",
          "replace_mode": "direct",
          "values": [
              0,
              3,
              1,
              2,
              4
          ]
        },
        {
          "layer_id": "375",
          "type": "Const",
          "replace_mode": "direct",
          "values": [
              0,
              3,
              1,
              2,
              4
          ]
        }
    ]
}

13. re-convert

openvino2tensorflow \
--model_path yolov5n.xml \
--output_saved_model \
--output_pb \
--non_verbose \
--output_no_quant_float32_tflite \
--weight_replacement_config replace_n.json

14. success

====================================================================================
layer_type: Result
layer_id: 394
input_layer0: layer_id=393: KerasTensor(type_spec=TensorSpec(shape=(1, 25200, 85), dtype=tf.float32, name=None), name='tf.concat_16/concat:0', description="created by layer 'tf.concat_16'")
tf_layers_dict: KerasTensor(type_spec=TensorSpec(shape=(1, 25200, 85), dtype=tf.float32, name=None), name='tf.identity/Identity:0', description="created by layer 'tf.identity'")
====================================================================================
TensorFlow/Keras model building process complete!
saved_model output started ==========================================================
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 60). These functions will not be directly callable after loading.
saved_model output complete!
.pb output started ==================================================================
.pb output complete! - saved_model/model_float32.pb
tflite Float32 convertion started ===================================================
WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 60). These functions will not be directly callable after loading.
Estimated count of arithmetic ops: 5.393 G  ops, equivalently 2.697 G  MACs
tflite Float32 convertion complete! - saved_model/model_float32.tflite
All the conversion process is finished! =============================================

[PINTO0309]

All issues of YOLOv5 asked the same question.
#115
#102
#99
#91
#88
#86
#82
#77
#54
#50
#48
#41
#38
#33
#21

[PINTO0309]

Btw, If you want to rename an input/output layer, you can easily rewrite it manually using this tool.
https://github.com/PINTO0309/tflite2json2tflite

[PINTO0309]

Closed due to no activity for over a week.

[PINTO0309]

https://github.com/PINTO0309/onnx2tf

• Straight conversion sample
  

• YOLOv7-tiny with Post-Process (NMS) ONNX to TFLite Float32