BlazePose frozen model about pinto_model_zoo

[PINTO0309]

@vpenades First, only the Accurate version of the model was committed.
https://github.com/PINTO0309/PINTO_model_zoo/tree/master/058_BlazePose_Full_Keypoints

[PINTO0309]

It's not a huge amount of work, so I generated a .pb and committed it. Use download.sh to download the materials again.

PINTO_model_zoo/53_BlazePose/01_float32/01_pose_detection_tflite2h5_weight_int_fullint_float16_quant.py

Lines 380 to 387 in 4b98944

	full_model = tf.function(lambda inputs: model(inputs))
	full_model = full_model.get_concrete_function(inputs = (tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype)))
	frozen_func = convert_variables_to_constants_v2(full_model, lower_control_flow=False)
	frozen_func.graph.as_graph_def()
	tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
	logdir=".",
	name="pose_detection_128x128_float32.pb",
	as_text=False)

PINTO_model_zoo/53_BlazePose/01_float32/02_pose_landmark_upper_body_tflite2h5_weight_int_fullint_float16_quant.py

Lines 759 to 766 in 4b98944

	full_model = tf.function(lambda inputs: model(inputs))
	full_model = full_model.get_concrete_function(inputs = (tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype)))
	frozen_func = convert_variables_to_constants_v2(full_model, lower_control_flow=False)
	frozen_func.graph.as_graph_def()
	tf.io.write_graph(graph_or_graph_def=frozen_func.graph,
	logdir=".",
	name="pose_landmark_upper_body_256x256_float32.pb",
	as_text=False)

https://github.com/PINTO0309/PINTO_model_zoo/tree/master/53_BlazePose/01_float32

[vpenades]

Thanks for the effort, yes, I suspected it's a new requirement that's been added to TF v2.3 and it's impossible to make it run in TF v2.2

what's funny is that the TFLite version does run with the current version.

I'll wait for TF v2.3 on my platform...

Thanks a lot!

[PINTO0309]

It looks like an Apache 2.0 license. I plan to do conversion work on weekends when I have time.
https://t.co/BU6b4cLHUu?amp=1

[vpenades]

Thanks a lot!

When I try to load the models, they give me this error:

'NodeDef mentions attr 'explicit_paddings' not in Op<name=DepthwiseConv2dNative; signature=input:T, filter:T -> output:T;
attr=T:type,allowed=[DT_HALF, DT_BFLOAT16, DT_FLOAT, DT_DOUBLE];
attr=strides:list(int); attr=padding:string,allowed=["SAME", "VALID"];
attr=data_format:string,default="NHWC",allowed=["NHWC", "NCHW"];
attr=dilations:list(int),default=[1, 1, 1, 1]>;
NodeDef: {{node model/depthwise_conv2d/depthwise}}.
(Check whether your GraphDef-interpreting binary is up to date with your GraphDef-generating binary.).'

My inderstanding is that the generated PB files require TensorFlow 2.3 and since I am using Emgu.CV, it's still using TensowFlow 2.2... so I'll have to wait for Emgu.CV to update to TensorFlow 2.3....

[PINTO0309]

If you're in a hurry, try generating a .pb file on your own using Tensorflow v2.2.0. Just run the script below.
https://github.com/PINTO0309/PINTO_model_zoo/blob/master/53_BlazePose/01_float32/01_pose_detection_tflite2h5_weight_int_fullint_float16_quant.py

I just tested loading with Tensorflow v2.3.0 and it completed loading successfully without any errors.

import tensorflow as tf
graph_def = tf.compat.v1.GraphDef()
# Import the TF graph
with tf.io.gfile.GFile('pose_detection_128x128_float32.pb', 'rb') as f:
    graph_def.ParseFromString(f.read())
    tf.import_graph_def(graph_def, name='')

In TF v2.3.0, explicit_paddings has been added.

[PINTO0309]

Unfortunately, generating .pb with Tensorflow v2.2.0 did not work correctly.

[vpenades]

@PINTO0309 It seems someone extracted the FULL BODY models from MLKIT: https://github.com/asus4/tf-lite-unity-sample/tree/master/Assets/StreamingAssets/mlkit

I've just been able to add them to my project and I can confirm that it's the full body blazepose models.

You might want to take a look and include these into your repository? (PBs too please! ^_^; I might need to find time to learn python)

[PINTO0309]

Thanks for sharing. I think I'll try to convert sooner or later. I've checked the structure of the model and it looks like it's only a little more OP's diagram below than the upper body only model. The structure of the model is almost identical.

What I'm wondering is what the license is for the tflite model of Blaze Pose Full.

[vpenades]

I have no idea about the license... my impression is that they're releasing the model straight away into MLKit to make it easy to use for developers and nothing else. What seems to be preventing them releasing the models for other platforms is that it would imply that they need to give maintenance and support for these platforms.

For example, developers have requested the full body to be available in MediaPipe for windows, and for TensorFlow.JS... but right now they're having serious issues trying to compile the windows version of MediaPipe.

[PINTO0309]

@vpenades
An additional Lite version has been committed.
https://github.com/PINTO0309/PINTO_model_zoo/tree/master/058_BlazePose_Full_Keypoints

[bhavikapanara]

Sorry to bother you again with this... I guess you're already working on converting the BlazePose TFLite models to Frozen model as long as saved model?

I've successfully ran the TfLite models, and even in desktop and using only CPU the models are very fast and responsive, it's definitely a much better solution than the PoseNet models.

My understanding is that PoseNet models are better suited for one shot detection, whereas BlazePose is better for continuous tracking, which makes it very good for realtime applications.

Hi @vpenades

How could you run BlazePose TFlite model on cpu? Can you please share the python script and TFLite model file for pose estimation?

Thanks a lot

[vpenades]

@bhavikapanara I'm not using python, my project is in c# with Emgu.TF.Lite , which can run tflite models on the CPU.

[bhavikapanara]

@vpenades Thanks for the quick reply.

Actually, I am able to run the TFLite model on the CPU. But I am facing issues to decode output.

This is the output format:

[{'name': 'Identity',
  'index': 285,
  'shape': array([  1, 128, 128,   1], dtype=int32),
  'shape_signature': array([ -1, 128, 128,   1], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}},
 {'name': 'Identity_1',
  'index': 379,
  'shape': array([1, 1, 1, 1], dtype=int32),
  'shape_signature': array([-1,  1,  1,  1], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}},
 {'name': 'Identity_2',
  'index': 381,
  'shape': array([  1, 156], dtype=int32),
  'shape_signature': array([  1, 156], dtype=int32),
  'dtype': numpy.float32,
  'quantization': (0.0, 0),
  'quantization_parameters': {'scales': array([], dtype=float32),
   'zero_points': array([], dtype=int32),
   'quantized_dimension': 0},
  'sparsity_parameters': {}}]

Form this how to get all detected ket-points coordinates

@PINTO0309 Can you please help me with this?

Thanks,
Bhavika

[vpenades]

I'm not an expert on python, I understand the official solution to use these models with python is this one: https://google.github.io/mediapipe/solutions/pose.html#python-solution-api