Comments (33)
Also I am using TF Lite Micro library generated by TF 2.2.0 branch. May be it is TF Lite Micro issue.
from pinto_model_zoo.
As per documentation pruning may not reduce the RAM requirement of the model, it will only reduce the model disk size (compression). It would be interesting to see if the size can be reduced further after you will change your script to TF v2. Thanks for your time!
from pinto_model_zoo.
Please try using Tensorflow v1.15.2.
from pinto_model_zoo.
Thanks! It works but I still get "Didn't find op for builtin opcode 'QUANTIZE' version '2'". while allocating tensors on the microcontroller. May be some ops are missing.
from pinto_model_zoo.
I think it's just that the OP on the TF Lite micro side doesn't support it. There is nothing wrong with the conversion script itself.
https://github.com/tensorflow/tensorflow/blob/3ebdec47df6c1afaa31ea6f0aa980f0f90729e87/tensorflow/lite/micro/kernels/quantize.cc#L156-L169
// This Op (QUANTIZE) quantizes the input and produces quantized output.
// AffineQuantize takes scale and zero point and quantizes the float value to
// quantized output, in int8 or uint8 format.
TfLiteRegistration* Register_QUANTIZE() {
static TfLiteRegistration r = {/*init=*/quantize::Init,
/*free=*/nullptr,
/*prepare=*/quantize::Prepare,
/*invoke=*/quantize::Eval,
/*profiling_string=*/nullptr,
/*builtin_code=*/0,
/*custom_name=*/nullptr,
/*version=*/0};
return &r;
}
from pinto_model_zoo.
from pinto_model_zoo.
Yes I used TF 2.2.0 for the conversion. But the models generated by 03_interger_quantization.py or 04_full_integer_quantization.py have same error.
from pinto_model_zoo.
I empathize with you.
from pinto_model_zoo.
Thanks for your understanding. I am struggling with this issue for 3 days.
from pinto_model_zoo.
Same issue. tensorflow/tensorflow - Didn't find op for builtin opcode 'QUANTIZE' version '2' #38488
from pinto_model_zoo.
Yes, I have seen that before. But I just saw TF Lite Micro has pushed new changes 1 hour ago: tensorflow/tensorflow@3ebdec4
I will try with the latest changes to see if the issue is resolved.
from pinto_model_zoo.
It's very interesting!!
from pinto_model_zoo.
With the latest changes, error has gone! Thanks for your help, I appreciate it. But now I am facing another issue which is related to the size of the model. The full integer quantized model needs a little over 500KB RAM (tensor allocation, weight/activation) but the microcontroller has only 512 KB RAM. Is there any way to decrease the size (Blazeface) further?
from pinto_model_zoo.
It is a very difficult problem. The following is a model pruning project, but it is still under development, and I could only find a sample of Keras. I haven't tried it yet, so I'm not sure if it's helpful yet.
https://github.com/tensorflow/model-optimization.git
https://blog.tensorflow.org/2019/05/tf-model-optimization-toolkit-pruning-API.html
from pinto_model_zoo.
Thanks for the clue! I guess I need to load "pb" saved model into TF Keras.
from pinto_model_zoo.
For example, there are model conversion tools like this one.
https://github.com/microsoft/MMdnn.git
from pinto_model_zoo.
I am trying to use model-optimization. I was able to load the model into Keras but can't see the model summary.
>>> import tensorflow as tf
>>> from tensorflow import keras
>>> print(tf.version.VERSION)
2.2.0
>>> model = tf.keras.models.load_model('saved_model')
>>> model
<tensorflow.python.training.tracking.tracking.AutoTrackable object at 0x101726890>
>>> model.summary()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'AutoTrackable' object has no attribute 'summary'
from pinto_model_zoo.
I suffer from the same phenomenon. It may be that I used tf.compat.v1.saved_model.simple_save when I first exported the model to the saved_model format. I may need to review the initial steps to generate saved_model using v2.x, but I don't see the answer right now.
>>> model = tf.keras.models.load_model('saved_model')
>>> model.
model.asset_paths model.initializer model.signatures model.variables
model.graph model.prune( model.tensorflow_git_version
model.graph_debug_info model.restore( model.tensorflow_version
from pinto_model_zoo.
Yes, it would be good if we can reduce the model size further for TFlite micro.
Also, there is no saved model for blazeface available from mediapipe repo.
from pinto_model_zoo.
The quantized model is only around 50% of the 32-bit float model. Ideally, it should be reduced by 70-75%.
face_detection_front.tflite: 413K
face_detection_front_128_full_integer_quant.tflite: 197K
If we can achieve at least 70% reduction, it would be really great. I am wondering if it is related to TF1 to TF2 conversion?
from pinto_model_zoo.
Ideally, it should be reduced by 70-75%.
It may be difficult to reduce the size to this level. Even if you have pruned the branches. The model I have generated is as follows.
- face_detection_front_128_weight_quant.tflite 163.8KB download here
- face_detection_front_128_integer_quant.tflite 201.6KB download here
What do you mean by the 70% reduction that you say is below?
- face_detection_front_128_weight_quant.tflite 49.14KB (70% reduced)
- face_detection_front_128_integer_quant.tflite 60.48KB (70% reduced)
I am wondering if it is related to TF1 to TF2 conversion?
My script that generates saved_model uses the v1 API. However, the saved_model generated by the v1 API seems to lack some information about Keras due to backward compatibility. If I could replace my .tflite to saved_model conversion script with a v2 base, I might be able to avoid the no attribute error.
from pinto_model_zoo.
I mean 70% of the default 32-bit float TFlite model.
For full integer quantization,
Currently: 413K --> 197K
Ideally: 413K --> 124K
Please see here for the reference:
https://www.tensorflow.org/lite/performance/model_optimization#quantization
from pinto_model_zoo.
I understand. In a normal model, it is reduced to a quarter of the size after quantization. However, the original size of the BlazeFace is too small, so the size reduction effect after quantization seems to be small.
from pinto_model_zoo.
The size of the mediapipe/blazeface model is 224 KB which is available here: https://github.com/google/mediapipe/blob/master/mediapipe/models/face_detection_front.tflite.
I believe this model is FP16, so it is around 1/2 of the FP32. My assumption may not be correct but Int8 quantization should reduce the size further. From where did you get the TFLite FP32 model? I could not find it at the mediapipe GitHub repo.
from pinto_model_zoo.
Also, here is an example which shows a very simple model conversion,
https://www.tensorflow.org/lite/performance/post_training_integer_quant
83424 --> 23368
So I guess size of the model does not matter.
from pinto_model_zoo.
The .tflite in the link you presented to me is the Float16 model. It's about the same size as the Float16 model I produced on hand.
From where did you get the TFLite FP32 model? I could not find it at the mediapipe GitHub repo.
The model on MediaPipe's official site was updated from Float32 to Float16 10 days ago. The Float32 model I downloaded 15 days ago is the one I downloaded.
Also, there is an example here which shows a very simple model conversion,
I'm not sure about the internal behavior of the TensorflowLite Converter as I'm performing the same procedure to quantize it.
from pinto_model_zoo.
memorandum of understanding
https://github.com/minus31/BlazeFace
It may be possible to perform the training from the beginning with pruning enabled in Keras.
from pinto_model_zoo.
I am also thinking to train it from scratch. Is training data available somewhere? Or, schema of the training data so that I can prepare my own.
from pinto_model_zoo.
The six key points (eyes, nose, ears, and mouth) do not seem to be detectable, but a training script with FDDB dataset is available. I'm exhausted from my day job, so I'll try it tomorrow.
https://github.com/vietanhdev/blazeface_keras
from pinto_model_zoo.
I have started training for BlazeFace by Keras. I'm going to try pruning after the 160 epoch is done.
from pinto_model_zoo.
Great! It would be interesting to see the final model size.
from pinto_model_zoo.
Pruning doesn't seem to work. 😢
from pinto_model_zoo.
That's sad. But I think pruning won't help in the RAM usage, it would only make the model size smaller for storage.
from pinto_model_zoo.
Related Issues (20)
- License of RAFT models HOT 3
- Script to convert RAFT models HOT 3
- Blazeface onnx model HOT 1
- BodyPix on MacOS - Dilation not supported for AutoPadType::SAME_UPPER or AutoPadType::SAME_LOWER HOT 8
- TOPK operator for RKNN export HOT 1
- InstructIR
- 064_Dense_Depth seems to have wrong dimensions HOT 1
- Midas2 model on coral edge TPU HOT 1
- Difference on model outputs (tflite, openvino IR, and Onnx) in model 227_face-detection-adas-0001 HOT 1
- bad results for 342_ALIKE HOT 1
- Aborted (core dumped) for full quantized tinyhitnet model
- 091_gaze-estimation-adas-0002 network HOT 1
- Release new 303_FAN with heatmaps HOT 3
- dataset HOT 1
- 410_FaceMeshV2 quantized tflite models are not functional HOT 1
- 053_BlazePose / 058_BlazePose_Full_Keypoints source HOT 3
- How to retrain simple MLP (palm_detection_full_inf_post_192x192.onnx) model with custom hand dataset ? HOT 2
- How to retrain simple MLP (palm_detection_full_inf_post_192x192.onnx) model with custom hand dataset ? HOT 2
- MiDaS conversion script to onnx
- Mismatch between BlendshapeV2 TensorFlow and TFLite Model
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pinto_model_zoo.