Comments (4)
目前ppq仅支持离线量化(ptq)。但集成了一些“量化训练"算法比如lsq,brecq,advanced optimize等方法,区别于qat,上述算法仅仅在标定集上进行blockwise的微调,不会整个模型进行梯度反传训练,其优化效果也是不错的,帮助我们解决了不少模型掉点。
暂未支持qat的原因:
- 数据隐私,无法获取整个数据集进行qat训练;面对批量的快速部署需求,采用ptq部署效率更高;
- 当前硬件推理库大都仅支持8比特量化,ptq可满足绝大部分需求;
- 面对"难量化"的模型掉点,更重要的是定位量化误差的来源(后端图融合?算子联合定点?激活值或者weight异常值等),否则无脑qat的话,效果可能不尽如人意。ppq花了很多时间在图融合,联合定点的量化模拟,希望对你有帮助。
from ppq.
You may also have some interests to this one: https://github.com/intel/neural-compressor/tree/master/examples. Just FYI.
from ppq.
You may also have some interests to this one: https://github.com/intel/neural-compressor/tree/master/examples. Just FYI.
thanks...
from ppq.
目前ppq仅支持离线量化(ptq)。但集成了一些“量化训练"算法比如lsq,brecq,advanced optimize等方法,区别于qat,上述算法仅仅在标定集上进行blockwise的微调,不会整个模型进行梯度反传训练,其优化效果也是不错的,帮助我们解决了不少模型掉点。 暂未支持qat的原因:
- 数据隐私,无法获取整个数据集进行qat训练;面对批量的快速部署需求,采用ptq部署效率更高;
- 当前硬件推理库大都仅支持8比特量化,ptq可满足绝大部分需求;
- 面对"难量化"的模型掉点,更重要的是定位量化误差的来源(后端图融合?算子联合定点?激活值或者weight异常值等),否则无脑qat的话,效果可能不尽如人意。ppq花了很多时间在图融合,联合定点的量化模拟,希望对你有帮助。
非常感谢您的回复,我后面仔细学习了解下该项目。
from ppq.
Related Issues (20)
- vision transformer量化模型 HOT 1
- 如何将yolov8n-seg进行int8量化. onnx2trtint8
- 回归模型相关咨询 HOT 2
- AttributeError: ORT_OOS_INT8 HOT 1
- LSQ算法微调后,输出的scale量化参数存在负值,请问如何修改源码,使其输出的scale为正值? HOT 1
- PPQ installtion error: `TypeError: Descriptors cannot be created directly. If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.` HOT 5
- PPQ yolov5-7.0 :return torch.cat(concat_view, axis=axis) RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 28 but got size 27 for tensor number 1 in the list. HOT 1
- yolov8量化后无结果 HOT 3
- YOLOv6量化后仅能预测一类 HOT 3
- 能不能量化自己的模型啊?有教程吗 HOT 1
- 内存泄露 HOT 3
- yolov5 模型量化后的精度损失 HOT 2
- QLinearMatmul : weight zero point must be a scalar, 1D tensor of size 1, or last to second dimension is 1
- 量化时出现RandomNormalLike这个算子没有注册跟实现的问题,请问有没有简单的注册实现例子可以参考? HOT 1
- NotImplementedError: Graph op: /quantize/ArgMin(ArgMin) has no backend implementation HOT 2
- 请问model zoo的模型在哪里? HOT 1
- 调度部分算子至FP32时,生成engine失败 HOT 2
- Op Execution Error: PRelu_1 HOT 4
- 关于resize op的一个bug HOT 1
- 量化模型的时候出现TypeError HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ppq.