Comments (11)
问题已经知悉,这似乎是上一次修改调度器逻辑留下的后遗症之一。
如你所见dispatcher.py 第125行处所执行的操作是强制激活函数与前序计算算子同平台,对于神经网络部署而言这是必须的。
我会尝试用一个补丁来修复上述问题,于此同时我认为你可以简单地注释掉125行附近的语句看看能不能运行成功。
对于efficientnet而言,我记得我们执行器算出来的结果好像跟onnx runtime的不太一样,虽然我还不知道是什么问题。
from ppq.
不对啊兄弟,我跑了一下这个模型,它跑的开开心心没啥毛病啊...
使用的代码如下所示:
from ppq import *
from ppq.api import *
d = [torch.rand(1, 224, 224, 3) for _ in range(32)]
quantize_onnx_model(
onnx_import_file='Models/cls_model/efficientnet-lite4-11.onnx',
calib_dataloader=d,
calib_steps=32,
input_shape=[1,224,224,3],
collate_fn=lambda x: x.to('cuda'),
)
from ppq.
额,这样的执行过程有涉及到ConvBN fuse吗(我不太了解PPQ细节haha),我的问题我发现是做了ConvBN fuse 会导致的
from ppq.
好像做了吧...不融合BN应该报别的错误了...
要不瞅一眼你所下载的ppq是否是最新的版本?或者就先直接注释掉那几行应该就好了,你这个网络我看了并不需要做复杂调度,全网上int8就行了。
from ppq.
害嗨,我改了一下代码,判断一下dispatching_table有没有source_op然后分情况处理就好了。
我又遇到另外一个raise Excption:quantization/observer/range.py 254行报的,说torch.quantile can not handle such many values,这是啥意思勒
from ppq.
对了,能叨扰你罗列一下目前能支持(能跑完量化)的模型吗😣
from ppq.
torch.quantile
torch.quantile
在 CPU 上有 tensor values 数量的限制,可以用 GPU 运行试试?或者换一下 observer 试试(
from ppq.
torch.quantile 函数只能处理16777216个数据的tensor,这个好像是pytorch自己写的有问题,这个函数会用来做percentile的observer,你using cuda kernel然后我们自己写了个quantile函数没有这个限制。然后正如a1trl9所说的,你也可以换一个observer,通过下列代码:
QUANT_SETTING = QuantizationSettingFactory.pplcuda_setting()
QUANT_SETTING.quantize_activation_setting.calib_algorithm = 'minmax'
from ppq.
👌👌
from ppq.
我们是按算子支持的哦,只要你的模型是由下列这些算子组成的,没有超出范围的算子就可以
'BatchNormalization_forward', 'Cast_forward', 'Clip_forward', 'Concat_forward',
'Constant_forward', 'ConstantOfShape_forward', 'Conv_forward', 'Eltwise_forward', 'Equal_forward',
'UnaryEltwise_forward', 'Expand_forward', 'Flatten_forward', 'Gather_forward', 'GatherND_forward', 'Gemm_forward',
'Grid_sampler_forward', 'AveragePool_forward', 'Greater_forward', 'Less_forward', 'MatMul_forward',
'MaxPool2d_forward', '_NMS_forward', 'NonZero_forward', 'Not_forward', 'Range_forward',
'ReduceL2_forward', 'ReduceMax_forward', 'Reshape_forward', 'Resize_forward', 'ScatterElements_forward',
'ScatterND_forward', 'Shape_forward', 'Slice_forward', 'Softmax_forward', 'Squeeze_forward', 'Tile_forward',
'TopK_forward', 'Transpose_forward', 'Unsqueeze_forward', 'Where_forward', 'ReduceSum_forward', 'ArgMax_forward',
'Split_forward', 'ReduceMean_forward', 'PRelu_forward', 'Pad_forward', 'LeakyRelu_forward', 'ConvTranspose_forward',
'Sqrt_forward', 'Log_forward', 'Floor_forward', 'RoiAlign_forward', 'MMCVRoiAlign_forward', 'SpaceToDepth_forward',
'DepthToSpace_forward', 'Tanh_forward', 'Pow_forward', 'Crop_forward', 'ChannelShuffle_forward',
'InstanceNormalization_forward', 'Parameter_forward', 'Interp_forward', 'CaffeArgMax_forward'
不管你中间怎么连接的这些算子,ppq都可以正常量化(应该吧),对于复杂的模型ppq也会自动切图调度。
然后onnx model zoo里面有些模型有loop算子或者 if算子,这种我们不支持,后端也大概率支持不了
总体来说就是我们能量化的网络肯定比后端能跑的多,但这些网络量化出来并没有什么价值。
你需要避免网络中出现那些看起来就很不可思议的算子,比如swish, elu, sigmoid, tanh, if, loop, lstm, gru, nms, resize,后端的支持能力比我们弱得多,你的网络要真正能部署在硬件上才有价值哦。
from ppq.
好的,感谢👍👍👍
from ppq.
Related Issues (20)
- 请问如何使用ppq量化模型的部分模块,而不是整个模型呢?
- Op Execution Error: Einsum HOT 1
- yolov5量化,在ppq/samples/yolo中运行02_Quantization.py文件出现警告和报错,输出模型只有onnx没有engine文件 HOT 1
- 请问PPQ支持一维网络模型的量化吗 HOT 1
- [视频资料]使用profiler来分析网络性能,tensorboard无法解析日志文件 HOT 2
- PPQint8CPU量化bug HOT 1
- Op Execution Error MMCVModulatedDeformConv2d
- 关于QAT中的BN? HOT 1
- 是否有计划支持高通QNN(比较新的版本)和海思芯片呢? HOT 1
- ppq使用torch_mlir导出tosa格式的mlir HOT 1
- 这个工具链已经收敛了吗?我看已经比较久没有特性更新了,后续还有新特性计划吗? HOT 1
- 能否支持权重和激活位宽不同的量化? HOT 2
- VIT类模型量化感知训练loss跑飞 HOT 1
- PPQ已经支持了不少后端推理框架(TensorRT,ncnn,mnn等),这些后端的量化规格细节都是怎么获取的呢?现在PPQ与后端量化推理可以完全对齐吗?
- PPQ在画质类任务上(如低光降噪)量化效果咋样呢?
- AttributeError: 'NoneType' object has no attribute 'copy'
- 求助 为什么量化以后模型没有减小?
- ProgramEntrance_2.py读取imagenet图片reshape报错
- RuntimeError: Op Execution Error: /Slice(Type: Slice, Num of Input: 5, Num of Output: 1)
- Quantization for tflite HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ppq.