Comments (9)
nihui
commented on May 12, 2024
半精度计算现在的硬件还不支持，8bit实际只是量化存储，读出来还是单精度浮点数
from ncnn.
asurachqtan
commented on May 12, 2024
@nihui 我们目前想做的是计算转换为8bit来进行 不知道精度损失会有多少 正在实现中
from ncnn.
qfdong
commented on May 12, 2024
转换为8bit的网络，需要重新retraining一下可以提高model的精度，我也在做类似的工作，希望能多交流
from ncnn.
ccJia
commented on May 12, 2024
@qfdong 请问你们在做8bit的乘法的时候，溢出问题严重吗？怎么规避的？
from ncnn.
qfdong
commented on May 12, 2024
@ccJia 我们在做量化的时候，会统计每层的输入，输出，以及参数的最大值，会选取合适的bit数来保证没有溢出的现象发生，然而每一层的layer的计算过程中，的确有可能溢出，中间的乘法的结果是存储在int类型来避免溢出，你是在什么平台上开发？这个受限于指令集
from ncnn.
ccJia
commented on May 12, 2024
@qfdong 我是在V8上面搞的，我没有动态的划分BIT数，中间乘法也是转存了一下。
from ncnn.
quietsmile
commented on May 12, 2024
请问，8bit优化在armv7上(3516/3519)有加速空间么？@ccJia @qfdong @nihui
from ncnn.
BUG1989
commented on May 12, 2024
理论上耗时缩减为原来的25%
from ncnn.
nihui
commented on May 12, 2024
move to #230
from ncnn.
Related Issues (20)
- 在64核机器上跑ncnn模型，运行时只会调用其中一个核，设置哪些参数能跑指定数量的cpu核心 HOT 1
- mtcnn ncnn模型fp32转fp16为什么会段错误 HOT 8
- 用摄像头调用 yolov5 net.extractor时内存增长迅速 HOT 1
- MacOS下 new ncnn::Net() 崩溃 HOT 2
- Tile 算子似乎没有起作用
- yolov8训练完成的模型转成ncnn的模型后，推理不出结果，网上查了说要进行前后处理，确实不会，能发个cpp的例子看看么！ HOT 4
- ncnn-20240410使用-DNCNN_BF16=OFF编译报错 HOT 3
- rnn(lstm，gru)，解卷积的量化以及增加weight only的量化 HOT 1
- 可以给生成的pnnx.py中的pytorch model 增加初始化权重默认入参么，方便迁移到其他地方使用
- 预编译库中，希望能增加macOS下的动态库版本供下载
- [pnnx]:torch.clamp_min convert failed HOT 2
- 目前RISCV版本中用到的RVV intrinsic代码已经不是最新版本了，有升级riscv intrinsic代码的计划吗？
- Android下的build，为什么默认关闭exception呢？ HOT 5
- openmp冲突引起crash
- iPhone创建ncnn::net崩溃 HOT 2
- benchmark测试占用率低
- 手动创建的net，推理慢了很多
- yolov8n模型在鲲鹏ARM机器的检测结果和pytorch结果不一样 HOT 6
- 我有3个GPU，但get_gpu_count()=1 HOT 8
- pnnx和ncnn输出不一致
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ncnn.