rockchip-linux / rknpu2 Goto Github PK
View Code? Open in Web Editor NEWLicense: BSD 3-Clause "New" or "Revised" License
License: BSD 3-Clause "New" or "Revised" License
rk3588+linux 想要使用rknntoolkit进行调试 无法用adb连接linux系统的板子
(QUANTIZE_ON = True时输出的box坐标有值但所有框和类别的置信度都为0)
//输入格式为uint8
cv::Mat orig_img = cv::imread(image_name, 1);
cv::Mat img;
cv::cvtColor(orig_img, img, cv::COLOR_BGR2RGB);
inputs[0].type = RKNN_TENSOR_UINT8;
inputs[0].size = width * height * channel;
inputs[0].buf = (void*)img.data;
//输入格式为float32
cv::Mat orig_img = cv::imread(image_name, 1);
cv::Mat img;
cv::cvtColor(orig_img, img, cv::COLOR_BGR2RGB);
img.convertTo(img, CV_32F, 1.0 / 255, 0);
inputs[0].type = RKNN_TENSOR_FLOAT32;
inputs[0].size = width * height * channel * 4;
inputs[0].buf = (void*)img.data;
ret = rknn_run(ctx, NULL)
时报同一个错误post process config: box_conf_threshold = 0.50, nms_threshold = 0.60
Read model/000657.jpg ...
img width = 640, img height = 480
Loading mode...
sdk version: 1.3.0 (c193be371@2022-05-04T20:16:33) driver version: 0.6.4
model input num: 1, output num: 1
index=0, name=image_arrays, n_dims=4, dims=[1, 640, 640, 3], n_elems=1228800, size=1228800, fmt=NHWC, type=INT8, qnt_type=AFFINE, zp=-128, scale=0.003922
index=0, name=outputs, n_dims=4, dims=[1, 8400, 85, 1], n_elems=714000, size=714000, fmt=NCHW, type=INT8, qnt_type=AFFINE, zp=-128, scale=2.498195
model is NHWC input fmt
model input height=640, width=640, channel=3
resize with RGA!
failed to open RGA:Permission denied.
rga_api version 1.6.0_[4] (d041200 build: 2021-12-01 15:26:14 base: )
rga_im2d: The current RockchipRga singleton is destroyed. Please check if RkRgaInit/RkRgaDeInit are called, if so, please disable them.
rga im2d: rga2 get info failed!
242, check error! Fatal error: The current RockchipRga singleton is destroyed.Please check if RkRgaInit/RkRgaDeInit are called, if so, please disable them.
rknn_query
从模型获得的信息:
input tensors:
index=0 name=x.1_0 n_dims=4 dims=[1 3 640 640] n_elems=1228800 size=1228800 fmt=0 type=3 qnt_type=2 fl=0 zp=0 scale=1.000000
output tensors:
index=0 name=permute_at_1747_221_out0_223 n_dims=5 dims=[3 80 80 11] n_elems=211200 size=211200 fmt=0 type=3 qnt_type=2 fl=-29 zp=227 scale=0.598226
index=1 name=permute_at_1770_230_out0_224 n_dims=5 dims=[3 40 40 11] n_elems=52800 size=52800 fmt=0 type=3 qnt_type=2 fl=-72 zp=184 scale=0.588694
index=2 name=permute_at_1793_239_out0_225 n_dims=5 dims=[3 20 20 11] n_elems=13200 size=13200 fmt=0 type=3 qnt_type=2 fl=-60 zp=196 scale=1.220445
设置输出自动量化:
// set output
outputs = new rknn_output [io_num.n_output];
memset(outputs, 0, sizeof(outputs));
for (int i = 0; i < io_num.n_output; i++)
{
outputs[i].want_float = 1;
}
之后
cout << outputs[0].size << endl;
cout << outputs[1].size << endl;
cout << outputs[2].size << endl;
结果输出
844800
1163544915
5394255
如果输出float32的数据,第一个844800是可以理解,但后面两个是怎么回事?
另外,如果outputs[i].want_float = 1;
的话就会出现Segmentation fault (core dumped)
报错,是因为非量化输出不支持.size
参数吗?
谢谢
I think RK3566/RK3568 is more than capable of running python. It would be great if this platform can provide the access to all resources likes camera, display, and NPU all in python.
The works to do are pretty much wrapping the C APIs defined below into python.
https://github.com/rockchip-linux/rknpu2/blob/master/Linux/librknn_api/include/rknn_api.h
使用 NPU SDK 1.3.0 在 3588S上推理时 rknn_query 竟然报错:
E RKNN: [03:53:15.041] rknn_query, info_len(372) != sizeof(rknn_tensor_attr)(376)!
Loading RKNN Model ...
Initializing RKNN Envs ...
sdk version: 1.3.0 (c193be371@2022-05-04T20:16:33) driver version: 0.6.4
Querying Model Attributes ...
model input num: 1, output num: 1
sizeof(this->input_attrs[0]): 372
sizeof(rknn_tensor_attr): 372
E RKNN: [03:53:15.041] rknn_query, info_len(372) != sizeof(rknn_tensor_attr)(376)!
RKNN_QUERY_INPUT_ATTR error ret=-5
报错部分代码如下:
// - I/O Attributes
memset(this->input_attrs, 0, sizeof(rknn_tensor_attr));
memset(this->output_attrs, 0, sizeof(rknn_tensor_attr));
cout << "\nsizeof(this->input_attrs[0]): " << sizeof(this->input_attrs[0]) << endl;
cout << "sizeof(rknn_tensor_attr): " << sizeof(rknn_tensor_attr) << "\n" << endl;
this->input_attrs[0].index = 0;
ret = rknn_query(this->ctx, RKNN_QUERY_INPUT_ATTR, &(this->input_attrs[0]), sizeof(rknn_tensor_attr)); // Single I/O YOLOX
if (ret < 0)
{
printf("RKNN_QUERY_INPUT_ATTR error ret=%d\n", ret);
return;
}
this->output_attrs[0].index = 0;
ret = rknn_query(this->ctx, RKNN_QUERY_OUTPUT_ATTR, &(this->output_attrs[0]), sizeof(rknn_tensor_attr)); // Single I/O YOLOX
if (ret < 0)
{
printf("RKNN_QUERY_OUTPUT_ATTR error ret=%d\n", ret);
return;
}
模型和测试代码在TB-RK3399PRO-D上运行顺利,迁移过来后模型使用RKToolkit2重新转换过,不能理解为什么两个输出的 sizeof(rknn_tensor_attr)
不一致,麻烦大神们解答,谢谢!!
10-25 06:32:13.744 0 0 W : invalid destination resolution act_w = 120, act_h = 0
10-25 06:32:13.744 0 0 E rga2 : req argument is inval
10-25 06:32:13.639 2312 2312 E rkyolo4j: colorConvert failed. Ret: Fatal error: Failed to call RockChipRga interface, query log to find the cause of failure.
10-25 06:32:13.767 0 0 W : invalid destination resolution act_w = 120, act_h = 46080
10-25 06:32:13.767 0 0 E rga2 : req argument is inval
10-25 06:32:13.650 2312 2367 E rkyolo4j: run_yolo: resize image with rga failed: Fatal error: Failed to call RockChipRga interface, query log to find the cause of failure.
10-25 06:32:13.777 0 0 W : invalid destination resolution act_w = 120, act_h = 0
10-25 06:32:13.777 0 0 E rga2 : req argument is inval
10-25 06:32:13.675 2312 2312 E rkyolo4j: colorConvert failed. Ret: Fatal error: Failed to call RockChipRga interface, query log to find the cause of failure.
10-25 06:32:13.802 0 0 W : invalid destination resolution act_w = 120, act_h = 46080
10-25 06:32:13.802 0 0 E rga2 : req argument is inval
10-25 06:32:13.685 2312 2367 E rkyolo4j: run_yolo: resize image with rga failed: Fatal error: Failed to call RockChipRga interface, query log to find the cause of failure.
10-25 06:32:13.812 0 0 W : invalid destination resolution act_w = 120, act_h = 0
10-25 06:32:13.812 0 0 E rga2 : req argument is inval
10-25 06:32:13.711 2312 2312 E rkyolo4j: colorConvert failed. Ret: Fatal error: Failed to call RockChipRga interface, query log to find the cause of failure.
10-25 06:32:13.838 0 0 W : invalid destination resolution act_w = 120, act_h = 46080
10-25 06:32:13.838 0 0 E rga2 : req argument is inval
model input num: 4, output num: 1
input tensors:
index=0 name=input_imgs n_dims=4 dims=[12 256 128 1] n_elems=393216 size=786432 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
index=1 name=desire n_dims=2 dims=[0 0 8 1] n_elems=8 size=16 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
index=2 name=traffic_convention n_dims=2 dims=[0 0 2 1] n_elems=2 size=4 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
index=3 name=initial_state n_dims=4 dims=[512 1 1 1] n_elems=512 size=1024 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
output tensors:
index=0 name=outputs n_dims=2 dims=[0 0 6409 1] n_elems=6409 size=12818 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
E RKNN: [07:24:10.076] Meet unsupported split
Aborted
请问 RK356X 的 rknpu2 的 npu 驱动 galcore.ko 之类 哪里获取。rknpu [https://github.com/rockchip-linux/rknpu/tree/master/drivers/npu_ko] 就提供相对应的驱动。
Is it possible to load more than one model at the same time and run inference on these models in sequence (pipeline) or in parallel (multitasking)?
跑的简单图像超分辨率模型,模拟器结果正确,NPU结果差别很大,用的一样的代码和模型,分析得到的余弦距离都是0.99以上
麻烦问下rk3568有没有yolo得demo,可以在此提供一下么?
./build-android.sh
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/Desktop/rknpu2-master/examples/rknn_ssd_demo/build/build_android_v8a
[ 33%] Linking CXX executable rknn_ssd_demo
/opt/tools/rk/android-ndk-r17//toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld:/home/ubuntu/Desktop/rknpu2-master/examples/rknn_ssd_demo/../../Android/librknn_api/arm64-v8a/librknn_api.so: file format not recognized; treating as linker script
/opt/tools/rk/android-ndk-r17//toolchains/aarch64-linux-android-4.9/prebuilt/linux-x86_64/bin/../lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin/ld:/home/ubuntu/Desktop/rknpu2-master/examples/rknn_ssd_demo/../../Android/librknn_api/arm64-v8a/librknn_api.so:1: syntax error
collect2: error: ld returned 1 exit status
CMakeFiles/rknn_ssd_demo.dir/build.make:158: recipe for target 'rknn_ssd_demo' failed
make[2]: *** [rknn_ssd_demo] Error 1
CMakeFiles/Makefile2:67: recipe for target 'CMakeFiles/rknn_ssd_demo.dir/all' failed
make[1]: *** [CMakeFiles/rknn_ssd_demo.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
报错内容提示:failed to open rknpu module,need to insmod rknpu dirver failed to open rknn device
但是在官网更新了最新的3588J的驱动之后,还是会报这个错误,所使用的demo为这个demo,所使用的模型是这个仓库里面的模型https://github.com/rockchip-linux/rknn-toolkit2/tree/master/examples/onnx/yolov5
所使用的固件是这个网站上的ubuntu固件
https://www.t-firefly.com/doc/download/161.html
我需要在x86 pc上重写数据预处理和后处理代码,但是由于没有对应的x86api,我该怎么调试自己的代码以保证正确性呢?
代码:
int main() {
const char *model_path = "/home/white/CLionProjects/yolo5s/model/yolox.rknn";
const char *img_path = "/home/white/Documents/dataset/1716/VOCdevkit/VOC2007/JPEGImages/000002.jpg";
std::cout<<model_path<<std::endl;
rknn_context ctx;
int ret;
int model_len = 0;
unsigned char *model;
model = load_model(model_path, &model_len);
// ret = rknn_init(&ctx,model,model_len,0);
rknn_init(&ctx, nullptr, 0, 0);
std::cout<<ret<<std::endl;
std::cout << "Hello, World!" << std::endl;
return 0;
}
报错信息:
undefined reference to `rknn_init'
collect2: error: ld returned 1 exit status
Hi do I need to modprobe anything? What driver is needed for this? thanks!
模型都是用rknntoolkit2-1.3版本转换过来的,python版本的运行正常,C++版本的在初始化时遇到这个问题,与这个问题一样的问题#20
官方有解决的办法吗?不知道是什么原因。但是单独打印rknn_tensor_attr的size却是372。
E RKNN: [14:57:27.201] rknn_query, info_len(372) != sizeof(rknn_tensor_attr)(376)!
rknn_init error ret=-5
index=0, name=, n_dims=0, dims=[0, 0, 0, 0], n_elems=0, size=0, fmt=NCHW, type=FP32, qnt_type=NONE, zp=0, scale=0.000000
sizeof_input_attrs[0]): 372
sizeof(rknn_tensor_attr): 372
model input num: 4, output num: 1
input tensors:
index=0 name=input_imgs n_dims=4 dims=[12 256 128 1] n_elems=393216 size=786432 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
index=1 name=desire n_dims=2 dims=[0 0 8 1] n_elems=8 size=16 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
index=2 name=traffic_convention n_dims=2 dims=[0 0 2 1] n_elems=2 size=4 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
index=3 name=initial_state n_dims=4 dims=[512 1 1 1] n_elems=512 size=1024 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
output tensors:
index=0 name=outputs n_dims=2 dims=[0 0 6409 1] n_elems=6409 size=12818 fmt=0 type=1 qnt_type=2 fl=0 zp=0 scale=1.000000
E RKNN: [07:24:10.076] Meet unsupported split
Aborted
I downloaded latest rknpu2 and rknn-toolkit2 and run on rk3588 board. And I got the above error. I saw the same issue published but still left open. And there is no fix in official release. Please help!
发现rk3568 , 偶发推理不一致 (2g内存rk3568板子)
感觉似乎是在内存比较吃紧的时候, 启动推理, 确定推理的输入是一致的,
但是输出是会偶发出现推理不一致, 而且一旦出现推理不一致的情况, 后面的推理 75%的会出错。
推理不是完全错误的, 错误的推理里面 有一半的结果是正确的, 但是会多出几个错误的检测框
注: 可以多连几个vscode, vscode非常消耗系统内存
./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn model/bus.jpg
执行代码提示
./rknn_yolov5_demo: error while loading shared libraries: librga.so.2: cannot open shared object file: No such file or directory
文档中有librga.so文件的说明,请问librga.so.2这个文件如何获取?
在rknn_inputs_set处出现错误,
ret = rknn_inputs_set(ctx, io_num.n_input, inputs);
128800是640,640,3的tensor,而4915200是12800,12800,3的tensor,我的rknn模型在连板调试中使用Python推理正确,且输入确实是640的,但是这里提醒我模型期望输入却是4915200(1280,1280,3),不知道为什么,当我讲图片resize成1280后,又出现了新的错误,如下图,我想知道函数rknn_inputs_set中返回的模型期望输入是从哪里计算出来的?因为即使 ret = rknn_query(ctx, RKNN_QUERY_INPUT_ATTR, &(input_attrs[i]), sizeof(rknn_tensor_attr));查询模型输入,返回的也依旧是640,640,3
Hi, there. I was following the document doc/Rockchip_Quick_Start_RKNN_SDK_V1.3.0_CN.pdf
and installed rknn
through pip. However, I got the following when running python3 test.py
E build: Catch exception when building RKNN model!
E build: Traceback (most recent call last):
E build: File "rknn/api/rknn_base.py", line 1680, in rknn.api.rknn_base.RKNNBase.build
E build: File "rknn/api/rknn_base.py", line 363, in rknn.api.rknn_base.RKNNBase._generate_rknn
E build: File "rknn/api/rknn_base.py", line 272, in rknn.api.rknn_base.RKNNBase._build_rknn
E build: ImportError: /lib/x86_64-linux-gnu/libm.so.6: version `GLIBC_2.29' not found
模型转换:rknn-toolkit2 version: 1.3.0-11912b58
最小可复现网络:
class InstanceModel(nn.Module):
def init(self, channels):
super().init()
self.ins = nn.InstanceNorm2d(channels)
def forward(self, inputs):
out = self.ins(inputs)
return out
转换后模型:https://drive.google.com/file/d/1MGvJ9SHaY0KDKaMEf6XkhUiROLJk3HTv/view?usp=sharing
The RKNN runtime in this repo is still in version 1.0.0. It can't load model files generated by current rknn-toolkit2
, which is 1.1.0.
I saw rknpu2
version 1.1.0 is actually available in other places. e.g. https://gitee.com/caesar-wang/rknpu2
Please keep this repo up-to-date. Thank you.
Now that NPU is under Rockchip IP, it would be helpful to open sorce RKNN and Toolkit for devs to fix bugs and add OPs not coverd by vanilla models like yolo and resnet.
Hello,
I'm trying to inference YOLOv4 .rknn converted from .onnx with the help of rknn-toolkit2. It performs well on PC using test.py code from toolkit examples but fails on AIO-3568J using rknpu .cc code. It returns error during inference (rknn.run):
E RKNN: [02:49:39.674] failed to submit!, op id: 227, op name: OutputOperator:139_convolutional, flags: 0x1, task start: 0, task number: 2583, run task counter: 2322, int status: 0
Tried to convert original darknet model to .rknn but received the same mistake. Have anyone tried to inference YOLOv4 rknn model before?
This copyright looks wrong for a file in a public repository:
/****************************************************************************
*
*****************************************************************************/
https://github.com/rockchip-linux/rknpu2/blob/master/runtime/RK3588/Linux/librknn_api/include/rknn_api.h
Please update/fix!
3rdparty/opencv/opencv-linux-aarch64/lib/目录下没有提供 libopencv_highgui.a
Hi, this repo supplies runtime. Is there linux driver, whose file extension is .ko modules for rk356x?
I use the modified version of the yolov program to train my model and the results are very good, but after converting to rknn, the recognition output will be out of the box. Someone suggested to me because the output dimension of my onnx model is 4 dimensions, and the official one is 5 dimensional, after that I found that when exporting the onnx model without adding the --ignore_output_permute parameter the output is 5 dimensional. But after converting the model, the frame is still messed up in this demo, and there are a lot of coordinates output. Please how can I solve this problem
如题,仓库中rknpu2/runtime/RK3588/Linux/里面的东西都是3588的?3588的rknn_server 是升级到了1.40版本,但是sdk怎么版本对不上啊,提交信息中写的是Update librknnrt.so: fix compatibility of RK356X,明明是RK356X的不是3588的吧?找了一圈只找到了armhf的1.40版本的sdk,还是在rknpu2/runtime/RV1106/Linux)/librknn_api/里面看到的,我手上的是firefly itx3588j这个能用吗?我记得应该是aarch64的才对呀
We tried to inference ssd_mobilenet _v1 model with C++ inference on Rk3566 board. But got this error
Loading model ...
rknn_init ...
I RKNN: set log level to 0
E RKNN: Invalid RKNN format
E RKNN: rknn_query, load_model failed
rknn_init fail! ret=-1
Tried to search the error code it said its Model Execution error, but not able to understand why its not working, because same model worked with python inference on ubuntu 16.04.
想读取视频应该怎么解决
RK3566
Android 11
RAM 2G
运行到rknn_init 时候直接崩溃 运行代码
int HEADDetect::LoadRKNN(const char* cpath)
{
FILE *fp = fopen(cpath, "rb");
if (fp == NULL) {
printf("fopen %s fail!\n", cpath);
return -1;
}
fseek(fp, 0, SEEK_END);
int model_len = ftell(fp);
model = malloc(model_len);
fseek(fp, 0, SEEK_SET);
if (model_len != fread(model, 1, model_len, fp)) {
printf("fread %s fail!\n", cpath);
free(model);
return -1;
}
int ret = 0;
//ret = rknn_init(&rknn_ctx, model, model_len, RKNN_FLAG_PRIOR_MEDIUM | RKNN_FLAG_COLLECT_PERF_MASK);
ret = rknn_init(&rknn_ctx, model, model_len, RKNN_FLAG_PRIOR_HIGH, NULL);
if (ret < 0)
{
printf("rknn_init fail! ret=%d\n", ret);
// goto Error;
}
free(model);
fclose(fp);
printf("rknn_init ret=%d\n", ret);
return ret;
}
手上有个RK3568,做yolov5的算法移植,NPU驱动版本为0.4.2,怎么查询NPU利用率?
linaro@linaro-alip: ~/rknn_yolov5_demo_Linux$ sudo ./rknn_yolov5_demo model/RK356X/yolov5s-640-640.rknn model/bus.jpg
Rga built version:274b345
post process config: box_conf_threshold = 0.50, nms_threshold = 0.60
Read model/bus.jpg ...
img width = 640, img height = 640
Loading mode...
Message from syslogd@linaro-alip at Mar 9 09:14:06 ...
kernel:[ 1150.930469] Internal error: Oops: 96000046 [#3] SMP
Message from syslogd@linaro-alip at Mar 9 09:14:07 ...
kernel:[ 1150.935380] Process rknn_yolov5_dem (pid: 1446, stack limit = 0x00000000aeaa45cd)
Message from syslogd@linaro-alip at Mar 9 09:14:07 ...
kernel:[ 1151.760869] Code: a8c12027 a8c12829 a8c1302b a8c1382d (a88120c7)
文档中https://github.com/rockchip-linux/linux-rga
网址不可用,请及时更新文档
之前你们用芯原的NPU IP的时候可以用他们提供的 OpenVX 与 TIM-VX 以对接 Apache TVM 等第三方 DNN 部署框架,现在 RK35xx 系列换成瑞芯微自研 NPU 了,就只能用闭源的 RKNN 了,没办法自己扩展。请问你们未来有开发算子库与支持开放标准的计划吗?
Hi! I tried use flag RKNN_FLAG_ASYNC_MASK
, but it doesn't affec to rknn_run
.
Have you any example? Or is it not supported in rknpu2
?
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.