Is it possible to get a galcore.ko version compiled to work with kernel 4.19?

模型是IN8量化后的RVM模型
报错信息如下：
2023-03-23 16:49:33.057 5433-5457/com.baidu.paddle.lite.demo.rk_bg_matting E/RKNN: failed to submit!, op id: 29, op name: exDataConvert:428__cvt_float16_int8, flags: 0x5, task start: 15, task number: 48, run task counter: 39, int status: 0

想请问一下这个是什么问题，以及怎么解决

rknn_query从模型获得的信息：

input tensors:
index=0 name=x.1_0 n_dims=4 dims=[1 3 640 640] n_elems=1228800 size=1228800 fmt=0 type=3 qnt_type=2 fl=0 zp=0 scale=1.000000
output tensors:
index=0 name=permute_at_1747_221_out0_223 n_dims=5 dims=[3 80 80 11] n_elems=211200 size=211200 fmt=0 type=3 qnt_type=2 fl=-29 zp=227 scale=0.598226
index=1 name=permute_at_1770_230_out0_224 n_dims=5 dims=[3 40 40 11] n_elems=52800 size=52800 fmt=0 type=3 qnt_type=2 fl=-72 zp=184 scale=0.588694
index=2 name=permute_at_1793_239_out0_225 n_dims=5 dims=[3 20 20 11] n_elems=13200 size=13200 fmt=0 type=3 qnt_type=2 fl=-60 zp=196 scale=1.220445

设置输出自动量化:

    // set output
    outputs = new rknn_output [io_num.n_output];
    memset(outputs, 0, sizeof(outputs));
    for (int i = 0; i < io_num.n_output; i++)
    {
        outputs[i].want_float = 1;
    }

之后

cout << outputs[0].size << endl;
cout << outputs[1].size << endl;
cout << outputs[2].size << endl;

结果输出

844800
1163544915
5394255

如果输出float32的数据，第一个844800是可以理解，但后面两个是怎么回事？
另外，如果outputs[i].want_float = 1;的话就会出现Segmentation fault (core dumped)报错，是因为非量化输出不支持.size参数吗？
谢谢

I tried to run rknpu/rknn/rknn_api/examples/rknn_yolov5_demo on RK3399Pro (Tinker Edge R) with rknpu driver 1.7.1 installed from this repo.
However, as noted on README.md, I got following error messages:
Failed to open device: No such file or directory, Try again...
_OpenDevice(1091): FATAL: Failed to open device, errno=No such file or directory

Then, I found that the user mode library for RK3399pro is different from this repo. So I downloaded "rknn_api.h" and "librknn_api.so" from "RKNPUTools" and compiled rknn_yolov5_demo.

However, this time, the program crashed with the following messages:
E RKNNAPI: rknn_init, msg_load_ack fail, ack = 1 (ACK_FAIL), expect 0 (ACK_SUCC)!
E RKNNAPI: RKNN VERSION:
E RKNNAPI: API: 1.6.1 (00c4d8b build: 2021-03-15 16:31:37)
E RKNNAPI: DRV: 1.4.0 (b4a8096 build: 2020-09-14 11:15:57)
rknn_init error ret = -6

Seeing that the latest commit history from RKNPUTools is for 1.6.1, I believe that the user mode library for RK3399Pro for 1.7.1 is not yet supported?

If I am right, do you have any plan for the API release?
If not, is there any way to run yolov5 with NPU from lower versions such as 1.6.0?

Thank you in advance.

在rk1808计算棒上更新了npu驱动到1.7.1版本，并重启计算棒后执行rknn_server进程，显示如下：

# ps -A | grep rknn
  438 ?        00:00:00 start_rknn.sh
  440 ?        00:00:00 rknn_server
# rknn_server
start rknn server, version:1.7.1 (0cfd4a1 build: 2021-12-10 09:43:11)
1 NNPluginManager loadPlugins(41): No plugin find!
I NPUTransfer: Starting NPU Transfer Server, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)

我再ubuntu20.04主机上远程调用rk1808计算棒上的rknn的sdk，出现如下问题：

9731a2c72af5:python3 -u /opt/project/examples/tflite/mobilenet_v1/tflite_mbv1_cls_test.py
--> config model
done
--> Loading model
W The target_platform is not set in config, using default target platform rk1808.
done
--> Building model
W:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py:1034: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
2022-02-14 02:44:30.259587: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1483] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
W:tensorflow:From /usr/local/lib/python3.6/dist-packages/rknn/api/rknn.py:278: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
done
--> Export RKNN model
done
--> Init runtime environment
* daemon not running; starting now at tcp:5037
* daemon started successfully
error: device 'TM018083200400203' not found
E Connect to Device Failure (-1)
E Catch exception when init runtime!
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 868, in rknn.api.rknn_base.RKNNBase.init_runtime
E   File "rknn/api/rknn_runtime.py", line 171, in rknn.api.rknn_runtime.RKNNRuntime.__init__
E   File "rknn/api/rknn_platform_utils.py", line 318, in rknn.api.rknn_platform_utils.start_ntp_or_adb
E Exception: Init runtime environment failed!
E Please feedback the detailed log file <log_feedback_to_the_rknn_toolkit_dev_team.log> to the RKNN Toolkit development team.
E You can also check github issues: https://github.com/rockchip-linux/rknn-toolkit/issues
E Current device id is: TM018083200400203
E Devices connected:
Init runtime environment failed
E []

Process finished with exit code 255

请教一下这是什么原因导致的？如何解决？谢谢。

RK1808 NPU理论性能为3TOPS，但是自己用TIM-VX测试出来只有几个GOPS，是否可以提供测试NPU性能的demo？

按文档更新了NPU驱动，报了个错误
[ 1] HAL user version 6.4.3.5.293908
[ 2] HAL kernel version 6.4.3.5.293908
Segmentation fault (core dumped)
查文档是由于用户态驱动没有更新

宿主机ubuntu18.04系统登录rk1808计算棒后，通过如下指令对固件进行升级

sudo apt update -y
sudo  apt upgrade -y
sudo reboot

rk1808重启后的npu驱动是1.7.0，这个时候宿主机还可以通过ssh远程登录计算棒，同时也可以实现与宿主机实现网络共享。
然后通过如下指令升级计算棒npu驱动

adb push drivers/linux-aarch64/usr   /
adb push drivers/npu_ko/galcore.ko /lib/modules/galcore.ko

rk1808重启后的npu驱动是1.7.1，但发现宿主机无法通过ssh远程登录计算棒，指示灯也不亮了，
请教一下是什么原因导致的？如何解决？谢谢。

我的开发版是Tinker Edge R 当前运行系统为官方的debain V2.0.5
需要说明的是我升级的是 [email protected]:airockchip/RK3399Pro_npu.git这个链接下的boot.img
重装系统后我直接使用的是RKNN1.7.3版本，如果没有更新boot.img运行测试软件发现没有问题可以正常使用，但是我跟新boot.img重启后运行信息如下：

`linaro@linaro-alip:~/Desktop/yolov5-1.7.3$ python3 test.py
librga:RGA_GET_VERSION:3.02,3.020000
ctx=0x5af51a0,ctx->rgaFd=3
Rga built version:7bf9abf

---

all device(s) with ntb mode:
e4d8618710babd94

---

---

Target platforms filled in RKNN model: ['RK3399PRO']
Target platforms supported by this RKNN model: ['RK1808', 'RK1806', 'RK3399PRO']

---

Load RKNN model done
--> Init runtime environment
current target is rk3399pro
I NPUTransfer: Starting NPU Transfer Client, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
E RKNNAPI: rknn_init, recv(MsgLoadAck) fail, -9(ERROR_PIPE) != 368!
E Catch exception when init runtime!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 936, in rknn.api.rknn_base.RKNNBase.init_runtime
E File "rknn/api/rknn_runtime.py", line 341, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
E Exception: RKNN init failed. error code: RKNN_ERR_DEVICE_UNAVAILABLE
E Please feedback the detailed log file <log_feedback_to_the_rknn_toolkit_dev_team.log> to the RKNN Toolkit development team.
E You can also check github issues: https://github.com/rockchip-linux/rknn-toolkit/issues
E Runtime environment is not inited, please call init_runtime to init it first!
None
Init runtime environment failed`

我实在是不知道该怎么解决，我不用更新是不是也能正常使用？请指教一下？

如果使用rknn-toolkit的example文件夹下的代码去转换YOLOv5的Pytorch模型应该具体怎么操作呢？

请问瑞芯微官方有 librknn_runtime version 1.7.0的库吗

当我使用了预分配的内存作为输出缓存时，设置了is_prealloc为true，size大小为正确的大小，index为输出的对应索引，此时无法获得正确的结果，获得结果均为0。
不使用预分配内存时能够获取正确的结果，请问是什么原因？
如果不使用预分配内存的情况下，是不是每次推理结束都要调用rknn_outputs_release这个接口去释放内存，如果不在每次推理结束后调用，而是等整个进程结束后调用，是否会引起内存泄漏？

ret = rknn_outputs_get(ctx, 2, outputs, nullptr)
this API need too much time, the same rknn model ,Python need 9ms one imgs ,but C++ API need 20ms C++ API do not use NPU?

    double timeConsume,start;
    start = static_cast<double>(cv::getTickCount());    //记录当前时刻CPU时钟周期

    for (int j=0;j<300;j++){
        // Load image
        inputs[0].buf = img.data;
        ret = rknn_inputs_set(ctx, 1, inputs);
        if(ret < 0) {
            printf("rknn_input_set fail! ret=%d\n", ret);
            goto Error;
        }

        // Start Inference
        ret = rknn_run(ctx, nullptr);
        if(ret < 0) {
            printf("rknn_run fail! ret=%d\n", ret);
            goto Error;
        }
    
        outputs[0].want_float = true;
        outputs[0].is_prealloc = false;
        outputs[1].want_float = true;
        outputs[1].is_prealloc = false;
        ret = rknn_outputs_get(ctx, 2, outputs, nullptr);
        if(ret < 0) {
            printf("rknn_outputs_get fail! ret=%d\n", ret);
            goto Error;
        }
    }
    timeConsume = ((double)cv::getTickCount() - start) / cv::getTickFrequency();
    printf("\n timeSonsume: %f\n" ,timeConsume    );

RKNNToolkit API 更新到了1.7，但DRV版本是1.3，toybrick社区使用update,upgrade方法都不能升级。请问下如何在不升级固件的情况下升级驱动。

Dears Rockchip

We got insmod fail galcore.ko after kernel configuration change.
Could you support this ?

We can see unstable error from galcore inside. attached is galcore NULL point error with scheduler.

[ 6323.430845] Unable to handle kernel NULL pointer dereference at virtual address 000000f1
[ 6323.431591] pgd = 44c0c21e
[ 6323.431916] [000000f1] *pgd=6f493835
[ 6323.432286] Internal error: Oops: 17 [#1] PREEMPT SMP ARM
[ 6323.432794] Modules linked in: galcore(O)
[ 6323.433194] CPU: 0 PID: 340 Comm: Detect_Thread Tainted: G O 4.19.111 #1
[ 6323.433894] Hardware name: Generic DT based system
[ 6323.434336] PC is at __switch_to+0x8/0x40
[ 6323.434709] LR is at __schedule+0x204/0x70c
[ 6323.435089] pc : [] lr : [] psr: a00e0093
[ 6323.435647] sp : ed7fdf40 ip : ed7fc040 fp : ed7fdf8c
[ 6323.436106] r10: b0b02428 r9 : ed7f0000 r8 : b0101204
[ 6323.436566] r7 : 00000000 r6 : ed460b80 r5 : ed660b80 r4 : eef8c240
[ 6323.437144] r3 : 00000000 r2 : 00000095 r1 : ed7fc000 r0 : ed660b80
[ 6323.437725] Flags: NzCv IRQs off FIQs on Mode SVC_32 ISA ARM Segment user
[ 6323.438358] Control: 10c5387d Table: 3d7e406a DAC: 00000055

Thanks
Best Regards

Since the Versilicon's TIM-VX NN inference lib has supported creating tensor from DMABUF memory, I wonder if RV1126's NPU can support this?

因为之前在开发板上运行代码出现过推理消耗20s左右时间，看了官方的trouble shoting

想着升级驱动试试看

按照readme进行手动升级，push完成后，重启开发板

发现运行原来的代码，会报错

然后百度发现有人遇到类似的情况
https://blog.csdn.net/u013171226/article/details/121104880

登录开发板，发现现象一致，是libOpenVX.so.1 ibOpenVX.so的软连接失效了，重新建立软连接后，再运行代码正常

···
ln -snf libOpenVX.so.1.2 libOpenVX.so.1
ln -snf libOpenVX.so.1 libOpenVX.so
···

Thanks for your great work! Your provided rknn model (yolov5s_relu_rv1109_rv1126_out_opt.rknn) can be successfully used for interference in the RK platform! I meet "ValidationError: Your model ir_version is higher than the checker's" when transforming my own onnx model to the rknn one, as differences between environments of the RK platform and YOLOv5. If convenient, can you show how you solve this problem in detail.
中文：对于Yolo-v5 demo, 可否详细说明一下，你是如何解决RK平台和Yolov5环境不同的问题。抑或是，可否分享一下模型训练时的环境配置并说明一下如何‘modifying the original pytorch model’.

在：postprocess.cc中nms错误：
源代码：
static int nms(int validCount, std::vector &outputLocations, std::vector classIds, std::vector &order, int filterId, float threshold)
{
for (int i = 0; i < validCount; ++i)
{
if (order[i] == -1 || classIds[i] != filterId)
{
continue;
}
int n = order[i];
for (int j = i + 1; j < validCount; ++j)
{
int m = order[j];
if (m == -1 || classIds[i] != filterId)
{
continue;
}
float xmin0 = outputLocations[n * 4 + 0];
float ymin0 = outputLocations[n * 4 + 1];
float xmax0 = outputLocations[n * 4 + 0] + outputLocations[n * 4 + 2];
float ymax0 = outputLocations[n * 4 + 1] + outputLocations[n * 4 + 3];

        float xmin1 = outputLocations[m * 4 + 0];
        float ymin1 = outputLocations[m * 4 + 1];
        float xmax1 = outputLocations[m * 4 + 0] + outputLocations[m * 4 + 2];
        float ymax1 = outputLocations[m * 4 + 1] + outputLocations[m * 4 + 3];

        float iou = CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1);

        if (iou > threshold)
        {
            order[j] = -1;
        }
    }
}
return 0;

}
应该改为：
static int nms(int validCount, std::vector &outputLocations, std::vector classIds, std::vector &order, int filterId, float threshold)
{
for (int i = 0; i < validCount; ++i)
{
int n = order[i];
if (n == -1 || classIds[n] != filterId)
{
continue;
}
for (int j = i + 1; j < validCount; ++j)
{
int m = order[j];
if (m == -1 || classIds[m] != filterId)
{
continue;
}
float xmin0 = outputLocations[n * 4 + 0];
float ymin0 = outputLocations[n * 4 + 1];
float xmax0 = outputLocations[n * 4 + 0] + outputLocations[n * 4 + 2];
float ymax0 = outputLocations[n * 4 + 1] + outputLocations[n * 4 + 3];

        float xmin1 = outputLocations[m * 4 + 0];
        float ymin1 = outputLocations[m * 4 + 1];
        float xmax1 = outputLocations[m * 4 + 0] + outputLocations[m * 4 + 2];
        float ymax1 = outputLocations[m * 4 + 1] + outputLocations[m * 4 + 3];

        float iou = CalculateOverlap(xmin0, ymin0, xmax0, ymax0, xmin1, ymin1, xmax1, ymax1);

        if (iou > threshold)
        {
            order[j] = -1;
        }
    }
}
return 0;

}

现在我手里面有一个带rk1808芯片的ununtu aarch64板子，不是计算棒，也不是rk3399，是一个厂家集成的（厂家领导不说），现在需要升级一下驱动的版本。
我已经把/driver/linux-aarch64/usr/*下面的文件放到了 板子的文件下面发现在pc上进行预编译还是不行，RKNNAPI: rknn_init, driver open fail! ret = -9(ERROR_PIPE)! Exception: RKNN init failed. error code: RKNN_ERR_DEVICE_UNAVAILABLE，会报这样的错误， 查看文档，我好像需要输入这样的命令 dmesg | grep -i galcore 查询 NPU 驱动版本
结果我的什么输出都没有，就算是说我的板子开机启动的时候根本没有启动npu driver

所以感觉上应该是rknpu文件夹下面一些ko文件的缘故，所以相文以下 ，这个rknpu文件夹下的东西是如何使用的，像这种找不到厂家的板子，如何才能升级固件驱动呢？

我用3399pro运行报错can't find librknn_runtime.so，我看这里面文档和demo也都是针对1808或1126的。

hi guys.
as you may know there is a big difference between yolov3 and yolov5 post process function.
I'm looking for an example written for using rknn and yolov3 .
do you know whether there is an example of yolov3 post process in c++ or not?

系统：ubuntu18.04
板子：Rk1808
python版本：python3.6
安装完rknn_toolkit_lite-1.7.3-cp36-cp36m-linux_aarch64.whl，进行测试报下面的错误！
--> Init runtime environment
E Catch exception when init runtime!
E Traceback (most recent call last):
File "/home/zlg/.local/lib/python3.6/site-packages/rknnlite/api/rknn_lite.py", line 144, in init_runtime
async_mode=async_mode, rknn2precompile=rknn2precompile)
File "rknnlite/api/rknn_runtime.py", line 292, in rknnlite.api.rknn_runtime.RKNNRuntime.init
File "rknnlite/api/rknn_runtime.py", line 491, in rknnlite.api.rknn_runtime.RKNNRuntime._load_library
File "rknnlite/api/rknn_runtime.py", line 467, in rknnlite.api.rknn_runtime.RKNNRuntime._get_rknn_api_lib_path
Exception: Can not find dynamic library on RK1808!

help me and thanks!

rt

在目录rknpu/rknn/doc文档中介绍，SDK中提供了linux平台 yolov3Tiny目标检测Demo, 但是实际目录下却没有该demo文件， 请问有相应链接供我参考下载yolov3tiny的DEMO吗？

在RV1106的板子上，用rknpu2出现这个错误是什么原因啊

在rv1126跑demo: https://github.com/rockchip-linux/rknpu/tree/master/rknn/rknn_api/examples/rknn_yolov5_demo.

当持续运行时，一开始能正常推理，但是运行一段时间（大概1500次推理）后，就报错了：
dlopen /usr/lib/librga.so failed
once run use 71.088000 ms

程序还能继续运行，但是无法正常推理。

信息：

1. rga_api version 1.2.6_[0] (RGA is compiling with meson base: $PRODUCT_BASE)
2. librknn_runtime version 1.7.0 (0bef7b3 build: 2021-08-18 19:53:17 base: 1131)

使用fp16模型，在PC端python模拟的结果是正确的，但是在板端c++推理的结果不正确

请问我的输入数据输入为float类型，在设置rknn_input时设置RKNN_TENSOR_FLOAT32
和RKNN_TENSOR_UINT8会有什么区别呢？
我的数据类型为float浮点数，是语音特征，有办法转成int8，然后输入到量化模型里吗？

同一个rknn模型，batchsize为4的图像推理时间大约是batchsize为1的 四倍左右，呈线性增长，
是否有优化推理速度的方式，感谢回复！

rknn_init, msg_load_ack fail, ack = 6(ACK_PERF_TOO_MANY_CLIENT), expect 0(ACK_SUCC)

running demo in rv1126: https://github.com/rockchip-linux/rknpu/tree/master/rknn/rknn_api/examples/rknn_yolov5_demo

At the beginning, the program can run and infer normally, but after running for a period of time, an error is reported:

dlopen /usr/lib/librga.so failed

在1126上跑yolov5例子，出现这个问题

Hi,

Can RV1126 NPU drivers be open to using for the latest kernel versions or any galcore modules for v5,10 or v6,1 ?

1. 参考rknn-toolkit
   并以sudo权限执行update_rk1808_usb_rule.sh脚本以获取USB设备的读写权限，并重启宿主机系统
2. 执行adb devides脚本获取rk1808计算棒设备id

List of devices attached
TM018083200400203	device

3. 执行ls -ltr /dev/bus/usb/003/001查看rk1808计算棒设备权限

crw-rw-r-- 1 root root 189, 256 2月  14 10:10 /dev/bus/usb/003/001

4. 参照rknpu更新RK1808的驱动至1.7.1版本

adb push drivers/linux-aarch64/   /
adb push drivers/npu_ko/galcore.ko /lib/modules/galcore.ko

5. 重启rk1808计算棒系统adb shell && sudo reboot并启动rknn_server进程adb shell && rknn_server,显示如下

# rknn_server
start rknn server, version:1.7.1 (0cfd4a1 build: 2021-12-10 09:43:11)
1 NNPluginManager loadPlugins(41): No plugin find!
I NPUTransfer: Starting NPU Transfer Server, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)

执行ps -A | grep rknn_server，显示如下

440 ?        00:00:00 rknn_server
488 pts/0    00:00:00 rknn_server

rknn_server进程已启动rknn的npu驱动已更新至1.7.1版本

6. 从宿主机ubuntu20.04系统上远程链接rk1808计算棒的npu驱动程序，python3 examples/tflite/mobilenet_v1/test.py执行结果如下

78b0f390f34c:python3 -u /opt/project/examples/tflite/mobilenet_v1/test.py
--> config model
done
--> Loading model
W The target_platform is not set in config, using default target platform rk1808.
done
--> Building model
W:tensorflow:From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/control_flow_ops.py:1034: py_func (from tensorflow.python.ops.script_ops) is deprecated and will be removed in a future version.
Instructions for updating:
tf.py_func is deprecated in TF V2. Instead, there are two
    options available in V2.
    - tf.py_function takes a python function which manipulates tf eager
    tensors instead of numpy arrays. It's easy to convert a tf eager tensor to
    an ndarray (just call tensor.numpy()) but having access to eager tensors
    means `tf.py_function`s can use accelerators such as GPUs as well as
    being differentiable using a gradient tape.
    - tf.numpy_function maintains the semantics of the deprecated tf.py_func
    (it is not differentiable, and manipulates numpy arrays). It drops the
    stateful argument making all functions stateful.
    
2022-02-14 06:27:56.534017: W tensorflow/compiler/jit/mark_for_compilation_pass.cc:1483] (One-time warning): Not using XLA:CPU for cluster because envvar TF_XLA_FLAGS=--tf_xla_cpu_global_jit was not set.  If you want XLA:CPU, either set that envvar, or use experimental_jit_scope to enable XLA:CPU.  To confirm that XLA is active, pass --vmodule=xla_compilation_cache=1 (as a proper command-line flag, not via TF_XLA_FLAGS) or set the envvar XLA_FLAGS=--xla_hlo_profile.
W:tensorflow:From /usr/local/lib/python3.6/dist-packages/rknn/api/rknn.py:278: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where
done
--> Export RKNN model
done
--> Init runtime environment
* daemon not running; starting now at tcp:5037
* daemon started successfully
error: device 'TM018083200400203' not found
E Connect to Device Failure (-1)
E Catch exception when init runtime!
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 868, in rknn.api.rknn_base.RKNNBase.init_runtime
E   File "rknn/api/rknn_runtime.py", line 171, in rknn.api.rknn_runtime.RKNNRuntime.__init__
E   File "rknn/api/rknn_platform_utils.py", line 318, in rknn.api.rknn_platform_utils.start_ntp_or_adb
E Exception: Init runtime environment failed!
E Please feedback the detailed log file <log_feedback_to_the_rknn_toolkit_dev_team.log> to the RKNN Toolkit development team.
E You can also check github issues: https://github.com/rockchip-linux/rknn-toolkit/issues
E Current device id is: TM018083200400203
E Devices connected:
E []
Init runtime environment failed

Process finished with exit code 255

想请教一下这个问题是什么原因导致的？如何解决？谢谢。

转换的rknn模型，可以在rknn-toolkit工具中inference，但是在RK1808开发板c++推理会报错

我在贵公司买了一块pro rv1126的主板，算力是2tops，但在推理一张自己模型（yolov3的模型），图片为440*700，时间在1s多，测试yolov3的官方模型和图片，时间在765ms，想知道这是正常的时间还是哪里有问题呢？感谢

我是严格按照md文件中的过程来的

我的板子是RK3399Pro debian系统

1、更新https://github.com/rockchip-linux/rknpu 替换rk3399pro代码仓库下的npu/external/rknpu，然后编译替换之前的npu固件
2、rk3399pro开发板上安装rknn_toolkit-1.7.1-cp37-cp37m-linux_aarch64.whl
3、adb push rknn-toolkit/platform-tools/ntp/linux-aarch64/npu_transfer_proxy 到debian下的/usr/bin 替换旧的
4、运行mobilenet_v1，报一下错误。

done
--> Export RKNN model
done
--> Init runtime environment
E Catch exception when init runtime!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 868, in rknn.api.rknn_base.RKNNBase.init_runtime
E File "rknn/api/rknn_runtime.py", line 163, in rknn.api.rknn_runtime.RKNNRuntime.init
E File "rknn/api/rknn_runtime.py", line 228, in rknn.api.rknn_runtime.RKNNRuntime._load_library
E File "/usr/lib/python3.7/ctypes/init.py", line 356, in init
E self._handle = _dlopen(self._name, mode)
E OSError: libOpenVX.so.1.2: cannot open shared object file: No such file or directory
E Please feedback the detailed log file <log_feedback_to_the_rknn_toolkit_dev_team.log> to the RKNN Toolkit development team.
E You can also check github issues: https://github.com/rockchip-linux/rknn-toolkit/issues
Init runtime environment failed

我操作的步骤完全参考于：
https://github.com/rockchip-linux/rknn-toolkit/tree/master/examples/onnx/yolov5

1. 创建conda env with python3.7
   activate 此 env, 同时安装python dependencies by pip3 install -r requirements.txt.

2. clone yolov5 并依RKNN文档DEMO时的版本check out到此commit id上:

   使用yolov5官方仓库导出模型，链接：https://github.com/ultralytics/yolov5。该demo创建时yolov5的最新节点的commit id为:c5360f6e7009eb4d05f14d1cc9dae0963e949213
   并在私有数据集上训练了模型，通过detect.py一切正常。

3. 安装_onnx==1.6.0_ (原因是rknn toolkit中用的是此版本的_onnx_), 并执行了export转.pt转至.onnx.

   pip3 list |grep onnx
   onnx                    1.6.0
   

   虽然最后_.onnx_生成了。但同时export过程中可以看到有个failure：

   PyTorch: starting from runs/train/exp/weights/last.pt (14.5 MB)

   TorchScript: starting export with torch 1.11.0+cu102...
   ...
   ...
   ONNX: starting export with onnx 1.6.0...
   ...
   ONNX: export failure: Your model ir_version is higher than the checker's.
   Torch version 1.11.0+cu102 has not been tested with coremltools. You may run into unexpected errors. Torch 1.10.2 is the most recent version that has been tested.

   CoreML: starting export with coremltools 5.2.0...
   ...
   ...

   不管它，仍将生成的.onnx拷贝至example/onnx/yolov5/下.

4. 换到rknn toolkit的conda env:

   执行python3 test.py时，得到以下错误：

   W Call onnx.optimizer.optimize fail, skip optimize
   W Please confirm that your onnx opset_version <= 11 (current opset_verison = 12)!!!
   E Catch exception when loading onnx model: elenet_yolov5s.onnx!
   E Traceback (most recent call last):
   E File "rknn/base/RKNNlib/RK_nn.py", line 141, in rknn.base.RKNNlib.RK_nn.RKnn.load_onnx
   E File "rknn/base/RKNNlib/app/importer/import_onnx.py", line 118, in rknn.base.RKNNlib.app.importer.import_onnx.Importonnx.run
   E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 111, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx.init
   E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 1059, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx._build_subgraph
   E File "rknn/base/RKNNlib/converter/onnx_util.py", line 366, in rknn.base.RKNNlib.converter.onnx_util.onnx_subgraph.build_subgraph
   E KeyError: '378'
   E Please feedback the detailed log file <log_feedback_to_the_rknn_toolkit_dev_team.log> to the RKNN Toolkit development team.
   E You can also check github issues: https://github.com/rockchip-linux/rknn-toolkit/issues
   Load yolov5 failed!

5. 尝试在export yolov4时使用最新的onnx， 即版本1.11.0， 则export无错：

   ONNX: starting export with onnx 1.11.0...
   /home/shao/yolov5/models/yolo.py:58: TracerWarning: Converting a tensor to a Python boolean might cause the trace to be incorrect. We can't record the data flow of Python values, so this value will be treated as a constant in the future. This means that the trace might not generalize to other inputs!
   if self.grid[i].shape[2:4] != x[i].shape[2:4] or self.onnx_dynamic:
   ONNX: export success, saved as runs/train/exp/weights/last.onnx (28.4 MB)
   ONNX: run --dynamic ONNX model inference with: 'python detect.py --weights runs/train/exp/weights/last.onnx'
   Torch version 1.11.0+cu102 has not been tested with coremltools. You may run into unexpected errors. Torch 1.10.2 is the most recent version that has been tested.
   ...
   ...

   但在rknn中执行 python3 test.py, 同样的错误。

Questions:

1. 以上错误如何避免呢?
2. 在yolov4中，需要使用什么版本的onnx来做export呢？1.6.0 or 1.11.0?

我买了贵公司一块rv1126的主板，算力在2tops。我的模型是基于yolov3的模型，图片的大小是440*700的分辨率，但发现推理的时间要1s多，然后测试了yolov3的自带例子，耗费765ms，所以想问一下原因......我提供一下模型和推理图，能帮我测试一下具体时间吗？因为想看看性能能不能达到项目的需求（请问一下如何上传模型和推理图）

这里和后面的图片说的是，如果是nchw的模型，nhwc的图片，会将nhwc转换成nchw，
但是在后文又有一句 “当RKNN 模型输入tensor 的属性是NHWC 布局时，没有NHWC 转换成NCHW
的流程”

请问这句是不是写反了，我结合头文件注释来看，应该默认是nchw的输入吧？

I wish to use libOpenVX.so with RK3399PRo (Toybrick),
I see rknpu/drivers
having galcore_rk3399pro-npu.ko dependencies for NPU support. Please let me know how can i use it.

I wish to flash it, Please provide me guideline how to ?

Regards,
Harsh

I use the rknn_yolov5_demo to do the loop test, just as below:

for (int iii = 0; iii < 1000; ++iii)
{
    gettimeofday(&start_time, NULL);
    img_resize_slow(&rga_ctx, drm_buf, img_width, img_height, resize_buf, width, height);
    rknn_inputs_set(ctx, io_num.n_input, inputs);
    ret = rknn_run(ctx, NULL);
    ret = rknn_outputs_get(ctx, io_num.n_output, outputs, NULL);
    ret = rknn_outputs_release(ctx, io_num.n_output, outputs);
    gettimeofday(&stop_time, NULL);
    if (((__get_us(stop_time) - __get_us(start_time)) / 1000) > 50) {
        printf("iii: %d, once run use %f ms\n", iii, (__get_us(stop_time) - __get_us(start_time)) / 1000);
    }
}

sometimes the run use time may 20+ seconds;
the output as below:
iii: 20, once run use 20673.275000 ms
iii: 69, once run use 28998.568000 ms
iii: 77, once run use 29920.682000 ms
iii: 533, once run use 39776.206000 ms
iii: 611, once run use 28304.172000 ms

I print the rockchip info as follow:
[61124.179619] [galcore]: GPU[0] hang, automatic recovery.
[61124.189393] [galcore]: recovery done

It may the GPU recovery, so how can I solve the problem?

Development environment

Host PC :
Linux d3fau1t-cvs 4.15.0-48-generic #51~16.04.1-Ubuntu SMP Fri Apr 5 12:01:12 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Evaluation Board :
rk3399pro1p3v11_016
using buildroot rootfs

On Host PC

My local Python

d3fau1t@d3fau1t-cvs:~/rknpu$ which python
/usr/bin/python
d3fau1t@d3fau1t-cvs:~/rknpu$ python -V
Python 3.5.2

Create new venv

d3fau1t@d3fau1t-cvs:~/rknpu$ virtualenv -p /usr/bin/python venv
Running virtualenv with interpreter /usr/bin/python
Using base prefix '/usr'
New python executable in /home/d3fau1t/rknpu/venv/bin/python
Installing setuptools, pkg_resources, pip, wheel...done.

Entry venv and install packages (with rknn-toolkit-0.9.8-*.whl)

d3fau1t@d3fau1t-cvs:~/rknpu$ source venv/bin/activate
(venv) d3fau1t@d3fau1t-cvs:~/rknpu$ 
(venv) d3fau1t@d3fau1t-cvs:~/rknpu$ cd rknn-toolkit/packages/
(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit/packages$ pip install -r requirements-cpu.txt 

...
Successfully installed Cython-0.29.7 Jinja2-2.10.1 MarkupSafe-1.1.1 Pillow-6.0.0 Werkzeug-0.15.2 absl-py-0.7.1 astor-0.7.1 click-7.0 cycler-0.10.0 decorator-4.4.0 dill-0.2.8.2 flask-1.0.2 flatbuffers-1.9 gast-0.2.2 grpcio-1.20.1 h5py-2.9.0 itsdangerous-1.1.0 keras-applications-1.0.7 keras-preprocessing-1.0.9 kiwisolver-1.1.0 lmdb-0.94 markdown-3.1 matplotlib-3.0.3 mock-3.0.5 networkx-1.11 numpy-1.16.3 onnx-1.5.0 opencv-python-4.1.0.25 protobuf-3.7.1 pyparsing-2.4.0 python-dateutil-2.8.0 scipy-1.2.1 six-1.12.0 tensorboard-1.13.1 tensorflow-1.13.1 tensorflow-estimator-1.13.0 termcolor-1.1.0 typing-3.6.6 typing-extensions-3.7.2
(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit/packages$
(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit/packages$ pip install rknn_toolkit-0.9.8-cp35-cp35m-linux_x86_64.whl
Processing ./rknn_toolkit-0.9.8-cp35-cp35m-linux_x86_64.whl
...
Installing collected packages: rknn-toolkit
Successfully installed rknn-toolkit-0.9.8
(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit/packages$ 

Run mobilenet_v1 example

(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit$ cd example/mobilenet_v1
(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit/example/mobilenet_v1$ python test.py
--> config model
done
--> Loading model
done
--> Building model
E Catch exception when building RKNN model!
T Traceback (most recent call last):
T   File "rknn/api/rknn_base.py", line 470, in rknn.api.rknn_base.RKNNBase.build
T   File "rknn/api/rknn_base.py", line 887, in rknn.api.rknn_base.RKNNBase._quantize
T   File "rknn/base/rknnlib/app/tensorzone/workspace.py", line 231, in rknn.base.rknnlib.app.tensorzone.workspace.Workspace.load_data
T   File "rknn/base/rknnlib/app/tensorzone/graph.py", line 32, in rknn.base.rknnlib.app.tensorzone.graph.Graph.load_data
T   File "rknn/base/rknnlib/rknnnet.py", line 323, in rknn.base.rknnlib.rknnnet.rknnNet.load_data
T   File "rknn/base/rknnlib/rknnnet.py", line 335, in rknn.base.rknnlib.rknnnet.rknnNet._load_old_data
T   File "rknn/base/rknnlib/rknnnet.py", line 336, in rknn.base.rknnlib.rknnnet.rknnNet._load_old_data
T   File "/home/d3fau1t/rknpu/venv/lib/python3.5/site-packages/numpy/lib/npyio.py", line 447, in load
T     pickle_kwargs=pickle_kwargs)
T   File "/home/d3fau1t/rknpu/venv/lib/python3.5/site-packages/numpy/lib/format.py", line 692, in read_array
T     raise ValueError("Object arrays cannot be loaded when "
T ValueError: Object arrays cannot be loaded when allow_pickle=False
Build mobilenet_v1 failed!
(venv) d3fau1t@d3fau1t-cvs:~/rknpu/rknn-toolkit/example/mobilenet_v1$ 

On Evaluation board

I modified rk3399pro menuconfig file( enable OpenCV,rknn,numpy..) before build rootfs image*

It is the same result that occurred on PC.

Is it really runnable?