rockchip-linux / rknn-toolkit Goto Github PK

我的tensorflow有两个输入，一个是图片，另外一个是is_training,值是False，我的input_size_list应该怎么写

I Try match LeakyRelu_LeakyRelu_171:out0
I Match r_leakrelu [['LeakyRelu_LeakyRelu_171']] [['LeakyRelu']] to [['leakyrelu']]
I Try match LeakyRelu_LeakyRelu_153:out0
I Match r_leakrelu [['LeakyRelu_LeakyRelu_153']] [['LeakyRelu']] to [['leakyrelu']]
I Try match Conv_Conv_161:out0
I Match r_conv [['Conv_Conv_161', 'Initializer_model.17.m.0.cv2.conv.weight', 'Initializer_model.17.m.0.cv2.conv.bias']] [['Conv', 'Constant_0', 'Constant_1']] to [['convolution']]
I Try match Resize_Resize_155:out0
W Not match tensor Resize_Resize_155:out0
E Try match Resize_Resize_155:out0 failed, catch exception!
W ----------------Warning(4)----------------
E Catch exception when loading onnx model: ./yolov5s.onnx!
E Traceback (most recent call last):
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 826, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx.match_paragraph_and_param
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 728, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx._onnx_push_ready_tensor
E TypeError: 'NoneType' object is not iterable
E During handling of the above exception, another exception occurred:
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 264, in rknn.api.rknn_base.RKNNBase.load_onnx
E File "rknn/base/RKNNlib/RK_nn.py", line 135, in rknn.base.RKNNlib.RK_nn.RKnn.load_onnx
E File "rknn/base/RKNNlib/app/importer/import_onnx.py", line 121, in rknn.base.RKNNlib.app.importer.import_onnx.Importonnx.run
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 832, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx.match_paragraph_and_param
E File "rknn/api/rknn_log.py", line 312, in rknn.api.rknn_log.RKNNLog.e
E ValueError: Try match Resize_Resize_155:out0 failed, catch exception!
Load model failed!

模型中的位置

看代码大概是位于模型中的上采样位置，pytorch中的操作是nn.Upsample。

你好，
yolov5s模型经过onnx到rknn，输入大小设置320，rk3399pro测试batch=1与batch等于8平均单张图片推理时间一致，并没有加速，请问是由于网络复杂还是因为板子本身对batch推理的兼容不太好，谢谢

I notice that all the wheels for x86_64 have crashed. Can you upload these wheels again? Thank you.

rknpu/rknn/python only have below file.

• rknn1808-1.2.0-cp36-cp36m-linux_aarch64.whl
• rknn1808-1.2.0-cp37-cp37m-linux_aarch64.whl

If using 32bit platform, it generate compile error.

How can we use 32bit python RKNN ?

Thanks

Does RKNN-toolkit support Tensorflow 1.15 or 2.0 ?

I have 5 RK1808 sticks.
But how do I programming with them ?

When i execute /examples/onnx/resnet50v2/test.py i got an error:

--> config model done --> Loading model Calc node Conv : resnetv24_conv0_fwd output shape fail E Catch exception when loading onnx model: resnet50v2.onnx! E Traceback (most recent call last): E File "rknn/base/RKNNlib/onnx_ir/onnx_numpy_backend/shape_inference.py", line 57, in rknn.base.RKNNlib.onnx_ir.onnx_numpy_backend.shape_inference.infer_shape E File "rknn/base/RKNNlib/onnx_ir/onnx_numpy_backend/ops/conv.py", line 49, in rknn.base.RKNNlib.onnx_ir.onnx_numpy_backend.ops.conv.Conv E ImportError: No module named 'torch' E During handling of the above exception, another exception occurred: E Traceback (most recent call last): E File "rknn/api/rknn_base.py", line 264, in rknn.api.rknn_base.RKNNBase.load_onnx E File "rknn/base/RKNNlib/RK_nn.py", line 135, in rknn.base.RKNNlib.RK_nn.RKnn.load_onnx E File "rknn/base/RKNNlib/app/importer/import_onnx.py", line 111, in rknn.base.RKNNlib.app.importer.import_onnx.Importonnx.run E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 85, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx.__init__ E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 434, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx._shape_inference E File "rknn/base/RKNNlib/onnx_ir/onnx_numpy_backend/shape_inference.py", line 62, in rknn.base.RKNNlib.onnx_ir.onnx_numpy_backend.shape_inference.infer_shape E File "rknn/api/rknn_log.py", line 312, in rknn.api.rknn_log.RKNNLog.e E ValueError: Calc node Conv : resnetv24_conv0_fwd output shape fail Load resnet50v2 failed!

Where i can found torch for RK3399Pro(D) if thats needed?

使用rknn-toolkit-v1.3.0\examples\pytorch例子转换pytorch模型失败，具体日志如下：

root@169b24048060:/examples/pytorch/resnet18# python test.py
--> config model
done
--> Loading model
/usr/local/lib/python3.5/dist-packages/onnx_tf/common/init.py:87: UserWarning: FrontendHandler.get_outputs_names is deprecated. It will be removed in future release.. Use node.outputs instead.
warnings.warn(message)
./resnet18.pt ********************
WARNING: Token 'COMMENT' defined, but not used
WARNING: There is 1 unused token
Syntax error in input! LexToken(NAMED_IDENTIFIER,'None',5,263)
done
--> Building model
W Genreate input meta fail, please check model.
W External input meta file "/tmp/tmp6iu3la8h/resnet18_inputmeta.yml" is not exists.
Traceback (most recent call last):
File "test.py", line 66, in
ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
File "/usr/local/lib/python3.5/dist-packages/rknn/api/rknn.py", line 238, in build
ret = self.rknn_base.build(do_quantization=do_quantization, dataset=dataset, pack_vdata=pre_compile, batch_size=rknn_batch_size)
File "rknn/api/rknn_base.py", line 719, in rknn.api.rknn_base.RKNNBase.build
File "rknn/api/rknn_base.py", line 2249, in rknn.api.rknn_base.RKNNBase._generate_inputmeta
IndexError: list index out of range

请问是什么问题?谢谢

Hi

I made a single layer network:

def export_pytorch_model():
	net = Conv3d(3, 64, kernel_size=[2,3,3], stride=[1,2,2])
	net.eval()
	trace_model = torch.jit.trace(net, torch.Tensor(1,3,2,180,360))
	trace_model.save('./sqnet.pt')

Then follow the rknn api as below:

input_size_list = [[3,2,180,360]] #stacked 3D images
# Create RKNN object
rknn = RKNN()
# pre-process config
print('--> config model')
rknn.config(channel_mean_value='123.675 116.28 103.53 58.395', reorder_channel='0 1 2') 
print('done')

if not os.path.exists('./sqnet.rknn'):
	export_pytorch_model()
	model = './sqnet.pt'
	# Load pytorch model
	print('--> Loading model')
	ret = rknn.load_pytorch(model=model, input_size_list=input_size_list)
	if ret != 0:
		print('Load pytorch model failed!')
		exit(ret)
	print('done')

When run the above code, I got following error message:
--> config model
done
--> Loading model
./sqnet.pt ********************
W Channels(360) of input node: input.1 > 128, mean/std values will be set to default 0/1.
W Please do pre-processing manually before inference.
E convolution output_padding only suport [0,0] for now, but meet [0,0]
W ----------------Warning(2)----------------
E Catch exception when loading pytorch model: ./sqnet.pt!
E Traceback (most recent call last):
E File "rknn/api/rknn_bI made a single layer network:

def export_pytorch_model():
	net = Conv3d(3, 64, kernel_size=[2,3,3], stride=[1,2,2])
	net.eval()
	trace_model = torch.jit.trace(net, torch.Tensor(1,3,2,180,360))
	trace_model.save('./sase.py", line 339, in rknn.api.rknn_base.RKNNBase.load_pytorch
E   File "rknn/base/RKNNlib/RK_nn.py", line 146, in rknn.base.RKNNlib.RK_nn.RKnn.load_pytorch
E   File "rknn/base/RKNNlib/app/importer/import_pytorch.py", line 128, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run
E   File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 2255, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load
E   File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 2370, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.parse_nets
E   File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 2059, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.convert_operators
E   File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 648, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.convolution
E   File "rknn/api/rknn_log.py", line 312, in rknn.api.rknn_log.RKNNLog.e
E ValueError: convolution output_padding only suport [0,0] for now, but meet [0,0]
Load pytorch model failed!

Question: what can I do to fix this [0,0] padding issue? Thanks for your help.

1.3.2和1.3.0 中不支持conv2d的weight为另外一个卷积的输出，导致做不下去了

onnx LSTM model failed loading into RKNN. Model in the below link could reproduce the issue.
https://raw.githubusercontent.com/microsoft/onnxruntime/master/onnxruntime/test/testdata/CNTK/test_LSTM.tanh.bidirectional/model.onnx

E Catch exception when loading onnx model: ../models/model.onnx!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 510, in rknn.api.rknn_base.RKNNBase.load_onnx
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 514, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx.init
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 176, in rknn.base.RKNNlib.converter.convert_onnx.onnx_shape_infer_engine.infer_shape
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 131, in rknn.base.RKNNlib.converter.convert_onnx.onnx_shape_infer_engine.__status_update
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 165, in rknn.base.RKNNlib.converter.convert_onnx.onnx_shape_infer_engine.__model_rebuild
E File "rknn/base/RKNNlib/converter/tensor_model.py", line 330, in rknn.base.RKNNlib.converter.tensor_model.TensorModel.build_onnx_model
E KeyError: ''
Load failed!

Hello Rockchip team,

I have been experiencing an issue converting Darknet models. YOLO-based models are successfully converted, however, the problem comes when converting image classification models (ex: Darknet-59, MobileNet-v2, ResNet, ...).

The problem shows up when using any ImageNet pre-trained models; for example, Cross Stage Partial Networks and others available here.

The models (.cfg) and the weights (.weights) are downloaded and used without any additional training or modification. These models work perfectly when executed with the original Darknet framework by Alexey.

The RKNN-Toolkit debug/error output looks always the same (size and shape may vary regarding the model):

...
D Process output_156 ...
D RKNN output shape(output): (0 1 1 1000)
I Build test complete.
E Catch exception when loading darknet model: ./test.cfg!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 555, in rknn.api.rknn_base.RKNNBase.load_darknet
E File "rknn/base/RKNNlib/converter/convert_darknet.py", line 685, in rknn.base.RKNNlib.converter.convert_darknet.Converter.load_blobs
E File "rknn/base/RKNNlib/converter/convert_darknet.py", line 403, in rknn.base.RKNNlib.converter.convert_darknet.Convolution_Blob_Assign.assign
E File "<__array_function__ internals>", line 6, in reshape
E File "/home/dxxx/.virtualenvs/python3.6/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 299, in reshape
E return _wrapfunc(a, 'reshape', newshape, order=order)
E File "/home/dxxx/.virtualenvs/python3.6/lib/python3.6/site-packages/numpy/core/fromnumeric.py", line 58, in _wrapfunc
E return bound(*args, **kwds)
E ValueError: cannot reshape array of size 598984 into shape (384,384,3,3)

It looks like the problem appears when loading the weights into the model but there is no way to fix this from the RKNN-Toolkit 1.3.2 interface. All dependencies are installed and their versions match as specified in RKNN-Toolkit User Guide 1.3.2 document.

Thanks.

Hi，thanks for the excellent job!Can u guys make a list for the support OP and the interrelated parameter bound?Most time when i choose or design a net, i am not ensure whether it could pass the inference on board,Thanks again!

解压缩后并没有这些档案

8ab6ffcb1c356d5ab593ca838456fb37 rknn_toolkit_lite-1.4.0-cp35-cp35m-linux_aarch64.whl
3a1e727e5ae091e89908a98c9b02225d rknn_toolkit_lite-1.4.0-cp35-cp35m-linux_x86_64.whl
9efe46ee33f8d90a9d5ac847c1f1a358 rknn_toolkit_lite-1.4.0-cp36-cp36m-linux_x86_64.whl
9d6d561fcca1fcd5e2bfefb86ad44fda rknn_toolkit_lite-1.4.0-cp36-cp36m-macosx_10_15_x86_64.whl
86922085f7ba43ed3b210255a188e653 rknn_toolkit_lite-1.4.0-cp36-cp36m-win_amd64.whl
23b6ac94c834e77fcbe5d7c2cd98245d rknn_toolkit_lite-1.4.0-cp37-cp37m-linux_aarch64.whl
130efb12b7e0b6ea7760047c9924800d rknn_toolkit_lite-1.4.0-cp37-cp37m-macosx_10_15_x86_64.whl

How many frame per second is the vgg face detection able to achieve?

Hi there,
Does the RKNN-Toolkit support pruning?

If skip the zeroes during compress/conversion model, we can achieve more latency improvements.

rknn已经开源了yolov5s的模型和测试代码，但toolkit没有发布yolov5s的模型转换代码，请问什么时候能在toolkit1.6.0版本更新一下yolov5s的转换呢？

Hi RKNN team,

I've recently tried to load and convert the official YOLOv4 model in Darknet format by using the RKNN Toolkit 1.3.2. The model is quite new so I supposed it would not work; but surprisingly, the RKNN Toolkit did not complain anytime even in verbose mode. After a successful .RKNN conversion, an inference simulation was done taking as reference the YOLOv3 provided example. However, the output results from the YOLOv4 model were not consistent.

The YOLOv4 model uses the MISH activation ([https://arxiv.org/abs/1908.08681]) which I supposed it was not supported by the latest RKNN-Toolkit. After changing all the MISH activations to "Leaky" (as in YOLOv3), the YOLOv4 model worked.

It is normal that the RKNN-Toolkit will not warn the developers that there are "unsupported functions" when loading models?

By the way, it would be nice to have the MISH function supported in the following RKNN-Toolkit updates which generally gives better accuracy as shown in the literature.

Best,

rknn_init, msg_load_ack fail, ack = 6(ACK_PERF_TOO_MANY_CLIENT), expect 0(ACK_SUCC)

Hi

where to find the source code for "rknn_base.cpython-35m-x86_64-linux-gnu.so" ?

did download rknn-toolkit-1.6.0.zip but did not find the source code of above .so lib.

Please help. Thanks.

加载caffe模型的时候报 W Warning: Axis may need to be adjusted according to original model shape. 这个警告 hybrid_quantization_step1 的时候 ValueError: cannot reshape array of size 2359296 into shape (1,576,1024,3)

The ARM version of npu_transfer_proxy appears to assume that it is running on a RK3399 and it looks for the internal RK1808. We tried putting a NPU compute stick into a Raspberry Pi and it won't work. npu_transfer_proxy works fine on x86 and Mac host.

fstat(1, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0
write(1, "I NPUTransfer: Starting NPU Tran"..., 140I NPUTransfer: Starting NPU Transfer Proxy, Transfer version 1.9.5 (e8ffa6c@2019-07-01T11:43:29), devid = 0123456789ABCDEF, pid = 2752:2750
) = 140
rt_sigaction(SIGCHLD, {sa_handler=SIG_IGN, sa_mask=[CHLD], sa_flags=SA_RESTART}, {sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
faccessat(AT_FDCWD, "/sys/devices/platform/ff690000.efuse/rockchip-efuse0/nvmem", R_OK) = -1 ENOENT (No such file or directory)
faccessat(AT_FDCWD, "/sys/devices/system/cpu/soc", R_OK) = -1 ENOENT (No such file or directory)
faccessat(AT_FDCWD, "/sys/devices/system/cpu/socid", R_OK) = -1 ENOENT (No such file or directory)
faccessat(AT_FDCWD, "/proc/device-tree/compatible", R_OK) = 0
openat(AT_FDCWD, "/proc/device-tree/compatible", O_RDONLY) = 8
fstat(8, {st_mode=S_IFREG|0444, st_size=35, ...}) = 0
read(8, "raspberrypi,4-model-b\0brcm,bcm28"..., 4096) = 35
read(8, "", 4096) = 0

debian10@debian10:~/rknpu/rknn/rknn_api/examples/rknn_yolov5_demo$ ./rknn_yolov5_demo model/rk180x/yolov5s_relu_rk180x_out_opt.rknn model/test2.jpg Read model/test2.jpg ... img width = 1280, img height = 720 Loading mode... [ 1] HAL user version 6.4.3.5.293908 [ 2] HAL kernel version 6.4.3.5.293908 Segmentation fault

We have a model that uses 6-channel data. Is there any way to convert a model with quantization? The API seems to ask for a dataset.txt containing a list of image files, which is not an option for us. It would be great if we could pass data samples directly to the build() function as np arrays or something. Any possibilities there?

当tensorflow框架下使用量化感知训练时，在原有模型上加入伪量化节点后转换rknn模型过程中会报错，应该怎么解决这个问题呢？
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 193, in rknn.api.rknn_base.RKNNBase.load_tensorflow
E File "rknn/base/RKNNlib/RK_nn.py", line 47, in rknn.base.RKNNlib.RK_nn.RKnn.load_tensorflow
E File "rknn/base/RKNNlib/app/importer/import_tensorflow.py", line 231, in rknn.base.RKNNlib.app.importer.import_tensorflow.Importensorflow.run
E ZeroDivisionError: float division by zero

Hi!

I'm working with R3399Pro now and trying to load an execute more than one model at this device. As the documentation says, there is exists a "Segmentation model". It says, that we can load model segments aka multiple models at one device, and inference it o asynchronously. But I can't found any detailed description of how to load and execute 2 or more models at one time.

Can someone provide instruction or example for this case?
Thanks!

I receive the following error when initializing rknn for a 2nd time since device connection, and after a successful first run of the rknn SSD demo:
"E RKNNAPI: rknn_init, input 0 reorder = INPUT_REORDER_NA when n_dims = 3!"

I noticed that this ONLY happens for input images that have a "white" background (i.e. 255,255,255). If I change the background color of the input image, then I can run this model on that input multiple times without error.

Using TB-RK1808S0 USB stick.
Model: mobilenet_ssd.rknn

D RKNNAPI: ==============================================
D RKNNAPI: RKNN VERSION:
D RKNNAPI: API: 1.3.2 (9eebd73 build: 2020-04-02 14:54:06)
D RKNNAPI: DRV: 0.9.8 (943f8e8 build: 2019-07-03 17:51:17)
D RKNNAPI: ==============================================

The example code I used is shown below, as are two demo input images. The "apple_white.raw" (300x300x3) fails on the second run, the "apple_grey.raw" succeeds.

Code:
rknn_core_test_example_c.txt

Images:
apple_.zip

use rv1126(armv7l) python3.7 run rknn_toolkit_lite-1.6.0/rknn-toolkit-lite/examples/inference_with_lite raise librknn_api.so: undefined symbol: rknn_get_output_num

[root@RV1126_RV1109:/inference_with_lite]# python test.py 
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_platform_utils' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_model' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_log' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_utils' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.npu_config.cpu_npu_mapper' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.utils.yaml_parser' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_runtime' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_perf' does not match runtime version 3.7
--> Load RKNN model
done
--> Init runtime environment
E Catch exception when init runtime!
E Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/rknnlite/api/rknn_lite.py", line 140, in init_runtime
    async_mode=async_mode, rknn2precompile=rknn2precompile)
  File "rknnlite/api/rknn_runtime.py", line 162, in rknnlite.api.rknn_runtime.RKNNRuntime.__init__
  File "rknnlite/api/rknn_runtime.py", line 250, in rknnlite.api.rknn_runtime.RKNNRuntime._load_library
  File "/home/juwan/rv1126_project/buildroot/output/rockchip_rv1126_rv1109/target/usr/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
  File "/home/juwan/rv1126_project/buildroot/output/rockchip_rv1126_rv1109/target/usr/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
AttributeError: /usr/lib/python3.7/site-packages/rknnlite/api/lib/hardware/PUMA/linux-x86_64/librknn_api.so: undefined symbol: rknn_get_output_num

Init runtime environment failed
[root@RV1126_RV1109:/inference_with_lite]# 

librknn_api.so use

• ./rv1126_project/external/rknpu/rknn/rknn_api/librknn_api/lib/librknn_api.so
• ./rknn-toolkit-v1.6.0/rknn-toolkit-lite/packages/rknn_toolkit_lite-1.6.0/rknnlite/api/lib/hardware/PUMA/linux-x86_64/librknn_api.so

pytorch中lstm的代码如下：
self.rnn = nn.LSTM(nIn, nHidden, num_layers=1, bidirectional=True)
能够成功转换到onnx，但是在onnx转到rknn的时候fail，代码如下

报错如下：

通过Netron查看onnx该层的结构图如下：

我猜想会不会是sequence_lens参数没有导致的？如果是的话，要如何对代码进行修改呢？如果不是的话，又是什么原因导致这个问题的出现，要如何去修改？

mxnet中pooling实现: {
"op": "Pooling",
"name": "vargface_head_head_pooling_se_pool1",
"attrs": {
"global_pool": "True",
"pool_type": "avg"
},
"inputs": 45, 0, 0
}

报错：
root@ba1aee2d3c30:/desktop/RK1808/examples/mxnet/resnext50# python test_mx1.py
--> config model
done
--> Loading model
Load MXnet model
/usr/local/lib/python3.5/dist-packages/mxnet/gluon/block.py:1159: UserWarning: Cannot decide type for the following arguments. Consider providing them as input:
data: None
input_sym_arg_type = in_param.infer_type()[0]
Parsing net
W Rename minusscalar0_variable to rknn_minusscalar0_variable
W Rename minusscalar0 to rknn_minusscalar0
W Rename mulscalar0_variable to rknn_mulscalar0_variable
W Rename mulscalar0 to rknn_mulscalar0
W Rename mul0 to rknn_mul0
W Rename mul2 to rknn_mul2
W Rename mul3 to rknn_mul3
W Rename mul4 to rknn_mul4
E Catch exception when loading mxnet model: ./model-v-arcface-clean-original-symbol.json!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 677, in rknn.api.rknn_base.RKNNBase.load_mxnet
E File "rknn/base/RKNNlib/app/importer/mxnetloader.py", line 661, in rknn.base.RKNNlib.app.importer.mxnetloader.MxnetLoader.load
E File "rknn/base/RKNNlib/app/importer/mxnetloader.py", line 549, in rknn.base.RKNNlib.app.importer.mxnetloader.MxnetLoader.parse_nets
E File "rknn/base/RKNNlib/app/importer/mxnetloader.py", line 380, in rknn.base.RKNNlib.app.importer.mxnetloader.MxnetLoader.parse_layer_params
E File "rknn/base/RKNNlib/app/importer/mxlayer_parse.py", line 228, in rknn.base.RKNNlib.app.importer.mxlayer_parse.parse_pool_param
E KeyError: 'kernel'
Load mxnet model failed!

我有个caffe分类模型，但是Fp16下网络的输出结果不能精确到小数点后三四位

zd_deploy.prototxt 网络描述
zd_4000.caffemodel 模型权重
main.cc 测试程序
convertcaffe.py 转换脚本
2.bmp 测试图像

网络输入 BGR NCHW 减去均值128 直接预测
本来应该的结果 0.958134 0.041866
rknn出来的结果 0.947266 , 0.052887 不能精确到小数点后三位

文件链接：链接：https://pan.baidu.com/s/1hGYF9gg2JyGu3jdeQAQd8Q
提取码：zvie

1.enter example mobilenet_v1
2.python3 test.py
3.-->config model
done
-->Loading model
done
-->Building model
Segmentation fault

How can I compile my C++ code which uses rknn_api c++ library and simulate it on Ubuntu x86-64 machine instead of target device. Is there any way I can run my c++ code similar to rknn python simulator in Ubuntu or windows?

import os
import urllib
import traceback
import time
import sys
import numpy as np
from rknn.api import RKNN
import torch
from torch import nn

def convert(rknn, quan, onnx_model, rknn_model):
    # pre-process config
    print('--> config model')
    rknn.config(channel_mean_value='123.675 116.28 103.53 58.82', reorder_channel='0 1 2')
    print('done')

    # Load tensorflow model
    print('--> Loading model')
    ret = rknn.load_onnx(model=onnx_model)
    if ret != 0:
        print('Load %r failed!' % onnx_model)
        exit(ret)
    print('done')

    # Build model
    print('--> Building model')
    if quan == True:
        ret = rknn.build(do_quantization=True, dataset='./dataset.txt')
    else:
        ret = rknn.build(do_quantization=False)
    if ret != 0:
        print('Build %r failed!' % onnx_model)
        exit(ret)
    print('done')

    # Export rknn model
    print('--> Export RKNN model')
    ret = rknn.export_rknn(rknn_model)
    if ret != 0:
        print('Export %r failed!' % rknn_model)
        exit(ret)
    print('done')



def foo(onnx_model, rk_name:str=None, quan=False):
    quan = False
    if rk_name == None:
        rk_name = os.path.splitext(os.path.basename(onnx_model))[0]
    #onnx_model = "../onnx/clean0901-resnet18-rk-simp-ir4.onnx"
    if quan == True:
        rknn_model = f"{rk_name}_quan.rknn"
    else:
        rknn_model = f"{rk_name}.rknn"

    # Create RKNN object
    rknn = RKNN()

    convert(rknn, quan, onnx_model, rknn_model)

    rknn.release()


def main():
    net = nn.Linear(1024, 512, bias=False) # TODO the bug will be fixed if bias=True
    torch.onnx.export(net, torch.rand(1,1024), "linear.onnx")
    foo("linear.onnx")


if __name__ =="__main__":
    main()

It will raise error:

KeyError: 'data_bias'

And it can be fixed if replace bias=False to bias=True

Using rknn.load_pytorch(), found a operate not supported.
The model is trained in torch.version=1.6.0
E Catch exception when loading pytorch model: ./nanodet_eff0.pt! E Traceback (most recent call last): E File "rknn/api/rknn_base.py", line 339, in rknn.api.rknn_base.RKNNBase.load_pytorch E File "rknn/base/RKNNlib/RK_nn.py", line 146, in rknn.base.RKNNlib.RK_nn.RKnn.load_pytorch E File "rknn/base/RKNNlib/app/importer/import_pytorch.py", line 128, in rknn.base.RKNNlib.app.importer.import_pytorch.ImportPytorch.run E File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 2250, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load E File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 2055, in rknn.base.RKNNlib.converter.convert_pytorch_new.PyTorchOpConverter.report_missing_conversion E NotImplementedError: The following operators are not implemented: ['aten::split_with_sizes']

I can run test.py successfully when use torchvision.models.AlexNet() with py36, pytorch==1.2.0, then I replaced the model and got an unexpected error when ret = rknn.init_runtime(), I can't find where the problem is, could you help me?
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 1154, in rknn.api.rknn_base.RKNNBase.init_runtime
E File "rknn/api/rknn_runtime.py", line 356, in rknn.api.rknn_runtime.RKNNRuntime.build_graph
E Exception: RKNN init failed. error code: RKNN_ERR_MODEL_INVALID

I'm using ubuntu 18.04 on CORE-3399PRO-JD4, which the python version is 3.6. But I can't find any package for arm python3.6 in 1.3.2 release. Are you still supporting arm python3.6 ?

rknn_inputs_set error: input[0] has 36 mean values, which is out of range (1-15),

when i execute "rknn.load_onnx()" i got an error

i train model and make onnx model in torch 1.2.0 ,and now i make rknn in a new environment which is rknn-toolkit 1.6.0 torch 1.5.1，is that something wrong?

W Not get any connection of Dropout_717:out0
E Catch exception when loading onnx model: ./model/model.onnx!
E Traceback (most recent call last):
E File "rknn/api/rknn_base.py", line 264, in rknn.api.rknn_base.RKNNBase.load_onnx
E File "rknn/base/RKNNlib/RK_nn.py", line 135, in rknn.base.RKNNlib.RK_nn.RKnn.load_onnx
E File "rknn/base/RKNNlib/app/importer/import_onnx.py", line 122, in rknn.base.RKNNlib.app.importer.import_onnx.Importonnx.run
E File "rknn/base/RKNNlib/converter/convert_onnx.py", line 904, in rknn.base.RKNNlib.converter.convert_onnx.convert_onnx.graph_connection_build
E AttributeError: 'NoneType' object has no attribute 'lid'
Load model failed!

I experience errors while converting tflite models to rknn using RKNN-Toolkit 1.3.2.

While converting tflite 1.14 model:

--> Loading model
E Catch exception when loading tflite model: ./supercombo_1.14.tflite!
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 396, in rknn.api.rknn_base.RKNNBase.load_tflite
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 75, in rknn.base.RKNNlib.converter.tflite_loader.TFLiteLoader.load
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 396, in rknn.base.RKNNlib.converter.tflite_loader.ModelParser.parse
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 342, in rknn.base.RKNNlib.converter.tflite_loader.ModelParser._build_layers
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 238, in rknn.base.RKNNlib.converter.tflite_loader.ModelParser._get_layer_RKNN_op
E   File "rknn/base/RKNNlib/converter/lite/RKNN_map_op.py", line 510, in rknn.base.RKNNlib.converter.lite.RKNN_map_op.RKNNMapOp.map_RKNN_ops
E   File "rknn/base/RKNNlib/converter/lite/RKNN_map_op.py", line 275, in rknn.base.RKNNlib.converter.lite.RKNN_map_op._permute
E   File "rknn/base/RKNNlib/converter/lite/utils.py", line 180, in rknn.base.RKNNlib.converter.lite.utils.get_builtin_options
E   File "rknn/base/RKNNlib/converter/lite/utils.py", line 174, in rknn.base.RKNNlib.converter.lite.utils.get_builtin_options_type
E AttributeError: module 'RKNNlib.converter.lite.tflite' has no attribute 'TransposeOptions'
Load failed!

While converting tflite 2.1 model:

--> Loading model
E Catch exception when loading tflite model: ./supercombo_2.1.tflite!
E Traceback (most recent call last):
E   File "rknn/api/rknn_base.py", line 396, in rknn.api.rknn_base.RKNNBase.load_tflite
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 75, in rknn.base.RKNNlib.converter.tflite_loader.TFLiteLoader.load
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 396, in rknn.base.RKNNlib.converter.tflite_loader.ModelParser.parse
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 342, in rknn.base.RKNNlib.converter.tflite_loader.ModelParser._build_layers
E   File "rknn/base/RKNNlib/converter/tflite_loader.py", line 238, in rknn.base.RKNNlib.converter.tflite_loader.ModelParser._get_layer_RKNN_op
E   File "rknn/base/RKNNlib/converter/lite/RKNN_map_op.py", line 510, in rknn.base.RKNNlib.converter.lite.RKNN_map_op.RKNNMapOp.map_RKNN_ops
E   File "rknn/base/RKNNlib/converter/lite/RKNN_map_op.py", line 276, in rknn.base.RKNNlib.converter.lite.RKNN_map_op._permute
E AttributeError: 'NoneType' object has no attribute 'PermAsNumpy'
Load failed!

I can't use lower version than 1.14, because ELU operation was introduced in tensorflow 1.14.

Attachment includes:
supercombo.keras-original model.
supercombo_1.14.tflite – tflite model converted with tensorflow 1.14.
supercombo_2.1.tflite – tflite model converted with tensorflow 2.1.
convert2.py – python script to convert tflite to rknn.
https://1drv.ms/u/s!Ai_yk0dieAUPgytiDDqvlv4hwS4b?e=TXUl1V

Hi there

Is there a list of supported operations in rknn-toolkit? What happens if one operation is not allowed? Is there a software fallback to execute the missing hw accelerated ops?

Best regards,

layer {
  name: "scale"
  type: "Scale"
  bottom: "fc8_m"
  top: "scale"
  scale_param {
    bias_term: true
  }
}

rknn's scale layer doesn't use bias term by default, when add "bias_term: true" manually, returns "Segmentation fault"
error.

install rknn_toolkit_lite-1.6.0-cp36-cp36m-linux_armv7l.whl change to rknn_toolkit_lite-1.6.0-cp37-cp37m-linux_armv7l.whl

use rv1126(armv7l) python3.7 run rknn_toolkit_lite-1.6.0/rknn-toolkit-lite/examples/inference_with_lite raise librknn_api.so: undefined symbol: rknn_get_output_num

[root@RV1126_RV1109:/inference_with_lite]# python test.py 
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_platform_utils' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_model' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_log' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_utils' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.npu_config.cpu_npu_mapper' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.utils.yaml_parser' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_runtime' does not match runtime version 3.7
/usr/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: compiletime version 3.6 of module 'rknnlite.api.rknn_perf' does not match runtime version 3.7
--> Load RKNN model
done
--> Init runtime environment
E Catch exception when init runtime!
E Traceback (most recent call last):
  File "/usr/lib/python3.7/site-packages/rknnlite/api/rknn_lite.py", line 140, in init_runtime
    async_mode=async_mode, rknn2precompile=rknn2precompile)
  File "rknnlite/api/rknn_runtime.py", line 162, in rknnlite.api.rknn_runtime.RKNNRuntime.__init__
  File "rknnlite/api/rknn_runtime.py", line 250, in rknnlite.api.rknn_runtime.RKNNRuntime._load_library
  File "/home/juwan/rv1126_project/buildroot/output/rockchip_rv1126_rv1109/target/usr/lib/python3.7/ctypes/__init__.py", line 377, in __getattr__
  File "/home/juwan/rv1126_project/buildroot/output/rockchip_rv1126_rv1109/target/usr/lib/python3.7/ctypes/__init__.py", line 382, in __getitem__
AttributeError: /usr/lib/python3.7/site-packages/rknnlite/api/lib/hardware/PUMA/linux-x86_64/librknn_api.so: undefined symbol: rknn_get_output_num

Init runtime environment failed
[root@RV1126_RV1109:/inference_with_lite]# 

librknn_api.so use

• ./rv1126_project/external/rknpu/rknn/rknn_api/librknn_api/lib/librknn_api.so
• ./rknn-toolkit-v1.6.0/rknn-toolkit-lite/packages/rknn_toolkit_lite-1.6.0/rknnlite/api/lib/hardware/PUMA/linux-x86_64/librknn_api.so

I am a pt model trained by yolov5, and the training environment is torch1.6. When I use the toolkit's onnx conversion, it prompts an error Your model ir_version is higher than the checker's. It should be that the onnx version is too low, I try to increase the onnx version to 1.7. 0, but it is not compatible with the toolkit, it only supports the highest version to 1.4.1. What should I do, can someone help me?
这是翻译：我是yolov5训练的pt模型，训练环境是torch1.6.在我使用toolkit的onnx转换时候，提示报错Your model ir_version is higher than the checker's.应该是onnx版本太低，我尝试提高onnx版本到1.7.0，但是与toolkit工具不兼容，它只支持最高版本到1.4.1.我应该怎么做，有人可以帮助我吗？

Hi there,
What is the full name of RKNN?

rt