xboot / libonnx Goto Github PK

It'd be nice to have python bindings, since onnxruntime by Micro$oft has telemetry and so it is a bit unethical to depend on it. Fortunately there can be a thin abstraction layer.

To maximize compatibility to various python impls and to spare users from compiling the lib themselves it may make sense to implement it via ctypes. There are packages generating ctypes bindings from headers automatically, but usually they need extensive postprocessing.

https://github.com/Maratyszcza/NNPACK

Hello!
I am working on a project where I have to deploy an inference engine in a very minimal environment where even the math library is not present. While compiling (onnxconf.h) for the platform I found out that the math.h header file was missing in the platform libraries. Here are some of the headers being used in that header file - onnxconf.h:

#include <stdio.h>
#include <stdlib.h>  
#include <stdint.h>  
#include <stddef.h>  
#include <string.h>  
#include <malloc.h>  
#include <float.h>  
#include <math.h>  
#include <list.h>  
#include <hmap.h>  

Also I have found that in this header file no math function has been used - I hope I am write on this! Correct me if I am wrong
So I was wondering if I could just remove this inclusion of math.h and still have a functional inference engine?

Just to make things simple compiling a math library would not be desirable but is possible.

hi,
I try to load 'yolov5n.onnx' like this:

#include "onnx.h"

int main(void)
{
    struct onnx_context_t *sess = onnx_context_alloc_from_file("yolov5n.onnx", NULL, 0);
    onnx_context_dump(sess, 1);
    return 0;
}

but nothing was output, including warnings and errors.

At first , I got error when complie this project:
main.c:1:10: fatal error: SDL2/SDL.h: 没有那个文件或目录
1 | #include <SDL2/SDL.h>
| ^~~~~~~~~~~~
then I use sudo apt-get install libsdl2-gfx-dev fixed this error.

I think this project should explain this dependence.

THANKS!

Hi, there.

I came across this project and try to compile it on MacOs. But fail with the following error.

main.c:90:47: warning: format specifies type 'long' but the argument has type 'uint64_t' (aka 'unsigned long long') [-Wformat]
printf("%-32s %ld %12.3f(us)\r\n", e->key, p->count, (p->count > 0) ? ((double)p->elapsed / 1000.0f) / (double)p->count : 0);
~~~ ^~~~~~~~
%llu
1 warning generated.
[LD] Linking benchmark
ld: library not found for -lcrt0.o
clang: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [benchmark] Error 1
make[1]: *** [all] Error 2
make: *** [all] Error 2

https://github.com/microsoft/onnxruntime/blob/main/onnxruntime/core/flatbuffers/schema/ort.fbs

Hi,

I'm trying to run the hello example on a small embedded system but im unsure of the memory required to allocate this model ( when running onnx_context_alloc).

I have roughly 2MB, is that enough?
Is there a smaller model that I can test with the model defined as a const char array?
Like the static const unsigned char mnist_onnx[] = { ... }

I downloaded tiny yolo v2 model from https://github.com/onnx/models/tree/main/vision/object_detection_segmentation/tiny-yolov2
And when inferencing this, I got those outputs from Valgrind

==178736== Invalid read of size 1
==178736== at 0x162DF9: shash (onnxconf.h:146)
==178736== by 0x162F11: MaxPool_init (MaxPool.c:38)
==178736== by 0x113FF0: onnx_graph_alloc (onnx.c:1238)
==178736== by 0x10FCFA: onnx_context_alloc (onnx.c:102)
==178736== by 0x10FF35: onnx_context_alloc_from_file (onnx.c:145)

==178736== Invalid write of size 1
==178736== at 0x1154F1: onnx_attribute_read_string (onnx.c:1747)
==178736== by 0x162F09: MaxPool_init (MaxPool.c:38)
==178736== by 0x113FF0: onnx_graph_alloc (onnx.c:1238)
==178736== by 0x10FCFA: onnx_context_alloc (onnx.c:102)
==178736== by 0x10FF35: onnx_context_alloc_from_file (onnx.c:145)

==178736== Invalid read of size 1
==178736== at 0x13BEB8: shash (onnxconf.h:146)
==178736== by 0x13BFD1: Conv_init (Conv.c:43)
==178736== by 0x113FF0: onnx_graph_alloc (onnx.c:1238)
==178736== by 0x10FCFA: onnx_context_alloc (onnx.c:102)
==178736== by 0x10FF35: onnx_context_alloc_from_file (onnx.c:145)

==178736== Invalid write of size 1
==178736== at 0x1154F1: onnx_attribute_read_string (onnx.c:1747)
==178736== by 0x13BFC9: Conv_init (Conv.c:43)
==178736== by 0x113FF0: onnx_graph_alloc (onnx.c:1238)
==178736== by 0x10FCFA: onnx_context_alloc (onnx.c:102)
==178736== by 0x10FF35: onnx_context_alloc_from_file (onnx.c:145)

==178736== ERROR SUMMARY: 30 errors from 4 contexts (suppressed: 0 from 0)

I have a model converted from Tensorflow that uses opset 12. (using tf2onnx.convert)
The model opens fine in Netron and elsewhere but crashes somewhere in Concat_reshape when I try to load it with onnx_context_alloc_from_file. I tried compiling for both x86 and x64 with the same result.

Here are the model properties as viewed through Netron:

Opening the models supplied in the libonnx test directory seemed to work fine. Do you have any suggestions for how to get this working? Thanks.

main.c:2241:14: error: ‘SDL_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR’ undeclared (first use in this function)
SDL_SetHint(SDL_HINT_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR, "0");

This is really a question, I don't think there is a bug here, just something I'm not understanding.

I'm looking at the code for maxpool and how it handles dilations. The spec has this example:

"""
input_shape: [1, 1, 4, 4]
output_shape: [1, 1, 2, 2]
"""
node = onnx.helper.make_node(
    'MaxPool',
    inputs=['x'],
    outputs=['y'],
    kernel_shape=[2, 2],
    strides=[1, 1],
    dilations=[2, 2]
)
x = np.array([[[
    [1, 2, 3, 4],
    [5, 6, 7, 8],
    [9, 10, 11, 12],
    [13, 14, 15, 16],
]]]).astype(np.float32)
y = np.array([[[
    [11, 12],
    [15, 16]]]]).astype(np.float32)

expect(node, inputs=[x], outputs=[y], name='test_maxpool_2d_dilations')

This should implicitly use AUTO_PAD_NOTSET. Now what I tried is getting the MaxPool_float32 to give the [ 11, 12, 15, 16 ] result by hardcoding the inputs, for the full code + output see this godbolt:

int strides[] = { 1, 1 };
int kernels[] = { 2, 2 };
int cpads[] = { 0, 0, 0, 0 };

int x_ndim = 4;
int x_dims[] = { 1, 1, 4, 4 };
int y_dims[] = { 1, 1, 2, 2 };

From my code reading, the dilation is only used to determine the output dimensions, which I've hardcoded here.

But with these inputs I get the incorrect output:

6.000000 7.000000 10.000000 11.000000

So, what is the way that dilations influence the end result that I am missing?

I have compiled libonnx on a fresh installation of Ubuntu - installed all the prerequisites ran just make and tried running the tests one by one. But I find the every test that I run fails and I am not able to figure out why.

Here is what I did:

• Installed the latest LTS release of Ubuntu
• Installed make, build-essential, git and libsdl2-gfx (Did do other stuff but those would not mess with this)
• Ran make all to compile
• Ran the tests on many examples: ./tests ./model/mnist_8/
• But every test that I have tried has just failed!

$ ./TESTING/libonnx/tests/tests ./TESTING/libonnx/tests/model/mnist_8/
[test_data_set_0]                                                                       [FAIL]
[test_data_set_1]                                                                       [FAIL]
[test_data_set_2]                                                                       [FAIL]

• All those in the simple and model folders fail but those in pytorch-* succeed partially
• Is this because of a missing operator? (no Unsupported opset message has been displayed as in the pytorch tests)
• Nonetheless the example for handwriting recognition has identified the number correctly most of the time

Hi, Can I build and use libonnx on gpu ?

https://github.com/ONNC/onnc/blob/master/docs/ONNC-C-Backend-Guide.md

Hi,

When I run the test/model/test_mnist_8 once it works and I get a OKAY result.
I then re-run it and it FAILS.

Any suggestion why this might be and what to look for?

IR Version: v6
Producer: pytorch 1.11.0
Domain:
Imports:
ai.onnx v11
Conv_0: Conv-11 (ai.onnx)
Inputs:
input.1: float32[1 x 3 x 352 x 352] = [...]
onnx::Conv_760: float32[24 x 3 x 3 x 3] = [...]
onnx::Conv_761: float32[24] = [...]
Outputs:
input.4: float32[1 x 24 x 176 x 176] = [...]

...

Concat_264: Concat-11 (ai.onnx)
Inputs:
onnx::Concat_744: float32[1 x 1 x 22 x 22] = [...]
onnx::Concat_960: float32[1 x 4 x 22 x 22] = [...]
onnx::Concat_757: float32[1 x 80 x 22 x 22] = [...]
Outputs:
758: float32[1 x 85 x 22 x 22] = [...]
110592
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Gather-11 (ai.onnx)
Unsupported opset => Resize-11 (ai.onnx)
Unsupported opset => Pad-11 (ai.onnx)

Issue:
Using https://github.com/dog-qiuqiu/FastestDet Fastest yolo, unsupported opset. Onnx model in /FastestDet/example/onnx-runtime.

算子不支持，在yolotiny测试中没问题，采用另一种轻量级模型出现该问题，想问下老哥是算子实现的问题还是别的。

I find that the GRU.c is nearly empty. The libonnx doesn't support GRU now, does it?

Dear friends,
Thanks for good job.
We have two question: 1) Does libonnx support dynamic shape input ? If not, how to implement it ? (Just some hints.);
2) How to support CUDA inference on libonnx ?

In the main.c file in the examples/hello folder, how did you convert the MNSIT model to a unsigned char array and use it?:

#include <onnx.h>

static const unsigned char mnist_onnx[] = {
	0x08, 0x03, 0x12, 0x04, 0x43, 0x4e, 0x54, 0x4b, 0x1a, 0x05, 0x32, 0x2e,
	0x35, 0x2e, 0x31, 0x22, 0x07, 0x61, 0x69, 0x2e, 0x63, 0x6e, 0x74, 0x6b,
	0x28, 0x01, 0x3a, 0xb2, 0xce, 0x01, 0x0a, 0x62, 0x0a, 0x0c, 0x50, 0x61,
	0x72, 0x61, 0x6d, 0x65, 0x74, 0x65, 0x72, 0x31, 0x39, 0x33, 0x0a, 0x1b,
	0x50, 0x61, 0x72, 0x61, 0x6d, 0x65, 0x74, 0x65, 0x72, 0x31, 0x39, 0x33,

If you have any code to do it can you share it?

I came across an error when compling it on my Mac.

[CC] helper.c
In file included from helper.c:28:
./helper.h:13:10: fatal error: 'malloc.h' file not found
#include <malloc.h>
^~~~~~~~~~
1 error generated.
make[1]: *** [helper.o] Error 1
make: *** [all] Error 2

Hi,

When I compile I get the following error with clang. I then try to link the library and it complains. Not quite sure why, I added in the main.c isnan and isinf and compiles ok. -lm is added to the linker.

Library compilation:
default/IsNaN.c:34:11: warning: implicit declaration of function 'isnanf' is invalid in C99 [-Wimplicit-function-declaration] py[i] = isnanf(v) ? 1 : 0;

Linker:
libonnx.a(.text+0x598): undefined reference to isnanf'

Hi,

In src/default folder, we could found the implementation C code of most operators(e.g. conv, matmul and etc. ), however, we could not found the implementation C code of Upsamle.

where is the implementation C code of Upsample?