[Bug]: OpenVINOInferencer does not work with ONNX classification model about anomalib

[haimat]

Update: It seems to work when creating the OpenVINOInferencer with device="CPU".
Does it not work with Nvidia GPUs?

[haimat]

Hey guys, any news on this?
Or can you suggest any other way to run an exported anomlib model in ONNX format on the Nvidia GPU?

[ashwinvaidya17]

Hey @haimat, the team is quite busy at this moment, so we didn't have time to look into it. We will return to is as soon as possible. Meanwhile, you can take a look at the OpenVINO documentation to get an idea of the supported hardware, https://docs.openvino.ai/2024/home.html.

[haimat]

@ashwinvaidya17 Thanks for your response. I can see that Nvidia GPUs are not (fully) supported by OpenVINO, so I guess the best (= fastest) approach would be to convert the ONNX model into a TensorRT model. I guess that should be doable. But how can I read the output of the ONNX model, can I follow 1:1 what happens in the OpenVINOInferencer?

[haimat]

It seems the problem is related to Nvidia GPUs being not fully supported in OpenVINO.
After I exported the model to ONNX / TensorRT format, everything works fine now.