openvinotoolkit / openvino_tensorflow Goto Github PK

Is their any plan to integrate openvino to Pytorch also?

while running following command after installing openvino_tensorflow.

python -c "import openvino_tensorflow; print(openvino_tensorflow.version)"

tensorflow.python.framework.errors_impl.NotFoundError: XXXXXXXXXXXXXXXXX \lib\site-packages\openvino_tensorflow\openvino_tensorflow.dll not found

no issues with tensorflow:

python -c "import tensorflow as tf; print('TensorFlow version: ',tf.version)"
2022-04-07 08:00:06.091197: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2022-04-07 08:00:06.091694: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
TensorFlow version: 2.8.0

I tried running this command to compile from the source.

python3 --version
>>Python 3.8.5

Command used: 
python3 build_ovtf.py --use_prebuilt_tensorflow

Package             Version
------------------- -------
Keras               2.4.3
Keras-Applications  1.0.8
Keras-Preprocessing 1.1.2
mock                4.0.3
numpy               1.18.5
pip                 21.0.1
psutil              5.8.0
setuptools          54.1.2
six                 1.15.0
termcolor           1.1.0
wheel               0.36.2
yapf                0.26.0
Target Arch: native
Using TensorFlow version v2.2.2
Install TensorFlow
Looking in indexes: https://test.pypi.org/simple/, https://pypi.org/simple
ERROR: Could not find a version that satisfies the requirement tensorflow-custom-abi0
ERROR: No matching distribution found for tensorflow-custom-abi0
Traceback (most recent call last):
  File "build_ovtf.py", line 501, in <module>
    main()
  File "build_ovtf.py", line 281, in main
    command_executor(
  File "/Users/rlo/Documents/openvino_dev/openvino_tensorflow/tools/build_utils.py", line 62, in command_executor
    assert retcode == 0, "dir:" + os.getcwd(
AssertionError: dir:/Users/rlo/Documents/openvino_dev/openvino_tensorflow/build_cmake. Error in running command: pip install --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple tensorflow-custom-abi0

Installed openvino using build_ov.py in D:\openvino2\artifacts\openvino. But getting error when executing following command:
python build_ovtf.py --use_openvino_from_location="D:\openvino2\artifacts\openvino"

Traceback (most recent call last):
File "D:\openvino_tensorflow-2.3.0\build_ovtf.py", line 642, in
main()
File "D:\openvino_tensorflow-2.3.0\build_ovtf.py", line 212, in main
raise AssertionError("Path doesn't exist {0}".format(ver_file))
AssertionError: Path doesn't exist D:\openvino2\artifacts\openvino/runtime/version.txt

OpenVINO TensorFlow Frontend does not support Reverse or ReverseV2 with multiple axes for the reversing

Hi, I'm trying to use openvino_tensorflow on my core i7-6700 CPU but if I use openvino tensorflow is around 10x slower than using only tf.
OS : ubuntu 20.04
CPU: core i7-6700

I just installed openvino_tensorflow and if I run my code using only TF I got an inference time of 0.10 s.
Then I add the line

import openvino_tensorflow openvino_tensorflow.set_backend('CPU')

My inference time is 1.5 s, log says
2022-07-11 09:50:48.690843: OVTF Summary -> 90 out of 624 nodes in the graph (14%) are now running with OpenVINO™ backend

I was able to complete the example on 12th Gen Core Processor using CPU backend.
But when I change to GPU backend, the example SegFault.

I have verified that OpenVINO 2022.1 was correctly installed by running the OpenVINO benchmark_app against the GPU device.

2022-07-21 15:47:12.926336: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2022-07-21 15:47:12.929013: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
2022-07-21 15:47:12.929026: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-07-21 15:47:14.114683: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2022.1.0.643/extras/opencv/python/cv2/../../bin:/opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
2022-07-21 15:47:14.114703: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-07-21 15:47:14.114715: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (my-adl-s): /proc/driver/nvidia/version does not exist
2022-07-21 15:47:14.114839: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Available Backends:
CPU
GPU
2022-07-21 15:47:20.447067: OVTF Summary -> 59 out of 1326 nodes in the graph (4%) are now running with OpenVINO™ backend
2022-07-21 15:47:21.487357: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:354] MLIR V1 optimization pass is not enabled
Inference time in ms: 205.22
person 0.98
tie 0.81
Output image is saved in detections.jpg

2022-07-21 15:47:23.855600: I tensorflow/core/util/util.cc:169] oneDNN custom operations are on. You may see slightly different numerical results due to floating-point round-off errors from different computation orders. To turn them off, set the environment variable `TF_ENABLE_ONEDNN_OPTS=0`.
2022-07-21 15:47:23.858267: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
2022-07-21 15:47:23.858281: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-07-21 15:47:25.036728: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: :/opt/intel/openvino_2022.1.0.643/extras/opencv/python/cv2/../../bin:/opt/intel/openvino_2022/extras/opencv/lib:/opt/intel/openvino_2022/tools/compile_tool:/opt/intel/openvino_2022/runtime/3rdparty/tbb/lib::/opt/intel/openvino_2022/runtime/3rdparty/hddl/lib:/opt/intel/openvino_2022/runtime/lib/intel64
2022-07-21 15:47:25.036750: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
2022-07-21 15:47:25.036761: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (my-adl-s): /proc/driver/nvidia/version does not exist
2022-07-21 15:47:25.036889: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 AVX_VNNI FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
Available Backends:
CPU
GPU
2022-07-21 15:47:31.407075: OVTF Summary -> 59 out of 1326 nodes in the graph (4%) are now running with OpenVINO™ backend
Segmentation fault (core dumped)

Is there a specific Tensorflow version required? Here, I meet a question that my other AI components don't work well when developing projects, they only support up to Tensorflow 2.3.1, but I'd love to accelerate with openvino. Is there a way to speed it up? I would appreciate it if possible.

In this notebook, when executing native tensorflow inference, it reports latency and printing:

OpenVINO TensorFlow is disabled
Loading input image...
Loading efficientdet_d6_coco17_tpu-32...
efficientdet_d6_coco17_tpu-32 loaded successfully!
Running 5 warmup inference iterations...
Running 20 inference iterations...
Inference Successfully completed on OpenVINO TensorFlow..! efficientdet_d6_coco17_tpu-32 model run on CPU in 3554.2 ms

Since OpenVINO Tensorflow is disabled, it should print:

Inference Successfully completed on native TensorFlow..! efficientdet_d6_coco17_tpu-32 model run on CPU in 3554.2 ms

According to openvinotoolkit/openvino#16472 (comment) OpenVINO integrates with PyTorch for training, but at the same time openvino_tensorflow package is abandoned in favor of Intel Extension for TensorFlow (ITEX)

I've browsed ITEX and looks like it is not based on OpenVINO.
As for me OpenVINO is completely suitable for such plugin like PluggableDevice for Tensorflow ...

For me it looks like 1 step forward and 1 step backward ...

It is obvious that OpenVINO will benefit from integration with Tensorflow and PyTorch as backend for training.

See also my comment here openvinotoolkit/openvino#16472 (reply in thread)

Greetings,

notebooks has problems to run in Windows 10 x64,

Thanks in advance

When the OpenVINO environment are set, the package can not be imported, failing with the error below. It seems to be a conflict on some binaries.

import openvino_tensorflow
Traceback (most recent call last):
File "", line 1, in
File "/home/nuc_openvino/.local/lib/python3.8/site-packages/openvino_tensorflow/init.py", line 92, in
_ = load_library.load_op_library(full_lib_path)
File "/home/nuc_openvino/.local/lib/python3.8/site-packages/tensorflow/python/framework/load_library.py", line 58, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: /home/nuc_openvino/.local/lib/python3.8/site-packages/openvino_tensorflow/libopenvino_tensorflow.so: undefined symbol: _ZNK6ngraph4Node11descriptionEv

When the OV environment variables are not set, it works fine.
Test performed with Python 3.8 on Ubuntu 20 with OpenVINO 2021.2

Readme suggests "pip3 install -U tensorflow==2.5.0" when running the sample code it fails and says OVTF built with 2.4.1

I followed the exact procedure described here, which installed without error.

However, openvino_tensorflow.list_backends() only lists my CPU. The BIOS describes my graphics card inside a Dell Latitude 7430 as "Intel Iris Xe Graphics". This webpage lists Iris graphics as supported by Openvino.

How do I proceed from here?

We observed that the program fails whenever the same image has been passed to sess.run() more than once. This error only happens with the SSD object detection models. For example:

Error occurs here:

OVTF Summary -> 2093 out of 4245 nodes in the graph (49%) are now running with OpenVINO™ backend
Predict Image [10]
Predict Image [89]
Predict Image [10] --> Segmentation fault

Error doesn't occur if we reinitialise the session:

OVTF Summary -> 2093 out of 4245 nodes in the graph (49%) are now running with OpenVINO™ backend
Predict Image [10]
Predict Image [89]
OVTF Summary -> 2093 out of 4245 nodes in the graph (49%) are now running with OpenVINO™ backend
Predict Image [10] 
...

Here is a code snippet for where the error was at:

Model: ssd_inception_v2,ssd_mobilenet_v1,ssd_mobilenet_v1_fpn,ssd_mobilenet_v2,ssd_resnet_50_fpn, ssdlite_mobilenet_v2
TensorFlow version: 2.6.0
openvino-tensorflow version: 1.0.0

class BackendTensorflowOpenvino(...)
  ...
  def load(self, model_path, inputs=None, outputs=None):
      # there is no input/output meta data i the graph so it need to come from config.
      if not inputs:
          raise ValueError("BackendTensorflow needs inputs")
      if not outputs:
          raise ValueError("BackendTensorflow needs outputs")
      self.outputs = outputs
      self.inputs = inputs

      infer_config = tf.compat.v1.ConfigProto()
      infer_config.intra_op_parallelism_threads = int(os.environ['TF_INTRA_OP_PARALLELISM_THREADS']) \
              if 'TF_INTRA_OP_PARALLELISM_THREADS' in os.environ else os.cpu_count()
      infer_config.inter_op_parallelism_threads = int(os.environ['TF_INTER_OP_PARALLELISM_THREADS']) \
              if 'TF_INTER_OP_PARALLELISM_THREADS' in os.environ else os.cpu_count()
      infer_config.use_per_session_threads = 1

      graph_def = tf.compat.v1.GraphDef()
      with tf.compat.v1.gfile.FastGFile(model_path, "rb") as f:
          graph_def.ParseFromString(f.read())
          g = tf.compat.v1.import_graph_def(graph_def, name='')
          
      self.sess = tf.compat.v1.Session(graph=g, config=infer_config)
      return self

  def predict(self, feed):
      ans = self.sess.run(self.outputs, feed_dict=feed)
      return ans

import openvino_tensorflow as ovtf
ovtf.set_backend('CPU')
model = BackendTensorflowOpenvino(...)
model.predict({"image_tensor:0": img1})
model.predict({"image_tensor:0": img1})

The program fails at the second model.predict() call. Error Message:

./tmp-e9_0_ipe.sh: line 42: 65995 Segmentation fault      (core dumped)

Also, the above code works with the original tensorflow backend and other models ( e.g. faster_rcnn_inception_v2_coco, faster_rcnn_resnet50_coco) from TensorFlow Object Detection Model Zoo and yolo-v3.

To reproduce the error, we can use ck - an automated workflow for designing ML systems.

pip install ck

Then, pull the relevant program:

ck pull repo --url=https://github.com/krai/ck-mlperf.git
ck pull repo --url=https://github.com/krai/ck-object-detection.git

Then, follow the building instruction here. And finally, run the following command

time docker run -it --rm ${CK_IMAGE} \
"ck run program:mlperf-inference-vision --cmd_key=direct --skip_print_timers \
  --env.CK_LOADGEN_SCENARIO=SingleStream \
  --env.CK_LOADGEN_MODE='--accuracy' \
  --env.CK_LOADGEN_EXTRA_PARAMS='--count 50' \
  \
  --env.CK_MODEL_PROFILE=default_tf_object_det_zoo \
  --dep_add_tags.weights=ssd_mobilenet_v1_coco \
  \
  --env.CK_INFERENCE_ENGINE=tensorflow \
  --env.CK_INFERENCE_ENGINE_BACKEND=openvino-cpu \
  --env.CUDA_VISIBLE_DEVICES=-1"

Connecting to pjreddie.com (pjreddie.com)|128.208.4.108|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 248007048 (237M) [application/octet-stream]
Saving to: ‘weights/yolov3.weights’

weights/yolov3.weights 100%[=================================================>] 236.52M 40.8MB/s in 6.9s

2021-09-03 10:17:16 (34.3 MB/s) - ‘weights/yolov3.weights’ saved [248007048/248007048]

./convert_yolov3.sh: line 17: python: command not found
./convert_yolov3.sh: line 18: python: command not found
cp: cannot stat 'weights/yolo_v3_darknet_2.pb': No such file or directory
cp: cannot stat 'weights/darknet53.h5': No such file or directory

Hi! Using tf.Module we are not able to run the layer through OpenVINO:

import time
import openvino_tensorflow as ovtf
ovtf.set_backend('CPU')
ovtf.start_logging_placement()

import numpy as np
import tensorflow as tf

w = tf.constant(np.random.normal(size=([1000,8000])).astype(np.float32))

class MyFunction(tf.Module):

  @tf.function(input_signature=[tf.TensorSpec(shape=[1000,1000], dtype=tf.float32)])
  def fun(self, x):
    y=tf.matmul(x, w)
    return {"custom_output_name":y}

to_export = MyFunction()
tf.saved_model.save(to_export, '/tmp/myfunction')

saved_model_loaded = tf.saved_model.load('/tmp/myfunction')
infer = saved_model_loaded.signatures["serving_default"]
inp=tf.constant(np.random.normal(size=([1000,1000])).astype(np.float32))

res=infer(x=inp)

logs:

2021-04-29 10:31:28.461500: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: NOTANOP: _SOURCE<NoOp>[-1] -> _SINK<NoOp>[-1]
2021-04-29 10:31:28.461518: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: UNSUPPORTED: x<_Arg>[0] -> PartitionedCall/MatMul<MatMul>[0]
2021-04-29 10:31:28.461543: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: UNSUPPORTED: unknown<_Arg>[0] -> PartitionedCall/MatMul<MatMul>[1]
2021-04-29 10:31:28.461553: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: UNSUPPORTED: PartitionedCall/MatMul<MatMul>[0] -> identity_RetVal<_Retval>[0]
2021-04-29 10:31:28.461561: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: NOTANOP: _SOURCE<NoOp>[-1] -> x<_Arg>[-1]
2021-04-29 10:31:28.461569: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: NOTANOP: _SOURCE<NoOp>[-1] -> unknown<_Arg>[-1]
2021-04-29 10:31:28.461577: I /home/dkurt/openvino_tensorflow/openvino_tensorflow/assign_clusters.cc:504] NONCONTRACTION: NOTANOP: identity_RetVal<_Retval>[-1] -> _SINK<NoOp>[-1]
Encapsulate i->j: non contraction reason histogram (Cannot be UNSUPPORTED, NOTANOP or SAMECLUSTER because unsupported ops will not be assigned an encapsulate)

OVTF_SUMMARY: Summary of reasons why a pair of edge connected encapsulates did not merge
OVTF_SUMMARY: DEADNESS: 0, STATICINPUT: 0, PATHEXISTS: 0
OVTF_SUMMARY: Summary of reasons why a pair of edge connected clusters did not merge
OVTF_SUMMARY: NOTANOP: 4, UNSUPPORTED: 3, DEADNESS: 0, SAMECLUSTER: 0, STATICINPUT: 0, PATHEXISTS: 0

OVTF_SUMMARY: Number of nodes in the graph: 6
OVTF_SUMMARY: Number of nodes marked for clustering: 1 (16% of total nodes)
OVTF_SUMMARY: Number of nodes assigned a cluster: 0 (0% of total nodes)          (0% of nodes marked for clustering) 
OVTF_SUMMARY: Number of ngraph clusters :0
OVTF_SUMMARY: Nodes per cluster: 0
OVTF_SUMMARY: Op_deassigned:  MatMul -> 1 
OP_placement:   Host    _SOURCE (NoOp)
OP_placement:   Host    _SINK (NoOp)
OP_placement:   Host    x (_Arg)
OP_placement:   Host    unknown (_Arg)
OP_placement:   Host    PartitionedCall/MatMul (MatMul)
OP_placement:   Host    identity_RetVal (_Retval)

OVTF_SUMMARY: Types of edges:: args: 0, retvals: 0, both arg and retval: 0, free: 4, encapsulated: 3, total: 7, computed total: 7

=============Ending sub-graph logs=============

import tensorflow as tf
import openvino_tensorflow
print('TensorFlow version: ',tf.version)

2023-06-29 13:17:11.072582: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'cudart64_110.dll'; dlerror: cudart64_110.dll not found
2023-06-29 13:17:11.072667: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
Traceback (most recent call last):
File "C:\Users\Smi1e\Desktop\文件\ont\NANOCALLER\NanoCaller-master\nanocaller_src\try.py", line 125, in
import openvino_tensorflow
File "D:\applications\anaconda\envs\arc_tf\lib\site-packages\openvino_tensorflow_init_.py", line 122, in
_ = load_library.load_op_library(full_lib_path)
File "D:\applications\anaconda\envs\arc_tf\lib\site-packages\tensorflow\python\framework\load_library.py", line 54, in load_op_library
lib_handle = py_tf.TF_LoadLibrary(library_filename)
tensorflow.python.framework.errors_impl.NotFoundError: D:\applications\anaconda\envs\arc_tf\lib\site-packages\openvino_tensorflow\openvino_tensorflow.dll not found

On my machine, the classification_sample example returns this error :

_Create kernel failed: Not found: No registered 'nGraphEncapsulate' OpKernel for 'GPU' devices compatible with node

I am able to successfully run this command :

python3 -c "import tensorflow as tf; print('TensorFlow version: ',tf.version);
import openvino_tensorflow; print(openvino_tensorflow.version)"

The full error message is :

$ python3 examples/classification_sample.py
2021-09-27 14:45:03.705017: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
Available Backends:
CPU
MYRIAD
2021-09-27 14:45:04.998781: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-09-27 14:45:04.999214: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcuda.so.1
2021-09-27 14:45:05.005135: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-09-27 14:45:05.005512: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: Quadro M2200 computeCapability: 5.2
coreClock: 1.036GHz coreCount: 8 deviceMemorySize: 3.94GiB deviceMemoryBandwidth: 82.08GiB/s
2021-09-27 14:45:05.005547: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-09-27 14:45:05.008327: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublas.so.11
2021-09-27 14:45:05.008417: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcublasLt.so.11
2021-09-27 14:45:05.009473: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcufft.so.10
2021-09-27 14:45:05.009687: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcurand.so.10
2021-09-27 14:45:05.010126: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusolver.so.11
2021-09-27 14:45:05.010675: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcusparse.so.11
2021-09-27 14:45:05.010834: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudnn.so.8
2021-09-27 14:45:05.010936: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-09-27 14:45:05.011675: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-09-27 14:45:05.012200: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0
2021-09-27 14:45:05.012250: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2021-09-27 14:45:05.344618: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix:
2021-09-27 14:45:05.344663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0
2021-09-27 14:45:05.344670: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N
2021-09-27 14:45:05.344869: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-09-27 14:45:05.345291: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-09-27 14:45:05.345681: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:937] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero
2021-09-27 14:45:05.346057: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3113 MB memory) -> physical GPU (device: 0, name: Quadro M2200, pci bus id: 0000:01:00.0, compute capability: 5.2)
2021-09-27 14:45:05.380470: I tensorflow/core/platform/profile_utils/cpu_utils.cc:114] CPU Frequency: 3000000000 Hz
2021-09-27 14:45:06.246353: OVTF Summary -> 476 out of 480 nodes in the graph (99%) are now running with OpenVINO™ backend
2021-09-27 14:45:06.289576: E tensorflow/core/framework/op_segment.cc:54] Create kernel failed: Not found: No registered '_nGraphEncapsulate' OpKernel for 'GPU' devices compatible with node {{node ovtf_cluster_0}}
. Registered: device='CPU'

Want to enable macro “TF_SRC_DIR” and compile the C++ classification example, so can anyone give me a simple guide? Thanks a lot.

Can we consider renaming set_backend to set_device? This change will make this interface consistent with what TF and PT have and avoid confusion:

• https://www.tensorflow.org/api_docs/python/tf/device
• https://pytorch.org/docs/stable/tensors.html#torch.Tensor.to

OpenVINO is already a backend for TF and calling set_backend for it looks really confusing.

I find it will crash when I use transpose_sinking pass through a ngraph function which only have one Conv op and two inserted transpose op which is used to handle the nhwc input and output. I suppose it should do nothing in this situation instead of crash.

Greetings,

When I tried to install openvino_tensorlow, the following error came out:

ERROR: tensorflow-2.9.1-cp39-cp39-win_amd64.whl is not a supported wheel on this platform.

Any clue? Thanks in advance

my system:

windows 10
python 3.8, in the venv Im trying to install
pip command: pip install

I try changing 3.8 with 3.9, NOTHING....

Had anyone installed openvino_tensorflow in Windows 10 x64?

Thanks in advance

Command:
python build_ov.py --output_dir="D:\openvino2"

Traceback (most recent call last):
File "D:\openvino_tensorflow-2.3.0\build_ov.py", line 90, in
main()
File "D:\openvino_tensorflow-2.3.0\build_ov.py", line 77, in main
build_openvino(build_dir, openvino_src_dir, cxx_abi, arguments.target_arch,
TypeError: build_openvino() missing 1 required positional argument: 'threading'

Error:
Abort was called at 22 line in file:
../neo/runtime/gmm_helper/gmm_interface.cpp
Aborted (core dumped)

Code:
import openvino_tensorflow
openvino_tensorflow.list_backends()

Installation steps followed:
pip3 install -U --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ tensorflow-custom-abi1==2.2.2
source /bin/setupvars.sh
pip3 install -U --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ openvino-tensorflow-addon-abi1

I am trying to install Openvino_tensorflow library but facing incompatible TF library issue

Code sample:
import tensorflow as tf
print('TensorFlow version: ',tf.version)
import openvino_tensorflow
print(openvino_tensorflow.version)

Error:

TensorFlow version: 2.4.1
Traceback (most recent call last):
File "test.py", line 6, in
import openvino_tensorflow
File "/home/nuc2/.local/lib/python3.6/site-packages/openvino_tensorflow/init.py", line 97, in
.format(TF_VERSION, TF_VERSION_NEEDED))
ValueError: Error: Installed TensorFlow version 2.4.1
openvino_tensorflow built with: 2.2.2

Steps used for installation in a conda environment:
pip3 install -U pip==21.0.1
pip3 install -U tensorflow==2.4.1
pip3 install openvino-tensorflow

Is there a specific Python version required? Right now when I try this on Mac 10.15.7 it won't work.

(venv) (base) rlo@rlo-mac01 openvino_tensorflow % python
Python 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.


pip3 install -U --index-url https://test.pypi.org/simple/ --extra-index-url https://pypi.org/simple/ openvino-tensorflow-addon-abi0

Looking in indexes: https://test.pypi.org/simple/, https://pypi.org/simple/
ERROR: Could not find a version that satisfies the requirement openvino-tensorflow-addon-abi0
ERROR: No matching distribution found for openvino-tensorflow-addon-abi0

After installing openvino_tensorflow, the message "Entry Point not Found" shows up on the command
python "import openvino_tensorflow as ovtf"

Installation:
python 3.9,windows10
install with to
pip3 install -U pip
pip3 install tensorflow==2.8.0
pip3 install openvino-tensorflow==2.0.0

Test under python:
import openvino_tensorflow
See image for error message

Did I missed something simple?

I tested the notebook from the TF beginner page: https://www.tensorflow.org/tutorials/quickstart/beginner
Every call with .numpy() fails with the error:
'Tensor´ object has no attribute ´numpy´
They are for value display, so the training and inference works if we withdraw them, but since it is a basic TF code, we should expect compatibility.

I have attached the notebook where I have added the openvino_tensorflow package.
beginner.zip

when I pip installed the latest version of of tensorflow 2.91 along with openvino-tensorflow 2.1.0 I get this errow that is thrown when importing openvino_tensorflow.
libtbb.so.2: cannot open shared object file: No such file or directory
the libopenvino is looking for libtbb.so.2 whereas the latest libtbb is libtbb.so.12 which is found in the lib folder
I am surprised this has not been tested !!!