This library contains methods to send data from your C++ application directly into the TensorFlow graph without any large dependency. Unfortunately, generating data within a C++ application for TensorFlow is not envisaged and communicated as not requested for the TF core. Even the discussion about the redesign of the input pipeline seems to converge to put all data-processing into the graph.
This is hilarious as most solutions for interesting problems are not based on a mindless iteration over a set of images. This small library helps to send data directly from your favorite C++ application (game-engine, renderer, emulator, [put here what you want]) to TensorFlow. Reading data is as simple as (see read_tf.py):
import tensorflow as tf
import zmq_op
image, label = zmq_op.pull('ipc:///tmp/ipc-socket-0', [tf.float32, tf.int32])
with tf.Session() as sess:
print(sess.run([image, label]))The C++ code to send data is about 45 lines in write.cpp. It uses ZMQ for an easy distributed messaging and msgPack for a fast serialization. You are free to change the shape of each tensor over time. If you want to receive the tensors in native python without TensorFlow dependencies, you can use read_py.py. This library supports all combinations:
| sender | receiver |
|---|---|
| write.py | TF (read_tf.py) |
| write.cpp | TF (read_tf.py) |
| write.py | native python (read_py.py) |
| write.cpp | native python (read_py.py) |
| write.py | plain C++ (read.cpp) |
| write.cpp | plain C++ (read.cpp) |
You basically need two dependencies (see .travis.yml).
# compile ZMQ library for c++
cd /path/to/your_lib_folder
git clone https://github.com/zeromq/libzmq
cd libzmq
./autogen.sh
./configure
./configure --prefix=/path/to/your_lib_folder/libzmq/dist
make
make install
# compile MSGPACK library for c++
cd ../
git clone https://github.com/msgpack/msgpack-c
cd msgpack-c
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX:PATH=/path/to/your_lib_folder/msgpack-c/dist
make
make installexport PKG_CONFIG_PATH=/path/to/your_lib_folder/libzmq/dist/lib/pkgconfig/:$PKG_CONFIG_PATH
export PKG_CONFIG_PATH=/path/to/your_lib_folder/msgpack-c/dist/lib/pkgconfig/:$PKG_CONFIG_PATH
git clone https://github.com/PatWie/tf_zmq.git
cd tf_zmq
./compile.shThere are 2 alternative attempts for solving this problem:
- Tensorpack: This solution requires a python script to send data (it is based on TensorProto protobuf (currently, symbols for this are not exported by TF))
- pull-request TF: Here you need to specify the shape dimensions in advance (also pure-python because the TensorProto dependency)
You probably do not want to link a game engine or another large codebase with the TensorFlow core using bazel, given compiling TensorFlow alone can be a nightmare and is only recommended for adventurer.
React
Typescript
Django
Laravel
Facebook
Microsoft
Google
Alibaba
D3
Tencent