Coder Social home page Coder Social logo

contributon2019-nns's People

Contributors

wooksong avatar

Stargazers

avatar avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar avatar avatar

Forkers

mojunsang26 lee-wonjun niklasjang hwanseok-dev vincent-kwon l2zz

contributon2019-nns's Issues

How to run nnstreamer-examples (native)

Prerequisites

If you've already done some of the following steps in #1 and #2, just skip those steps.

$ sudo apt-add-repository ppa:nnstreamer
$ sudo apt install git meson ninja-build cmake
$ sudo apt install nnstreamer-*
$ sudo apt install libglib2.0-dev libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base gstreamer1.0-plugins-good,
$ sudo apt install tensorflow-c-dev tensorflow-c-dev protobuf-dev libcairo2-dev libopencv-dev

$ git clone https://github.com/nnsuite/nnstreamer-example.git
$ cd nnstreamer-example

Build

$ meson build
The Meson build system
Version: 0.50.1
Source dir: /home/wook/Work/nnstreamer-example
Build dir: /home/wook/Work/nnstreamer-example/build
Build type: native build
Project name: nnstreamer-example
Project version: 0.1.0
Native C compiler: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
Native C++ compiler: c++ (gcc 7.4.0 "c++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
Build machine cpu family: x86_64
Build machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Dependency glib-2.0 found: YES 2.56.4
Dependency gstreamer-1.0 found: YES 1.14.5
Dependency gstreamer-base-1.0 found: YES 1.14.5
Dependency gstreamer-video-1.0 found: YES 1.14.5
Dependency gstreamer-audio-1.0 found: YES 1.14.5
Dependency gstreamer-app-1.0 found: YES 1.14.5
Dependency cairo found: YES 1.15.10
Library m found: YES
Library dl found: YES
Dependency threads found: YES 
Dependency nnstreamer found: YES 0.3.0
Dependency tensorflow found: YES 1.13.1
Dependency protobuf found: YES 3.6.1
Dependency tensorflow-lite found: YES 1.13.1
Dependency caffe2 found: YES 1.1.0
Has header "nnstreamer/tensor_filter_custom.h" : YES
Build targets in project: 12
$ ninja -C build

Run

$ cd build/native
$ ls -1
example_cam
example_decoder_image_labelling
example_filter_performance_profile
example_image_classification_caffe2
example_image_classification_tflite
example_object_detection_tensorflow
example_object_detection_tensorflow_lite
example_sink
example_speech_command_tensorflow_lite
example_two_tensor_stream
  1. Basic examples
  • example_cam
$ pushd example_cam
$ ./nnstreamer_example_cam 
$ popd
  • example_sink
$ pushd example_sink
$ ./nnstreamer_sink_example_play 
$ popd
  1. Object detection examples
  • example_object_detection_tensorflow
$ pushd example_object_detection_tensorflow
$ bash ../../../bash_script/example_models/get-model-object-detection-tf.sh 
$ ./nnstreamer_example_object_detection_tf
$ popd
  • example_object_detection_tensorflow_lite
$ pushd example_object_detection_tensorflow_lite
$ bash ../../../bash_script/example_models/get-model-object-detection-tflite.sh 
$ ./nnstreamer_example_object_detection_tflite 
$ popd
  • example_filter_performance_profile: skipped
  1. Image classification examples
  • example_image_classification_tflite
$ pushd example_image_classification_tflite
$ bash ../../../bash_script/example_models/get-model-image-classification-tflite.sh 
$ ./nnstreamer_example_image_classification_tflite 
$ popd
  • example_image_classification_caffe2: skipped

[Tests/Sink] Casting from unsigned to signed is dangerous

In ./tests/nnstreamer_sink/unittest_sink.cpp, the variable, num_buffers, in the struct TestOption is treated as int. It is very dangerous.

For example,

 444   switch (option.test_type) {                                                   
 445     case TEST_TYPE_VIDEO_RGB:                                                   
 446       /** video 160x120 RGB */                                                  
 447       str_pipeline =                                                            
 448           g_strdup_printf                                                       
 449           ("videotestsrc num-buffers=%d ! videoconvert ! video/x-raw,width=160,height=120,format=RGB,framerate=(fraction)30/1 ! "
 450           "tensor_converter ! tensor_sink name=test_sink", option.num_buffers); 
 451       break;

could we guarantee that this code would work as we intended?

Apply regular expression if possible

In order to check matching patterns in a variable, we have used combined if-statements a lot. This is obviously very simple way, but also inefficient way IMO.

For example, Talse could be handled as True in the following code.

 71 static gboolean                                                                 
 72 _parse_bool_string (const gchar * strval, gboolean def)                         
 73 {                                                                               
 74   gboolean res = def;                                                           
 75                                                                                 
 76   if (strval) {                                                                 
 77     /* 1/0, true/false, t/f, yes/no, on/off. case incensitive. */               
 78     if (strval[0] == '1' || strval[0] == 't' || strval[0] == 'T' ||             
 79         strval[0] == 'y' || strval[0] == 'Y' ||                                 
 80         !g_ascii_strncasecmp ("on", strval, 2)) {                               
 81       res = TRUE;                                                               
 82     } else if (strval[0] == '0' || strval[0] == 'f' || strval[0] == 'F' ||      
 83         strval[0] == 'n' || strval[0] == 'N' ||                                 
 84         !g_ascii_strncasecmp ("of", strval, 2)) {                               
 85       res = FALSE;                                                              
 86     }                                                                           
 87   }                                                                             
 88                                                                                 
 89   return res;                                                                   
 90 }

How about applying regular expressions to such logics in our project?

Opinion about NNStreamer Android guide

NNStreamer API Library for Android

Android GUI NNStreamer Application Examples

How to build Android applications on Ubuntu

위의 가이드를 진행해본결과 개선의 여지가 필요하다고 생각되었습니다.

  1. NNStreamer API Library for Android 에서 권한 문제

Build Android API

Run the build script in NNStreamer.
After building the Android API, you can find the library(.aar) in $NNSTREAMER_ROOT/android_lib.

$ cd $NNSTREAMER_ROOT
$ bash ./api/android/build-android-lib.sh

해당 부분을 진행할떄
./gradlew 파일의 퍼미션 문제로 fail이 발생합니다.
생성하는 build-android-lib.sh 가 생성하는 ./gradlew 파일의 퍼미션을 변경하는 sudo chmod +x ./gradlew 의 코드가 필요하다고 생각됩니다.

# Build Android library.
sudo chmod +x ./gradlew    #Add
./gradlew api:build

  1. NNStreamer API Library for Android Prerequisite 관련
    Android SDK Build-Tools 28.0.3 을 필요합니다.
    위와 같은 부분을 진행할떄 Android SDK Build-Tools 28.0.3 설치를 요구합니다
    Prerequisite 에 추가되면 좋겠습니다.

  2. SVN 관련
    해당 가이드들을 진행할때 파일설치시 SVN을 사용합니다.
    파일을 받아올때 SVN 말고 다른 방식으로 guild 및 스크립트가 작성되있으면 좋겠습니다.

  3. How to build Android applications on Ubuntu 가이드 관련
    해당 가이드를 보고 예제및 api빌드하기가 어렵습니다. 수정되거나
    NNStreamer API Library for Android,Android GUI NNStreamer Application Examples 등의 링크를 첨부하는 식으로 개선할수있을듯 합니다,

  4. Android GUI NNStreamer Application Examples 에서 common 폴더 관련

해당 가이드의

Extract external libraries into common directory.

extfiles.tar.xz includes external library such as 'ahc'.

tensorflow-lite-1.13.tar.xz includes the libraries and header files of tensorflow-lite.

$ cd $ANDROID_DEV_ROOT/workspace/nnstreamer-example/android/example_app/common/jni
$ tar xJf ./extfiles.tar.xz
$ svn --force export https://github.com/nnsuite/nnstreamer-android-resource/trunk/android_api/ext-files/tensorflow-lite-1.13.tar.xz
$ tar xJf ./tensorflow-lite-1.13.tar.xz # Check tensorflow-lite version and extract prebuilt library
$ ls ahc tensorflow-lite

common 폴더까지 들어가서
해당부분의 extfiles 는 포함된 압축파일을 직접 풀고
tensorflow-lite 는 svn으로 받은뒤 압축을 푸는 과정을 거칩니다.
일관성이 맞지않고 자동화 스크립트를 만들어주어 개선할수 있을것 같습니다.

[CodeClean] Improve the readability

nnstreamer/nnstreamer#1786 (comment)

static gboolean
gst_tensor_transform_convert_dimension (GstTensorTransform * filter,
    GstPadDirection direction, const GstTensorInfo * in_info,
    GstTensorInfo * out_info)
{
  int i;

  switch (filter->mode) {
    case GTT_DIMCHG:
      out_info->type = in_info->type;

      if (direction == GST_PAD_SINK) {
        int a = filter->data_dimchg.from;
        int b = filter->data_dimchg.to;

        for (i = 0; i < NNS_TENSOR_RANK_LIMIT; i++) {
          if ((i < a && i < b) || (i > a && i > b) || a == b) {
            out_info->dimension[i] = in_info->dimension[i];
          } else if (i == b) {
            out_info->dimension[i] = in_info->dimension[a];
          } else {
            if (a > b) {
              g_assert (i > 0 && i > b);
              out_info->dimension[i] = in_info->dimension[i - 1];
            } else {
              g_assert (i < b && i < (NNS_TENSOR_RANK_LIMIT - 1));
              out_info->dimension[i] = in_info->dimension[i + 1];
            }
          }
        }
      } else {
        int a = filter->data_dimchg.from;
        int b = filter->data_dimchg.to;

        for (i = 0; i < NNS_TENSOR_RANK_LIMIT; i++) {
          if ((i < a && i < b) || (i > a && i > b) || a == b) {
            out_info->dimension[i] = in_info->dimension[i];
          } else if (i == a) {
            out_info->dimension[i] = in_info->dimension[b];
          } else {
            if (a > b) {
              g_assert (i < a && i < (NNS_TENSOR_RANK_LIMIT - 1));
              out_info->dimension[i] = in_info->dimension[i + 1];
            } else {
              g_assert (i > 0 && i > a);
              out_info->dimension[i] = in_info->dimension[i - 1];
            }
          }
        }
      }
      break;
...omission...

위 함수의 변수 a, b를 좀 더 의미 있는 이름으로 바꿔서 가독성을 높이면 좋을 것 같습니다!

How to run nnstreamer-examples (bash_script)

Prerequisites

If you've already done some of the following steps in #1 and #2, just skip those steps.

$ sudo apt-add-repository ppa:nnstreamer
$ sudo apt install nnstreamer-*
$ sudo apt install gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-plugins-bad

$ git clone https://github.com/nnsuite/nnstreamer-example.git
$ cd nnstreamer-example

TensorFlow-Lite

  1. Prepare model files
$ cd bash_script/example_object_detection_tensorflow_lite
$ bash ../example_models/get-model-object-detection-tflite.sh
...omission...
2019-09-22 00:13:00 (15.7 MB/s) - ‘ssd_mobilenet_v2_coco.tflite’ saved [67312764/67312764]
...omission...
2019-09-22 00:13:01 (61.4 MB/s) - ‘coco_labels_list.txt’ saved [665/665]
...omission...
2019-09-22 00:13:02 (600 KB/s) - ‘box_priors.txt’ saved [76932/76932]
  1. Run
$ ./gst-launch-object-detection-tflite.sh

TensorFlow

  1. Prepare model files
$ cd bash_script/example_object_detection_tensorflow
$ bash ../example_models/get-model-object-detection-tf.sh
...omission...
2019-09-22 00:28:32 (11.1 MB/s) - ‘ssdlite_mobilenet_v2.pb’ saved [19911343/19911343]
...omission...
2019-09-22 00:28:33 (70.4 MB/s) - ‘coco_labels_list.txt’ saved [665/665]
  1. Run
$ ./gst-launch-object-detection-tf.sh

[Meson] Bug: Could not find numpy/arrayobject.h when the numpy is installed using pip

https://github.com/nnsuite/nnstreamer/blob/7adb064334019680fe3b4b752d34844ebc7608f5/meson.build#L211

...omission...

if cc.has_header('numpy/arrayobject.h', args: python2_incs)

...omission...

https://github.com/nnsuite/nnstreamer/blob/7adb064334019680fe3b4b752d34844ebc7608f5/meson.build#L220

...omission...

if cc.has_header('numpy/arrayobject.h', args: python3_incs)
...omission...

Above code snippet has failed to find 'numpy/arrayobject.h' when numpy is installed via pip2/pip3 instead of apt.

[Tools] Make a tool to check whether a model can be loaded on TF-Lite plugin

While developing a new example, we got to know that some TF-Lite models are not loaded on current NNStreamer. It was due to the versions of some operators, which are beyond the version of TF-Lite plugin.

Now I'm working to make a new tool to check whether a specific model can be loaded on TF-Lite plugin of NNStreamer installed.
Using this tool, one can check if a model is usable on NNStreamer without writing a pipeline script.

Plus, this tool let us know the dimensions of input and output tensors, for models that are loaded successfully.

How to build NNStreamer form the scratch

0. Prerequisites

$ sudo apt-add-repository ppa:nnstreamer
$ sudo apt install devscripts git equivs

$ mkdir -p ~/Work
$ cd ~/Work

$ git clone https://github.com/nnsuite/nnstreamer.git
Cloning into 'nnstreamer'...
remote: Enumerating objects: 32, done.
remote: Counting objects: 100% (32/32), done.
remote: Compressing objects: 100% (26/26), done.
remote: Total 9766 (delta 12), reused 16 (delta 5), pack-reused 9734
Receiving objects: 100% (9766/9766), 25.32 MiB | 8.81 MiB/s, done.
Resolving deltas: 100% (6572/6572), done.

$ ls
nnstreamer

1. Let's create a package satisfying NNStreamer's build-dependencies

$ cd nnstreamer

$ mk-build-deps -i ./debian/control --root-cmd sudo
dh_testdir
dh_testroot
dh_prep
dh_testdir
dh_testroot
dh_install
dh_installdocs
dh_installchangelogs
dh_compress
dh_fixperms
dh_installdeb
dh_gencontrol
dh_md5sums
dh_builddeb
dpkg-deb: building package 'nnstreamer-build-deps' in '../nnstreamer-build-deps_0.3.0.0_amd64.deb'.

The package has been created.
Attention, the package has been created in the current directory,
not in ".." as indicated by the message above!
(Reading database ... 181103 files and directories currently installed.)
Preparing to unpack nnstreamer-build-deps_0.3.0.0_amd64.deb ...
Unpacking nnstreamer-build-deps (0.3.0.0) over (0.3.0.0) ...
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Correcting dependencies...Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
 Done
Starting pkgProblemResolver with broken count: 0
Starting 2 pkgProblemResolver with broken count: 0
Done
The following additional packages will be installed:
...omission...
Setting up libgstreamer1.0-dev:amd64 (1.14.5-0ubuntu1~18.04.1) ...
Setting up libgstreamer-plugins-base1.0-dev:amd64 (1.14.5-0ubuntu1~18.04.1) ...
Setting up nnstreamer-build-deps (0.3.0.0) ...

2. Do build using meson and ninja

$ meson build
The Meson build system
Version: 0.50.1
Source dir: /home/wook/Work/nnstreamer
Build dir: /home/wook/Work/nnstreamer/build
Build type: native build
Project name: nnstreamer
Project version: 0.3.0
Native C compiler: cc (gcc 7.4.0 "cc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
Native C++ compiler: c++ (gcc 7.4.0 "c++ (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0")
Build machine cpu family: x86_64
Build machine cpu: x86_64
Compiler for C supports arguments -Wredundant-decls: YES
Compiler for C++ supports arguments -Wredundant-decls: YES
Compiler for C supports arguments -Wwrite-strings: YES
Compiler for C++ supports arguments -Wwrite-strings: YES
Compiler for C supports arguments -Wformat: YES
Compiler for C++ supports arguments -Wformat: YES
Compiler for C supports arguments -Wformat-nonliteral: YES
Compiler for C++ supports arguments -Wformat-nonliteral: YES
Compiler for C supports arguments -Wformat-security: YES
Compiler for C++ supports arguments -Wformat-security: YES
Compiler for C supports arguments -Winit-self: YES
Compiler for C++ supports arguments -Winit-self: YES
Compiler for C supports arguments -Waddress: YES
Compiler for C++ supports arguments -Waddress: YES
Compiler for C supports arguments -Wno-multichar -Wmultichar: YES
Compiler for C++ supports arguments -Wno-multichar -Wmultichar: YES
Compiler for C supports arguments -Wvla: YES
Compiler for C++ supports arguments -Wvla: YES
Compiler for C supports arguments -Wpointer-arith: YES
Compiler for C++ supports arguments -Wpointer-arith: YES
Compiler for C supports arguments -Wmissing-declarations: YES
Compiler for C supports arguments -Wmissing-prototypes: YES
Compiler for C supports arguments -Wnested-externs: YES
Compiler for C supports arguments -Waggregate-return: YES
Compiler for C supports arguments -Wold-style-definition: YES
Compiler for C supports arguments -Wdeclaration-after-statement: YES
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Dependency glib-2.0 found: YES 2.56.4
Dependency gobject-2.0 found: YES 2.56.4
Dependency gstreamer-1.0 found: YES 1.14.5
Dependency gstreamer-base-1.0 found: YES 1.14.5
Dependency gstreamer-controller-1.0 found: YES 1.14.5
Dependency gstreamer-video-1.0 found: YES 1.14.5
Dependency gstreamer-audio-1.0 found: YES 1.14.5
Dependency gstreamer-app-1.0 found: YES 1.14.5
Dependency gstreamer-check-1.0 found: YES 1.14.5
Library m found: YES
Library dl found: YES
Dependency threads found: YES 
Dependency protobuf found: YES 3.6.1
Dependency orc-0.4 found: YES 0.4.28
Program orcc found: YES (/usr/bin/orcc)
Dependency tensorflow found: YES 1.13.1
Dependency tensorflow-lite found: YES 1.13.1
Dependency pytorch found: YES 1.1.0
Dependency caffe2 found: YES 1.1.0
Program pkg-config found: YES (/usr/bin/pkg-config)
Dependency python-2.7 found: YES 2.7
Has header "numpy/arrayobject.h" : YES
Dependency python3 found: YES 3.6
Has header "numpy/arrayobject.h" : YES
Dependency opencv found: YES 3.2.0
Dependency libpng found: YES 1.6.34
Program cp found: YES (/bin/cp)
Dependency GTest found: YES (building self)
Configuring nnstreamer-test.ini using configuration
Configuring nnstreamer.ini using configuration
Configuring nnstreamer.pc using configuration
Build targets in project: 44
Found ninja-1.8.2 at /usr/bin/ninja

$ ninja -C build
ninja: Entering directory `build'
[146/146] Linking target tests/unittest_src_iio.

The target files are placed in ./build.

$ cd build
$ ls -l
total 352
drwxrwxr-x  2 wook wook   4096  9월 20 22:11 api
-rw-rw-r--  1 wook wook 177566  9월 20 22:11 build.ninja
-rw-rw-r--  1 wook wook 131870  9월 20 22:11 compile_commands.json
drwxrwxr-x  3 wook wook   4096  9월 20 22:11 ext
drwxrwxr-x  3 wook wook   4096  9월 20 22:11 gst
drwxrwxr-x  2 wook wook   4096  9월 20 22:11 meson-info
drwxrwxr-x  2 wook wook   4096  9월 20 22:11 meson-logs
drwxrwxr-x  2 wook wook   4096  9월 20 22:11 meson-private
drwxrwxr-x  8 wook wook   4096  9월 20 22:11 nnstreamer_example
-rw-rw-r--  1 wook wook    663  9월 20 22:11 nnstreamer.ini
-rw-rw-r--  1 wook wook    371  9월 20 22:11 nnstreamer.pc
-rw-rw-r--  1 wook wook    664  9월 20 22:11 nnstreamer-test.ini
drwxrwxr-x 10 wook wook   4096  9월 20 22:12 tests

How to build NNStreamer using pdebuild

0. Prerequisites

$ sudo apt install pbuilder debootstrap devscripts

1. Create .pbuilderrc, which a configuration file for pbuilder

$ ls ~/.pbuilderrc
ls: cannot access '/home/wook/.pbuilderrc': No such file or directory
  • For Ubuntu 18.04 (bionic),
$ cat << EOF > ~/.pbuilderrc
> DISTRIBUTION=bionic
> COMPONENTS="main restricted universe multiverse"
> OTHERMIRROR="deb http://archive.ubuntu.com/ubuntu bionic-backports universe | deb [trusted=yes] http://ppa.launchpad.net/nnstreamer/ppa/ubuntu bionic main"
> EOF
  • For Ubuntu 16.04 (xenial),
$ cat << EOF > ~/.pbuilderrc
> DISTRIBUTION=xenial
> COMPONENTS="main restricted universe multiverse"
> OTHERMIRROR="deb http://archive.ubuntu.com/ubuntu xenial-backports universe | deb [trusted=yes] http://ppa.launchpad.net/nnstreamer/ppa/ubuntu xenial main"
> EOF
  • You can see that .pbuilderrc is located in your home directory.
$ ls ~/.pbuilderrc 
/home/wook/.pbuilderrc
$ cat ~/.pbuilderrc 
DISTRIBUTION=bionic
COMPONENTS="main restricted universe multiverse"
OTHERMIRROR="deb http://archive.ubuntu.com/ubuntu bionic-backports universe | deb [trusted=yes] http://ppa.launchpad.net/nnstreamer/ppa/ubuntu bionic main"

2. Create a base.tgz for Ubuntu 18.04 or Ubuntu 16.04

$ sudo ln -s ~/.pbuilderrc  /root/.pbuilderrc
$ sudo pbuilder create
I: Distribution is bionic.
I: Current time: Fri Sep 20 22:40:18 KST 2019
I: pbuilder-time-stamp: 1568986818
I: Building the build environment
I: running debootstrap
/usr/sbin/debootstrap
I: Retrieving InRelease 
I: Checking Release signature
I: Valid Release signature (key id 790BC7277767219C42C86F933B4FE6ACC0B21F32)
I: Retrieving Packages 
I: Validating Packages 
I: Resolving dependencies of required packages...
I: Resolving dependencies of base packages...
...omission...
I: creating base tarball [/var/cache/pbuilder/base.tgz]
I: cleaning the build env 
I: removing directory /var/cache/pbuilder/build/19414 and its subdirectories

3. Generate debian package files (.deb) for NNStreamer

$ pdebuild
$ time pdebuild
dh clean --parallel
   debian/rules override_dh_auto_clean
make[1]: Entering directory '/home/wook/Work/nnstreamer'
rm -rf build
make[1]: Leaving directory '/home/wook/Work/nnstreamer'
   dh_clean
dpkg-source: warning: no source format specified in debian/source/format, see dpkg-source(1)
dpkg-source: warning: source directory 'nnstreamer' is not <sourcepackage>-<upstreamversion> 'nnstreamer-0.3.0.0'
dpkg-source: info: using source format '1.0'
dpkg-source: info: building nnstreamer in nnstreamer_0.3.0.0.tar.gz
dpkg-source: info: building nnstreamer in nnstreamer_0.3.0.0.dsc
I: Generating source changes file for original dsc
dpkg-genchanges: info: including full source code in upload
I: pbuilder: network access will be disabled during build
I: Current time: Fri Sep 20 22:46:08 KST 2019
I: pbuilder-time-stamp: 1568987168
I: Building the build Environment
...omission...
I: copying local configuration
I: Copying back the cached apt archive contents
I: unmounting dev/ptmx filesystem
I: unmounting dev/pts filesystem
I: unmounting dev/shm filesystem
I: unmounting proc filesystem
I: unmounting sys filesystem
I: cleaning the build env 
I: removing directory /var/cache/pbuilder/build/11971 and its subdirectories
I: Current time: Fri Sep 20 22:49:22 KST 2019
I: pbuilder-time-stamp: 1568987362

real	3m16.039s
user	3m34.430s
sys	0m27.615s

Q. Where are the generated .deb files?

$ ls -al /var/cache/pbuilder/result/*.deb

How to test your webcam with gstreamer-1.0

Before running the executables under the base_script and native directories in nnstreamer-example, let's make sure that your webcam properly works with gstreamer-1.0.

$ sudo apt install gstreamer1.0-tools
$ sudo apt install libgstreamer-plugins-base1.0 libgstreamer-plugins-good1.0
$ ls /dev/video*
/dev/video0  /dev/video1

In my case, there are two device nodes you can see. If you have just one webcam device, this is normal (video0 for input and video1 for output).

If you could not see any video device nodes in your /dev directory as follows, the device or drivers of your webcam are not compatible with your linux system.

$ ls /dev/video*
ls: cannot access '/dev/video*': No such file or directory

With simple pipeline construction with gst-launch-1.0, you can test your webcam with the gstreamer-1.0 framework.

$ gst-launch-1.0 v4l2src name=cam_src ! videoconvert ! videoscale ! video/x-raw,format=RGB ! queue ! videoconvert ! ximagesink name=img_origin

If you can see a window displaying the video streams from your webcam, most of executables in the base_script and native would works properly.

Warnings

In the case of Linux running on VMWare, your webcam could not work well with gstreamer-1.0. I've tested this on two laptops running Ubuntu 18.04 on VMWare.

  • The embedded front webcam in both laptops did not work well with gstreamer.
  • With an external webcam (using USB interface), which works perfectly in the native environment, it is possible to run the pipeline mentioned above. However, I could only see a window displaying a black screen. Note that this webcam is XBox Live Vision (2017). I guess it might work if you are using recent Logitech webcams.

[Android/Api] is "zip" need?

api/android/build-android-lib.sh

해당 파일에서

zip이 패키지에 포함되어 있는지 검사를 진행합니다.

그러나 zip 패키지의 경우
run_unittest 옵션이 'yes' 인 경우에만 사용하는 패키지입니다.

run_unittest 의 기본 옵션이 'no'인데 no일때는 검사할 필요가 없지 않을까요?

Improve examples by sharing your experience

We hoped that the native and bash-script examples for NNStreamer works fine with few efforts. However, as none of you could not run the examples, we need to improve our examples to be easier to run as we intended. IMO, it seems to be your good first issue and also a contribution chance.

[Ext/Decoders/BoundingBox] Bug: enum variables are always larger than -1 

 382     if (NULL == param || *param == '\0' || bdata->mode < 0) {                   
 383       GST_ERROR ("Please set the valid mode at option1");                       
 384       return FALSE;                                                             
 385     }

bdata->mode is a variable of enum type so that it is always a positive value or zero.
Some compilers (i.e., clang) would complain about it. Need to fix this logical bug.

Correct typo

It is always a good and easy first PR to correct typo in the projects. Just read our source code and find typo and white spaces.

Style transfer example

Overall Design

https://www.tensorflow.org/lite/models/style_transfer/overview

image

Style transfer model은 위의 그림과 같이 구성됩니다.

  • Input:

    • content_image
      : [description] style을 변경할 대상 image
      [format] type=float32, range=[0~1], RGB

    • style_image
      : [description] 적용하고자 하는 style image
      [format] type=float32, range=[0~1], fixed dimens=(256, 256, 3), RGB

(주의)
dimension이 고정되어 있기 때문에 preprocessing 필요 tensorflow-lite
예제에서는 center cropping을 진행

  • Output:
    • stylized_image
      : [description] style이 변경된 이미지
      [format] type=float32, range=[0~1], RGB, dimens=content_image.dimens

Models

Brief Pipeline design

filesrc (style)
   |
tensor_converter
   |
tensor_filter (Style predict model)
   |
tensor_mux - tensor_filter (Style transform model) - tensor_decoder - videoconvert - ximagesink (stylized) 
   |
tensor_converter
   |
videoscale
   |
v4l2src (content)

Build NNStreamer examples for Android

Now is the time for Team#3 members to do some work.

Based on the wiki page, build the android examples for NNStreamer on your platform. You need to follow Step 0, Step 1, Step 2, and Step 3 of this.

  • Is it possible to build NNStreamer examples for Android on Windows? @Lee-WonJun @soheel
    • GCC?
    • llvm/clang?
  • Is it possible to build NNStreamer examples for Android on native Linux? @abcinje
  • TBD

Improve emitting error policy about not exist proper model.

"/nnsuite/nnstreamer-example"의 예제들이 모델이 부재한 경우에 대하여 서로 다른 방식으로 Error를 출력하고 있습니다. 그래서 해당 깃에 이슈를 포스팅했고 아래의 코멘트를 받았습니다.

https://github.com/nnsuite/nnstreamer-example/issues/60#issuecomment-535470296

위의 Method 2에 따라 응용 레벨이 아닌 nnstreamer의 tensor_filter 수준에서 Model, Label을 체크하고, 없을 경우 g_critical함수를 이용하여 Error를 출력하도록 tensor_filter 업데이트 진행하겠습니다.

[Ext/Filter/TFLite] Bug: Do not use array designators under c++11 

In file included from /home/wook/Work/prj_nnstreamer/nnstreamer/ext/nnstreamer/tensor_filter/tensor_filter_tensorflow_lite_core.cc:29:
../ext/nnstreamer/tensor_filter/tensor_filter_tensorflow_lite_core.h:88:5: warning: array designators are a C99 extension [-Wc99-designator]
    [NNAPI_CPU] = "cpu",
    ^~~~~~~~~~~
../ext/nnstreamer/tensor_filter/tensor_filter_tensorflow_lite_core.h:88:5: warning: mixture of designated and non-designated initializers in the same initializer list is a C99 extension [-Wc99-designator]
    [NNAPI_CPU] = "cpu",
    ^~~~~~~~~~~~~~~~~~~
../ext/nnstreamer/tensor_filter/tensor_filter_tensorflow_lite_core.h:92:5: note: first non-designated initializer is here
    __null
    ^~~~~~
3 warnings generated.

[Android/Api] Windows Build

리눅스 배시 스크립트를 윈도우 배치스크립트로 바꾸는 과정을 진행중입니다.

관련이슈는 다음과 같습니다.

  1. git apply
    non-tizen 빌드이기때문에 non_tizen_build.patch 파일을 적용시켜야합니다.
    배시 스크립에서는 patch 를 사용하는데
    git에서 기본적으로 apply 라는 diff 적용 도구가 존재하여 이를 사용하고자합니다.

다만 문제로 non_tizen_build.patch 의 개행문자가 Windows버전이면 git apply에서 오류를 발생시켜서 개행 문자를 LF 로 변경해주어야합니다.

  1. svn
    sh 스크립트에서 svn을 이용해서 필요한 파일 을 가져옵니다.
    git으로 가져와도 될꺼같은데 git svn 명령어가 있어서 그냥 git svn사용해서 받아와도 될것 같습니다.

  2. tar.xz
    이 부분이 고민이 많은 부분입니다.
    윈도우 지원 tar로는 xz 파일압축을 풀수가 없습니다.
    따라서 7zip의 커멘드 라인파일 (7z.exe)을 이용하서 풀고 있습니다.
    근데 7z.exe 파일을 그냥 폴더에 포함시켜도 괜찮은 것인가요?

4.심볼릭 링크
텐서플로우 라이트 1.13/1.9 를 선택할수있습니다.
근데 여기서 1.13의 경우 압축에 심볼릭 링크가 포함되어있습니다.
\tensorflow\contrib\lite 가 tensorflow\lite 로 연결되어있습니다.
근데 이게 압축파일을 해재하면 그냥 *.symbollink 로 되어있는데 윈도우에서 사용하는 심볼릭 링크가 아니여서 gradle에서 빌드할때 인식하지 않습니다.

1.13 버전인경우에는 직접 명령어로 심볼릭링크를 만들면 될까요?

  1. 빌드오류
    5-1. error 87이 발생할때가 있습니다. gsteamer,nnsteamer, sdk, ndk 등의 폴더명이 길어지면 윈도우 최대범위를 넘어서 오류가 발생합니다.

5-2 이전에는 빌드가 됬는데 다시 프로젝트를 만들어서하는 도중에 error 3 가 발생합니다. 원인을 확인해보는 중입니다.

[Models] Improve get-model-* scripts to be more general

Currently, we have four scripts to get models in nnstreamer-example/bash_script/example_models, but do we really need to keep those scripts in separate files?

Could we use a single script that cover all the examples in the nnstreamer-exmaple repository.

How about

$ ./get-model.sh tensorflow ssdlite_mobilenet_v2

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.