nnstreamer / nnstreamer-example

Example applications of nnstreamer. Note that we have to enable the 'apptest" CI module in the near future.

NNStreamer Examples

This repository shows developers how to create their applications with nnstreamer/gstreamer. We recommend to install nnstreamer by downloading prebuilt binary packages from Launchpad/PPA (Ubuntu) or Download.Tizen.org (Tizen). If you want to build nnstreamer in your system for your example application builds, pdebuild (Ubuntu) with PPA or gbs (Tizen) are recommended for building nnstreamer. This repository has been detached from nnstreamer.git to build examples independently from the nnstreamer source code since Jan-09-2019.

Ubuntu PPA: nnstreamer/ppa [PPA Main]
Tizen Project: devel:Tizen:6.0:AI [OBS Project] [RPM Repo]

We provide example nnstreamer applications:

• Traditional Linux native applications
  • Linux/Ubuntu: GTK+ application
  • gst-launch-1.0 based scripts
• Tizen GUI Application
  • Tizen C/C++ application
  • Tizen .NET (C#) application
  • Tizen Web application
• Android applications
  • NDK based C/C++ CLI applicaton
  • JNI based GUI application

Quick start guide for NNStreamer example

Use PPA

• Download nnstreamer :

$ sudo add-apt-repository ppa:nnstreamer/ppa
$ sudo apt-get update
$ sudo apt-get install nnstreamer
$ sudo apt-get install nnstreamer-example
$ cd /usr/lib/nnstreamer/bin # binary install directory

*note: nnstreamer-example ppa was integrated into nnstreamer ppa.

As of 2018/10/13, we support 16.04 and 18.04

If you want to build nnstreamer example yourself, please refer to the link : [Build example]

Usage Examples

Text classification

$ cd /usr/lib/nnstreamer/bin
$ ./nnstreamer_example_text_classification_tflite

Refer to this link for more examples : [NNStreamer example]

Example List

Ubuntu Native

Application	Implementations	Used gstreamer/nnstreamer feature
Image Classification	- C (tflite) - Python (tflite) - C (ONE) - C (caffe2)	- v4l2src for input image stream - textoverlay for showing labels
Object Detection	- C++ (tf) - C++ (tflite) - Python (tflite)	- v4l2src for input image stream - cairooverlay for drawing boxes
Pose Estimation	- C++ (tflite) - Python (tflite)	- v4l2src for input image stream - cairooverlay for drawing body points
Image Classification with tensor_decoder	- C++ (tflite)	- tensor_decoder mode=image_labeling for postprocessing
Text Classification	- C (tflite)	- appsrc for input text data
Object Detection with 2 cameras	- C++ (tflite)	- v4l2src for input image stream - cairooverlay for drawing boxes

With gst-launch-1.0

Application	Implementations	Used gstreamer/nnstreamer feature
Object Detection	- tflite - tf	- v4l2src for input image stream - tensor_decoder mode=bounding_boxes for postprocessing - compositor for drawing decoded boxes
Image Segmentation	- tflite - Edge TPU     * Edge-AI server     * Edge-AI client	- v4l2src for input image stream - tensor_decoder mode=image_segment for postprocessing - tcpclientsrc / tcpserversink / gdppay / gdpdepay for networking between devices - tensor_converter / tensor_decoder mode=flatbuf for using Flatbuffers
Pipeline Flow Control in Face Detection	- OpenVINO + tflite + passthrough - OpenVINO + tflite + tensorpick	- v4l2src for input image stream - tensor_if for flow control

Tizen Applications

Application	Implementations	Used gstreamer/nnstreamer feature
Image Classification	- Native App with Pipeline API - Native App with Single API	- appsrc for input image data from hardward camera - Machine Learning Inference Native API
Object Detection	- Native App with Pipeline API	- appsrc for input image data from hardward camera - tensor_decoder mode=bounding_boxes for postprocessing - Pipeline API
Text Classification	- Native App with Pipeline API - .NET App with C# API	- appsrc for input text data - Pipeline API - C# API
Orientation Detection	- Native App with Pipeline API - .NET App with C# API	- tensor_src_tizensensor type=accelerometer for hardware sensor data - Pipeline API - C# API
Face Landmark	- Tizen IoT App	- v4l2src for input data from camera - tizenwlsink for lendering video to display - cairooverlay for drawing dots

Android Applications

Application	Implementations	Used gstreamer/nnstreamer feature
Object Detection	- w/ JNI + ahc2src - w/ JNI + amcsrc	- ahc2src for hardware camera input - amcsrc for media file input
NNStreamer Multi Model Face Detection + Hand Detection + Object Detection + Pose Estimation	- JNI App - GooglePlay App	- ahc2src for hardware camera input - application code for multiple pipeline control
Sample App using NNStreamer JAVA API	- JAVA w/o JNI	- appsrc for input image data from hardward camera - videoflip for flipping image - NNStreamer JAVA API

nnstreamer-example's Issues

nnstreamer_example_cam gst_mini_object_unref: assertion 'mini_object != NULL' failed

When run the example on Mac 10.15

Any idea how to fix it?

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.544: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.544: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.544: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps1)' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.544: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.544: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps2)' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_pad_template_new: assertion 'caps != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_element_request_compatible_pad: assertion 'GST_IS_PAD_TEMPLATE (templ)' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_object_unref: assertion 'object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.545: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps2)' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_pad_template_new: assertion 'caps != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_element_request_compatible_pad: assertion 'GST_IS_PAD_TEMPLATE (templ)' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.546: gst_object_unref: assertion 'object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.548: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.548: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.548: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps1)' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.548: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.549: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(nnstreamer_example_cam:4410): GStreamer-CRITICAL **: 15:02:39.549: gst_mini_object_ref: assertion 'mini_object != NULL' failed

** (nnstreamer_example_cam:4410): CRITICAL **: 15:02:39.571: _set_window_title: assertion 'element != NULL' failed

** (nnstreamer_example_cam:4410): CRITICAL **: 15:02:39.571: _set_window_title: assertion 'element != NULL' failed
^C

nnstreamer_example_speech_command_tflite failed on ubuntu 18

** Message: 15:35:13.188: start app..
** Message: 15:35:13.188: finished to load labels, total 12
** Message: 15:35:13.201: alsasrc name=audio_src device=hw:0,0 ! audioconvert ! audio/x-raw,rate=16000,format=S16LE,channels=1 ! tee name=t_raw t_raw. ! queue ! goom ! textoverlay name=tensor_res font-desc=Sans,24 ! videoconvert ! ximagesink t_raw. ! queue ! tensor_converter frames-per-tensor=1600 ! tensor_aggregator frames-in=1600 frames-out=16000 frames-flush=3200 frames-dim=1 ! tensor_transform mode=arithmetic option=typecast:float32,div:32767.0 ! tensor_filter framework=custom model=./libnnscustom_speech_command_tflite.so ! tensor_filter framework=tensorflow-lite model=./speech_model/conv_actions_frozen.tflite ! tensor_sink name=tensor_sink

(nnstreamer_example_speech_command_tflite:4655): GLib-CRITICAL **: 15:35:13.212: g_file_test: assertion 'filename != NULL' failed
** Message: 15:35:13.229: device [hw:0,0]
** Message: 15:35:13.229: device name [ES1371 DAC2/ADC]
** Message: 15:35:13.229: card name [(NULL)]
** Message: 15:35:13.229: received start message

Errors during 'gradlew build'

Dear NNStreamer maintainers,

I faced following logs when I tried to build a sample app (android/nnstreamer-ssd) using gradlew.
(Though I temporarily made "try {} catch {}" as comments and succeeded the build.)

~/nnstreamer/nnstreamer/gst/nnstreamer/tensor_filter/tensor_filter_support_cc.cc:75:3: error: cannot use 'try' with exceptions disabled
try {
^
~/nnstreamer/nnstreamer/gst/nnstreamer/tensor_filter/tensor_filter_support_cc.cc:137:3: error: cannot use 'try' with exceptions disabled
try {
^
~/nnstreamer/nnstreamer/gst/nnstreamer/tensor_filter/tensor_filter_support_cc.cc:173:3: error: cannot use 'try' with exceptions disabled
try {
^
3 errors generated.
make: *** [/home/chanhee/Android/Sdk/ndk/21.1.6352462/build/core/build-binary.mk:478:

How could I resolve this issue..?

Note that I'm using Android Sdk and Ndk as follows.
android {
compileSdkVersion 24
ndkVersion "21.1.6352462"
buildToolsVersion '28.0.3'
... }

[Question] How to get labels.txt for example_image_classification_caffe2

There is following guidance in nnstreamer_example_image_classification_caffe2.c

 * Required model and resources are stored at below link
 * https://github.com/caffe2/models/tree/master/mobilenet_v2

But, I cannot find label.txt in the above link. How can I get label.txt file?
@wooksong

Cannot build Android App.

Following https://github.com/nnsuite/nnstreamer-example/tree/master/android

At the step of make, after installing unzip,

Unify the notice about not exist the proper models and labels.

After building nnstreamer-example, I tried to execute the executables in "builddir/native/" without proper models and labels.

The executables about object_detection(nnstreamer_example_object_detection_*) returned the following error message:

$ ./nnstreamer_example_object_detection_tf
** (process:6390): CRITICAL **: 19:46:41.502: the file of model_path is not valid: ./tf_model/ssdlite_mobilenet_v2.pb
$ ./nnstreamer_example_object_detection_tflite
** (process:6444): CRITICAL **: 19:48:37.962: the file of model_path is not valid: ./tflite_model/ssd_mobilenet_v2_coco.tflite

However, Nothing happened when I executed the executables about image_classification(nnstreamer_example_image_classification_*). Because these make a message only in debugging mode.

In my opinion, It would be better that the examples even make an error message when it is not in debugging mode.

example_decoder_image_labelling does not check for the existance of label file.

/**
 * @brief Check tflite model and load labels.
 *
 * This example uses 'Mobilenet_1.0_224_quant' for image classification.
 */
static gboolean
_tflite_init_info (tflite_info_s * tflite_info, const gchar * path)
{
  const gchar tflite_model[] = "mobilenet_v1_1.0_224_quant.tflite";

  g_return_val_if_fail (tflite_info != NULL, FALSE);

  tflite_info->model_path = NULL;

  /* check model file exists */
  tflite_info->model_path = g_strdup_printf ("%s/%s", path, tflite_model);

  if (access (tflite_info->model_path, F_OK) != 0) {
    g_critical ("cannot find tflite model [%s]", tflite_info->model_path);
    return FALSE;
  }

  return TRUE;
}

This is the function checks tflite model and loads labels in the example.
However, there is no constructions that check the label path is valid.

So, I got following error when I remove label.txt.
IMO, It is better to add constructions check the label path is valid as another examples.

**
ERROR:/home/mojunsang/Work/nnstreamer/ext/nnstreamer/tensor_decoder/tensordec-imagelabel.c:256:il_decode: assertion failed: (max_index < data->total_labels)

@wooksong

Need a proper policy on install-able binaries

After building and installing nnstreamer-example in my local machine, I got the following files under ${prefix}/bin:

-rw-r--r--    get-model-image-classification-tflite.sh
-rw-r--r--    get-model-object-detection-tflite.sh
-rw-r--r--    get-model-object-detection-tf.sh
-rw-r--r--    get-model-speech-command.sh
-rwxr-xr-x    gst-launch-object-detection-tflite.sh
-rwxr-xr-x    gst-launch-object-detection-tf.sh
-rwxr-xr-x    libnnscustom_speech_command_tflite.so
-rwxr-xr-x    nnstreamer_example_cam
-rwxr-xr-x    nnstreamer_example_decoder_image_labelling
-rwxr-xr-x    nnstreamer_example_filter_performance_profile
-rwxr-xr-x    nnstreamer_example_image_classification_caffe2
-rwxr-xr-x    nnstreamer_example_image_classification_tflite
-rw-r--r--    nnstreamer_example_image_classification_tflite.py
-rwxr-xr-x    nnstreamer_example_object_detection_tf
-rwxr-xr-x    nnstreamer_example_object_detection_tflite
-rwxr-xr-x    nnstreamer_example_speech_command_tflite
-rwxr-xr-x    nnstreamer_example_two_tensor_stream
-rwxr-xr-x    nnstreamer_sink_example
-rwxr-xr-x    nnstreamer_sink_example_play

As we can see, some files have execution permission and others do not. In addition, naming conventions are mixed. They do not looks intuitive.

In my opinion, it would be better to formulate a proper policy that affects

• naming conventions
• install locations of binaries/get-model-* scripts/wget-ed model files
• permissions for the scripts.

apptest: this module is not enabled at first.

It is strange. It seems that the 'apptest' module is not enabled at first. @jaeyun-jung , We you come back from the winter vacation, Please share us tell on your work that you updated the existing apptest module. I am going to disable the 'apptest' module temporarily.

@helloahn , @myungjoo PTAL.

Log message:

When a PR is failed due to unstable 'apptest' module, we have got the error message from the TAOS-CI robot as following:

################################################################################################################################################################################
[MODULE] TAOS/pr-audit-nnstreamer-apptest: Starting a sample app test
Checking for meson...
/usr/bin/meson
Checking for ninja...
/usr/bin/ninja
Checking for wget...
/usr/bin/wget
Checking for python...
/usr/bin/python
Checking for Xvnc...
/usr/bin/Xvnc
Checking for git...
/usr/bin/git
Checking for insmod...
/sbin/insmod
Checking for cat...
/bin/cat
Checking for grep...
/bin/grep
Checking for usermod...
/usr/sbin/usermod
Checking for xauth...
/usr/bin/xauth
Checking for touch...
/usr/bin/touch
Checking for awk...
/usr/bin/awk
Checking for grep...
/bin/grep
Checking for ps...
/bin/ps
[DEBUG] NNST_ROOT is '/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example'
[DEBUG] LD_LIBRARY_PATH is ':/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example/lib'
[DEBUG] GST_PLUGIN_PATH is ':/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example/lib/gstreamer-1.0'
/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example /var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
The Meson build system
Version: 0.43.0
Source dir: /var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
Build dir: /var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example/build
Build type: native build
Project name: nnstreamer-example
Native C compiler: ccache cc (gcc 5.4.0)
Native C++ compiler: ccache c++ (gcc 5.4.0)
Build machine cpu family: x86_64
Build machine cpu: x86_64
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Native dependency glib-2.0 found: YES 2.48.2
Native dependency gstreamer-1.0 found: YES 1.8.3
Native dependency gstreamer-base-1.0 found: YES 1.8.3
Native dependency gstreamer-video-1.0 found: YES 1.8.3
Native dependency gstreamer-audio-1.0 found: YES 1.8.3
Native dependency gstreamer-app-1.0 found: YES 1.8.3
Native dependency cairo found: YES 1.14.6
Library m found: YES
Library dl found: YES
Dependency threads found: YES
Native dependency tensorflow found: YES 1.9.0
Dependency protobuf found: NO
Native dependency tensorflow-lite found: YES 0.1
Build targets in project: 9
Found ninja-1.8.2 at /usr/bin/ninja
ninja: Entering directory `build'
[1/19] Compiling C object 'native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite@sha/nnscustom_speech_command_tflite.c.o'.
FAILED: native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite@sha/nnscustom_speech_command_tflite.c.o 
ccache cc  -Inative/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite@sha -Inative/example_speech_command_tensorflow_lite -I../native/example_speech_command_tensorflow_lite -fdiagnostics-color=always -pipe -D_FILE_OFFSET_BITS=64 -Werror -std=gnu89 -fPIC -MMD -MQ 'native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite@sha/nnscustom_speech_command_tflite.c.o' -MF 'native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite@sha/nnscustom_speech_command_tflite.c.o.d' -o 'native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite@sha/nnscustom_speech_command_tflite.c.o' -c ../native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite.c
../native/example_speech_command_tensorflow_lite/nnscustom_speech_command_tflite.c:18:45: fatal error: nnstreamer/tensor_filter_custom.h: No such file or directory
compilation terminated.
[2/19] Compiling C object 'native/example_cam/nnstreamer_example_cam@exe/nnstreamer_example_cam.c.o'.
[3/19] Compiling C object 'native/example_sink/nnstreamer_sink_example@exe/nnstreamer_sink_example.c.o'.
[4/19] Compiling C object 'native/example_sink/nnstreamer_sink_example_play@exe/nnstreamer_sink_example_play.c.o'.
[5/19] Compiling C object 'native/example_decoder_image_labelling/nnstreamer_example_decoder_image_labelling@exe/nnstreamer_example_decoder_image_labelling.c.o'.
[6/19] Compiling C object 'native/example_filter_performance_profile/nnstreamer_example_filter_performance_profile@exe/nnstreamer_example_filter_performance_profile.c.o'.
[7/19] Compiling C object 'native/example_speech_command_tensorflow_lite/nnstreamer_example_speech_command_tflite@exe/nnstreamer_example_speech_command_tflite.c.o'.
[8/19] Compiling C object 'native/example_image_classification/nnstreamer_example_image_classification@exe/nnstreamer_example_image_classification.c.o'.
[9/19] Compiling C++ object 'native/example_object_detection_tensorflow_lite/nnstreamer_example_object_detection_tflite@exe/nnstreamer_example_object_detection_tflite.cc.o'.
ninja: build stopped: subcommand failed.
[DEBUG] An USB Camera device is not enabled. It is required by {nnstreamer_example_image_classification|nnstreamer_example_cam}.
[DEBUG] Enabling virtual cam camera...
/var/www/html/nnstreamer-example/v4l2loopback /var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example /var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
rm -f *~
rm -f Module.symvers Module.markers modules.order
make -C /lib/modules/`uname -r`/build M=/var/www/html/nnstreamer-example/v4l2loopback clean
make[1]: Entering directory '/usr/src/linux-headers-4.4.0-138-generic'
  CLEAN   /var/www/html/nnstreamer-example/v4l2loopback/.tmp_versions
make[1]: Leaving directory '/usr/src/linux-headers-4.4.0-138-generic'
Building v4l2-loopback driver...
make -C /lib/modules/`uname -r`/build M=/var/www/html/nnstreamer-example/v4l2loopback modules
make[1]: Entering directory '/usr/src/linux-headers-4.4.0-138-generic'
  CC [M]  /var/www/html/nnstreamer-example/v4l2loopback/v4l2loopback.o
  Building modules, stage 2.
  MODPOST 1 modules
  CC      /var/www/html/nnstreamer-example/v4l2loopback/v4l2loopback.mod.o
  LD [M]  /var/www/html/nnstreamer-example/v4l2loopback/v4l2loopback.ko
make[1]: Leaving directory '/usr/src/linux-headers-4.4.0-138-generic'
[DEBUG] The group 'video' has to include 'www-data' in the 'video' group.
video:x:44:ubuntu,www-data
/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example /var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
[DEBUG][PASS] It's okay. ~/.Xauthority exists.
[DEBUG] It's okay. The producer (pid: 61500) is successfully started.
[DEBUG] -------------------- locale: start --------------------
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8
[DEBUG] -------------------- locale: start --------------------
[DEBUG] -------------------- netstat(Xvnc:59xx): start --------------------
tcp        0      0 0.0.0.0:5900            0.0.0.0:*               LISTEN      -               
tcp        0      0 172.31.27.164:5900      62.212.90.229:51084     ESTABLISHED -               
tcp        0      0 172.31.27.164:5900      62.212.90.229:18121     TIME_WAIT   -               
tcp        0      0 172.31.27.164:5900      62.212.90.229:53227     TIME_WAIT   -               
tcp        0      0 172.31.27.164:5900      62.212.90.229:5910      TIME_WAIT   -               
tcp        0      0 172.31.27.164:5900      62.212.90.229:29618     TIME_WAIT   -               
tcp        0      0 172.31.27.164:5900      62.212.90.229:39822     TIME_WAIT   -               
tcp6       0      0 :::5931                 :::*                    LISTEN      61499/Xvnc      
tcp6       0      0 :::5900                 :::*                    LISTEN      -               
tcp6       0      0 :::5901                 :::*                    LISTEN      -               
tcp6       0      0 172.31.27.164:5901      23.95.198.142:39819     TIME_WAIT   -               
[DEBUG] -------------------- netstat(Xvnc:59xx): end   --------------------
[DEBUG] -------------------- netstat(Xvnc:60xx): start --------------------
tcp        0      0 0.0.0.0:6031            0.0.0.0:*               LISTEN      61499/Xvnc      
tcp        0      0 0.0.0.0:6001            0.0.0.0:*               LISTEN      -               
[DEBUG] -------------------- netstat(Xvnc:60xx): end   --------------------
[DEBUG] -------------------- env: start ------------------------
APACHE_PID_FILE=/var/run/apache2/apache2.pid
SHELL=/bin/bash
TERM=xterm
APACHE_RUN_USER=www-data
dir_ci=/var/www/html/nnstreamer-example/TAOS-CI/ci
OLDPWD=/var/www/html/nnstreamer-example/v4l2loopback
LC_ALL=en_US.UTF-8
LD_LIBRARY_PATH=:/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example/lib
dir_worker=repo-workers/pr-checker
dir_commit=repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
APACHE_LOG_DIR=/var/log/apache2
PWD=/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
LANG=en_US.UTF-8
APACHE_RUN_GROUP=www-data
SHLVL=2
LANGUAGE=en_US:en
NNST_ROOT=/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
GST_PLUGIN_PATH=:/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example/lib/gstreamer-1.0
DISPLAY=0.0:31
APACHE_RUN_DIR=/var/run/apache2
APACHE_LOCK_DIR=/var/lock/apache2
_=/usr/bin/env
[DEBUG] -------------------- env: end   ------------------------
[DEBUG] -------------------- xauth: start ----------------------
[DEBUG] -------------------- xauth: end   ----------------------
Setting pipeline to PAUSED ...
Setting pipeline to NULL ...
Freeing pipeline ...
/var/www/html/nnstreamer-example/TAOS-CI/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/nnstreamer-example
[DEBUG][FAILED] Oooops!!!!!! apptest is failed. Resubmit the PR after fixing correctly.

[DEBUG] report the execution result of apptest. The result value is 132. 
[DEBUG] Running the curl-based PR status change procedure.
[DEBUG] The length of a webhook message is "126".
[DEBUG] The original DESCRIPTION is "Oooops. apptest is failed. Resubmit the PR after fixing correctly. Commit number is 2f958b07b1ba906ce2124454660222ba16653904.".
[DEBUG] The edited TRIM_DESCRIPTION is "Oooops. apptest is failed. Resubmit the PR after fixing correctly. Commit number is 2f958b07b1ba906ce2124454660222ba1665 ...".
{
  "url": "https://api.github.com/repos/nnsuite/nnstreamer-example/statuses/2f958b07b1ba906ce2124454660222ba16653904",
  "avatar_url": "https://avatars2.githubusercontent.com/u/42455533?v=4",
  "id": 6309876490,
  "node_id": "MDEzOlN0YXR1c0NvbnRleHQ2MzA5ODc2NDkw",
  "state": "failure",
  "description": "Oooops. apptest is failed. Resubmit the PR after fixing correctly. Commit number is 2f958b07b1ba906ce2124454660222ba1665 ...",
  "target_url": "http://nnsuite.mooo.com/nnstreamer-example/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/",
  "context": "TAOS/pr-audit-nnstreamer-apptest",
  "created_at": "2019-02-25T04:44:41Z",
  "updated_at": "2019-02-25T04:44:41Z",
  "creator": {
    "login": "taos-ci",
    "id": 42455533,
    "node_id": "MDQ6VXNlcjQyNDU1NTMz",
    "avatar_url": "https://avatars2.githubusercontent.com/u/42455533?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/taos-ci",
    "html_url": "https://github.com/taos-ci",
    "followers_url": "https://api.github.com/users/taos-ci/followers",
    "following_url": "https://api.github.com/users/taos-ci/following{/other_user}",
    "gists_url": "https://api.github.com/users/taos-ci/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/taos-ci/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/taos-ci/subscriptions",
    "organizations_url": "https://api.github.com/users/taos-ci/orgs",
    "repos_url": "https://api.github.com/users/taos-ci/repos",
    "events_url": "https://api.github.com/users/taos-ci/events{/privacy}",
    "received_events_url": "https://api.github.com/users/taos-ci/received_events",
    "type": "User",
    "site_admin": false
  }
}
[DEBUG] Return value of the curl webhook command is '0'. If the value is 0, it means that webhook operation is normal.
[DEBUG] Note: If webhook server replies "message": "Not Found", add a privileged user id at 'Setting - Collaborators'.
[DEBUG] Note: The privileged user id has to be appended by "Write" permission.
[DEBUG] Note: If webhook server replies "message": "Bad credentials", try do it again with a correct token key.
[DEBUG] Running the curl-based commenting procedure.
[DEBUG] TOKEN: ea799dca0899673f7da8ed876ee41f5f809b7fc5
[DEBUG] MESSAGE: :octocat: **cibot**: @leemgs, apptest could not be completed. To find out the reasons, please go to http://nnsuite.mooo.com//nnstreamer-example/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/checker-pr-audit.log
[DEBUG] COMMIT_ADDRESS: https://api.github.com/repos/nnsuite/nnstreamer-example/issues/13/comments
{
  "url": "https://api.github.com/repos/nnsuite/nnstreamer-example/issues/comments/466870440",
  "html_url": "https://github.com/nnsuite/nnstreamer-example/pull/13#issuecomment-466870440",
  "issue_url": "https://api.github.com/repos/nnsuite/nnstreamer-example/issues/13",
  "id": 466870440,
  "node_id": "MDEyOklzc3VlQ29tbWVudDQ2Njg3MDQ0MA==",
  "user": {
    "login": "taos-ci",
    "id": 42455533,
    "node_id": "MDQ6VXNlcjQyNDU1NTMz",
    "avatar_url": "https://avatars2.githubusercontent.com/u/42455533?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/taos-ci",
    "html_url": "https://github.com/taos-ci",
    "followers_url": "https://api.github.com/users/taos-ci/followers",
    "following_url": "https://api.github.com/users/taos-ci/following{/other_user}",
    "gists_url": "https://api.github.com/users/taos-ci/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/taos-ci/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/taos-ci/subscriptions",
    "organizations_url": "https://api.github.com/users/taos-ci/orgs",
    "repos_url": "https://api.github.com/users/taos-ci/repos",
    "events_url": "https://api.github.com/users/taos-ci/events{/privacy}",
    "received_events_url": "https://api.github.com/users/taos-ci/received_events",
    "type": "User",
    "site_admin": false
  },
  "created_at": "2019-02-25T04:44:42Z",
  "updated_at": "2019-02-25T04:44:42Z",
  "author_association": "MEMBER",
  "body": ":octocat: **cibot**: @leemgs, apptest could not be completed. To find out the reasons, please go to http://nnsuite.mooo.com//nnstreamer-example/ci/repo-workers/pr-checker/13-201902251336060.69754195213318-2f958b07b1ba906ce2124454660222ba16653904/checker-pr-audit.log"
}
[DEBUG] Return value of the curl webhook command is '0'. If the value is 0, it means that webhook operation is normal.
[DEBUG] Note: If webhook server replies "message": "Not Found", add a privileged user id at 'Setting - Collaborators'.
[DEBUG] Note: The privileged user id has to be appended by "Write" permission.
[DEBUG] Note: If webhook server replies "message": "Bad credentials", try do it again with a correct token key.

Screenshot:

[Models] Improve get-model-* scripts to be more general

Currently, There are four scripts to get models in nnstreamer-example/bash_script/example_models.

Could we use a single script that cover all the examples in the nnstreamer-exmaple repository.

How about

$ ./get-model.sh image-classification-tflite

nnstreamer_example_object_detection_tf does core dump when switching models

I tried switching from ssdlite_mobilenet_v2.pb to ssd_inception_v2.pb

I got the pb file from the model zoo:

https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/detection_model_zoo.md

Error:

ERROR:/home/aaron/src/nnstreamer/gst/nnstreamer/tensor_filter/tensor_filter_tensorflow.c:104:tf_open: assertion failed: (retval == 0)
Aborted (core dumped)

tensor_filter_subplugin example is not registered correctly

When creating a filter sub plugin with nnstreamer-example and trying to run I get:
nnstreamer_subplugin of temp(/usr/local/lib/nnstreamer/filters/libnnstreamer_filter_temp.so) is broken. It does not call register_subplugin with its init function.

While debugging I found that the nnstreamer_filter_probe (&NNS_support_temp);
exits with an error.
Diving to nnstreamer_subplugin.c :: register_subplugin
I found that this line returns with error (1):
g_hash_table_insert (subplugins[type], g_strdup (name), spdata);

The name is propagated correctly to this function and the type is 0.
I would appreciate your help!
Thanks

example_object_detection_tensorflow bash script not working

Running the bash script gives me the following error:

$ ./bash_script/example_object_detection_tensorflow/gst-launch-object-detection-tf.sh

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps1)' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps2)' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_pad_template_new: assertion 'caps != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_element_request_compatible_pad: assertion 'GST_IS_PAD_TEMPLATE (templ)' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_object_unref: assertion 'object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps2)' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_pad_template_new: assertion 'caps != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_element_request_compatible_pad: assertion 'GST_IS_PAD_TEMPLATE (templ)' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_object_unref: assertion 'object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_caps_can_intersect: assertion 'GST_IS_CAPS (caps1)' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_unref: assertion 'mini_object != NULL' failed

(gst-launch-1.0:6): GStreamer-CRITICAL **: gst_mini_object_ref: assertion 'mini_object != NULL' failed

** (gst-launch-1.0:6): CRITICAL **: _check_tensors: assertion 'config->info.num_tensors >= limit' failed

** (gst-launch-1.0:6): CRITICAL **: _check_tensors: assertion 'config->info.num_tensors >= limit' failed
WARNING: erroneous pipeline: could not link tensorconverter0 to tensorfilter0

I am using the current ppa of nnstreamer, which according to gust-inspect-1.0 is version 0.1.1

[tflite2] example of object detection with quantized SSD model.

Below is the float point model for object detection. Do you have example for quantized SSD model?

bash_script/example_object_detection_tensorflow_lite/gst-launch-object-detection-tflite.sh

update Android App build guide

In https://github.com/nnsuite/nnstreamer-example/tree/master/android unzip is missing.

[Contributhon] updating available languages in "example_text_classification"

Problem
Currently, "example_text_classification" example only supports English, so I would like to add a code to test the example in Korean.

Solution
Through Python, I will write an example code that can be text classified in Korean and then rewrite it in c language.
I am going to pre-processing Korean using KoNLPy library.

Move example Wiki to the source tree

I suggest moving the example Wiki (https://github.com/nnstreamer/nnstreamer/wiki/usage-examples-screenshots) to the source tree in this repo (maybe in README.md).

To be clear, I am talking about the specific wiki page, not the Wiki as a whole. I think the other pages are in the right place.

When I was having fun with the examples, I noticed the wiki page is out of sync with the repo. For example, s/nnsuite/nnstreamer/g, the use of wget when get_model.sh is available, some undocumented examples, and etc.

Although Github Wiki is a convenient tool to quickly update notes but Github Wiki has some drawbacks. I could not submit a change even though I noticed some errors, because the wiki currently allows edits only from contributors.

Moving the example Wiki to README.md will have advantages:

1. Allows other developers to correct errors through a PR.
2. It is easier to keep it updated. You can ask a PR to add descriptions of new examples in README.md during the review process. I believe it is quite important.

Thanks for reading :)

Update the license!

nnstreamer-example/Tizen.platform/Vivante_single_NonGUI_inceptionV3/packaging/vivante-inceptionv3-single.spec

Line 7 in f932efc

License: Priprietary

All examples should be provided as LGPL 2.1
If you want other licenses, we can discuss it (e.g., Apache 2.0, Flora, or so on)
But, we cannot accept proprietary. I'm sure "proprietary" is inserted by mistake (that's just a simple example/reference code).

Search for other sample apps, too.

Create .gitignore to ignore build directory

How about creating .gitignore to ignore build directory made by meson build command?

Converting the C example sources to python3

This is an project of team-3 in OSS Contributhon.

Team Member
hy-kiera
JawThrow
ddeokho

The current examples based C language are too difficult to follow them by beginner.
Therefore, we decided to convert all of them to examples based Python 3.

TODO

• Converting the examples to python3
• Release to PYPI
• Write python tutorials

Is it possible to implement this code in Ubuntu 18.04 LTS?

Hello,

I'm interested in deploy and develop this code in Android.

But, I have a different PC environment.

So I wonder that it is possible for me to implement this code in my Ubuntu 18.04 LTS environment.

Does it have some bugs or some issues when I implement this code in Ubuntu 18.04 LTS?

Please let me know it is possible or not.

Thank you:)

(gst-launch-1.0:31): GStreamer-CRITICAL **: 05:24:58.650: gst_mini_object_unref: assertion 'mini_object != NULL' failed

I encountered an error executing the following command

root@ht-2288H-V5:/opt/nvidia/deepstream/deepstream-5.0# gst-launch-1.0 rtspsrc location=rtsp://admin:[email protected]:554/h264/ch1/main/av_stream latency=200 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! video/x-raw, format=(string)BGRx ! videoconvert ! fakesink sync=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://admin:[email protected]:554/h264/ch1/main/av_stream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request

(gst-launch-1.0:31): GStreamer-CRITICAL **: 05:24:58.650: gst_mini_object_unref: assertion 'mini_object != NULL' failed
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:03.065412691
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
^C

However, changing the decoder command does not report an error，I believe nvV4L2Decoder is the source of the problem

root@ht-2288H-V5:/opt/nvidia/deepstream/deepstream-5.0# gst-launch-1.0 rtspsrc location=rtsp://admin:[email protected]:554/h264/ch1/main/av_stream latency=200 ! rtph264depay ! h264parse ! avdec_h264 ! nvvideoconvert ! video/x-raw, format=(string)BGRx ! videoconvert ! fakesink sync=false
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://admin:[email protected]:554/h264/ch1/main/av_stream
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Redistribute latency...
Redistribute latency...
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:34.124400984
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I use a docker
https://docs.nvidia.com/metropolis/deepstream/5.0.1/dev-guide/text/DS_docker_containers.html
https://catalog.ngc.nvidia.com/orgs/nvidia/containers/deepstream
docker run --gpus all -it --rm --net=host --privileged -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=$DISPLAY -w /opt/nvidia/deepstream/deepstream-5.0 nvcr.io/nvidia/deepstream:5.0.1-20.09-devel

root@ht-2288H-V5:/opt/nvidia/deepstream/deepstream-5.0# gst-launch-1.0 --version
gst-launch-1.0 version 1.14.5
GStreamer 1.14.5
https://launchpad.net/distros/ubuntu/+source/gstreamer1.0

New nnstreamer-examples are not included after build

Following the build instruction described in nnstreamer build examples, newly created examples are not included in build products.

The followings are new nnstreamer examples.

• nnstreamer-example/native/example_object_detection_tensorflow_lite/nnstreamer_example_object_detection_tflite.py
• nnstreamer-example/native/example_face_detection_tflite/nnstreamer_example_face_detection_tflite.py

I think these new examples are not included because path for git clone command in the build instruction is from nnsuite/nnstreamer-example, not from nnstreamer/nnstreamer-example.
Therefore, the problem will be solved if nnsuite/nnstreamer-example is updated or if the path in build instruction is changed to git clone https://github.com/nnstreamer/nnstreamer-example.git nnstreamer-example.git.

[WIP][bashscript/android] Support tflite segmentation model

It is a sub-issue of #62 Support tensorflow-lite basic model with bash_script .

As mentioned in style transfer reallocate issue, there is a realloc problem with style transfer model. So I am going to try to support segmentation example.

I am considering this segmentation model which is supported by tensorflow.

• Input: content image (257, 257, 3)
• Output: type: float32 | dimen: (257,257,21) // a probability each of 21 labels

//supported labels
LABEL_NAMES = np.asarray([
    'background', 'aeroplane', 'bicycle', 'bird', 'boat', 'bottle', 'bus',
    'car', 'cat', 'chair', 'cow', 'diningtable', 'dog', 'horse', 'motorbike',
    'person', 'pottedplant', 'sheep', 'sofa', 'train', 'tv'
])

Brief pipeline

gst-launch-1.0 -v \
      filesrc location=cat.jpg ! jpegdec ! videoconvert ! videoscale ! imagefreeze !\
      video/x-raw,format=RGB,width=257,height=257,framerate=10/1 ! tee name=t \
      t. tensor_converter !\
      tensor_transform mode=arithmetic option=typecast:float32,add:0.0,div:255.0 !\
      tensor_filter framework=tensorflow-lite model=deeplabv3_257_mv_gpu.tflite !\
      tensor_decoder mode=segmentation ! compositor name=mix sink_0::zorder=2 sink_1::zorder=1 !\
      videoconvert ! ximagesink
      t. ! queue leaky=2 ! mix.

At first, I will make decoder(output array -> image type) for this model and then construct bash script example.
If it is done, I will expand it to android application example.

[contributhon][team-2] example with multiple camera/sensor inputs on Raspberry Pi

Team-2 is planning to:

Make examples with multiple camera/sensor inputs on Raspberry Pi.

Expected Result

• work on Raspberry Pi
  Raspberry Pi 에서 동작하는 Edge AI 데모 시나리오 예제들을 몇 개 구현한 뒤 제출하려고 합니다.
• process multiple camera input
  다중 카메라 입력을 받아서 동시에 처리하는 데모 시나리오 예제 구현
• process camera/sensor compound input
  카메라와 센서 입력을 동시에 받아서 상황을 파악하고 반응하는 데모 시나리오 예제 구현

Team members

@cloudhs7
@JeonChangmin
@suehdn
@tony-jinwoo-ahn

question

• if you have purchased any usb camera/sensor, please recommend.
  구매에 앞서, 추천하실만한 카메라/센서 모듈이 있다면 공유 부탁드립니다.
  이걸 써보니 가성비가 괜찮다든지, 이 카메라를 여러대 연결해봤는데 무사히 잘 동작했다든지...
• if you have any experience with Raspberry Pi, please guide us for environment setup.
  최신 라즈비안 설치 후 우분투와 동일하게 환경세팅을 하면 되는 것인지, nnstreamer 기본 동작성을 이미 검증해보신 분이 있으신지 문의드립니다.

PLEASE WRITE ISSUES AT github.com/nnstreamer/nnstreamer

We are going to close issues tab in this repo.
Please describe issues at https://github.com/nnstreamer/nnstreamer instead.

안드로이드 예제 관련해서 질문 드립니다.

안녕하세요.

안드로이드, NNStreamer-ssd 예제를 수정하며 공부 중 궁금한 것이 있어 질문을 남깁니다.

1. nnstreamer-ex.cpp 내의 파이프 라인을(507번째 줄) 바꾸어 카메라를 여는 것 대신 영상을 가지고 올 수 있도록 하려 합니다. 그리고 그 영상을 디텍션할 수는 없을까 고민하고 있습니다.

그래서 아래의 파이프 라인을 확인 후 해당 코드에 적용했습니다.
sudo gst-launch-1.0 souphttpsrc location=http://aarkapp.iptime.org/aark_ob/test.webm ! decodebin ! autovideosink

적용전

str_pipeline = g_strdup_printf
          (ahc2src ! videoconvert ! video/x-raw,format=RGB,width=640,height=480,framerate=30/1 ! tee name=traw "

적용 후

str_pipeline = g_strdup_printf
          ("uridecodebin uri=http://aarkapp.iptime.org/aark_ob/test.webm  ! decodebin ! tee name=traw "      

그 후 핸드폰에 빌드를 하니 "Failed to build pipeline [1]"와 같은 결과가 나옵니다. 파이프 라인이 잘못 적용된 거 같은데 파이프라인 외에도 변경해 주어야 할 게 있을 지 궁급합니다.

2. 위의 내용과 다른 질문입니다. 라이다의 실시간 데이터를 불러오려면 'appsrc'를 사용하면 된다고 알고 있습니다. NNStreamer 예제 중 카메라를 사용한 예제의 파이프라인 안에 'ahc2src' 또는 'v4l2src' 대신 'appsrc'로만 바꾸어 주면 되는 것일까요?

너무 기초적인 질문을 드려 죄송합니다. 혹시 제가 참고할 만한 것을 추천해 주셔도 좋습니다.
그리고 긴 글 읽어주셔서 감사합니다.

How can I input the lidar sensor data in jni file?

I have a question to develop with 'nnstreamer-ssd' example in android.

I want to input the lidar sensor data instread of video streaming data.

I analyzed the code structure and I thought System.loadLibrary("nnstreamer-jni") function in the 'NNStreamerActivity' supports to connect Java with C source code. Is it right?

If it is right, can I make or modify the inferface with gstreamer in the nnstreamer-jni.c for lidar sensor data?

Compared Gstreamer tutorial[Android tutorials- video],

I found that it it added several linked lists related to pipeline 'get_pipeline_info(), init_pipeline_info(), free_pipeline_info(), nns_register_pipeline()' and functions 'run_pipeline()'

I try to modify this part to insert 'lidar sensor data'. But, I'm not sure.

I'm looking forward to any advices.

Thank you!

From hayley.shim

Need example for depth

I need support with these two:

1. The main repo claims that the tensor decoder supports depth too. But I do not see an example here. Can you provide an example of a depth model with a suitable tensor decoder mode?

2. Where are the various tensor decoder modes explained? I cannot find it in the documentation.

Thanks.

nnstreamer-multi app cannot download model files on S8/S9/S10 devices.

It's strange. the nnstreamer-multi app cannot download model files on S8/S9/S10 devices.
You can download the application by searching "nnstreamer multi" at Google Store.

• Debug message with adb/logcat:

initiate android service application

Currently, we do not have andorid service application. I think we should make a folder for the android service application without front.

Use shellcheck to verify shell scripts?

This repo has a lot of (16) shell scripts. So it would be great if we have certain rules for shell scripts to improve maintainability.
The useful tool is Shellchek. We can easily integrate this using ShellCheck Github action.
To run this locally you can install it using sudo snap install shellcheck. I use snap because the version of APT package is too old.

Here is an example of running shellcheck for all shell scripts in the repo. I excluded rule SC2164 because it triggers too much warnings that doesn't seem to worth it to me:

(click to expand) find . -name *.sh -exec shellcheck -e SC2164 {} +

In ./bash_script/example_image_segmentation_tensorflow_lite/gst-launch-image-seg-flatbuf-edgetpu-server.sh line 15:
tensor_decoder mode=flatbuf ! gdppay ! tcpserversink host=${ip} port=${port}
                                                          ^---^ SC2086: Double quote to prevent globbing and word splitting.
                                                                     ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
tensor_decoder mode=flatbuf ! gdppay ! tcpserversink host="${ip}" port="${port}"


In ./bash_script/example_image_segmentation_tensorflow_lite/gst-launch-image-seg-flatbuf-edgetpu-client.sh line 25:
    tcpclientsrc host=${ip} port=${port} ! gdpdepay ! other/flatbuf-tensor ! tensor_converter ! tee name=t \
                      ^---^ SC2086: Double quote to prevent globbing and word splitting.
                                 ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
    tcpclientsrc host="${ip}" port="${port}" ! gdpdepay ! other/flatbuf-tensor ! tensor_converter ! tee name=t \


In ./bash_script/example_models/get-model-image-classification-tflite.sh line 18:
if echo $0 | grep -q -w "get-model-image-classification-tflite.sh"; then 
        ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
if echo "$0" | grep -q -w "get-model-image-classification-tflite.sh"; then 


In ./bash_script/example_models/get-model.sh line 18:
if echo $0 | grep -q -w "get-model-image-classification-tflite.sh"; then 
        ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
if echo "$0" | grep -q -w "get-model-image-classification-tflite.sh"; then 


In ./android/IoT-ARM64/02-gst-android-with-cerbero/additional-patches/build-all.sh line 9:
if [[ $? -eq 0 ]]; then
      ^-- SC2181: Check exit code directly with e.g. 'if mycmd;', not indirectly with $?.


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 30:
if [ -f $rpmfile ]; then
        ^------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
if [ -f "$rpmfile" ]; then


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 31:
  rm $rpmfile
     ^------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
  rm "$rpmfile"


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 36:
sofilepath=$(rpm2cpio $rpmfile | cpio -t | grep $sofile)
                      ^------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
sofilepath=$(rpm2cpio "$rpmfile" | cpio -t | grep $sofile)


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 37:
rpm2cpio $rpmfile | cpio -id $sofilepath
         ^------^ SC2086: Double quote to prevent globbing and word splitting.
                             ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
rpm2cpio "$rpmfile" | cpio -id "$sofilepath"


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 38:
chmod 755 $sofilepath
          ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
chmod 755 "$sofilepath"


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 39:
mv -f $sofilepath ./res/.
      ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
mv -f "$sofilepath" ./res/.


In ./Tizen.native/PipelineSample/fetch_shared_object.sh line 44:
rm -rf $rpmfile
       ^------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
rm -rf "$rpmfile"


In ./Tizen.platform/Tizen_IoT_text_classification_NonGUI/gen_tizen_iot_text_classification_rpm.sh line 10:
git add *
        ^-- SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options.


In ./Tizen.platform/Vivante_single_NonGUI_inceptionV3/gen_vivante-inceptionv3-single_rpm.sh line 10:
git add *
        ^-- SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options.


In ./Tizen.platform/Vivante_pipeline_experiments/gen_vivante-pipeline-experiment_rpm.sh line 10:
git add *
        ^-- SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options.


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 33:
time vivante-pipeline-experiment $ENABLE_VNN_INCEPTION $ENABLE_VNN_YOLO $ENABLE_TFLITE_INCEPTION $SLEEP_TIME &
                                 ^-------------------^ SC2086: Double quote to prevent globbing and word splitting.
                                                       ^--------------^ SC2086: Double quote to prevent globbing and word splitting.
                                                                        ^----------------------^ SC2086: Double quote to prevent globbing and word splitting.
                                                                                                 ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
time vivante-pipeline-experiment "$ENABLE_VNN_INCEPTION" "$ENABLE_VNN_YOLO" "$ENABLE_TFLITE_INCEPTION" "$SLEEP_TIME" &


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 34:
process_id=`/bin/ps -fu $USER| grep "vivante-pipeline-experiment" | grep -v "grep" | awk '{print $2}'`
           ^-- SC2006: Use $(...) notation instead of legacy backticked `...`.
                        ^---^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
process_id=$(/bin/ps -fu "$USER"| grep "vivante-pipeline-experiment" | grep -v "grep" | awk '{print $2}')


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 36:
taskset -cp 2-5 $process_id # set affinity
                ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
taskset -cp 2-5 "$process_id" # set affinity


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 37:
end=$((SECONDS+$SLEEP_TIME+1))
^-^ SC2034: end appears unused. Verify use (or export if used externally).
               ^---------^ SC2004: $/${} is unnecessary on arithmetic variables.


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 39:
PEAK=0
^--^ SC2034: PEAK appears unused. Verify use (or export if used externally).


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 40:
MIN=10000000000000
^-^ SC2034: MIN appears unused. Verify use (or export if used externally).


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 41:
CNT=0
^-^ SC2034: CNT appears unused. Verify use (or export if used externally).


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 42:
SUM_MEM=0
^-----^ SC2034: SUM_MEM appears unused. Verify use (or export if used externally).


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 45:
  sleep $SLEEP_TIME
        ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
  sleep "$SLEEP_TIME"


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 46:
  output=`grep VmRSS /proc/$process_id/status`
         ^-- SC2006: Use $(...) notation instead of legacy backticked `...`.
                           ^---------^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
  output=$(grep VmRSS /proc/"$process_id"/status)


In ./Tizen.platform/Vivante_pipeline_experiments/check_usage.sh line 48:
  mem_usage=$((${mem_str[1]}))
               ^-----------^ SC2004: $/${} is unnecessary on arithmetic variables.


In ./Tizen.platform/Vivante_pipeline_NonGUI_yoloV3/gen_vivante-yolov3-pipeline_rpm.sh line 10:
git add *
        ^-- SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options.


In ./Tizen.platform/Vivante_pipeline_NonGUI_inceptionV3/gen_vivante-inceptionv3-pipeline_rpm.sh line 10:
git add *
        ^-- SC2035: Use ./*glob* or -- *glob* so names with dashes won't become options.


In ./templates/tensor_filter_subplugin/deploy.sh line 7:
TARGET=$(pwd)
^----^ SC2034: TARGET appears unused. Verify use (or export if used externally).


In ./templates/tensor_filter_subplugin/deploy.sh line 8:
BASEPATH=`dirname "$0"`
^------^ SC2034: BASEPATH appears unused. Verify use (or export if used externally).
         ^------------^ SC2006: Use $(...) notation instead of legacy backticked `...`.

Did you mean: 
BASEPATH=$(dirname "$0")


In ./templates/tensor_filter_subplugin/deploy.sh line 9:
BASENAME=`basename "$0"`
         ^-------------^ SC2006: Use $(...) notation instead of legacy backticked `...`.

Did you mean: 
BASENAME=$(basename "$0")


In ./templates/tensor_filter_subplugin/deploy.sh line 14:
        printf "usage: ${BASENAME} <name> <path to create>\n\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".


In ./templates/tensor_filter_subplugin/deploy.sh line 21:
path="$2"
^--^ SC2034: path appears unused. Verify use (or export if used externally).


In ./templates/tensor_filter_subplugin/deploy.sh line 26:
        printf "The name \"$1\" contains a whitespace. Cannot proceed.\n\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".


In ./templates/tensor_filter_subplugin/deploy.sh line 32:
        printf "Please provide name with A-Z, a-z, -, _, 0-9 only. The name \"$1\" is not such a name. Cannot proceed.\n\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".


In ./templates/tensor_filter_subplugin/deploy.sh line 37:
if [ -e $2 ]
        ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
if [ -e "$2" ]


In ./templates/tensor_filter_subplugin/deploy.sh line 39:
        printf "The path $2 already exists. Please designate a new path.\n\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".


In ./templates/tensor_filter_subplugin/deploy.sh line 43:
mkdir -p $2
         ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
mkdir -p "$2"


In ./templates/tensor_filter_subplugin/deploy.sh line 44:
if [ ! -d $2 ]
          ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
if [ ! -d "$2" ]


In ./templates/tensor_filter_subplugin/deploy.sh line 46:
        printf "Failed to create a new directory $2.\n\n"
               ^-- SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".


In ./templates/tensor_filter_subplugin/deploy.sh line 50:
printf "Initializing a git repo of tensor-filter at $2.\n"
       ^-- SC2059: Don't use variables in the printf format string. Use printf '..%s..' "$foo".


In ./templates/tensor_filter_subplugin/deploy.sh line 51:
pushd $2
      ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
pushd "$2"


In ./templates/tensor_filter_subplugin/deploy.sh line 55:
cp -R src packaging meson.build $2/
                                ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
cp -R src packaging meson.build "$2"/


In ./templates/tensor_filter_subplugin/deploy.sh line 57:
pushd $2
      ^-- SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
pushd "$2"


In ./templates/tensor_filter_subplugin/deploy.sh line 60:
mv tensor-filter-TEMPLATE.manifest tensor-filter-${name}.manifest
                                                 ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
mv tensor-filter-TEMPLATE.manifest tensor-filter-"${name}".manifest


In ./templates/tensor_filter_subplugin/deploy.sh line 61:
mv tensor-filter-TEMPLATE.spec.in tensor-filter-${name}.spec
                                                ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
mv tensor-filter-TEMPLATE.spec.in tensor-filter-"${name}".spec


In ./templates/tensor_filter_subplugin/deploy.sh line 63:
sed -i "s|TEMPLATE|${name}|" tensor-filter-${name}.spec
                                           ^-----^ SC2086: Double quote to prevent globbing and word splitting.

Did you mean: 
sed -i "s|TEMPLATE|${name}|" tensor-filter-"${name}".spec

For more information:
  https://www.shellcheck.net/wiki/SC2034 -- BASEPATH appears unused. Verify u...
  https://www.shellcheck.net/wiki/SC2035 -- Use ./*glob* or -- *glob* so name...
  https://www.shellcheck.net/wiki/SC2059 -- Don't use variables in the printf...

Some notable rules are SC2034 (unused variables) and SC2006 (use $() instead of non-posix backticks)

What do you think about this? I could make a PR to fix warnings if you like it.

update the get model guidances of native example sources.

Get model scripts was merged into one script(#66).
So, the existing get model guidances are invalid.
We need to update the guidance.

example)
before:

 * Get model by
 * $ cd $NNST_ROOT/bin
 * $ bash get-model-image-classification-tflite.sh
 *

after:

 * Get model by
 *  $bash ../../../bash_script/example_models/get-model.sh  image-classification-tflite
 *

[native] port example_sink to Python

Issue Description

Port example_sink to python

Expected Result

I'm working on porting nnstreamer-example/native/example_sink/ to Python.

• nnstreamer_sink_example.c
• nnstreamer_sink_example_play.c

I'd send PR for each file.

[tizen_iot_text_classification] libnnstreamer_protobuf.so: cannot open shared object file: No such file or directory.

hello, I've built nnstreamer_tizen_iot_text_classification for ubuntu 18.04 on "RaspberryPi4B" and ubuntu 18.04 on "laptop".
I've tested using Rpi4/PC for server/client and vice versa, and got error:

$ ./nnstreamer_tizen_iot_text_classification server tensorflow-lite $MYIP $MYPORT

** (nnstreamer_tizen_iot_text_classification:4824): CRITICAL **: 23:57:48.203: Cannot open protobuf(/usr/lib/nnstreamer/decoders/libnnstreamer_decoder_protobuf.so) with error libnnstreamer_protobuf.so: cannot open shared object file: No such file or directory.

(nnstreamer_tizen_iot_text_classification:4824): GStreamer-CRITICAL **: 23:57:48.208: gst_mini_object_unref: assertion 'mini_object != NULL' failed

... and error chain...

I have a question: what package should I install exactly?
I've installed:

sudo apt install libprotobuf-dev 
sudo apt install libprotobuf-c-dev

but seems not the answer.

Thank you.

[PPA] arm64 build failure

Hello, @gichan-jang I have a question.
I tried ubuntu PPA for nnstreamer-example: https://launchpad.net/~nnstreamer-example/+archive/ubuntu/ppa/+packages

But there is build failure for arm64: https://launchpadlibrarian.net/480209386/buildlog_ubuntu-bionic-arm64.nnstreamer-example_0.1.1-0~202005161235~ubuntu18.04.1_BUILDING.txt.gz

Is it possible to resolve build failure?
Do you have plan to support arm64 ubuntu PPA?

(FYI. I'm using headless ubuntu18.04 on raspberry pi 4b)
Thank you.

[Feature request] Support tensorflow-lite basic model with bash_script

I am new to NNS. I think that I am not good enough at understanding NNS. To help someone like me, I suggest support all of six tensorflow-lite basic models as bash_script example.

it contains 6 models.

1. Image Classification
2. Object detection
3. pose estimation
4. segmentation
5. style transfer
6. smart reply

With this support, NNS will be able to prove that it can be applied to various models. And also it helps newbies understand faster and repository star increase fast.

It is first time to request feature for me. Let me know if it doesn't have enough information about what I am getting at.

[Question] how to check v4l2 device deliver jpeg/image or not.

I want to help someone new to NNS understand and easily build nnstreamer-example.

My problem is in 'nnstreamer-example/bashscript/example_object_detection_tensorflow_lite/gst-launch-object-detection-tflite.sh'

#!/usr/bin/env bash
gst-launch-1.0 \
	v4l2src name=cam_src !  videoconvert ! videoscale ! video/x-raw

... omission ... 

when I run script mentioned above, this error occured.

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:cam_src: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:cam_src:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.006039041
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I found that it is caused by webcam data format. My labtop wemcam deliver only jpeg/image data.

v4l2 gstv4l2object.c:4238:gst_v4l2_object_probe_caps:<cam_src:src> probed caps:
image/jpeg, width=(int)1280, height=(int)720,  ... omission...
image/jpeg, width=(int)960, height=(int)540,  ... omission...
image/jpeg, width=(int)848, height=(int)480, ... omission...
image/jpeg, width=(int)640, height=(int)480,  ... omission...
image/jpeg, width=(int)640, height=(int)360, ... omission...
image/jpeg, width=(int)424, height=(int)240,  ... omission...
image/jpeg, width=(int)352, height=(int)288,  ... omission...
image/jpeg, width=(int)320, height=(int)240,  ... omission...
image/jpeg, width=(int)320, height=(int)180, ... omission...
image/jpeg, width=(int)176, height=(int)144,  ... omission...

After several tries, I get to know when it works for me.

1. Internal data stream error: original script
2. Show only my face : Add jpegdec element with framerate 30/1
3. Show my face with object-detection box : Add jpegdec element with framerate 10/1
4. Show only my face : Add decodebin with framerate 30/1
5. Internal data stream error : decodebin with framerate 10/1

//1. 
#!/usr/bin/env bash
gst-launch-1.0 \
	v4l2src name=cam_src ! videoconvert ! videoscale ! 
video/x-raw,width=640,height=480,format=RGB,framerate=30/1 ! tee name=t \
	... same with original script ...

2.
#!/usr/bin/env bash
gst-launch-1.0 \
	v4l2src name=cam_src ! jpegdec ! videoconvert ! videoscale ! 
video/x-raw,width=640,height=480,format=RGB,framerate=30/1 ! tee name=t \
	... same with original script ...

3.
#!/usr/bin/env bash
gst-launch-1.0 \
	v4l2src name=cam_src ! jpegdec ! videoconvert ! videoscale ! 
video/x-raw,width=640,height=480,format=RGB,framerate=10/1 ! tee name=t \
	... same with original script ...

4.
#!/usr/bin/env bash
gst-launch-1.0 \
	v4l2src name=cam_src ! decodebin! videoconvert ! videoscale ! 
video/x-raw,width=640,height=480,format=RGB,framerate=30/1 ! tee name=t \
	... same with original script ...

5.
#!/usr/bin/env bash
gst-launch-1.0 \
	v4l2src name=cam_src ! decodebin! videoconvert ! videoscale ! 
video/x-raw,width=640,height=480,format=RGB,framerate=10/1 ! tee name=t \
	... same with original script ...

How can I make this script work for everyone?
Please give me some tips.

• Oracle VM Virtual BOX
• GUEST : Ubuntu18.04LTS
• HOST : Window 10