intel-iot-devkit / people-counter-cpp

Create a smart video application using the Intel Distribution of OpenVINO toolkit. The toolkit uses models and inference to run single-class object detection.

DISCONTINUATION OF PROJECT

This project will no longer be maintained by Intel.

This project has been identified as having known security escapes.

Intel has ceased development and contributions including, but not limited to, maintenance, bug fixes, new releases, or updates, to this project.

Intel no longer accepts patches to this project.

People Counter

Details
Target OS:	Ubuntu* 18.04 LTS
Programming Language:	C++
Time to Complete:	45 min

What It Does

This application is one of a series of IoT reference implementations illustrating how to develop a working solution for a particular problem. It demonstrates how to create a smart video IoT solution using Intel® hardware and software tools. This people counter solution detects people in a designated area, providing number of people in the frame, their average duration in the frame, and the total count.

How It Works

The counter uses the Inference Engine included in the Intel® Distribution of OpenVINO™ toolkit. A trained neural network detects people within a designated area by displaying a bounding box over them. It counts the number of people in the current frame, the duration that a person is in the frame (time elapsed between entering and exiting a frame), and the total number of people detected, and then sends the data to a local web server using the Paho* MQTT C client libraries.

Requirements

Hardware

• 6th to 8th generation Intel® Core™ processors with Iris® Pro graphics or Intel® HD Graphics.

Software

• Ubuntu* 18.04 LTS
  
  Note: We recommend using a 4.14+ Linux* kernel with this software. Run the following command to determine your kernel version:

  uname -a
  

• OpenCL™ Runtime Package

• Intel® Distribution of OpenVINO™ toolkit 2020 R3 release

• Node v6.17.1

• Npm v3.10.10

• MQTT Mosca* server

Install Intel® Distribution of OpenVINO™ toolkit

Refer to Install Intel® Distribution of OpenVINO™ toolkit for Linux* to learn how to set up the toolkit.

Install the OpenCL™ Runtime Package to run inference on the GPU. It is not mandatory for CPU inference.

Setup

Get the code

Steps to clone the reference implementation:

sudo apt-get update && sudo apt-get install git
git clone https://github.com/intel-iot-devkit/people-counter-cpp.git 

Which model to use

This application uses the person-detection-retail-0013 Intel® model, that can be accessed using the model downloader. The model downloader downloads the .xml and .bin files that will be used by the application.

Install the dependencies

To install the dependencies of the RI and to download the person-detection-retail-0013 Intel® model, run the following command:

cd <path_to_the_people-counter-cpp_directory>
./setup.sh

Make sure the npm and node versions are exact, using the commands given below:

node -v

The version should be v6.17.1

npm -v

The version should be v3.10.10

Note: If the Node and Npm versions are different, run the following commands:

sudo npm install -g n
sudo n 6.17.1

Note: After running the above commands, please open a new terminal to proceed further. Also, verify the node and npm versions from the new terminal.

Installation

Go to people-counter-cpp directory:

cd <path_to_people-counter-cpp_directory>

MQTT Mosca* server

cd webservice/server
npm install
npm i [email protected]

Web Server

cd ../ui
npm install

Run the Application

There are three components that need to be running in separate terminals for this application to work:

• MQTT Mosca server
• Node.js* Web server
• FFmpeg server

Go to people-counter-cpp directory:

cd <path_to_people-counter-cpp_directory>

Step 1 - Start the Mosca server

Ensure that no process is running at port address 3000 using the following command:

sudo lsof -i:3000

Navigate to the node-server path and run the server using following commands:

cd webservice/server/node-server
node ./server.js

If successful, this message will appear in the terminal:

connected to ./db/data.db
Mosca server started.

Step 2 - Start the GUI

Open a new terminal and run the commands below:

cd ../../ui
npm run dev

If successful, this message will appear in the terminal:

webpack: Compiled successfully.

Step 3 - FFmpeg Server

Open a new terminal and run the below commands:

cd ../..
sudo ffserver -f ./ffmpeg/server.conf

Step 4 - Set Up the Environment

Open a new terminal in the current directory and run the below command to set up the environment variables required to run the Intel® Distribution of OpenVINO™ toolkit applications:

source /opt/intel/openvino/bin/setupvars.sh

Note: This command only needs to be executed once in the terminal where the application will be executed. If the terminal is closed, the command needs to be executed again.

Step 5 - Build and Start the Main Application

This application uses the SSD derived person detection model bundled with the Intel® Distribution of OpenVINO™ toolkit. To do a clean re-build, run the following commands:

cd ieservice
mkdir -p build && cd build
cmake ..
make

The new version of the software will be built as people-counter/ieservice/bin/intel64/Release/obj_recognition. Switch to the directory where the main application was built:

cd ../bin/intel64/Release

Set up the needed MQTT environment variables:

export MQTT_SERVER=localhost:1884
export MQTT_CLIENT_ID=cvservice

Run on the CPU

./obj_recognition -i Pedestrain_Detect_2_1_1.mp4 -m /opt/intel/openvino/deployment_tools/tools/model_downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml -d CPU -thresh 0.65 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -i - http://localhost:8090/fac.ffm

Note:To see the output on web based interface, open the link http://localhost:8080 on browser. Refresh the browser window if the video does not play automatically.

Run on the GPU

• To use GPU in 16-bit mode, use the following command:

  ./obj_recognition -i Pedestrain_Detect_2_1_1.mp4 -m /opt/intel/openvino/deployment_tools/tools/model_downloader/intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml -d GPU -thresh 0.65 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -i - http://localhost:8090/fac.ffm
  

  To see the output on web based interface, open the link http://localhost:8080 on browser.

• To use GPU in 32-bit mode, use the following command:

  ./obj_recognition -i Pedestrain_Detect_2_1_1.mp4 -m /opt/intel/openvino/deployment_tools/tools/model_downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml -d GPU -thresh 0.65 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -i - http://localhost:8090/fac.ffm
  

  To see the output on web based interface, open the link http://localhost:8080 on browser.

Note: The Loading time for GPU is more, so it might take few seconds to display the output. If request busy error is observed, please restart the ffmpeg server and try again.

Run on the Intel® Neural Compute Stick

./obj_recognition -i Pedestrain_Detect_2_1_1.mp4 -m /opt/intel/openvino/deployment_tools/tools/model_downloader/intel/person-detection-retail-0013/FP16/person-detection-retail-0013.xml -d MYRIAD -thresh 0.65 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -i - http://localhost:8090/fac.ffm

To see the output on web based interface, open the link http://localhost:8080 on browser.

Note: The Intel® Neural Compute Stick can only run FP16 models. The model that is passed to the application, through the -m <path_to_model> command-line argument, must be of data type FP16.

Use a Camera Stream

Use the camera ID followed by -i , where the ID is taken from the video device (the number X in /dev/videoX). On Ubuntu, to list all available video devices use the following command:

ls /dev/video*

For example, if the output of the above command is /dev/video0, then camera ID would be: 0

Run the application:

./obj_recognition -i 0 -m /opt/intel/openvino/deployment_tools/tools/model_downloader/intel/person-detection-retail-0013/FP32/person-detection-retail-0013.xml -d CPU -thresh 0.65 | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 768x432 -i - http://localhost:8090/fac.ffm

To see the output on web based interface, open the link http://localhost:8080 on browser.

Note: Use the camera's resolution with -video_size to observe the output on the web based interface.

people-counter-cpp's Issues

Not Able to see any video on the UI interface

Could anybody help me out with this

Can not link libaho-mqtt3cs.so in my project

HI,all

• Thanks for your project providing a effective way to link the paho.mqtt.c in my project with cmake.
• But when I use it in my project, I got the error in link step /usr/bin/ld: 找不到 -lpaho-mqtt3cs. I would appreciate it if you could give me some guidance

detail information

• pahomqtt.cmake

include(ExternalProject)
find_package(Git REQUIRED)
find_package(PkgConfig REQUIRED)
pkg_search_module(OPENSSL REQUIRED openssl)

ExternalProject_add(
    pahomqtt
    GIT_REPOSITORY "https://github.com/eclipse/paho.mqtt.c.git"
    GIT_TAG "master"
    UPDATE_COMMAND ""
    PATCH_COMMAND ""
    SOURCE_DIR "${CMAKE_BINARY_DIR}/paho-src"
    CMAKE_ARGS -DPAHO_WITH_SSL=TRUE -DCMAKE_INSTALL_PREFIX=${CMAKE_BINARY_DIR}/paho-build
)

include_directories(${CMAKE_BINARY_DIR}/paho-build/include)
link_directories(${CMAKE_BINARY_DIR}/paho-build/lib)

• project CMakeLists.txt

include(pahomqtt)
add_dependencies(connect pahomqtt)
target_link_libraries(connect PUBLIC paho-mqtt3cs)  # connect is a dynamic library name compiled

env

• ubuntu 18.04
• cmake version 3.16.0

running

./obj_recognition -i Pedestrain_Detect_2_1_1.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/person-detection-retail-0012/FP16/person-detection-retail-0012.xml -l /opt/intel/computer_vision_sdk/deployment_tools/intel_models/person-detection-retail-0012/FP16/person-detection-retail-0012.bin -d GPU -t SSD -thresh 0.7 0 2>/dev/null | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 544x320 -i - http://localhost:8090/fac.ffm
Output file is empty, nothing was encoded (check -ss / -t / -frames parameters if used)

Using CV SDK include path

Using CV SDK include path /opt/intel//computer_vision_sdk_2018.5.445/deployment_tools/inference_engine
-- CMAKE_BUILD_TYPE not defined, 'Release' will be used
-- /etc/*-release distrib: Ubuntu 16.04
CMake Error at CMakeLists.txt:149 (include):
include could not find load file:

feature_defs

CMake Error at CMakeLists.txt:151 (add_subdirectory):
add_subdirectory given source
"/opt/intel//computer_vision_sdk_2018.5.445/deployment_tools/inference_engine/samples/extension"
which is not an existing directory.

CMake Error at sample/CMakeLists.txt:27 (find_package):
Could not find a configuration file for package "OpenCV" that is compatible
with requested version "3.2".

The following configuration files were considered but not accepted:

/opt/intel/computer_vision_sdk_2018.5.445/opencv/cmake/OpenCVConfig.cmake, version: 4.0.1

Link in README.md is broken

At the beginning README.md file there is a line:

This reference implementation is also available in Python.

The link in this line is broken.

Link to python implementation is 404

As title, the link on the readme is incorrect, and should point to https://github.com/intel-iot-devkit/people-counter-python

Cmake error

CMakeFiles/cmTC_22551.dir/build.make:97: recipe for target 'cmTC_22551' failed
make[1]: *** [cmTC_22551] Error 1
make[1]: Leaving directory '/home/shivaji/Desktop/people-counter-master/ieservice/build/CMakeFiles/CMakeTmp'
Makefile:126: recipe for target 'cmTC_22551/fast' failed
make: *** [cmTC_22551/fast] Error 2

can not install Paho* MQTT

follow the instructions but when install MQTT with 'sudo make install' it fails.

on Intel(R) Core(TM) i5-6400 desktop with ubuntu16.04.5, kernel is 4.4.0-134-generic

install -m 644 doc/man/man1/paho_c_pub.1 /usr/local/share/man/man1
install -m 644 doc/man/man1/paho_c_sub.1 /usr/local/share/man/man1
install -m 644 doc/man/man1/paho_cs_pub.1 /usr/local/share/man/man1
install -m 644 doc/man/man1/paho_cs_sub.1 /usr/local/share/man/man1
install -m 644 build/output/doc/MQTTClient/man/man3/MQTTClient.h.3 /usr/local/share/man/man3
install: cannot stat 'build/output/doc/MQTTClient/man/man3/MQTTClient.h.3': No such file or directory
Makefile:273: recipe for target 'install' failed
make: [install] Error 1 (ignored)
install -m 644 build/output/doc/MQTTAsync/man/man3/MQTTAsync.h.3 /usr/local/share/man/man3
install: cannot stat 'build/output/doc/MQTTAsync/man/man3/MQTTAsync.h.3': No such file or directory
Makefile:273: recipe for target 'install' failed
make: [install] Error 1 (ignored)

Not able to see video in demo

In People-counter demo, I am not able to see visual.
In object detection, there is not option to pass input as a camera. Only path of video can be passed.

Below is the command I am running.
./obj_recognition -i Pedestrain_Detect_2_1_1.mp4 -m /opt/intel/computer_vision_sdk/deployment_tools/intel_models/person-detection-retail-0013/FP32/person-detection-retail-0013.xml -l /opt/intel/computer_vision_sdk/deployment_tools/intel_models/person-detection-retail-0013/FP32/person-detection-retail-0013.bin -t SSD -thresh 0.7 -d CPU 0 2>/dev/null | ffmpeg -v warning -f rawvideo -pixel_format bgr24 -video_size 544x320 -i - http://localhost:8090/fac.ffm

I am only able to see people count. The processed video is not visible.

Running without argument i gives below error:
ERROR: image(s) for inference - not set

Can we run this demo for camera setup?
Please help !!