z00389191 / sample-videoanalysisperson

Codes for video analysis: person, age and so on

EN|CN

Developers can deploy the application on the Atlas 200 DK AI acceleration cloud server to decode the local MP4 file or RTSP video streams, detect pedestrians, and human faces in video frames, predict their attributes, generate structured information, and send the structured information to the server for storage and display.

Prerequisites

Before using an open source application, ensure that:

• Mind Studio has been installed. For details, see Mind Studio Installation Guide.
• The Atlas 200 DK developer board has been connected to Mind Studio, the cross compiler has been installed, the SD card has been prepared, and basic information has been configured. For details, see Atlas 200 DK User Guide.

Software Preparation

Before running the application, obtain the source code package and configure the environment as follows.

1. Obtain the source code package.

   Download all the code in the sample-facedetection repository at https://github.com/Ascend/sample-videoanalysisperson to any directory on Ubuntu Server where Mind Studio is located as the Mind Studio installation user, for example, /home/ascend/sample-videoanalysisperson.

2. Log in to Ubuntu Server where Mind Studio is located as the Mind Studio installation user and set the environment variable DDK_HOME.

   vim ~/.bashrc

   Run the following commands to add the environment variables DDK_HOME and LD_LIBRARY_PATH to the last line:

   export DDK_HOME=/home/XXX/tools/che/ddk/ddk

   export LD_LIBRARY_PATH=$DDK_HOME/uihost/lib

   NOTE:

   • XXX indicates the Mind Studio installation user, and /home/XXX/tools indicates the default installation path of the DDK.
   • If the environment variables have been added, skip this step.

   Enter :wq! to save and exit.

   Run the following command for the environment variable to take effect:

   source ~/.bashrc

Deployment

1. Access the root directory where the video analysis application code is located as the Mind Studio installation user, for example, /home/ascend/sample-videoanalysisperson

2. Run the deployment script to prepare the project environment, including compiling and deploying the ascenddk public library, downloading the network model, and configuring Presenter Server.

   bash deploy.sh host_ip model_mode

   • host_ip: For the Atlas 200 DK developer board, this parameter indicates the IP address of the developer board.For the AI acceleration cloud server, this parameter indicates the IP address of the host.

   • model_mode indicates the deployment mode of the model file. The value can be local or internet. The default setting is internet.

     • local: If the Ubuntu system where Mind Studio is located is not connected to the network, use the local mode. In this case, you need to have downloaded the model file and the dependent common code library to the sample-videoanalysisperson/script directory by referring to Downloading Network Models and Dependency Code Library.
     • internet: If the Ubuntu system where Mind Studio is located is connected to the network, use the Internet mode. In this case, download the model file and dependency code library online.

   Example command:

   bash deploy.sh 192.168.1.2 internet

   • When the message Please choose one to show the presenter in browser(default: 127.0.0.1): is displayed, enter the IP address used for accessing the Presenter Server service in the browser. Generally, the IP address is the IP address for accessing the Mind Studio service.
   • When the message Please input a absolute path to storage video analysis data: is displayed, enter the existing absolute path for storing video analysis data in Mind Studio. The Mind Studio user must have the read and write permissions.

   Select the IP address used by the browser to access the Presenter Server service in Current environment valid ip list and enter the path for storing video analysis data, as shown in Figure 1.

   Figure 1 Project deployment
   

3. Start Presenter Server.

   Run the following command to start the Presenter Server program of the video analysis application in the background:

   python3 presenterserver/presenter_server.py --app video_analysis_person &

   NOTE:
   presenter_server.py is located in the presenterserve directory. You can run the python3 presenter_server.py -h or python3 presenter_server.py --help command in this directory to view the usage method of presenter_server.py.

   Figure 2 shows that the presenter_server service is started successfully.

   Figure 2 Starting the Presenter Server process
   

   Use the URL shown in the preceding figure to log in to Presenter Server (only the Chrome browser is supported). The IP address is that entered in 2 and the default port number is 7011. The following figure indicates that Presenter Server is started successfully.

   Figure 3 Home page
   

4. The video structured application can parse local videos and RTSP video streams.

   • To parse a local video, upload the video file to the Host.

     For example, upload the video file person.mp4 to the /home/HwHiAiUser/sample directory on the host.

     NOTE:
     H.264 and H.265 MP4 files are supported. If an MP4 file needs to be edited, you are advised to use FFmpeg. If a video file is edited by other tools, FFmpeg may fail to parse the file.

   • If only RTSP video streams need to be parsed, skip this step.

Running

1. Run the video analysis application.

   Run the following command in the /home/ascend/sample-videoanalysiscar directory to start the video analysis application:

   bash run_videoanalysispersonapp.sh host_ip _ presenter_view_appname channel1_ [channel2] &

   • host_ip: For the Atlas 200 DK developer board, this parameter indicates the IP address of the developer board.For the AI acceleration cloud server, this parameter indicates the IP address of the host.
   • presenter_view_app_name: Indicates View Name displayed on the Presenter Server page, which is user-defined.
   • channel1: absolute path of a video file on the host
   • channel2: URL of an RTSP video stream

   Example command:

   bash run_videoanalysispersonapp.sh 192.168.1.2 video /home/HwHiAiUser/sample/person.mp4 &

2. Use the URL that is displayed when you start the Presenter Server service to log in to the Presenter Server website (only the Chrome browser is supported). For details, see 3.

   NOTE:
   The Presenter Server of the video analysis application supports the display of a maximum of two presenter_view_app_name at the same time.

   The navigation tree on the left displays the app name and channel name of the video. The large image of the extracted video frame and the detected target small image are displayed in the middle. After you click the small image, the detailed inference result and score are displayed on the right.

   This application supports human body attribute detection, and facial attribute detection.

   • Human body attributes include:

     Ages (ages 16–30, 31–45, 46–60 as well as 61 and above); Backpack; Carrying other; Casual lower; Casual upper; Formal lower; Hat; Jacket; Jeans; Leather Shoes; Logo; Short hair/Long hair; Male/Female; Messenger Bag; Muffler; No accessory; No carrying; Plaid; Plastic bags; Sandals; Shoes; Shorts; Short Sleeve; Skirt; Sneaker; Stripes; Sunglasses; Trousers; T-shirt; Upper other; V-Neck.

     In the detailed inference result display area on the video analysis interface, Age, Male/Female, and Short hair/Long hair are mandatory. Other attributes are displayed only when the confidence level is greater than 0.5.

   • Facial attribute detection supports age and gender identification.

Follow-up Operations

• Stopping the Video Structured Analysis Application

  To stop the video analysis application, perform the following operations:

  Run the following command in the sample-videoanalysisperson directory as the Mind Studio installation user:

  bash stop_videoanalysispersonapp.sh host_ip

  host_ip: For the Atlas 200 DK developer board, this parameter indicates the IP address of the developer board.For the AI acceleration cloud server, this parameter indicates the IP address of the host.

  Example command:

  bash stop_videoanalysispersonapp.sh 192.168.1.2

• Stopping the Presenter Server Service

  The Presenter Server service is always in the running state after being started. To stop the Presenter Server service of the video structured analysis application, perform the following operations:

  Run the following command to check the process of the Presenter Server service corresponding to the video structured analysis application as the Mind Studio installation user:

  ps -ef | grep presenter | grep video_analysis_person

  ascend@ascend-HP-ProDesk-600-G4-PCI-MT:~/sample-videoanalysisperson$ ps -ef | grep presenter | grep video_analysis_car
  ascend 3656 20313 0 15:10 pts/24?? 00:00:00 python3 presenterserver/presenter_server.py --app video_analysis_person
  

  In the preceding information, 3656 indicates the process ID of the Presenter Server service corresponding to the facial recognition application.

  To stop the service, run the following command:

  kill -9 3656

Downloading Network Models and Dependency Code Library

• Downloading network model

  The models used in the Ascend DK open source applications are converted models that adapt to the Ascend 310 chipset. For details about how to download this kind of models and the original network models, see Table 1. If you have a better model solution, you are welcome to share it at https://github.com/Ascend/models.

  Download the network models files (.om files) to the sample-videoanalysisperson/script directory.

  Table 1 Models used in Atlas DK open source applications

  Model Name	Description	Model Download Path	Original Network Download Address
  Network model for face detection (face_detection.om)	It is a network model converted from ResNet0-SSD300 model based on Caffe.	Download the model from the computer_vision/object_detect/face_detection directory in the https://github.com/Ascend/models/ repository. For the version description, see the README.md file in the current directory.	For details, see the README.md file of the computer_vision/object_detect/face_detection directory in the https://github.com/Ascend/models/ repository. Precautions during model conversion: During the conversion, a message is displayed indicating that the conversion fails. You only need to select SSDDetectionOutput from the drop-down list box for the last layer and click Retry.
  Network model for performing inference on human body features (pedestrian.om)	It is a VeSPA model based on Caffe.	Download the model from the computer_vision/classification/pedestrian directory in the https://github.com/Ascend/models/ repository. For the version description, see the README.md file in the current directory.	For details, see the README.md file of the computer_vision/classification/pedestrian directory in the https://github.com/Ascend/models/ repository.
  Network model for identifying the age (inception_age.om)	It is an Inception V3 model based on TensorFlow.	Download the model from the computer_vision/classification/inception_age directory in the https://github.com/Ascend/models/ repository. For the version description, see the README.md file in the current directory.	For details, see the README.md file of the computer_vision/classification/inception_age directory in the https://github.com/Ascend/models/ repository. Precautions during model conversion: The age_inference node processes ten pictures at a time. Therefore, the value of N in Input Shaple needs to be changed to 10 during conversion.
  Network model for identifying the gender (inception_gender.om)	It is an Inception V3 model based on TensorFlow.	Download the model from the computer_vision/classification/inception_gender directory in the https://github.com/Ascend/models/ repository. For the version description, see the README.md file in the current directory.	For details, see the README.md file of the computer_vision/classification/inception_gender directory in the https://github.com/Ascend/models/ repository. Precautions during model conversion: The gender_inference node processes ten pictures at a time. Therefore, the value of N in Input Shaple needs to be changed to 10 during conversion.
  Network model for object detection (vgg_ssd.om)	It is an SSD512 model based on Caffe.	Download the model from the computer_vision/object_detect/vgg_ssd directory in the https://github.com/Ascend/models/ repository. For the version description, see the README.md file in the current directory.	For details, see the README.md file of the computer_vision/object_detect/vgg_ssd directory in the https://github.com/Ascend/models/ repository.

  • Download the dependent software libraries

    Download the dependent software libraries to the sample-videoanalysisperson/script directory.

    Table 2 Download the dependent software library

    Module Name	Module Description	Download Address
    EZDVPP	Encapsulates the dvpp interface and provides image and video processing capabilities, such as color gamut conversion and image / video conversion	https://github.com/Ascend/sdk-ezdvpp After the download, keep the folder name ezdvpp。
    Presenter Agent	API for interacting with the Presenter Server.	https://github.com/Ascend/sdk-presenter/tree/master/presenteragent After the download, keep the folder name presenteragent.
    FFmpeg code	The current application uses FFmpeg to decapsulate video files.	The URL for downloading the FFmpeg 4.0 code is https://github.com/FFmpeg/FFmpeg/tree/release/4.0. After the download, keep the folder name ffmpeg.
    tornado (5.1.0) protobuf (3.5.1) numpy (1.14.2)	Python libraries that Presenter Server depends on..	Search for related sources and install them.