conchylicultor / clientonlinereid

Mqtt client which get features from other client and try to compute the reidentification

ClientOnlineReid

Introduction

Mqtt client which get features from other client and try to compute the reidentification

Installation

Dependencies: In order to compile, the program will require QMake (tested with v3.0), OpenCv (tested with 2.4.8) and Mosquittopp (tested with v1.3.4). It also require a compiler compatible with C++11.

The path structure must look like this:

• ReidPath/ClientOnlineReid/ : This repository.
• ReidPath/ClientOnlineReid/build/ : Contain the executable file (must be the working directory when running the program !)
• ReidPath/Data/Training : Contain the informations used to train the program. Those files can be generated in training mode. Otherwise, here is an example of training file, if you don't want to generate yours.
• ReidPath/Data/Received : Store the received sequences. This folder contain temporary files used by the program but useless for the final user (except eventually for debugging).
• ReidPath/Data/OutputReid : Folder where the result of the reidentification will be stored.

In order to run the program, the folder ClientOnlineReid must contain a config.yml indicating the ip adress of the mqtt brocker:

%YAML:1.0
brokerIp:'192.168.100.13'

Running

Multiple modes are proposed by this software:

• Training mode: the program record all person with there correct identity. No recognition is made. This mode is used to generate the differents positive and negative samples in order to train our binary classifier.
• Testing mode: Test our adaptative database and record the errors (kind, number,...). In this mode (as the previous one), the received sequences must have been labelized.
• Release mode: Same as training mode but without any kind of verification.

Here are the controls of the program:

• s: Switch between the modes (Release, Testing, Training)
• q: Exit the program
• For the training mode:
  • t: Generate and record positive and negative sample from the received data in order to generate the training set. The training data are saved on the Training/ folder.
  • g: Generate a testing set using all the received sequences and test the efficiency of the binary classifier.
  • b: Do both training and testing on the current received sequence by using half-half of the received sequences. This has to be used only for estimating the efficiency of our classifier. No training file is saved.
  • a: Activate the calibration mode. If the calibration mode is activated, all incoming sequences will be concidered as one person (no recognition possible but useful to see what path this person has done in order to compute the network topology). It is not require to be in calibration mode to compute the transitions.
  • c: Use the camera information of the received sequence to compute the transitions between the camera and having an idea Calibrate the camera. The results are saved into the Training/ folder.
  • p: Plot the different transitions between the camera which have been computed.
• For the testing mode:
  • e: In testing mode, Evaluate and plot the result to show how well our algorithm perform. The data are also saved in a .csv file (on the OutputReid/ directory) which can be imported in any spreadsheet software.
• For both testing and release mode:
  • a: Switch between the two recognition modes: either each sequence is added as a new person (whatever the person already exist in the dataset or not) or if there is a match (reidentification score above threshold), the sequence is only added to the most similar one.
  • n: Record the network (in the OutputReid/ directory) of the current recognition database.
  • d: Activate/desactivate the debug mode. If enable, all recognitions will be saved to precisely see where the recognition fails.

Warning: Each time the program save a file, it replace the previous file without asking for confirmation. Be carful if you don't want to lost data.

clientonlinereid's Issues

Bug when disconnect properly

Last will not send to the broker (call loop ?)

Recording mode

Record all sequences instead of remove it when loading

Add first appearance as new transition

From 0 to camX

Test to only add the reid sequence to the most similar person

Instead of having multiple recognition.

Crash when plot transitions (if new cam)

When Transition::checkCam(), we do not update the background.
Create problem when we call Transition::plotTransitions()

Transition prediction algorithm

For the exit vector

1. Look for the closest end of the arrow vector + add the angle parameter
2. Deduce the next appearance

Update the person last transition (if not already done)

Inteligent person update

When a person is recognized, update his database images (features), position (cam infos) in a more intelligent way

Share features configuration

Send over the network all configuration value used by the features (which kind of feature ? , which constant ?).
Thanks to that, all clients will share the same configuration, without any need to rebuild all.

Warning: if the camera client is connected while the features are changed, the client has to update some values.

Person contain list of Sequence

Instead of list of features and camInfo as actual.

Check trainAndTest

Are the features vectors correctly moved when we swap the database ?

Add detail success to the evaluation plot

(Differentiate between success due to reidentification or success due to non-reidentification)

Log scaling for time

The exit and entrance time can be really close (1-2s) or really long (3h+). It could be relevant to try to scale the duration with a logarithmic scale.

New debug mode

Record the sequences images in folder "Result/Pers/idPers/Correct" or "Result/Pers/idPers/Error"

Verify that I use const ref when needed

For more proper code

for (auto const &x : vec)

instead of:

for (auto x : vec)

Reverse transitions

In order to double the number of transition.

Camera training

Training:
Check the number of cameras
Associate each camera name/hashcode with a number
Record the association in the training file.
When loading the training file (loadmachineLearning), read the association

Testing:
Add a categorical feature camera id of dimention nbOfCam. For instance (0,0,1,0) for cam3. (1,0,0,0) for cam1.
When receive the feature, compute his id for the distance.

Received sequences ignored

Some sequences are ignored. Pb with the recording/loading on disk

Merge features and camInfos

Instead of two list (feature and camInfosList) into a person element, add a struct which contain the two. Like:

struct SequenceElement
{
    vector<FeaturesElement> features;
    CamInfosElement camInfosList;
};