zqpei / deep_sort_pytorch

MOT using deepsort and yolov3 with pytorch

Hi there 👋, this is ZQPei's github.

• 🔭 I’m currently working on ML systems.
• 🌱 I’m currently learning tvm.
• 📫 How to reach me: [email protected].

Github Homepage Statistics

deep_sort_pytorch's Issues

how to train YOLOV3 for my own object.

thanks for your works. and how to train YOLOV3 for my own object. follow the official step?

Object detection issue

When I run the code, it detects the object more than one time as showing below can you please fix this issue? also, can you continue improving the speed?

deep_sort on custom video

hello @ZQPei , i hope that you are doing well. i am going to use deep_sort to track custom objects (human head), i have darknet on my dataset , i have yolov3 model trained on head dataset. when i tried to use deep-sort_yolov3 repository, the identification is erroneous. there is a huge difference between the ids of tracked objects. i want to use the original deep_sort repository, in order to track human in my own video. in deep_sort ,the detection ground truth is required. since i don't have the groundtruth label of my video, i am wondering , how can i use deep_sort to track objects in custom video. your suggestion will be helpful.

针对不同目标

@ZQPei,你好，我想请问一下，您的检测预训练模型只能检测行人吗，能不能检测其他目标比如说汽车，谢谢

Can you add some details about train.py and data to README.md?Thanks a lot.

@ZQPei
==================================== ERRORS ====================================
__________________________ ERROR collecting train.py ___________________________
train.py:44: in
torchvision.datasets.ImageFolder(train_dir, transform=transform_train),
../../anaconda3/envs/CenterNet/lib/python3.6/site-packages/torchvision/datasets/folder.py:178: in init
target_transform=target_transform)
../../anaconda3/envs/CenterNet/lib/python3.6/site-packages/torchvision/datasets/folder.py:79: in init
"Supported extensions are: " + ",".join(extensions)))
E RuntimeError: Found 0 files in subfolders of: /home/ying/centerNet-deep-sort-master/deep/data/train
E Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.49 seconds ============================
Process finished with exit code 0

realtime testing

Hello ZQPei,

How can i modify the code in demo.py to use the live stream directly from the webcamera in real time?

feature extractor batch vs single inference results

in my test case,

image1, image2

im_crops = [image1, image2]
features = feature_extractor(im_crops)
feature1 = features[0]
feature2 = features[1]

feature3 = feature_extractor([image1])[0]
feature4 = feature_extractor([image2])[0]

feature1 and feature3 are different.

is my test wrong ? or am i using extractor incorrectly?
i think feature1 and feature3 are must be same .. because of 'batch inference'

Dear ZqPei

Hello ZqPei,

I wanted to run train.py.
So I downloaded the dataset Mars and put it as input.
However, there are 625 classes in the train dataset and 636 classes in the test dataset, resulting in a CUDA_error.

So I removed 11 test dataset folders and run the code
Looking at train.jpg, it seems that learning is not working properly
I would appreciate if you could give me an answer.

RuntimeError: Attempting to deserialize object on CUDA device 2 but torch.cuda.device_count() is 1.

Hi,
I am trying to run the demo_yolov3_deepsort.py with the original_ckpt.t7 model but received this following error as it's not the case for the ckpt.t7 model, do you have any idea?
RuntimeError: Attempting to deserialize object on CUDA device 2 but torch.cuda.device_count() is 1. Please use torch.load with map_location to map your storages to an existing device.
I am doing this because I noticed that the tracking performance is not as good as the official implementation in TF. Specifically, the ID switching problem is more noticeable.

the mode of the feature extractor?

In your code, the feature extractor extract appearance feature for one patch in one forward propagation. Why not do this process in a batch mode?

Did someone test it on MOT16 train set？I got an unreasonable result.

This is the result by testing the model that is provided by the official repo
********************* Your MOT16 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
37.3 74.7 24.9| 31.3 94.0 0.42| 517 30 180 307| 2211 75823 241 1189| 29.1 78.5 29.3

However，I get a different result by testing the model of the current repo.
********************* Your MOT16 Results *********************
IDF1 IDP IDR| Rcll Prcn FAR| GT MT PT ML| FP FN IDs FM| MOTA MOTP MOTAL
1.0 1.5 0.7| 1.0 1.9 10.23| 517 0 22 495| 54369109354 40 49| -48.3 60.6 -48.3

What is wrong with it? And could someone help me?
Thank you very much!

Explanation of deep sort architecture

Dear ZQPei,
In the train chart, there are two loss charts named loss(left) and top1err(right). if the top1err represents the error after softmax, then what does the the "loss" chart represent?
Also, what should be the general output of the mAP value of toperror while running evaluation.py?

Good Job !

来学习了 ，沙发！ 我也一直想做些扩展优化，无奈 tensorflow 不熟悉，再忙点就一直没有推进，多谢你的分享！

_next_id init error ?

Hi. Thanks for this cool repo. Shouldn't this https://github.com/ZQPei/deep_sort_pytorch/blob/master/sort/tracker.py#L48

self._next_id = 1

be

self._next_id = 0

?

: cannot connect to X server localhost:10.0？？

When I run the demo_yolo3_deepsort program under linux, this error occurs: Can not connect to X server localhost: 10.0, I have commented out the cv2.imshow() function in the source code, or reported the same error, and what else needs to be modified?

Track IDs getting updated too quickly

Hi @ZQPei. Thanks for the great implementation of Deep Sort. However I had a concern, the track IDs are getting updated way too quickly in some videos, like the track IDs change from 28 to 38 to 51 in a few seconds. Could you please tell me how can I avoid this?

Enable more than person class?

Is it possible to detect and track more than one class? Specifically the Car class as well.
I had a glance at the relevant files but wasn't able to find where to change.
Would be glad if you can help. Thanks!

用Mars数据集训练的模型，怎么验证其accuracy?

Mars数据集中,Train的数据集和Test的数据集的标签不一样，怎么验证训练出来的模型在Test数据集上的Accuracy?

No residual connection

@ZQPei, I cannot figure out where you have applied residual connections in deep/model.py. Is there any different logic to this?

。

重新训练Re-ID模型

你好，我在尝试重新训练Re-ID模型，将Market-1501数据集放置于data/train下运行train.py报错。

Traceback (most recent call last):
File "train.py", line 43, in
torchvision.datasets.ImageFolder(train_dir, transform=transform_train),
File "/home/mxc-lab-ubuntu/anaconda3/envs/pytorch_041/lib/python3.5/site-packages/torchvision/datasets/folder.py", line 209, in init
target_transform=target_transform)
File "/home/mxc-lab-ubuntu/anaconda3/envs/pytorch_041/lib/python3.5/site-packages/torchvision/datasets/folder.py", line 87, in init
"Supported extensions are: " + ",".join(extensions)))
RuntimeError: Found 0 files in subfolders of: data/train
Supported extensions are: .jpg,.jpeg,.png,.ppm,.bmp,.pgm,.tif,.tiff,webp

A bug in feature extractor?

deep_sort_pytorch/deep/feature_extractor.py

Line 20 in 5dd84c0

img = img.astype(np.float)#/255.

CV2读取的图像数值范围是0-255
这里应该需要将数值缩放到0-1
因为后面做一个标准化时用的是0-1的范围？
请问下/255这个是不是不应该注释？？

How can I train deep sort using my own images?

Hello ZqPei,

Is it possible to finetune the Marketdataset? Or how can I use my own data for training DeepSort from scratch to generate checkpoints?

请问train代码是否使用 cosine metric learning

非常感谢您的工作，对我非常有帮助。
想详细的请教您一下，在这版代码中，是否支持原作者DeepSort的训练方式， cosine metric learning？这个度量学习方式是一种不错的训练方法，请问您这版代码训练过程中是使用的这个算法训练的吗？

How can I make the detecting and tracking faster?

Is there any way to make it more faster?

Evaluation for custom dataset

I am working on my custom dataset, the objects are license plates. I used YOLO for detection, can you tell me how to evaluate for such custom dataset, not Market 1501.

How to change the class name and display all the classes?

Hello ZQPei,
What changes do I have to make in your code if I want to display all the classes with their class name? I have 5 classes. E.g. human, tv on, tv off, fan on and fan off. I need to display their names along with their respective object id. In your code, only persons are displayed by bounding box and it is labelled as 'object 1', 'object 2' and so on. If need to display as ' human 1', 'human 2', 'fan on 1', 'fan on 2' etc..... How can I do that?

support cosine metric learning？

It seems that the train.py does not support cosine metric learning in deepsort, is there a plan to support that? thank you

Does increased accuracy result in imrpoved tracking performance?

Hello ZQPei,
Does the performance of tracking improve if the accuracy of classification model is increased?

ValueError: need at least one array to stack

I'm so sorry to ask you for help owing to the error I got during the study of your code. Can you help me with that,please? Thanks a lot
Traceback (most recent call last):
File "demo_yolo3_deepsort.py", line 68, in
det.detect()
File "demo_yolo3_deepsort.py", line 42, in detect
outputs = self.deepsort.update(bbox_xywh, cls_conf, im)
File "/home/gongyuan/Downloads/DEMO/deep_sort_pytorch-master/deep_sort.py", line 25, in update
features = self._get_features(bbox_xywh, ori_img)
File "/home/gongyuan/Downloads/DEMO/deep_sort_pytorch-master/deep_sort.py", line 66, in _get_features
features = np.stack(features, axis=0)
File "/home/gongyuan/anaconda3/envs/py3/lib/python3.6/site-packages/numpy/core/shape_base.py", line 349, in stack
raise ValueError('need at least one array to stack')
ValueError: need at least one array to stack

merge two checkpoints

Dear ZQPei,

Is it possible to implement tracking two different classes in the same video? I have separately trained the re-identification model for humans and vehicles so I got two separate checkpoints. How can I track both humans and vehicles in the same video?

Training on VeRi Dataset

How to use the train.py train the VeRi dataset?

Improve the speed for tracking

Can you please improve the speed for tracking?
Thank you

Error using test.py

Dear ZQPei,
When i try to test my model using test.py, the following error occurs:

can u please tell me how can i fix this?

bug in the format of bbox coordinate

Thanks for sharing the well-written code. I find a small bug that can be fixed easily.

1. The bbox output of YOLO is (x_c, y_c, w, h), where (x_c, y_c) is center point coordinate.

2. The initialization of a Detection object uses (x_tl, y_tl, w, h), where (x_tl, y_tl) is top-left coordinate,
   The mismatch of 1-2 is a bug here, which will cause inaccuracy in nms in and the later initialization of the track.

3. This two lines are inconsistent. the box is in (x_tl, y_tl, w, h), but the self._xywh_to_xyxy(box) in line 44 assume box is in (x_c, y_c, w, h). It will cause coordinate offset in visulization.

why *= 1.2 for yolo detection result?

Line 40 in demo_yolo3_deepsort.py: bbox_xywh[:,3] *= 1.2. Can I ask why multiply 1.2 for height of yolo detection result?

index out of bounds in drad_bbox() function

In util.py,
x1, y1, x2, y2 = [int(i) for i in box]
should convert to
x1, y1, x2, y2 = [int(j) for j in box] to solver the problem.

how to test the MOTA in MOT challenge

Cannot detect from my own weight file

Hello ZQPei,
When i run the demo with the available yolov3 original weight file and cfg file, it works perfectly. But when i use my custom trained weight file and cfg file, it doesnot detect anything. What might be the problem?

suggestion on skip detection for a speed-up?

Great work, thanks a lot for sharing the code!

Could you give some suggestions on how to modify the code if to skip detection for every n frames, say n=3?
The purpose is to save the detection time for every frame, which might not be needed.

It seems that the line
will effectively become the following:
self.tracker.update([])
will your deep_sort support this?

Thanks again!

Thanks.

请问我想训练车辆跟踪的模型、我应该怎么做呢？

你好、谢谢你提供的分享！我最近在学习车辆跟踪方面的内容、请问如果我想要用你的代码进行实现、需要训练和替换哪些东西呢？谢谢啦

您好，请问你的fps是多少？

你好！我用你的代码跑了个video，但是帧率只有大概6fps，效果看起来不是很好。我用的显卡是1080Ti。对了，请问可以使用yolov3-tiny的权重放入YOLO文件夹嘛？

new_covariance update correct??

hello, I was learning kalman filter. ref: https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/

At the bottom the blog, I saw its new_covariance update formula is different from yours.
yours is: new_covariance = cov - K * project_cov * K.T
in blog: new_covariance = cov - K * project_matrix*cov, where cov is P in the blog.

could you please explain it?

About the result on MOT16

Hi @ZQPei, Could you share the evaluation result of modified Deep SORT model on MOT16? Thanks.

ckpt.t7's Training Data

Which dataset that ckpt.t7(download from https://drive.google.com/drive/folders/1xhG0kRH1EX5B9_Iz8gQJb7UNnn_riXi6) is training on?

这个可以用来跟踪其他目标嘛？比如汽车等？

测试视频时跟踪框不能十分准确框出人

大神你好，我在测试时发现跟踪框总是和真实检测框有一点偏移，偏向前进方向一点.请问这是什么原因？我要将yolo改为maskrcnn，但是跟踪框和检测框不重合导致我无法进行下一步．

在追踪时候，出现很多跟踪框的问题

您好，我在跑您的demo时，发现当一个人站着动的时候，他的周围会出现很多跟踪框，而且id都不一样，我想问下我是哪里做的不对吗，谢谢！

是否有必要每帧都做检测？

我知道deepSort比Sort算法多了ReID的模型，在关联跟踪框和检测框时多了一个指标。但是每帧检测+ReID速度只能达到10FPS左右（RTX2080TI），想请问下可以实现隔帧做检测吗？代码中哪一部分需要做修改？

Usage: python demo_yolo3_deepsort.py [YOUR_VIDEO_PATH]

My video_path should where to change or add?