VitTracker crush about opencv_zoo

[fengyuentau]

Thanks for reporting this issue. May I know what is the input resolution in your case?

…

________________________________ 发件人: Vardan ***@***.***> 发送时间: Wednesday, June 12, 2024 5:45:59 PM 收件人: opencv/opencv_zoo ***@***.***> 抄送: Subscribed ***@***.***> 主题: [opencv/opencv_zoo] VitTracker crush (Issue #261) Hi. I am encountering the problem when some number of pixels from video input are black. The error message is "error: (-4:Insufficient memory) Failed to allocate 39768204675(this number is variable) bytes in function 'OutOfMemoryError". I have tried to manually change the number of black pixels and depending on that the tracker crushes or works correctly. Error is occuring when calling model.infer() ― Reply to this email directly, view it on GitHub<#261>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AEDL63QYSHN3BRAIQHGPKSLZHAKFPAVCNFSM6AAAAABJF7VDWWVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM2DQMZRG43TQNA>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[saghavart]

Thank you for quick response. It's (640,480). I've also tried with (1280,720)

[saghavart]

To reproduce , you can set number of pixels to 0 manually like this

frame[:,:,:] = 122
frame[:-padd, :-padd, :] = 0

[fengyuentau]

Could you share the minimal reproducer?

[saghavart]

Sure. Just insert following 3 lines in line 108 of demo.py file

padd = 70
frame[:,:,:]=122
frame[:-padd, :-padd, :] = 0

and run demo.py file

[fengyuentau]

I tried with these three lines on my m1 macbook. Things are it did not crash but went very slow with your three lines of code. Also tried commented out frame[:,:,:]=122, it went smooth again. I guess some internal computation has problem with a massive area of pixel value 122. So the problem isn't with black pixels.

[saghavart]

Maybe this can help to reproduce. Remove those three lines, start camera, select an object and then close the camera with your hand or something black for 2-3 seconds. In my cases it writes target lost and then crushes

[fengyuentau]

Okay, it can be reproduced on my side. Let me look into the issue tomorrow.

[fengyuentau]

Okay, the problem is confirmed. Problem is when the target is lost, the bounding box gets larger and larger (the wrong bounding box pattern is recorded regardless of score) and the size of roi grows until out-of-memory error. See crop_image and update:

https://github.com/opencv/opencv/blob/a03b81316782ae30038b288fd3568993fa1e3538/modules/video/src/tracking/tracker_vit.cpp#L75-L93

https://github.com/opencv/opencv/blob/a03b81316782ae30038b288fd3568993fa1e3538/modules/video/src/tracking/tracker_vit.cpp#L199-L203

[fengyuentau]

I will submit a PR to fix this problem.

[fengyuentau]

@saghavart I submitted a PR to fix the problem hopefully. Feel free to review and test it opencv/opencv#25760.

[fengyuentau]

Hello @saghavart , my patch to fix the problem has been merged. Feel free to have a try on your side. Closing this issue for now.