Comments (9)
Hi @feiwu77777,
I think there's now a better solution to your problem. You can run CoTracker in online mode. This way, you don't have to keep the whole video in memory, only the current sliding window and estimated tracks. Please see https://github.com/facebookresearch/co-tracker?tab=readme-ov-file#online-mode and check out the online demo.
Let me know if this helps!
from co-tracker.
Hi @feiwu77777, I think there's now a better solution to your problem. You can run CoTracker in online mode. This way, you don't have to keep the whole video in memory, only the current sliding window and estimated tracks. Please see https://github.com/facebookresearch/co-tracker?tab=readme-ov-file#online-mode and check out the online demo. Let me know if this helps!
Thank you, that works for me!
from co-tracker.
Hi,
Thank you so much for sharing your interesting work!
At present, I am trying to understand about online_demo code.
In particular, the variable "window_frames" will be appended continuously as long as new frames are fed.
Therefore, if I need to track points for several hours/days, "window_frames" will be very huge and might cause Out-of-memory.
In this case, is it possible if I discard previous frames and keep the number of frames in "window_frames" to be fixed,
and the tracking accuracy is still OK?
from co-tracker.
Hi @pvtoan, this variable exists for one reason - to make it possible to visualize the output using the existing tools in this repository. You can safely discard the previous frames.
from co-tracker.
Hi @nikitakaraevv,
Thank you for your clear answer.
Btw, when I already "discarded" previous frames of variable "window_frames" (it means the input of function "CoTrackerOnlinePredictor" has a fixed size)
However, the output including "pred_tracks" and "pred_visibility" are still appended continuously.
-
Could you please let me know which commands cause appending continuously for these two variables?
Also, is it safe too if I discard previous frames for these two variables? -
Next, could you please roughly explain about the meaning of the below command when returning values for these two variables in the last command of function "CoTrackerOnlinePredictor" (I just rewrite them for an easier view)?
pred_tracks = tracks * tracks.new_tensor([(W - 1) / (self.interp_shape[1] - 1), (H - 1) / (self.interp_shape[0] - 1),])
from co-tracker.
Hi @pvtoan, yes, this is done for the same reason, and we should probably fix it.
- Here we continuously update tracks an visibilities inside the model in the online mode:
The fix is to do something like:
self.online_coords_predicted = coords_predicted[:, ind : ind + S]
self.online_vis_predicted = vis_predicted[:, ind : ind + S]
and make sure it doesn't break anything.
- We resize each input video to the model resolution, and tracks are predicted in this resolution. This line transforms the predictions back to the original resolution.
from co-tracker.
Hi @nikitakaraevv ,
I tried to update with the two commands in "cotracker.py", as you showed
self.online_coords_predicted = coords_predicted[:, ind : ind + S]
self.online_vis_predicted = vis_predicted[:, ind : ind + S]
However, I got the following errors
Traceback (most recent call last):
File "./CoTracker/codes/cotracker/testcode.py", line 207, in
pred_tracks, pred_visibility = _process_step_points(window_frames, is_first_step, queries = queries)
File "./CoTracker/codes/cotracker/testcode.py", line 165, in _process_step_points
return cotracker(video_chunk, is_first_step=is_first_step, queries=queries[None], add_support_grid=True)
File "./lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "./lib/python3.9/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "./CoTracker/codes/cotracker/cotracker/predictor.py", line 256, in forward
tracks, visibilities, __ = self.model(video=video_chunk, queries=self.queries, iters=6, is_online=True,)
File "./lib/python3.9/site-packages/torch/nn/modules/module.py", line 1501, in _call_impl
return forward_call(*args, **kwargs)
File "./CoTracker/codes/cotracker/cotracker/models/core/cotracker/cotracker.py", line 335, in forward
coords_predicted[:, ind : ind + S] = coords[-1][:, :S_trimmed]
RuntimeError: The expanded size of the tensor (4) must match the existing size (8) at non-singleton dimension 1. Target sizes: [1, 4, 63, 2]. Tensor sizes: [8, 63, 2]
Could you please help me fix this issue?
from co-tracker.
Hi @nikitakaraevv ,
If you have time, please take a look at my issue.
Thank you for your help!
from co-tracker.
Hi @pvtoan, this will take some time, I'll try to take a look over the weekend!
from co-tracker.
Related Issues (20)
- add co-track to hf hub HOT 1
- tracking grid_points frame by frame.
- tracking query ponts not correctly HOT 1
- how to track a specified x-y point? HOT 3
- Add points as they get visible throughout the video HOT 5
- Sample occluded points during training HOT 3
- where can I find the confidence value of prediction? HOT 1
- huggingface space down?
- ValueError: mutable default <class 'hydra.conf.JobConf.JobConfig.OverrideDirname'> for field override_dirname is not allowed: use default_factory HOT 1
- About reproducing the paper HOT 2
- Why is it not tracking correctly? The track points are shaking abnormally HOT 1
- trouble accessing pixel colors of tracked locations HOT 1
- It seems that the model can not track the balls after they change the direction. HOT 3
- The huggingface demo doesn't work.
- Training batch size HOT 1
- Questions about loss computation HOT 3
- Point tracking with different frame initializations HOT 2
- Evaluate on custom dataset HOT 1
- Significant Increase in Loss when Adjusting traj_per_sample Parameter HOT 2
- TypeError when tracing model by TorchScript HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from co-tracker.