fourms / mgt-python Goto Github PK

I get an error message when using crop='Auto' in motionvideo. What works is crop='auto'. Would it be better to use title case for all (like for color) or switch everything to lowercase?

Does it work on Windows and Ubuntu (both local machine and server)?

This is important as we want to move more and more functions over the ffmpeg "backend". Important aspects of this should be:

1. rendering progress should be piped back to display progress with MgProgressbar
2. subprocesses should be closed properly not to have hanging ffmpeg instances + unreleased video files
3. should generally work with any ffmpeg command

These are now implemented and will be pushed to the main branch in the next commit.

Any other file path than in the default directory raises an exception

Currently it says "Frame" in any case. It could show the file name it draws and maybe a hint how to close the window. That is another thing, currently it listens for the Q key, I think it is more intuitive to change it to Esc or Space. It can also be either of these (so we don't break "compatibility").

The motion history videos become quite dark at the moment:

if you compare to the motion video of the same example:

I think it would be good to have some kind of weighting of the frames (or normalization). I experimented with weighting when creating some (non-motion) history videos using FFmpeg. It could be done like a that, or perhaps even better would be to add the "tale" to the current frame. That may make the present frame stick out even more, yet still keep the motion history visible.

An explanation of how-to here: https://dzone.com/articles/executable-package-pip-install

I just tried importing a long (2h) video file, in which I wanted to trim out only 1 minute in the middle. It looks like MgObject converts the entire thing to AVI before doing the trimming. That takes a lot of time!

I think it would be better to trim first, and then convert. Would that cause problems for other objects?

It would be nice to have an option to choose whether you want white-on-black (normal), or black-on-white (inverted) motion videos and motiongrams coming out mgmotion. Now it returns inverted, which is nice, but sometimes the other is preferable.

It would be nice to have modules of the functions average and history as well, so that it is easy to run them on the mg object. Are there any problems with making this?

All subfolders, videos, readme.

If the user only provides starttime the endtime remains the default 0, which results in an empty video. Instead if endtime is smaller than starttime the function should fall back to using the end of the video as endtime.
(The problem doesn't occur when the user only provides endtime, since the default starttime is 0.)

Is there any normalization happening in the motion video. I see that the motiongrams are quite dark:

There is more information there, here doing a normalization of the image:

Perhaps we should add normalization as an option?

One thing is that in colab we need to fetch 'dance.avi' from the module specifically with like
dance_path = os.path.join(os.path.split(musicalgestures.__file__)[0], "dance.avi").
But the more annoying thing is that cv2.imshow is disabled in colab, and one has to use their own substitute, cv2_imshow. So whenever we are in colab mode, mg_show should use that when playing back videos. I imagine something like this:
musicalgestures.MgObject(dance_path).show(in_colab=True)

It would be really nice to have timecode in the first column of the TSV files. My previous experience is that timecodes vary dependent on the video format being used. Is it possible to get out timecode in milliseconds?

This is more of a "retrospective"/"note-to-self" issue, while reimplementing history, motion, and working on a safe, non-hanging ffmpeg_cmd which can pipe it's progess, I noticed that files were hanging after they went through the videoreader, motion (the cv2-based one), history, get_length and has_audio... These should all be fixed in the next commit.

When I am trying to run the setup file, I get the following problem:

$ python setup.py 
Traceback (most recent call last):
  File "setup.py", line 15, in <module>
    with open(os.path.join(_here, 'motionvideo', 'version.py')) as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/alexanje/github/MGT-python/motionvideo/version.py'

What is wrong?

Difficult for newbies to get things to run at all.

And change all references in notebooks, examples, etc.

Now that the package is on pip, this should be easy.

Since we decided to support the newer chaining workflow, we retire motionhistory(), since it is equivalent to motion().history(). It will be moved to a new Deprecated folder, which we include in the repo but don't package for PyPi. (So it's there in case we change our minds.)

Just noticed this while testing. Most of the time, the progress bars behave as they should, and they don't add much overhead. But sometimes, (notably the one which is for mg_skip_frames) the progress bar is actually harder on the cpu then the video process itself - at least on my old laptop. I think the problem is caused by calling sys.stdout.flush() (which is a bit expensive, but in turn -for some reason...- is much more reliable then printing inplace with print(end='\r')) too quickly too often which introduces a lot of backlogging. Especially when the video processing itself would not take much time this can result in situations when the resulting output files are finished minutes earlier than the (super-backlogged and lagging) progressbar shows.
On the other hand it is nice to see fast and responsive progress bars if they are affordable. So as a first attempt on the solution I will implement a time limit on the progress bar, so it will skip printing if the last print was issued less than let's say 30 ms ago.

I am testing some video files without audio (timelapses), and then i get into trouble:

FileNotFoundError: [Errno 2] No such file or directory: '../original/GH010095_motion_w_audio.avi' -> '../original/GH010095_motion.avi'

Looks like it assumes that there is an audio stream in the file.

I often record standing people with a rotated camera, so it would be very nice to have the opportunity to rotate the video (90,180,270 degrees). Either as a separate function, or as part of mg_motionvideo?

When running

mg.MgObject('dance.avi', skip=6).motion(inverted_motiongram=True, inverted_motionvideo=True).average().show()

I get a colored inverted motiongram, but a grayscale inverted average video. Strange...

It would be nice to create videograms in addition to motiongrams. That is, to run the same matrix averaging but on the original video file instead of motion video.

It would be very nice to add support for optical flow. This could be done either within the mgmotion module, or, perhaps better, as a separate module. It should be possible to generate a motion video with optical flow arrows, and possibly also use this for creating a motiongram.

When trying to run motionhistory on a greyscale video, I get the following error:

Traceback (most recent call last):
  File "test_mg.py", line 18, in <module>
    mg.mg_motionhistory()
  File "../../mgmodule/_motionhistory.py", line 41, in mg_motionhistory
    frame = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
cv2.error: OpenCV(4.1.0) /io/opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

I don't remember this being a problem before, but now suddenly it's happening consistently: When I load an MgObject with a video, trim it and set a skip value, I get an error when the skipping is done, and the _videoreader.py tries to remove the previous _trim.avi:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'dance2_trim.avi'

I could reproduce this 3 times in a row. It is really curious, since the notebook is full of similar lines, and I never got this problem before. Calling vidcap.release() on the input vidcap of mg_skip_frames() before I populate the vidcap varable with the resulting video does not help.

It would be nice to export a TSV file with quantitative data coming out of mgmotion, similar to what is done in MGT for Matlab.

It is a bit redundant, but I guess it is useful for people who don't install via pip, just download the package from this repo.

name MotionVideo is not an attribute of mg object - it is mg_motionvideo
Even this name is clunky imo, maybe we can agree upon a naming strategy, maybe avoid underscore, and add camelcase?

On Ubuntu (19.04): I get this error after rendering motionvideo:

Rendering motionvideo 100% 
QApplication: invalid style override passed, ignoring it.
Available styles: Windows, Fusion

What is this?

Testing out the notebook and getting an issues in cell 6:

  mg = mgmodule.MgObject('dance.avi', starttime = 5, endtime = 15)
  Moviepy - Running:      >>> "+ " ".join(cmd)      Moviepy - Command successful
  ---------------------------------------------------------------------------         NameError                                 Traceback (most recent call last)      <ipython-input-6-b65d648662ba> in <module>      ----> 1 mg = mgmodule.MgObject('dance.avi', starttime = 5, endtime = 15)
  ~/github/MGT-python/mgmodule/__init__.py in __init__(self, filename, method, filtertype, thresh, starttime, endtime, blur, skip, color, contrast, brightness, crop)
      38         self.crop = crop
      39         self.test_input()         ---> 40         self.get_video()
      41 
      42 
  ~/github/MGT-python/mgmodule/__init__.py in get_video(self)
      52     def get_video(self):
      53         """ Creates a video attribute to the Musical Gestures object with the given correct settings. """         ---> 54         self.video, self.length, self.width, self.height, self.fps, self.endtime, self.of = mg_videoreader(self.filename, self.starttime, self.endtime, self.skip, self.contrast, self.brightness, self.crop)
  ~/github/MGT-python/mgmodule/_videoreader.py in mg_videoreader(filename, starttime, endtime, skip, contrast, brightness, crop)
      31         trimvideo = ffmpeg_extract_subclip(filename, starttime, endtime, targetname= of +'_trim' + fex)
      32         of = of + '_trim'         ---> 33         vidcap = cv2.VideoCapture(of+fx)
      34 
      35     # Or just use whole video
  NameError: name 'fx' is not defined

Looks like several issues at once?

There are some functions which were using ffmpeg previously, but via the more naive os.system() command. They should be updated to use the new ffmpeg_cmd() which can report progress (could be useful when dealing with long videos).
These functions are:

• convert_to_avi
• extract_subclip
• rotate_video
• convert_to_grayscale

I see that MgObject converts the audio to MP3 when converting to AVI. Many MP4 files are using AAC for audio, so I wonder whether it would be better to just pass on the AAC, when possible. I guess the AVI container will be able to handle a combination of MJPEG and AAC?

It would be nice to have an option to keep audio for converted motion videos. To get this it is probably necessary to either

1. export the audio from the input video and then add it back in for the converted video.
2. use ffmpeg to copy audio from source video to output video after conversion.

It would be cool to add a skeleton extraction module, for example, based on Posenet.

It is based on the previous version and a lot of things have changed since then.

There is no message being shown when doing the contrast/brightness-adjustments. So if this happens first, the use wonders what is going on. Would be nice to have a status bar of the progression also for this.

It would be nice to have a normalization function, or be able to adjust the contrast in average_image. This is particularly useful when running it on motion videos, in which there is not so much content.

The methods described online don't seem to work, so as of now, there is a hacky solution in setup.py which does not prevent installation and also display some description on PyPi. But it would be nice to figure out how to just link the README.md correctly...

Need to create wiki page with detailed documentation of usage. Could copy some of the stuff from the Matlab wiki.

It would be nice to be able to choose between "samples" and "seconds" as the unit in the QoM plots coming out of mg_motion.

...but shows the correct percentage, and thinks that self.finished = True. So all is good, this is just a small display issue. Must be a rounding error somewhere.