Support for Live Video Stream (Non-Terminating) Processing Mode about pyscenedetect

[mhashim6]

I thought by this you wanted to limit the number of frames (or time) that a live stream is open for - or do you mean something else?

Aaah I get your idea now, no that's not what I meant. I meant a live-stream that's hypothetically going forever.

Thank you for the clarification!

I like the idea of a blocking generator, but couldn't this be accomplished by using the callback you added in the linked PR? My apologies if I'm mis-reading what you wrote, thanks for the feedback!

Exactly! I wrote this before implementing the feature with callbacks. Something about generators makes them more appealing even when they're not necessary:D

[Breakthrough]

Hey @mhashim6;

Alright, I will create a new issue to track that work. I need to figure out the underlying API changes to implement this cleanly - specifically, to support the future work of integrating motion detection (as opposed to just detecting scene changes, motion detection has explicit scene start/scene end times).

This may affect the API of the callback that you have proposed, or I will add an additional callback for event based detectors. I also need to figure out the best way to accomplish this for event-based detectors as well as threshold-based ones, but this is certainly possible.

Thank you for the suggestion, will try to have that implemented for v0.6! I'll try to merge your PR into the v0.6.x branch by the end of the weekend, and will ping you in the new issue I create to get your feedback on my API change proposals.

[Breakthrough]

This has been merged into the v0.6.x branch (thanks again @mhashim6), and will be included in the first release of PySceneDetect v0.6. Users wishing to use this feature can checkout the v0.6.x branch, and it can be used as follows:

cam = cv2.VideoCapture(0)
scene_manager.add_detector(ContentDetector())
scene_manager.detect_scenes(frame_source=cam, callback=lambda frame: print("a new scene: ", len(frame)))

Many thanks to @mhashim6 for the PR and feedback on the feature. Further breaking API changes are planned to support this feature in a more robust manner before release, so see #177 for additional upcoming changes, as well as to provide any feedback.

[Breakthrough]

Will be included in v0.5.5 release (cherry-picked into v0.5.5 branch as well).

[Breakthrough]

Giving this issue a bump, as the implementation of this feature will affect some parts of the upcoming API changes. Much of the original issue information is the same, with the exception that a specific SceneManager context will need to be initiated before performing live detection (with respect to the upcoming v0.5 API).

[Breakthrough]

This is now supported natively in v0.5 via the Python API by passing a cv2.VideoCapture object instead of a scenedetect.VideoManager object.

However, I'm not sure how to go about adding this feature to the command line - is there a use case for this, and if so, does anyone have any suggestions for how it should be implemented (e.g. what options/flags should be provided, what the output should be, and how to determine when to terminate? [can use the time command to do this, but other questions need investigation]).

[x-techlabs]

When I checked the documentation , they didn't say about the option for live. Kindly can you show me where this documentation is?

[Breakthrough]

Hi @x-techlabs;

This feature is only available from the Python API. Instead of using a VideoManager class, you can instead use a native OpenCV VideoCapture object. For example, see this page and replace the video_manager with an open VideoCapture (be sure to set the end time when detecting scenes, or it will run indefinitely).

For additional details, see the detect_scenes method documentation from the StatsManager.

Thanks!

[mhashim6]

@Breakthrough
How is it possible to set an end time for a live-stream?
Our use case is: We have a continuous camera feed from a raspberry pi, and we need to trigger an action when the scene change.
so perhaps something like a callback or a blocking python generator to iterate upon detected scenes would be ideal.

[Breakthrough]

Hi @mhashim6;

Sorry for taking so long to get back to you! You can accomplish this as follows:

vm = VideoManager([0])   # Open Device 0
time_zero = vm.get_base_timecode()
vm.set_duration(time_zero + 5.3)    # Limit to 5.3 seconds
# or alternatively: vm.set_duration(500)  # limit to 500 frames
vm.start()

Is this what you were asking for, or were you looking for a different functionality? My apologies again for the delay in responding to this comment!

[mhashim6]

Hi @mhashim6;

Sorry for taking so long to get back to you! You can accomplish this as follows:

Hi! no problem, the whole world is kind of frozen right now.

vm = VideoManager([0])   # Open Device 0
time_zero = vm.get_base_timecode()
vm.set_duration(time_zero + 5.3)    # Limit to 5.3 seconds
# or alternatively: vm.set_duration(500)  # limit to 500 frames
vm.start()

Is this what you were asking for, or were you looking for a different functionality? My apologies again for the delay in responding to this comment!

I hope this is not a dumb question, but doesn't that make the video finite?

[Breakthrough]

Hi @mhashim6;

Nope, not a dumb question - perhaps I misinterpreted your original question?

How is it possible to set an end time for a live-stream?

I thought by this you wanted to limit the number of frames (or time) that a live stream is open for - or do you mean something else?

Our use case is: We have a continuous camera feed from a raspberry pi, and we need to trigger an action when the scene change.
so perhaps something like a callback or a blocking python generator to iterate upon detected scenes would be ideal.

I like the idea of a blocking generator, but couldn't this be accomplished by using the callback you added in the linked PR? My apologies if I'm mis-reading what you wrote, thanks for the feedback!

[Breakthrough]

Hi @mhashim6;

After re-reading this, it may be useful for you to allowing detect_scenes() to return after it finds a new scene (rather than blocking until the video source has been exhausted). That would definitely be possible, I'm just not sure right now about how the logistics of that would work correctly for a live stream (as it may drop some frames unexpectedly), but it definitely seems possible. If you feel that's a nice feature to have, feel free to create a new issue to track that work so it can be added in the future. (Again, not sure about your exact use case, so if the callback method is sufficient, then perhaps I'll just suggest in the documentation that a generator could be created manually using the new callback)

Thanks again for your feedback!

[mhashim6]

Hi @mhashim6;

After re-reading this, do you instead mean something similar to allowing detect_scenes() to return after it finds a new scene (rather than blocking until the video source has been exhausted)? That would definitely be possible, and if that's the case, feel free to open a new issue/feature request!

Yep! And users can just iterate upon this function and it blocks until a new scene is detected.

[mhashim6]

This has been merged into the v0.6.x branch (thanks again @mhashim6), and will be included in the first release of PySceneDetect v0.6. Users wishing to use this feature can checkout the v0.6.x branch, and it can be used as follows:

cam = cv2.VideoCapture(0)

scene_manager.add_detector(ContentDetector())

scene_manager.detect_scenes(frame_source=cam, callback=lambda frame: print("a new scene: ", len(frame)))

Many thanks to @mhashim6 for the PR and feedback on the feature. Further breaking API changes are planned to support this feature in a more robust manner before release, so see #177 for additional upcoming changes, as well as to provide any feedback.

This is awesome! You're most welcome and I'm very happy to help.