Comments (7)
Closing this issue to cleanup the issue tracker. If anyone wishes to continue this discussion, however, feel free to either leave a comment here, reopen this issue, or create a new one.
Ideally, I'd like to eventually create a Wiki page that everyone can edit that contains references to PySceneDetect in literature, as well as references to other scene detection methodologies (if anyone would like to go for it, please feel free to do so!)
Thank you all!
from pyscenedetect.
Hello @leonwyang;
You may find the following papers useful:
https://ieeexplore.ieee.org/document/7040826/
https://ieeexplore.ieee.org/document/7079058/
You might also learn about some other algorithms which are more advanced by searching "scene change detection" on IEEEXplore. Currently there is a threshold (pixel value) based algorithm, content (changes in HSV colour space) detection algorithm, and motion-detection via background subtraction.
I am also investigating building a histogram instead of computing whole-frame delta scores, although the number of frequency bins would probably be small in order to have it not take up too much room in the statistics file, as well as have it easy for end users to determine optimal parameters.
Another big thing I want to look into is adaptive threshold values, or automatically detecting the optimal value by analyzing the peak values for a given video.
If you have any questions regarding your own application feel free to share them here and I will answer them to the best of my ability. Thank you for your interest.
from pyscenedetect.
Hello @Breakthrough
Thanks Brandon! I'll look into the literature you pointed to.
The type of video I am dealing with is surgery video, where basically the entire video consists of scenes that surgeons doing surgery inside human body, for example, in this video. The content based algorithm works pretty well for this type but makes false predictions when there is for example a reflection light came out from the equipment or there is a hand with white glove blocks part of the video.
It seems that the content based method does not work too well for gradual change, there are cases where
part of the surgery video is cut out of a whole procedure, but it has the gradual cut between who scenes that does not differ that much, and I suppose the dissolve detector maybe better in this case? Do you know if there is any scene detection algorithm that is robust to both abrupt and gradual change?
And It would be great if you can point to any package that can do dissolve scene change detection.
Thanks!
from pyscenedetect.
Hi @leonwyang;
One feature I have been looking into adding to PySceneDetect is a dissolve detector, hopefully once the v0.5 release is done I will have some more time to work on it. Other than these programs, I'm not really familiar with any other software that can accomplish what this does.
That being said, there is already an open issue (#35) regarding camera flashes. After reviewing the linked video, it appears that the reflection or white glove blocking issues may also be resolved once I determine an algorithm to deal with the flashing or momentary interruptions of videos.
That being said, there are some other parts of the video that aren't encompassed in the camera flash issue, so I will leave this issue open for further discussion/suggestions in regards to development of these features. I will also create a separate issue when development starts on the dissolve detector to track progress.
Thanks for providing some sample material to experiment with. If you yourself have any suggestions in regards to how some of these features should be implemented (esp. with respect to the algorithm choices), please feel free to share your opinion on the matter.
from pyscenedetect.
Hi Brandon,
Thanks for the detailed response regarding all the issues.
I am just wondering at this point do you have any algorithm for dissolve detector in mind(for example, any paper that may be helpful)? I could go look it up and implement it myself.
Thanks!
from pyscenedetect.
For dissolve, I was going to look at cases where in HSL colourspace, there is a consistent drop in luma (either linear or exp/log), and use that to signal a scene cut/break.
Another option would be to use histograms (see the notebook in #53) and detect a shift in the mean downwards, although I'm not sure if binning pixels is required for dissolve detection.
from pyscenedetect.
I've added a new page to the PySceneDetect site which aims to track uses of the project in literature, which will now allow tracking of proposed and better scene detection algorithms in the future:
https://pyscenedetect.readthedocs.io/en/latest/other/resources/
I have seen a few interesting papers comparing the baseline content-detection algorithm against other alternatives and new designs (usually NN-based, some hybrids) which are definitely worth considering going forwards. If anyone has any other pieces of literature they think would be relevant to be added to that page, please feel free to let me know.
Eventually I would also like that page to contain some of the sources contained in this issue as a way of providing some insight into the current detection algorithms as well as other common implementations.
from pyscenedetect.
Related Issues (20)
- Use csv as input for scene splitting HOT 1
- No output to list-scenes HOT 3
- horizontal wipe cuts
- ffmpeg segment_times options HOT 4
- QP file output HOT 6
- Can't get save_images to work HOT 1
- 'Error splitting video (ffmpeg returned 1) ' occurs while using 'split_video_ffmpeg' HOT 11
- Define minimum and maximum number of splits in split-video HOT 1
- Extend image_name_template to allow timestamp HOT 1
- Add Project Gallery to Website
- CITATION.cff file for citing repository HOT 1
- Return SyntaxWarning errors HOT 2
- Remove downscaling for perceptual hash
- Support histograms/hashes with AdaptiveDetector
- Group similar scenes together HOT 3
- The speed at which different backends run. HOT 2
- Two-Phase Command Architecture
- `min_scene_len` parameter isn't respected with AdaptiveDetector HOT 5
- Find start and end of black frame sequences HOT 1
- Get the progress in real time for scene_manager.detect_scenes HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyscenedetect.