Comments (7)
In latest version of pyannote.audio (v1.0), pyannote.audio.labeling.tasks.SpeechActivityDetection
has a overlap
boolean parameter that can be used to train SAD with 3 classes (non-speech vs. one speaker vs. 2+ speakers) instead of 2 (non-speech vs. speech).
from pyannote-audio.
Hi there! Thank you for opensourcing this project 👍 .
I want to detect overlapping speech regions using pyAnnote.Audio version 1.0.1.
I trained a model with speech activity detection and enabled overlap detection (as you proposed in the previous comment). Then I applied the model to AMI corpus.
Now it's unclear to me how/where I can see the detected overlapping speech regions.
When I load my precomputed SAD-scores, I would normally binarize these SAD-scores to obtain speech regions, but where can I see regions with amount of detected speakers > 1 ?
Best wishes
from pyannote-audio.
With overlap=True
, SAD scores are now 3-dimensional:
- 1st dimension is for non-speech
- 2nd dimension is for exactly one speaker
- 3rd dimension is for 2 or more speakers
Therefore, you should binarize the 3rd dimension, ie use dimension=2
in
pyannote-audio/pyannote/audio/signal.py
Line 161 in b678920
FYI, I never really used this overlap detection so I'd be curious to receive some feedback on how well it performs...
from pyannote-audio.
Hi,
I am also working on the problem of overlap detection for AMI. For training, I followed your tutorial, just adding overlap: True parameter to config.yml
, and I got a training loss curve like this (comparison b/w overlap and non-overlap)
However, I am not sure how to evaluate this model using pyannote-metrics
after binarizing the 3rd dimension.
from pyannote-audio.
You can take inspiration for pyannote-overlap-detection
validation code or use the script directly
$ pyannote-overlap-detection --help
It will compute recall at fixed precision.
from pyannote-audio.
Thanks a lot!
from pyannote-audio.
Closing this stalled issue. Please re-open if needed.
See https://arxiv.org/abs/1910.11646 for more up-to-date overlapped speech detection results (obtained with pyannote.audio).
from pyannote-audio.
Related Issues (20)
- RuntimeError with Mixed CUDA Devices for Multi-GPU Training HOT 1
- pyannote uninstalling torch cuda and install torch cpu HOT 8
- 'speechbrain' must be installed to use 'speechbrain/spkrec-ecapa-voxceleb' embeddings. HOT 3
- Error occurs following training tutorial HOT 1
- Speaker Diarizations get_segmentations() raises for several input type variants
- Speaker Diarization pipeline.get_segmentations produces integer ascending start/ends instead of something useful HOT 1
- Embeddings takes 3x the length of the audio length HOT 7
- numpy.ndarray audio input doesn't work? HOT 3
- Audio input as tensor or BytesIO is unexpectedly slow HOT 1
- Cannot instantiate parameters on finetuned model HOT 5
- Can not reproduce "adapting_pretrained_pipeline.ipynb" on local machine HOT 4
- What is the purpose of the Resegmentation and AdaptiveVoiceActivityDetection Pipeline? HOT 1
- Why is pyannote not using my GPU ro CPU? So slow too. HOT 5
- Wrong usage of meta-protocols subsets in segmentation tasks HOT 1
- Speakers with similar pitch are difficult to distinguish HOT 2
- DER above zero when using Oracle Segmentation & Oracle Clustering HOT 2
- How to map the transcribed text with their respective speakers in speaker diarization? HOT 2
- `torchaudio.info.num_frames` can give wrong results so it can provide false exceptions HOT 2
- 3.3 dependencies HOT 2
- outputs of separation module is clipping HOT 6
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyannote-audio.