majdoddin / nlp Goto Github PK

I really like your work. For the sake of readability it would be great if the text appears in blocks until the speaker changes. The link to each individual timestamp could be hidden under the actual text. In principle the same thing as otter.ai does.

It could look like this:

`

    }
	a {
		color: black;
	}
	a:link { 
		color: white;
		text-decoration: none; 
	}
	a:visited {
		text-decoration: none;
	}
	a:hover {
		color: white;
		background-color: #e8347d;
		text-decoration: none;
	}
	a:active {
		text-decoration: none; 
		}
    .l {
      color: #050;
    }
    .s {
        display: inline-block;
    }
    .e {
        display: inline-block;
    }
    .t {
        display: inline-block;
    }
	.text_block {
		width: 60%;
	}
	.name_1 {
		font-size: 23px;
		color: #fa4236;
		font-weight: bold;
	}
	.name_2 {
		font-size: 23px;
		color: #feea00;
		font-weight: bold;
	}
    #player {
	position: sticky;
	top: 20px;
	float: right;
}
</style>

Yann LeCun: Dark Matter of Intelligence and Self-Supervised Learning | Lex Fridman Podcast #258

		`

First off fantastic work. Thanks a lot for creating this. It worked much better than what i had previously tried and i have already used it quite a lot for transcribing podcasts.

It seems small parts of the Whisper-generated transcript get lost when matching transcription and diarization when i tried using it with other youtube-videos. So whisper correctly transcribes the full video, but then in the matching-phase small chunks aren't included in the final transcript.

E.g. the whisper-transcript reads:

"should just rein in their lawyers because they're I'm
sure racking up a fortune in legal fees.
And they're trying to.
It sounds like.
Sachs, the case is that they're trying to make"

The final output for the same spot reads

link | 00:07:3.61 [Lex] should just rein in their lawyers because they're I'm
link | 00:07:9.88 [Lex] It sounds like.
link | 00:07:9.99 [Lex] Sachs, the case is that they're trying to make

-> "sure racking up a fortune in legal fees. And they're trying to." gets lost/is missing.

(It isn't limited to this youtube-video/the specific spot as i tried it with several podcast and happened multiple times in all podcasts. This episode is just the first one i found below one hour of the podcast/randomly chosen.)

The only code-changes i made was changing the youtube-link and changing the audio-file cut-length from 20 to 50 minutes.
("#!yt-dlp -xv --ffmpeg-location ffmpeg-master-latest-linux64-gpl/bin --audio-format wav -o lecun.wav -- https://www.youtube.com/watch?v=GojTj91eLho" was changed to "!yt-dlp -xv --ffmpeg-location ffmpeg-master-latest-linux64-gpl/bin --audio-format wav -o lecun.wav -- https://www.youtube.com/watch?v=jlK5tsUuEP0" and "t2 = 20 * 60 * 1000" to "t2 = 50 * 60 * 1000", so basically no important code-changes.
I have also uploaded the full-code to this Google Colab[https://colab.research.google.com/drive/1BwST1H7sfvgAZ53KNL5QufoEMnPHelLk?usp=sharing], but as i said its barely changed.)

Do you know why this issue arises or/and how it might be fixable?

Thanks in advance for any help and in any case thanks for creating this.

https://github.com/Majdoddin/nlp/blob/main/Pyannote_plays_and_Whisper_rhymes_v_2_0.ipynb?short_path=22ffcf3#L2237
Here should be 06.3f

Hi, I'm trying to use the v2 Colab notebook but whenever I get to this section:

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
pipeline.to(device)

I get the following error:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
[<ipython-input-22-eabb39154ab5>](https://localhost:8080/#) in <cell line: 1>()
----> 1 device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
      2 pipeline.to(device)

NameError: name 'torch' is not defined

Not sure what I'm doing wrong. Any ideas? I believe I followed through all previous steps correctly.

Hello team,

Great project, Is there a possibility to have a real-time processing ?