fursovia / paraphrase_tutorial Goto Github PK

When I run the function ParaphraseDataset, got error (module 'Torchtext.vocab' has no attribute '_default_unk_index'). I am using Torchtext 0.4.0 and torch 1.2.0.

My understanding is that Torchtext version 0.4.0 the unk token isn’t hard coded, so I made a modification to the function as:
_#self.word2idx = defaultdict(torchtext.vocab.default_unk_index)
#self.word2idx.update(dict(self.text_field1.vocab.stoi))
self.word2idx = dict(self.text_field1.vocab.stoi)
self.word2idx.update(dict(self.text_field1.vocab.stoi))

I got this step run and got Vocabulary size = 10176. But when I ran the next step:
batch = next(iter(train_iter))

for batch in train_iter:
print(batch)
break_

I got error as (KeyError: 'bharath'), and this word 'bharath' changes every time I run.

Thank you!