bpyap / bert-wsd Goto Github PK

When trying to run the demo, I've run into three issues.

Firstly, when I run demo_model.py with any pre-trained model (in this case, bert_large-batch_size=128...), I get the following warning:

Some weights of the model checkpoint at DIR were not used when initializing BertWSD: ['similarity_loss_factor', 'ranking_loss_factor', 'similarity_linear.weight', 'similarity_linear.bias']

• This IS expected if you are initializing BertWSD from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
• This IS NOT expected if you are initializing BertWSD from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
  Special tokens have been added in the vocabulary, make sure the associated word embedding are fine-tuned or trained.

Then, when I run it with my CPU using the test sentence "He caught a [TGT] bass [TGT] yesterday.", it throws an error:

return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self

Finally, if I run it with my GPU (2080 Ti), it throws a ton of errors reading " Assertion srcIndex < srcSelectDimSize failed." It ends with this error: "RuntimeError: CUDA error: CUBLAS_STATUS_NOT_INITIALIZED when calling cublasCreate(handle)"

Was wondering if you had any fixes to get it working. Thanks a ton!

Edit: Running just "bert-large-uncased" worked. Is there a way to still use the pre-trained models included?

demo_model.py works well with transformers v2.3.0. However when I try a more recent version, I get the embedding error "IndexError: index out of range in self".

I localized the error to the value of tokenizer.additional_special_tokens_ids, which is [100] in v2.3.0 and [30523] in v4.2.2. If I replace all occurrences of 30523 with 100, I am able to reproduce the results. But this is just a dirty fix. Is there any elegant solution? Am I missing anything with respect to the tokenizer?

Sir, can you explain what should i do to implement WSD using BERT for any other language?

Hi,

I'm trying to use BERT-Base, Multilingual Cased from google-research

But it's taking very long to load. I suspect an infinite loop.

The files in the model folder have different names than the models available from BERT-WSD.

How can I make this other model compatible? I need multilingual support.

How do I use models that aren't in the specified list?

I would like to use this model:

https://huggingface.co/dccuchile/bert-base-spanish-wwm-uncased

How do I go about doing this?

Regards,

Jamie