instadeepai / tunbert Goto Github PK

Hello, Thank you for sharing and the models.

I was wondering if you can share the details about the pre-training data. Is it possible de share the data for pre-training?

Thank you in advance.

Hello! I'd love to use your model, but the links to the pre-trained weights (both PyTorch and TensorFlow) are showing the following error:

<Error>
  <Code>ProjectNotFound</Code>
  <Message>The requested project was not found.</Message>
  <Details>The requested project was not found.</Details>
</Error>

If you'd like, I can help you get the weights up on the Hugging Face Hub. We have documentation on how to do so, but I'm more than happy to help 😄

cc: @osanseviero

how test it after installation ?

After training the module using finetuning_sa_tdid.sh , then running predictions on the test.tsv, the output of the prediction which gets written in test_results.tsv I noticed that the test file and the test_results file don't have same number of rows.

for example the tunbert/dev-data/sentiment_analysis_tsac/test.tsv file that comes with your repo has 5 sentences/rows. When I run the predictions the tunbert/finetuning_tsac/test_results.tsv file outputs only 4 sentences/rows.

Update: After spending more time tinkering with the model, It truns out that I didn't spot a minor issue when I was dealing with headers. Sorry for the inconvenience and the false issue. Great work, I really enjoyed it ^^

While running the command python models/bert-nvidia/bert_finetuning_SA_DC.py --config-name "sentiment_analysis_config" model.language_model.lm_checkpoint="/path/to/checkpoints/PretrainingBERTFromText--end.ckpt" model.train_ds.file_path="/path/to/train.tsv" model.validation_ds.file_path="/path/to/valid.tsv" model.test_ds.file_path="/path/to/test.tsv" to fine-tune TunBERT-PyTorch on the SA task, I had the following error

I tried to install NeMo several times, but the error still persists.

Could you please provide me with the number of Steps/Epochs, and total run time taken to train your pytorch model? Also, do you have a published/pre-print paper by any chance? Thanks in advance.