Cannot load the model TransformerLanguageModelPrompt.from_pretrained about biogpt

[Husseinfadhel]

you need to install fastBPE with g++ compiler:
https://github.com/glample/fastBPE.git
probably fastBPE doesn't work on windows.

[tushar20121907]

@Husseinfadhel does fastBPE work with WSL ???

[bmazeraski]

fastBPE works fine under WSL/Ubuntu, but additional dependencies are either missing or need to be generated using other scripts. For example, data/BC5CDR/relis-bin doesn't exist in the repo and I'm not sure how to create it. It would be nice if the authors would provide instructions for building everything that's needed in order to successfully run the examples.

[bmazeraski]

I'm running Pycharm on Windows 10 using a conda environment configured under WSL/Ubuntu and this example from the BioGPT github page works fine for me. I haven't been able to run the second example though.

import torch
from fairseq.models.transformer_lm import TransformerLanguageModel
m = TransformerLanguageModel.from_pretrained(
"checkpoints/Pre-trained-BioGPT",
"checkpoint.pt",
"data",
tokenizer='moses',
bpe='fastbpe',
bpe_codes="data/bpecodes",
min_len=100,
max_len_b=1024)
m.cuda()
src_tokens = m.encode("COVID-19 is")
generate = m.generate([src_tokens], beam=5)[0]
output = m.decode(generate[0]["tokens"])
print(output)

[panamantis]

It looks like the relis-bin is the raw training data and may need to be regenerated via preprocess.sh which rebuilds the source data files from train/test/validate. I'm stuck on the FASTBPE/fast requirement (linux only I believe)

[bmazeraski]

Yes... it appears that fastBPE needs to run under Linux. My attempts to build it under Windows were not successful. My workaround was to install the Windows Subsystem for Linux (WSL) on my Windows 10 box and then create a BioGPT virtual environment on WSL/Ubuntu. All of the BioGPT dependencies are installed in that environment. I develop all of my python code on Windows, but run the code under WSL/Ubuntu by simply setting the virtual environment as my active python interpreter.