AH01215: OSError: Couldn't reach server at 'https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json' to download pretrained model configuration file about lm-scorer

[simonepri]

It seems to work in the terminal.

What do you mean?

Can you please explain step by step what are you doing and how to reproduce the problem?
If the problem only happen inside Vegrant can you provide a minimum Vegrantfile that generates the problem?

Thanks

[MSMOON]

Here is what I have just to see if I can get a language model(gpt2) to load.
I'm getting:

OSError: Couldn't reach server at 'https://s3.amazonaws.com/models.huggingface.co/bert/gpt2-config.json' to download pretrained model configuration file.

https://drive.google.com/file/d/1gRvVE13UhGBnvUZCsiiip5gGluGdMRnw/view?usp=sharing

[simonepri]

The link you shared is private.
Anyway it is likely to be a misconfiguration on your machine rather than a problem with this library.
What the errors is telling you is that the machine is unable to download the model from the server.

[BramVanroy]

It is for exactly this reason that I added the local_files_only directive. This avoids our going connections and only looks for the local files (in cache or in the specified location). It seems that the LMScorer currently doesn't pass kwargs down to the actual 'from_pretrained' method, hence the following would not work.

scorer = LMScorer.from_pretrained("gpt2", local_files_only=True)

It might be helpful if options were passed down to the actual from_pretrained method.

[MSMOON]

To you both, thanks for your help!
Currently this is my situation. I am using GPT2 with local files successfully.

However, this line..
scorer = LMScorer.from_pretrained("gpt2", device=device, batch_size=batch_size)
is failing because it still wants to access s3.amazonaws.com and is looking for the model names (["gpt2", "gpt2-medium", "gpt2-large", "gpt2-xl", distilgpt2"])

ValueError: Unrecognized model name.Can be one of: gpt2, gpt2-medium, gpt2-large, gpt2-xl, distilgpt2

However, I need it to recognize my gpt2 local files. Is there a way to use lm-scorer by using gpt2 local files?

My apologies. The link should work now. https://drive.google.com/file/d/1UEybSOF88vwcVVSYgut7WIOo2l4iprk0/view?usp=sharing

[MSMOON]

I was able to finally able to figure out my problem with loading gpt2 and other python modules by changing the envvars file described here: https://stackoverflow.com/questions/3783887/why-cant-python-find-some-modules-when-im-running-cgi-scripts-from-the-web