ashtraysoap / transformer-lm

transformer language model based on OpenAI's GPTs

Transformer language model for text generation

Code is heavily based on OpenAI's gpt-2 code samples and their original transformer language model

The initial codebase lacks code for training and the latter is oriented towards finetuning the model for supervised tasks. This repository aims to provide an implementation allowing straightforward text generation from the model.

Requirements

• numpy
• tensorflow>=1.12.0

Training

python3 train.py path/to/text/corpus

Sampling

python3 sample.py path/to/model/checkpoint path/to/vocab path/to/hyperparams

transformer-lm's Issues

use as lanuage model

how i can use is as a lanuage model to compute a sentence's ppl ,just as
ppl(i have a apple)=0.07
ppl(i a applle)=0.005
thanks