r-tensorflow / gpt2

R Interface to OpenAI's GPT-2 model

r-tensorflow website

This is a blogdown website and can be previewed locally with

blogdown::serve_site()

It relies on Hugo v0.58.3.

Updating the API reference:

The API reference can be built with:

Rscript /scripts/api.R

Updating the guides

You can update the Guide section with:

Rscript /scripts/guide.R

gpt2's Issues

Error in board_bin_store

When running:

reticulate::use_condaenv(required = TRUE, condaenv = "r-gpt2")
gpt2::gpt2(model = "124M")

I get

Error in board_pin_store(board, x, name, description, "files", list(extension = extension),  : 
  Client error: (404) Not Found. Failed to download remote file: https://storage.googleapis.com/gpt-2/models/124M/checkpoint

Following that link I get an XML doc that says

<Error>
<Code>NoSuchBucket</Code>
<Message>The specified bucket does not exist.</Message>
</Error>

Not sure, but from here it looks like maybe they moved the models to a new hosting?

Installation guide?

remotes::install_github("r-tensorflow/gpt2")
Downloading GitHub repo r-tensorflow/gpt2@master
These packages have more recent versions available.
Which would you like to update?

1: All
2: CRAN packages only
3: None
4: reticulate (1.12 -> 1.14 ) [CRAN]
5: mime (0.6 -> 0.8 ) [CRAN]
6: openssl (1.3 -> 1.4.1) [CRAN]
7: zip (2.0.1 -> 2.0.4) [CRAN]
8: Rcpp (1.0.1 -> 1.0.3) [CRAN]
9: processx (3.3.0 -> 3.4.1) [CRAN]
10: curl (3.3 -> 4.3 ) [CRAN]
11: sys (3.2 -> 3.3 ) [CRAN]
12: rlang (0.3.4 -> 0.4.2) [CRAN]
13: purrr (0.3.2 -> 0.3.3) [CRAN]

Enter one or more numbers, or an empty line to skip updates:

Installing 5 packages: pins, tensorflow, httr, R6, whisker
Installing packages into ‘C:/Users/HOME/Documents/R/win-library/3.4’
(as ‘lib’ is unspecified)

There are binary versions available but the source versions are later:
binary source needs_compilation
tensorflow 1.13.1 2.0.0 FALSE
httr 1.4.0 1.4.1 FALSE
R6 2.4.0 2.4.1 FALSE
whisker 0.3-2 0.4 FALSE

installing the source packages ‘pins’, ‘tensorflow’, ‘httr’, ‘R6’, ‘whisker’

trying URL 'https://cran.rstudio.com/src/contrib/pins_0.3.0.tar.gz'
Content type 'application/x-gzip' length 1879459 bytes (1.8 MB)
downloaded 1.8 MB

trying URL 'https://cran.rstudio.com/src/contrib/tensorflow_2.0.0.tar.gz'
Content type 'application/x-gzip' length 48236 bytes (47 KB)
downloaded 47 KB

trying URL 'https://cran.rstudio.com/src/contrib/httr_1.4.1.tar.gz'
Content type 'application/x-gzip' length 158465 bytes (154 KB)
downloaded 154 KB

trying URL 'https://cran.rstudio.com/src/contrib/R6_2.4.1.tar.gz'
Content type 'application/x-gzip' length 35536 bytes (34 KB)
downloaded 34 KB

trying URL 'https://cran.rstudio.com/src/contrib/whisker_0.4.tar.gz'
Content type 'application/x-gzip' length 28574 bytes (27 KB)
downloaded 27 KB

• installing source package 'tensorflow' ...
  ** package 'tensorflow' successfully unpacked and MD5 sums checked
  ** R
  ** inst
  ** preparing package for lazy loading
  Error in loadNamespace(i, c(lib.loc, .libPaths()), versionCheck = vI[[i]]) :
  there is no package called 'whisker'
  ERROR: lazy loading failed for package 'tensorflow'
• removing 'C:/Users/HOME/Documents/R/win-library/3.4/tensorflow'
  In R CMD INSTALL
  Error: Failed to install 'gpt2' from GitHub:
  (converted from warning) running command '"C:/PROGRA1/R/R-341.3/bin/x64/R" CMD INSTALL -l "C:\Users\HOME\Documents\R\win-library\3.4" C:\Users\Public\Documents\Wondershare\CreatorTemp\RtmpuUgIyk/downloaded_packages/tensorflow_2.0.0.tar.gz' had status 1

Hi, computer neanderthal here, I have a problem with tensorflow, can u help me plz?

Inform users that model will be stored as a pin

Does it make sense to inform users that models will be stored as pins?
Something like "Downloading ... and storing as ..." when a new model type is downloaded.

Possible values for top_p

Hello,

Thanks a lot for making this available.
I have a question regarding possible values for top_p. From my understanding reading the paper "The curious case of neural text degeneration" (Holtzman et al., 2020), top_p should be a probability between 0 and 1. Yet when I check the code in 'model.R', you turn the value into an integer with as.integer() before providing it to Python.
I tried various values between 0 and 1 with the same seed, and as I expected from the code, the outputs are the same. I get different outputs with values greater than 1, but I can't figure out how things work exactly. Could you please provide some more explanations?

Thanks a lot in advance.
Best,

Christophe C.

Works with TF 1.14

added to #2