An implementation of GPT2 Byte Pair Encoding Encoder/Decoder in golang
. Generally very fast, bottlenecked by the regex engine for whitespace separation.
aniol13 / gpt_bpe Goto Github PK
View Code? Open in Web Editor NEWThis project forked from wbrown/gpt_bpe
GPT2 Byte Pair Encoding implementation in Golang
License: GNU Lesser General Public License v2.1