jeongukjae / huggingface-to-tfhub Goto Github PK

View Code? Open in Web Editor NEW

5.0 1.0 0.0 122 KB

Converting Huggingface Models to TFHub

Python 97.15% Shell 2.85%

huggingface-to-tfhub's Issues

Convert DistilBERT Model

for toy project.

en-cased
en-uncased
multi-cased

Convert MiniLM models

L6xH768 MiniLMv2 distilled from RoBERTa-Large
L12xH384 MiniLMv2 distilled from RoBERTa-Large
L6xH384 MiniLMv2 distilled from RoBERTa-Large
L12xH384 MiniLMv2 distilled from XLMR-Large
L6xH384 MiniLMv2 distilled from XLMR-Large

https://github.com/microsoft/unilm/tree/master/minilm

-> to use in personal project (multilingual L6xH384 + "Load what you need")

Allow bert_pack_inputs to have a configurable seq_length

Hey there! First off I'd like to thank you on all the work you've done to convert these HF models into TF Hub models, we've found your work really useful within my organization.

We were trying to use the DistilBERT uncased model, and wanted to set a different seq_length for bert_pack_inputs. We noticed on the official tensorflow site, it's conventional to support an optional seq_length for the preprocessor's bert_pack_inputs implementation. Indeed in your underlying implementation, there is the option, but it's not exposed in the call function, which only allows the default length of 128.

For now, we have copied our own version of DistilBertPackInputs, but it would be nice to just use the TF Hub preprocessor as-is.

I was wondering if there was a reason you avoided including it - I'd be happy to open a PR to add it, but I don't have a good means of testing to make sure that the change is backwards compatible with anything else you may be using it for.