how to integrate models not available via huggingface or gluon? about mlm-scoring

[pstroe]

hi julian,
thanks a lot for your response! this helped a lot and i could make it work!

[pstroe]

hello there, since there has not been a response as of yet, i'd like to reinvoke this issue.

[JulianSlzr]

Hi Phillip, thanks for reaching out. If I understand correctly from this and your (indirect) private message, your model is still defined in Transformers; it's just one you trained yourself.

At least for Transformers v3, you can see in commit f33f240 how I added ALBERT support in response to issue #2 :

• Here is where support for a zoo model or local weights (via --weights) is added: https://github.com/awslabs/mlm-scoring/blob/master/src/mlm/models/__init__.py#L94-L102
• (Optional) Here is where I subclass ALBERT's masked LM head to improve memory use: https://github.com/awslabs/mlm-scoring/blob/master/src/mlm/models/bert.py#L195-L199

I have been on leave + working on other projects, but hope to revisit usability and a switch to v4 in Jan-Feb 2022.