Tokenizer class YiTokenizer does not exist or is not currently imported. about yi

[MarceloCorreiaData]

I had the same problem as you guys, so I reasoned reedcli meant with https://huggingface.co/01-ai/Yi-6B/discussions/6 they expected us to apply "trust_remote_code" to both instantiations.
So this code worked like a charm, please try it out:

from transformers import AutoTokenizer, AutoModelForCausalLM
model_path = '/path/to/your/model'
model = AutoModelForCausalLM.from_pretrained(model_path, trust_remote_code=True, device_map="auto")
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True, dtype="bfloat16", use_accelerate=True)
P.S.: (Add/replace your "device" snippet, I use mine).

[zl-liu]

@mallorbc Thanks for your reply! I already set it to True (I even tried hard-coding "True" for "trust_remote_code"

[oobabooga]

Is there any way to fix this in text-generation-webui-main for ExLlama_HF? What and where should I edit and add?

You need to launch the web UI with the --trust-remote-code flag (it is disabled by default as a security measure).

[loofahcus]

Unfortunately I can't reproduce this problem, maybe you can have a try with our Docker image (will be released soon: #3)

[zl-liu]

I did a deep dive into the code ---- I think this problem might be caused by loading errors in this code: [https://github.com/huggingface/transformers/blob/eef7ea98c31a333bacdc7ae7a2372bde772be8e4/src/transformers/models/auto/tokenization_auto.py] . I downloaded the Yi-34b tar file, decompressed it, and used that dir as the pretrained path.

In tokenizer_config.json:

"auto_map": {
--
  | "AutoTokenizer": ["tokenization_yi.YiTokenizer", null]
  | },

Perhaps it failed to find the YiTokenizer class?

[mallorbc]

Run with "trust_remote_code" being set to True

[romulowspp]

Hello! i'm receiving the same error, i also tried trust_remote_code to True and receive same error.

[DmitryVN]

Is there any way to fix this in text-generation-webui-main for ExLlama_HF? What and where should I edit and add?