请问Yi模型的微调数据组织格式如何获取 about self-llm

[KMnO4-zx]

不同模型的构造方式不同，建议自行探索，看看模型的源码

yi采用的是transformers包自带的tokenizer apply_chat_template 方法来构造prompttemplate的，所以可以去transformers仓库找到tokenizer的源码。

input_ids = tokenizer.apply_chat_template(conversation=messages, tokenize=True, add_generation_prompt=True, return_tensors='pt')
output_ids = model.generate(input_ids.to('cuda'))
response = tokenizer.decode(output_ids[0][input_ids.shape[1]:], skip_special_tokens=True)

源码如下：
https://github.com/huggingface/transformers/blob/866253f85eb95522c686881c04a9eb9bdf8fea4e/src/transformers/tokenization_utils_base.py#L1783

    def default_chat_template(self):
        """
        This template formats inputs in the standard ChatML format. See
        https://github.com/openai/openai-python/blob/main/chatml.md
        """
        logger.warning_once(
            "\nNo chat template is defined for this tokenizer - using a default chat template "
            "that implements the ChatML format (without BOS/EOS tokens!). If the default is not appropriate for "
            "your model, please set `tokenizer.chat_template` to an appropriate template. "
            "See https://huggingface.co/docs/transformers/main/chat_templating for more information.\n"
        )
        return (
            "{% for message in messages %}"
            "{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}"
            "{% endfor %}"
            "{% if add_generation_prompt %}"
            "{{ '<|im_start|>assistant\n' }}"
            "{% endif %}"
        )

[mst272]

感谢回复！