记录一些思路和使用的/参考的Link和信息。
SAM: https://huggingface.co/docs/transformers/main/en/model_doc/sam 不支持fine-tuning
Meta-Llama: https://huggingface.co/meta-llama
组织架构:
sam.py: Image - Visual Knowledge Injection - Image Encoder
llama.py: - Text Knowledge Injection
llm.py: Pre-trained LLM