ComfyUI InstantID

This is a thin wrapper custom node for Instant ID. It's providing basic testing interface for playing around with Instant ID functions. Forgive me for not implementing stepping progress indicator.

It's not following ComfyUI module design nicely, but I just want to set it up for quick testing. Hope IPAdapterPlus will do better integrating to ComfyUI ecosystems...

A little more explanation: Yes, I know it's great to break down nodes; but it's diffuser based implementation and its inputs / outputs are not compatible with existing other nodes. Even if you break down nodes, those nodes are just connecting each others within the group. Let's wait for better IPAdapterPlus implementation instead of introducing yet another bunch of fancy nodes just for one purpose.

Install

Just as other custom nodes:

cd ComfyUI/custom_nodes/
git clone https://github.com/huxiuhan/ComfyUI-InstantID.git
pip install -r requirements.txt

Download Models

Download

You can directly download the model from Huggingface. You also can download the model in python script:

from huggingface_hub import hf_hub_download
hf_hub_download(repo_id="InstantX/InstantID", filename="ControlNetModel/config.json", local_dir="./checkpoints")
hf_hub_download(repo_id="InstantX/InstantID", filename="ControlNetModel/diffusion_pytorch_model.safetensors", local_dir="./checkpoints")
hf_hub_download(repo_id="InstantX/InstantID", filename="ip-adapter.bin", local_dir="./checkpoints")

If you cannot access to Huggingface, you can use hf-mirror to download models.

export HF_ENDPOINT=https://hf-mirror.com
huggingface-cli download --resume-download InstantX/InstantID --local-dir checkpoints

For face encoder, you need to manually download via this URL to models/antelopev2 as the default link is invalid. Once you have prepared all models, the folder tree should be like:

  .
  ├── models
  ├── checkpoints
  ├── ip_adapter
  ├── pipeline_stable_diffusion_xl_instantid.py
  └── README.md

Usage

Choose a SDXL base ckpt. You can also try SDXL Turbo with 4 steps, very efficient for fast testing.

First time loading usually takes more than 60s, but the node will try its best to cache models.

Original Project

InstantID : Zero-shot Identity-Preserving Generation in Seconds

InstantID is a new state-of-the-art tuning-free method to achieve ID-Preserving generation with only single image, supporting various downstream tasks.

Release

[2024/1/22] 🔥 We release the pre-trained checkpoints, inference code and gradio demo!
[2024/1/15] 🔥 We release the technical report.
[2023/12/11] 🔥 We launch the project page.

Demos

Stylized Synthesis

Comparison with Previous Works

Comparison with existing tuning-free state-of-the-art techniques. InstantID achieves better fidelity and retain good text editability (faces and styles blend better).

Comparison with pre-trained character LoRAs. We don't need multiple images and still can achieve competitive results as LoRAs without any training.

Comparison with InsightFace Swapper (also known as ROOP or Refactor). However, in non-realistic style, our work is more flexible on the integration of face and background.

Usage Tips

For higher similarity, increase the weight of controlnet_conditioning_scale (IdentityNet) and ip_adapter_scale (Adapter).
For over-saturation, decrease the ip_adapter_scale. If not work, decrease controlnet_conditioning_scale.
For higher text control ability, decrease ip_adapter_scale.
For specific styles, choose corresponding base model makes differences.

Acknowledgements

Our work is highly inspired by IP-Adapter and ControlNet. Thanks for their great works!
Thanks to the HuggingFace team for their generous GPU support!

Disclaimer

This project is released under Apache License and aims to positively impact the field of AI-driven image generation. Users are granted the freedom to create images using this tool, but they are obligated to comply with local laws and utilize it responsibly. The developers will not assume any responsibility for potential misuse by users.

Cite

If you find InstantID useful for your research and applications, please cite us using this BibTeX:

@article{wang2024instantid,
  title={InstantID: Zero-shot Identity-Preserving Generation in Seconds},
  author={Wang, Qixun and Bai, Xu and Wang, Haofan and Qin, Zekui and Chen, Anthony},
  journal={arXiv preprint arXiv:2401.07519},
  year={2024}
}

cracdev / comfyui-instantid Goto Github PK