Read our article here: https://blib.la/blog/comfyui-on-runpod
โ Please also checkout Captain: The AI Platform
- Quickstart
- Features
- Config
- Use the Docker image on RunPod
- API specification
- Interact with your RunPod API
- How to get the workflow from ComfyUI?
- Bring Your Own Models and Nodes
- Local testing
- Automatically deploy to Docker hub with GitHub Actions
- Acknowledgments
- ๐ณ Choose one of the three available images for your serverless endpoint:
timpietruskyblibla/runpod-worker-comfy:3.0.0-base
: doesn't contain any checkpoints, just a clean ComfyUItimpietruskyblibla/runpod-worker-comfy:3.0.0-sdxl
: contains the checkpoints and VAE for Stable Diffusion XL- Checkpoint: sd_xl_base_1.0.safetensors
- VAEs:
timpietruskyblibla/runpod-worker-comfy:3.0.0-sd3
: contains the sd3_medium_incl_clips_t5xxlfp8.safetensors checkpoint for Stable Diffusion 3
- โน๏ธ Use the Docker image on RunPod
- Run any ComfyUI workflow to generate an image
- Provide input images as base64-encoded string
- The generated image is either:
- Returned as base64-encoded string (default)
- Uploaded to AWS S3 (if AWS S3 is configured)
- There are three different Docker images to choose from:
timpietruskyblibla/runpod-worker-comfy:3.0.0-base
: doesn't contain anything, just a clean ComfyUItimpietruskyblibla/runpod-worker-comfy:3.0.0-sdxl
: contains the checkpoint and VAE for Stable Diffusion XLtimpietruskyblibla/runpod-worker-comfy:3.0.0-sd3
: contains the checkpoint for Stable Diffusion 3
- Bring your own models
- Based on Ubuntu + NVIDIA CUDA
Environment Variable | Description | Default |
---|---|---|
REFRESH_WORKER |
When you want to stop the worker after each finished job to have a clean state, see official documentation. | false |
COMFY_POLLING_INTERVAL_MS |
Time to wait between poll attempts in milliseconds. | 250 |
COMFY_POLLING_MAX_RETRIES |
Maximum number of poll attempts. This should be increased the longer your workflow is running. | 500 |
SERVE_API_LOCALLY |
Enable local API server for development and testing. See Local Testing for more details. | disabled |
This is only needed if you want to upload the generated picture to AWS S3. If you don't configure this, your image will be exported as base64-encoded string.
- Create a bucket in region of your choice in AWS S3 (
BUCKET_ENDPOINT_URL
) - Create an IAM that has access rights to AWS S3
- Create an Access-Key (
BUCKET_ACCESS_KEY_ID
&BUCKET_SECRET_ACCESS_KEY
) for that IAM - Configure these environment variables for your RunPod worker:
Environment Variable | Description | Example |
---|---|---|
BUCKET_ENDPOINT_URL |
The endpoint URL of your S3 bucket. | https://<bucket>.s3.<region>.amazonaws.com |
BUCKET_ACCESS_KEY_ID |
Your AWS access key ID for accessing the S3 bucket. | AKIAIOSFODNN7EXAMPLE |
BUCKET_SECRET_ACCESS_KEY |
Your AWS secret access key for accessing the S3 bucket. | wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY |
- Create a new template by clicking on
New Template
- In the dialog, configure:
- Template Name:
runpod-worker-comfy
(it can be anything you want) - Template Type: serverless (change template type to "serverless")
- Container Image:
<dockerhub_username>/<repository_name>:tag
, in this case:timpietruskyblibla/runpod-worker-comfy:3.0.0-sd3
(or-base
for a clean image or-sdxl
for Stable Diffusion XL) - Container Registry Credentials: You can leave everything as it is, as this repo is public
- Container Disk:
20 GB
- (optional) Environment Variables: Configure S3
- Note: You can also not configure it, the images will then stay in the worker. In order to have them stored permanently, we have to add the network volume
- Template Name:
- Click on
Save Template
- Navigate to
Serverless > Endpoints
and click onNew Endpoint
- In the dialog, configure:
- Endpoint Name:
comfy
- Select Template:
runpod-worker-comfy
(or whatever name you gave your template) - Active Workers:
0
(whatever makes sense for you) - Max Workers:
3
(whatever makes sense for you) - Idle Timeout:
5
(you can leave the default) - Flash Boot:
enabled
(doesn't cost more, but provides faster boot of our worker, which is good) - (optional) Advanced: If you are using a Network Volume, select it under
Select Network Volume
. Otherwise leave the defaults. - Select a GPU that has some availability
- GPUs/Worker:
1
- Endpoint Name:
- Click
deploy
- Your endpoint will be created, you can click on it to see the dashboard
The following describes which fields exist when doing requests to the API. We only describe the fields that are sent via input
as those are needed by the worker itself. For a full list of fields, please take a look at the official documentation.
{
"input": {
"workflow": {},
"images": [
{
"name": "example_image_name.png",
"image": "base64_encoded_string"
}
]
}
}
Field Path | Type | Required | Description |
---|---|---|---|
input |
Object | Yes | The top-level object containing the request data. |
input.workflow |
Object | Yes | Contains the ComfyUI workflow configuration. |
input.images |
Array | No | An array of images. Each image will be added into the "input"-folder of ComfyUI and can then be used in the workflow by using it's name |
An array of images, where each image should have a different name.
๐จ The request body for a RunPod endpoint is 10 MB for /run
and 20 MB for /runsync
, so make sure that your input images are not super huge as this will be blocked by RunPod otherwise, see the official documentation
Field Name | Type | Required | Description |
---|---|---|---|
name |
String | Yes | The name of the image. Please use the same name in your workflow to reference the image. |
image |
String | Yes | A base64 encoded string of the image. |
-
Generate an API Key:
- In the User Settings, click on
API Keys
and then on theAPI Key
button. - Save the generated key somewhere safe, as you will not be able to see it again when you navigate away from the page.
- In the User Settings, click on
-
Use the API Key:
- Use cURL or any other tool to access the API using the API key and your Endpoint ID:
- Replace
<api_key>
with your key.
- Replace
- Use cURL or any other tool to access the API using the API key and your Endpoint ID:
-
Use your Endpoint:
- Replace
<endpoint_id>
with the ID of the endpoint. (You can find the endpoint ID by clicking on your endpoint; it is written underneath the name of the endpoint at the top and also part of the URLs shown at the bottom of the first box.)
- Replace
curl -H "Authorization: Bearer <api_key>" https://api.runpod.ai/v2/<endpoint_id>/health
You can either create a new job async by using /run
or a sync by using /runsync
. The example here is using a sync job and waits until the response is delivered.
The API expects a JSON in this form, where workflow
is the workflow from ComfyUI, exported as JSON and images
is optional.
Please also take a look at the test_input.json to see how the API input should look like.
curl -X POST -H "Authorization: Bearer <api_key>" -H "Content-Type: application/json" -d '{"input":{"workflow":{"3":{"inputs":{"seed":1337,"steps":20,"cfg":8,"sampler_name":"euler","scheduler":"normal","denoise":1,"model":["4",0],"positive":["6",0],"negative":["7",0],"latent_image":["5",0]},"class_type":"KSampler"},"4":{"inputs":{"ckpt_name":"sd_xl_base_1.0.safetensors"},"class_type":"CheckpointLoaderSimple"},"5":{"inputs":{"width":512,"height":512,"batch_size":1},"class_type":"EmptyLatentImage"},"6":{"inputs":{"text":"beautiful scenery nature glass bottle landscape, purple galaxy bottle,","clip":["4",1]},"class_type":"CLIPTextEncode"},"7":{"inputs":{"text":"text, watermark","clip":["4",1]},"class_type":"CLIPTextEncode"},"8":{"inputs":{"samples":["3",0],"vae":["4",2]},"class_type":"VAEDecode"},"9":{"inputs":{"filename_prefix":"ComfyUI","images":["8",0]},"class_type":"SaveImage"}}}}' https://api.runpod.ai/v2/<endpoint_id>/runsync
Example response with AWS S3 bucket configuration
{
"delayTime": 2188,
"executionTime": 2297,
"id": "sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1",
"output": {
"message": "https://bucket.s3.region.amazonaws.com/10-23/sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1/c67ad621.png",
"status": "success"
},
"status": "COMPLETED"
}
Example response as base64-encoded image
{
"delayTime": 2188,
"executionTime": 2297,
"id": "sync-c0cd1eb2-068f-4ecf-a99a-55770fc77391-e1",
"output": { "message": "base64encodedimage", "status": "success" },
"status": "COMPLETED"
}
- Open ComfyUI in the browser
- Open the
Settings
(gear icon in the top right of the menu) - In the dialog that appears configure:
Enable Dev mode Options
: enable- Close the
Settings
- In the menu, click on the
Save (API Format)
button, which will download a file namedworkflow_api.json
You can now take the content of this file and put it into your workflow
when interacting with the API.
Using a Network Volume allows you to store and access custom models:
-
Create a Network Volume:
- Follow the RunPod Network Volumes guide to create a volume.
-
Populate the Volume:
- Create a temporary GPU instance:
- Navigate to
Manage > Storage
, clickDeploy
under the volume, and deploy any GPU or CPU instance. - Navigate to
Manage > Pods
. Under the new pod, clickConnect
to open a shell (either via Jupyter notebook or SSH).
- Navigate to
- Populate the volume with your models:
cd /workspace for i in checkpoints clip clip_vision configs controlnet embeddings loras upscale_models vae; do mkdir -p models/$i; done wget -O models/checkpoints/sd_xl_turbo_1.0_fp16.safetensors https://huggingface.co/stabilityai/sdxl-turbo/resolve/main/sd_xl_turbo_1.0_fp16.safetensors
- Create a temporary GPU instance:
-
Delete the Temporary GPU Instance:
- Once populated, terminate the temporary GPU instance.
-
Configure Your Endpoint:
- Use the Network Volume in your endpoint configuration:
- Either create a new endpoint or update an existing one.
- In the endpoint configuration, under
Advanced > Select Network Volume
, select your Network Volume.
- Use the Network Volume in your endpoint configuration:
Note: The folders in the Network Volume are automatically available to ComfyUI when the network volume is configured and attached.
If you prefer to include your models directly in the Docker image, follow these steps:
-
Fork the Repository:
- Fork this repository to your own GitHub account.
-
Add Your Models in the Dockerfile:
- Edit the
Dockerfile
to include your models:RUN wget -O models/checkpoints/sd_xl_base_1.0.safetensors https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors
- You can also add custom nodes:
RUN git clone https://github.com/<username>/<custom-node-repo>.git custom_nodes/<custom-node-repo>
- Edit the
-
Build Your Docker Image:
- Build the base image locally:
docker build -t <your_dockerhub_username>/runpod-worker-comfy:dev-base --target base --platform linux/amd64 .
- Build the sdxl image locally:
docker build --build-arg MODEL_TYPE=sdxl -t <your_dockerhub_username>/runpod-worker-comfy:dev-sdxl --platform linux/amd64 .
- Build the sd3 image locally:
docker build --build-arg MODEL_TYPE=sd3 --build-arg HUGGINGFACE_ACCESS_TOKEN=<your-huggingface-token> -t <your_dockerhub_username>/runpod-worker-comfy:dev-sd3 --platform linux/amd64 .
- Build the base image locally:
Note
Ensure to specify --platform linux/amd64
to avoid errors on RunPod, see issue #13.
Both tests will use the data from test_input.json, so make your changes in there to test this properly.
- Make sure you have Python >= 3.10
- Create a virtual environment:
python -m venv venv
- Activate the virtual environment:
- Windows:
.\venv\Scripts\activate
- Mac / Linux:
source ./venv/bin/activate
- Windows:
- Install the dependencies:
pip install -r requirements.txt
- Install WSL2 and a Linux distro (like Ubuntu) following this guide. You can skip the "Install and use a GUI package" part.
- After installing Ubuntu, open the terminal and log in:
wsl -d Ubuntu
- Update the packages:
sudo apt update
- Install Docker in Ubuntu:
- Follow the official Docker installation guide.
- Install docker-compose:
sudo apt-get install docker-compose
- Install the NVIDIA Toolkit in Ubuntu:
Follow this guide and create the
nvidia
runtime.
- Enable GPU acceleration on Ubuntu on WSL2:
Follow this guide.
- If you already have your GPU driver installed on Windows, you can skip the "Install the appropriate Windows vGPU driver for WSL" step.
- Add your user to the
docker
group to use Docker withoutsudo
:sudo usermod -aG docker $USER
Once these steps are completed, switch to Ubuntu in the terminal and run the Docker image locally on your Windows computer via WSL:
wsl -d Ubuntu
- Run all tests:
python -m unittest discover
- If you want to run a specific test:
python -m unittest tests.test_rp_handler.TestRunpodWorkerComfy.test_bucket_endpoint_not_configured
You can also start the handler itself to have the local server running: python src/rp_handler.py
To get this to work you will also need to start "ComfyUI", otherwise the handler will not work.
For enhanced local development, you can start an API server that simulates the RunPod worker environment. This feature is particularly useful for debugging and testing your integrations locally.
Set the SERVE_API_LOCALLY
environment variable to true
to activate the local API server when running your Docker container. This is already the default value in the docker-compose.yml
, so you can get it running by executing:
docker-compose up
- With the local API server running, it's accessible at: localhost:8000
- When you open this in your browser, you can also see the API documentation and can interact with the API directly
- With the local API server running, you can access ComfyUI at: localhost:8188
The repo contains two workflows that publish the image to Docker hub using GitHub Actions:
- dev.yml: Creates the image and pushes it to Docker hub with the
dev
tag on every push to themain
branch - release.yml: Creates the image and pushes it to Docker hub with the
latest
and the release tag. It will only be triggered when you create a release on GitHub
If you want to use this, you should add these secrets to your repository:
Configuration Variable | Description | Example Value |
---|---|---|
DOCKERHUB_USERNAME |
Your Docker Hub username. | your-username |
DOCKERHUB_TOKEN |
Your Docker Hub token for authentication. | your-token |
HUGGINGFACE_ACCESS_TOKEN |
Your READ access token from Hugging Face | your-access-token |
And also make sure to add these variables to your repository:
Variable Name | Description | Example Value |
---|---|---|
DOCKERHUB_REPO |
The repository on Docker Hub where the image will be pushed. | timpietruskyblibla |
DOCKERHUB_IMG |
The name of the image to be pushed to Docker Hub. | runpod-worker-comfy |
- Thanks to all contributors for your awesome work
- Thanks to Justin Merrell from RunPod for worker-1111, which was used to get inspired on how to create this worker
- Thanks to Ashley Kleynhans for runpod-worker-a1111, which was used to get inspired on how to create this worker
- Thanks to comfyanonymous for creating ComfyUI, which provides such an awesome API to interact with Stable Diffusion