mason-mcgough / comfyui-mosaica

Create colorful mosaic images in ComfyUI by computing label images and applying lookup tables.

🎨 ComfyUI-Mosaica

Create colorful mosaic images in ComfyUI by computing label images and applying lookup tables.

Workflow Examples

K-Means

Generate an image using a stable diffusion model and apply the k-means clustering algorithm to convert it to a label image. The average color of each cluster is applied to the image's labels and a colorized image is returned.

K-means is quick and easy to use, but you must specify the number of clusters (i.e. unique labels) that you intend to find.

Mean Shift

Generate an image using a stable diffusion model and apply the mean shift clustering algorithm to convert it to a label image. The average color of each cluster is applied to the image's labels and a colorized image is returned.

Mean shift is much slower than k-means, especially for images greater than 512x512. However, you do not need to specify the number of clusters. Instead, you adjust the "bandwidth" parameter. From my experience, values in the range [0.0, 0.15] tend to produce the best results.

Watershed

Generate an image using a stable diffusion model and apply the watershed segmentation algorithm to convert it to a label image. The average color of each cluster is applied to the image's labels and a colorized image is returned.

Watershed is a fast region-based method and will only produce the best results on images with a lot of intensity variation. It does not account for the hue of the original image like k-means or mean shift.

Random LUT

Apply a randomly generated lookup table of RGB colors to colorize the label image from the mean shift clustering node.

Load LUT from Matplotlib

Apply a lookup table from Matplotlib to colorize the label image.

Label img2img

Apply an img2img with light denoising to the colorized label image.

Colorize an image with K-Means

This slightly more complex workflow uses a k-means label image and a Matplotlib LUT to colorize a generated image. The resulting image is then upscaled for a few additional denoising steps (similar to the hires fix technique) to smoothly blend the colors of the label image with the content from the generated image.

Nodes

• Mean Shift - Apply the Mean Shift clustering algorithm to an image.
• Apply LUT To Label Image - Converts a label image into an RGB image by applying a RGB lookup table (LUT).
• Random LUT - Randomly generate a LUT of RGB colors.
• Load LUT From Matplotlib - Load an RGB LUT from Matplotlib.

To do

• implement LoadLUTFromFile node
• implement MedianFilter node
• implement KMeans node
• implement Watershed node
• implement Resize Label Image node
• add support for Segment Anything labels
• write unit tests
• use LAB space in RandomLUT for better perceptual uniformity
• add random seed option to RandomLUT