yuanze-lin / learnable_regions

Official implementation of the work "Text-Driven Image Editing via Learnable Regions" (CVPR 2024)

Text-Driven Image Editing via Learnable Regions
(CVPR 2024)

Paper | Project Page | Youtube Video

Official implementation of "Text-Driven Image Editing via Learnable Regions"

Yuanze Lin, Yi-Wen Chen, Yi-Hsuan Tsai, Lu Jiang, Ming-Hsuan Yang

Sorry for delay, code will be released before 30th April!

Abstract: Language has emerged as a natural interface for image editing. In this paper, we introduce a method for region-based image editing driven by textual prompts, without the need for user-provided masks or sketches. Specifically, our approach leverages an existing pre-trained text-to-image model and introduces a bounding box generator to find the edit regions that are aligned with the textual prompts. We show that this simple approach enables flexible editing that is compatible with current image generation models, and is able to handle complex prompts featuring multiple objects, complex sentences, or long paragraphs. We conduct an extensive user study to compare our method against state-of-the-art methods. Experiments demonstrate the compet- itive performance of our method in manipulating images with high fidelity and realism that align with the language descriptions provided. Our project webpage: https://yuanze-lin.me/LearnableRegions_page.

Method Overview

Results Using Diverse Prompts

Additional Results

BibTeX

@article{lin2023text,
  title={Text-Driven Image Editing via Learnable Regions},
  author={Lin, Yuanze and Chen, Yi-Wen and Tsai, Yi-Hsuan and Jiang, Lu and Yang, Ming-Hsuan},
  journal={arXiv preprint arXiv:2311.16432},
  year={2023}
}

learnable_regions's Issues

code release

hi, 30 April is coming! Looking forward to your great work!

Code release?

good work！！！
your project looks very interesting, do you have a date for it's release?