Text-free diffusion inpainting using reference images for enhanced visual fidelity.

• Language-based Subject Generation faces challenge in accurate portrayal of subject. • Nowadays Reference Guided Generation lacks ability to preserve subject identity. • Exemplar-based instructions with visual tokens preserve visual details of subject. • Model based guidance samples better quality images with different pose. • Our model achieved highest CLIP, DINO score and user study compared to others. This paper presents a novel approach to subject-driven image generation that addresses the limitations of traditional text-to-image diffusion models. Our method generates images using reference images without relying on language-based prompts. We introduce a visual detail preserving module that captures intricate details and textures, addressing overfitting issues associated with limited training samples. The model's performance is further enhanced through a modified classifier-free guidance technique and feature concatenation, enabling the natural positioning and harmonization of subjects within diverse scenes. Quantitative assessments using CLIP, DINO and Quality scores (QS), along with a user study, demonstrate the superior quality of our generated images. Our work highlights the potential of pre-trained models and visual patch embeddings in subject-driven editing, balancing diversity and fidelity in image generation tasks. Our implementation is available at https://github.com/8eomio/Subject-Inpainting. [Display omitted] To create your abstract, type over the instructions in the template box below. Fonts or abstract dimensions should not be changed or altered. [ABSTRACT FROM AUTHOR]