ComfyUI.org

From Flawed to Flawless: Harnessing the Power of SDXL Inpainting and Style Fusion

CN
ComfyUI.org
2025-05-02 05:28:35

1. Workflow Overview

ma6cq36kozu4bsexxr7c1387582d276d4ee8abad33eca5a3eed22e07e55572813c8d52c5b6bab52dcc.png

This workflow combines SDXL inpainting with IPAdapter-based style fusion. Key features:

  • Repaints masked areas of an input image.

  • Transfers style from a reference image via IPAdapter.

  • Uses Fooocus's inpainting model for detail coherence.

2. Core Models

  • SDXL (万享XL_超写实摄影V8.4): Main text-to-image model.

  • IPAdapter (ip-adapter-plus_sdxl_vit-h): Injects reference image style.

  • Fooocus Inpaint (inpaint_v26.fooocus.patch): Optimizes local repairs.

  • CLIP Vision (CLIP-ViT-H-14-laion2B-s32B-b79K): Encodes reference image features.

3. Node Details

Key Nodes

  • LoadImage: Loads input image and mask (white=inpaint area).

  • PrepImageForClipVision: Preprocesses reference image for IPAdapter.

  • IPAdapterAdvanced: Blends style with main model (strength=1.0 by default).

  • INPAINT_LoadFooocusInpaint: Loads Fooocus inpainting components.

  • KSampler: Sampler (Euler, 30 steps) controls generation.

Installation

  • IPAdapter: Install via ComfyUI Manager (IPAdapter Plus).

  • Fooocus Models: Download manually from GitHub to ComfyUI/models/inpaint.

Dependencies

  • IPAdapter Models: Download from HuggingFace.

  • CLIP Vision: Auto-downloads (requires internet).

4. Workflow Structure

  1. Input Group

    • Nodes: LoadImage, PrepImageForClipVision.

  2. Model Loading

    • Nodes: CheckpointLoaderSimple, IPAdapterModelLoader, INPAINT_LoadFooocusInpaint.

  3. Inpainting Processing

    • Nodes: VAEEncodeForInpaint, InpaintModelConditioning, INPAINT_ApplyFooocusInpaint.

  4. Style Fusion

    • Node: IPAdapterAdvanced.

  5. Output Generation

    • Nodes: KSampler, VAEDecode, SaveImage.

5. Inputs & Outputs

  • Inputs:

    • Required: Base image, mask, reference image.

    • Optional: Prompt (e.g., a bronze round shield...), seed (e.g., 506247113697241).

  • Output: Saved to ComfyUI folder.

6. Notes

  1. Common Errors:

    • Missing IPAdapter files cause node failures.

    • Mask must match input image dimensions.

  2. Optimization:

    • Reduce KSampler steps (e.g., 20) for speed.

    • GPU with ≥12GB VRAM recommended.

  3. Compatibility:

    • Requires ComfyUI ≥0.3.30 and matching IPAdapter plugin versions.

previous
Revamp Your Visuals: Inpainting, Style Transfer, and Auto-Captioning Made Easy
next
From Dull to Dazzling: IC-Light V2 Advanced Lighting Control Workflow

Recommend