1. Workflow Overview

• Purpose:
  This workflow replaces materials in specified image regions (e.g., fabric, leather) using manual masking, leveraging IPAdapter for style transfer and ControlNet for structural consistency.

• Core Models:

  • juggernautXL_v9-Lightning_4S_V9 + RDPhoto 2: Base model for photorealistic generation.

  • IPAdapter: Transfers material style from reference images.

  • ControlNet (sai_xl_depth_256lora): Constrains structure via depth maps.

  • DepthAnythingPreprocessor: Generates depth maps for ControlNet.

2. Key Nodes

3. Workflow Structure

• Group 1: Image Input

  • Inputs:

    • Original image (e.g., furniture).

    • Material reference image.

    • Manually painted mask (right-click to edit).

  • Outputs: Cropped/aligned images and masks.

• Group 2: Material Processing

  • Steps:

    1. Encodes material style via IPAdapter.

    2. Generates depth maps with DepthAnything for ControlNet.

    3. Smooths mask edges using GrowMask and FeatherMask.

• Group 3: Generation & Composite

  • Generation:

    • KSampler merges IPAdapter and ControlNet outputs.

    • ImageCompositeMasked blends results with the original image.

  • Output: Final image (saved to ComfyUI/output).

4. Inputs & Outputs

• Input Parameters:

  • Resolution: Default 1024x1024 (adjustable).

  • Seed: Fixed or randomized.

  • IPAdapter weight: Default 0.85 (controls style strength).

• Output:

  • Image with replaced material (PNG).

5. Notes

• Common Errors:

  • Missing IPAdapter/ControlNet models triggers file path errors.

  • Incorrect masking causes misalignment.

• Optimization:

  • Tweak FeatherMask radius for natural edges.

  • Reduce IPAdapter weight (e.g., 0.6) to avoid overfitting.

• Hardware:

  • Recommended 12GB+ VRAM (due to IPAdapter + ControlNet).