Elevate Your Visual Storytelling: Wan2.1 Video Generation Workflow for Professionals

CN
ComfyUI.org
2025-04-07 15:33:32

1. Workflow Overview

m978dcketz65jbexrkr5958698cfab9f9d5f378948426f7df89df11314abff1abc8153aae5e52b9d670.gif

This is an advanced Wan2.1 video generation workflow specializing in "division/replication" visual effects. Key features:

  • Dual-modal input (text/image) β†’ video generation

  • Object splitting/cloning effects

  • Post-processing enhancement

2. Core Models

Model Name

Function

Hardware Req.

Wan2.1-I2V-14B-480P

Base video generation model

24GB+ VRAM

umt5-xxl-enc

Multilingual text encoder

BF16 required

open-clip-xlm-roberta-large

Multimodal image encoder

FP16 precision

3. Key Components

3.1 Custom Nodes

1. ComfyUI-WanVideoWrapper (Required):
   - Contains all Wan video processing nodes
   - Requires manual install: `sageattn` optimizer

2. Custom LoRA:
   - `Wan2.1_division_replication.safetensors`
   - Recommended strength: 0.9-1.1

3.2 Critical Nodes

  • WanVideoBlockSwap (ID:32): Controls frame block swapping

  • WanVideoTeaCache (ID:98): VRAM optimization

  • WanVideoSLG (ID:96): Smooth Latent Guidance

4. Workflow Stages

Phase 1: Input Processing

  • Text: "A smartphone splits into multiple apples"

  • Image: 480x720 resolution (auto-adjusted)

Phase 2: Feature Encoding

  • Text β†’ umt5 encoder

  • Image β†’ CLIP vision encoder

Phase 3: Video Generation

  • UniPC sampler (30 steps)

  • SLG smoothing (strength 0.6)

Phase 4: Output

  • 272 frames @16fps

  • Automatic VRAM cleanup

5. I/O Specifications

Input Parameters:

{
  "positive_prompt": "object splitting effect", # Required  
  "negative_prompt": "low quality...",  
  "seed": 594591065694543,          # Fixed seed  
  "resolution": [480,720]           # 16:9 recommended  
}

Output:

  • MP4 video (H.264)

  • Metadata with generation params

6. Important Notes

  1. Hardware:

    • Minimum: RTX 3090(24GB)

    • Recommended: A100 40GB

  2. Optimization:

    • Enable offload_device

    • Use fp8_e4m3fn quantization

  3. Troubleshooting:

    • VRAM overflow β†’ Reduce block_swap frequency

    • Frame flickering β†’ Increase SLG strength