1. Workflow Overview

This is a multi-functional video generation workflow integrating text-to-video, image-to-video, video super-resolution, frame interpolation and depth control. Key features:

• Supports multiple loading methods for Wan 2.1 models (GGUF/SAFETENSORS)

• 4x video super-resolution (using RealESRGAN)

• GIMM-VFI frame interpolation (up to 4x)

• Includes CFG-ZeroStar quality enhancement

• Multi-stage model acceleration (TeaCache/Torch compilation)

Core Models:

• Wan2.1-T2V-14B: 14B param text-to-video base

• RealESR-General-x4v3: Video super-resolution

• GIMMVFI-R-ARB: Adaptive motion compensation

• UMT5-xxl: Multilingual text encoder

2. Node Breakdown

Critical Components:

1. VHS Video Suite:

   • Includes VHS_LoadVideo & VHS_VideoCombine

   • Requires: comfyui-videohelpersuite

   • Handles frame extraction/audio preservation

2. GIMMVFI_interpolate:

   • Core frame interpolation node

   • Install: ComfyUI-GIMM-VFI

   • Params: 2-4x interpolation factor

3. DD-ModelOptimizer:

   • Model loading optimizer

   • Options: "Step loading"/"Smart mode"

   • Dependency: ComfyUI-DD-Nodes

Special Requirements:

• Wan2.1-VAE.safetensors: Dedicated video VAE

• CLIP-Vision-VIT-H: Image feature extractor

• Wan.2.1-Rotation.safetensors: Motion LoRA

3. Workflow Structure

Group Logic:

• Text-to-Video Group:

  • Initializes latent with DDEmptyWan21LatentVideo

  • Dual prompt encoding (CN/EN)

  • Uses UniPC sampler (40 steps)

• Video Upscale Group:

  • Pipeline: Load→Split Alpha→RealESRGAN

  • Output: 4K MP4 (CRF18)

• Image-to-Video Group:

  • Uses WanImageToVideo node

  • CLIP vision conditioning

  • Fixed 49FPS (motion optimization)

• Control Video Group:

  • Depth control pipeline: Ref+Control video

  • WanFunControlToVideo node

  • Adjustable control strength (0.2-1.0)

4. Inputs & Outputs

Parameters:

• Video Input: MP4/MOV (auto frame rate)

• Text Prompts: Bilingual support

• Control: CFG 5.8, 49/65/81 FPS presets

Output:

• Video: H.264, yuv420p

• Metadata: Embedded in MP4 header

• Resolution: 832x480 to 4096x2160

5. Notes

• Hardware: Minimum 16GB VRAM (14B model)

• Recommended:

  • 1.3B model: ≤720p

  • 14B model: ≤1280p

• Common Issues:

  • Frame rate mismatch causes audio sync problems

  • GGUF requires AVX512 support

• Optimization:

  • TeaCache reduces 30% inference time

  • Use --preview-method none for performance