Anime Video Magic: Unleashing the Potential of WAN2.1 for Stunning Visuals

CN
ComfyUI.org
2025-05-06 11:42:14

1. Workflow Overview

macfv3t3vdg8wk1t5zddb8a960fb80e565f2faf2ada02f131696bac4539c3aed6b2d0740373bb5ce9f.gif

This workflow uses Alibaba’s WAN2.1 model for Image-to-Video (I2V) generation, transforming a 2D anime-style character image (e.g., a girl cycling) into a dynamic video with camera motion.

Core Models:

  • Wan2.1-I2V-14B: Video generation model (480P output).

  • UMT5-XXL: Multilingual text encoder for Chinese prompts.

  • OpenCLIP-ViT-H: Image encoder for input analysis.

2. Key Nodes

Node

Function

Installation

WanVideoModelLoader

Loads WAN2.1 model (requires fp8_e4m3fn)

Manual download to models folder

WanVideoTextEncode

Processes bilingual prompts (e.g., "cycling scene")

Requires ComfyUI-WanVideoWrapper

RIFE VFI

Frame interpolation for smoother video

Install via ComfyUI Manager

VHS_VideoCombine

Combines frames into MP4

Install VideoHelperSuite plugin

Dependencies:

  • Manual model downloads:

    • Wan2_1-I2V-14B-480P_fp8_e4m3fn.safetensors

    • umt5-xxl-enc-bf16.safetensors

    • Save to: ComfyUI/models/wan_video/

3. Workflow Groups

  1. Load Image: Input character image (e.g., 00028-2257557710.png).

  2. Model Load: Load WAN2.1, VAE, and text encoders.

  3. Prompt Processing: Positive/Negative prompts (e.g., "speed effect").

  4. Video Generation: WanVideoSampler generates latent frames.

  5. Post-Processing: Upscaling (CR Upscale Image).

  6. Video Export: MP4 output (VHS_VideoCombine).

  7. Frame Interpolation: Boosts FPS to 32 via RIFE VFI.

4. Inputs & Outputs

  • Inputs:

    • Image: 480x768 PNG (anime character).

    • Prompts: Chinese/English descriptions.

    • Params: Frame rate (16→32FPS), CFG=8.0, Steps=30.

  • Output:

    • MP4 video (default: xiao.mp4).

5. Notes

  • VRAM: ≥16GB GPU recommended (use bf16 for lower usage).

  • Debugging:

    • Missing model → Check wan_video folder path.

    • RIFE error → Ensure rife47.pth is in ComfyUI/models/rife/.

  • Optimization:

    • Reduce resolution (e.g., 360P) for lower VRAM.

    • Disable experimental features to speed up.