1. Workflow Overview

• Purpose
  This workflow converts real-life videos into anime-style animations, featuring:

  • Input: Real videos (e.g., 10068-0228.mp4) → Output: Studio Ghibli-style animation.

  • Style control via LoRA (e.g., LiveActionAnim model).

  • Auto-generates prompts with MiniCPM_VQA_Polished or manual input.

• Core Models

  1. Wan2.1-T2V-1.3B: Base video generation model (bf16 precision).

  2. Wan2.1_VAE_bf16: Video decoder.

  3. umt5-xxl-enc-bf16: T5XXL text encoder for prompt processing.

  4. LoRA: LiveActionAnim for anime-style enhancement.

2. Key Nodes

• WanVideoSampler: Core sampler using unipc scheduler.

• WanVideoLoraSelect: Loads LoRA model (weight=0.7).

• MiniCPM_VQA_Polished: Analyzes input video to generate English descriptions (e.g., "Studio Ghibli style...").

• BaiduTranslateNode: Translates Chinese prompts to English.

• Dependencies

  • Models:

    • Place Wan2.1-T2V-1.3B.safetensors in models/wan_video/.

    • Download umt5-xxl-enc-bf16.safetensors to models/text_encoders/.

  • Plugins:

    • ComfyUI-WanVideoWrapper: Essential for video nodes.

    • comfyui-videohelpersuite: Video loading/combining.

3. Workflow Structure

• Group 1: Input Processing

  • VHS_LoadVideo: Loads video (832x480, 16fps).

  • DepthAnythingV2Preprocessor: Extracts depth maps.

• Group 2: Prompt Generation

  • Auto path: MiniCPM_VQA_Polished → Translation → Text encoding.

  • Manual path: Direct prompt input.

• Group 3: Video Generation

  • WanVideoModelLoader → WanVideoSampler → WanVideoDecode.

4. Inputs & Outputs

• Inputs:

  • Video: MP4 format (e.g., 10068-0228.mp4).

  • Prompts: Auto-generated or manual.

  • Resolution: 832x480 (adjustable via ImageResizeKJ).

• Output:

  • Anime-style video (MP4, H.264 encoded).

5. Notes

• Performance:

  • Requires ≥12GB VRAM; bf16 models recommended.

• Troubleshooting:

  • Missing umt5-xxl-enc-bf16? Download from HuggingFace.

• Compatibility:

  • NVIDIA GPU only (CUDA 12.1+).