Unlock Smooth Video Synthesis: A Step-by-Step Guide

CN
ComfyUI.org
2025-04-22 11:17:47

1. Workflow Overview

m9seu9ml6lnkh0jd24b7360686e8f30b1012b1eac3081109237225b26b49a9c6327a696dcd7dc365f7c.gif

This workflow generates high-quality character images and animates them with 360° rotation. Key features:

  • Character Generation: Creates realistic/anime characters from input images or text prompts.

  • Video Synthesis: Converts image sequences into smooth rotation animations.

  • Post-Processing: Includes upscaling, frame interpolation, and face detailing.

Core Models:

Model Name

Function

Source

Wan2.1-I2V-14B

Base video generation

NetEase Open Source

360_rotation_lora_flux

Controls rotation motion

Custom LoRA (Manual Install)

8x_NMKD-Faces

Face upscaling

CivitAI

GIMM-VFI

Frame interpolation

GitHub Plugin


2. Key Nodes & Installation

Critical Nodes:

  1. WanVideoModelLoader

    • Purpose: Loads base video models (e.g., Wan2.1-I2V-14B).

    • Install: Requires ComfyUI-WanVideoWrapper plugin via ComfyUI Manager.

    • Dependency: Download models from NetEase and place in models/wan_video.

  2. WanVideoLoraSelect

    • Purpose: Applies rotation LoRA (e.g., 360_rotation_lora_flux).

    • Install: Place LoRA files in models/loras.

  3. GIMMVFI_interpolate

    • Purpose: Frame interpolation for smoother videos.

    • Install: Clone ComfyUI-GIMM-VFI from GitHub.

  4. FaceDetailer

    • Purpose: Auto-refines facial details.

    • Install: Built-in node; requires UltralyticsDetectorProvider (YOLO model).


3. Workflow Structure

Group 1: Character Generation

  • Inputs:

    • Image (via LoadImage) or text prompt (e.g., "1girl, fashion model").

    • Parameters: Resolution (default 512x768), seed, steps (20).

  • Output: Generated character image (PNG).

Group 2: 360° Rotation Animation

  • Inputs:

    • Generated image + rotation LoRA.

    • Parameters: FPS (16), rotation control (via WanVideoBlockSwap).

  • Output: Rotation video (MP4).

Group 3: Post-Processing

  • Nodes:

    • CR Upscale Image: 8x upscaling.

    • GIMMVFI_interpolate: Frame interpolation to 32fps.

    • VHS_VideoCombine: Final video rendering.


4. Inputs & Outputs

Input Parameters: (json

{
  "prompt": "1girl, Asian fashion model",  
  "negative_prompt": "low quality, blurry",  
  "seed": -1,  // Random seed  
  "width": 512,  
  "height": 768  
}

Output:

  • Video file: /ComfyUI/output/XIAO_xxxx.mp4 (H.264).


5. Notes

  1. Model Dependencies:

    • Download Wan2.1 models (~14GB); VRAM ≥16GB (24GB recommended).

    • Missing LoRA triggers LORA not found; verify filenames.

  2. Optimization:

    • Run ComfyUI with --highvram flag.

    • Disable unused Groups (e.g., character generation) to save resources.

  3. Troubleshooting:

    • Resolution mismatch: Use ImageResizeKJ to unify dimensions.

    • VRAM overflow: Enable PurgeVRAM to clear cache.