Unlock Smooth Animation Transitions with AI-Powered Video Upscaling

CN
ComfyUI.org
2025-06-03 09:24:07

1. Workflow Overview

mbgba73ryl0i1s825fa27cd80a0e5e78a0be5f7c1f30cbedfa1e8bdb3219d15d7ee0317361767d98bf.gif

This workflow generates interpolated frames between start/end images and upscales the video. Key steps:

  1. Input start/end frames, generate intermediate frames via VACE model.

  2. Enhance video sequence using WanVideo sampler.

  3. Apply 4x super-resolution with NMKD-Siax model.
    Use Case: Smooth animation transitions (e.g., character movements, scene morphing).


2. Core Models

Model Name

Function

Source/Installation

WanVideo VACE

Frame interpolation

Requires ComfyUI-WanVideoWrapper plugin

WanVideo 2.1

Latent video generation

Built-in to plugin

4x_NMKD-Siax_200k

4x upscaling

Load via UpscaleModelLoader


3. Key Nodes

Node Name

Functionality

Installation

WanVideoVACEStartToEndFrame

Interpolates frames

Manual plugin install

WanVideoSampler

Controls video sampling

Built-in

ImageUpscaleWithModelBatched

Batch upscaling

Requires VideoHelperSuite

Dependencies:

  • Plugin ComfyUI-WanVideoWrapper (GitHub)

  • Models vace_preview_fp32.safetensors & Wan2_1_VAE_bf16.safetensors (manual download)


4. Workflow Groups

  • Group 1: Frame Input

    • Nodes: LoadImageImageResizeKJ

    • Output: Resized images (832x480).

  • Group 2: VACE Interpolation

    • Nodes: WanVideoVACEStartToEndFrameGetImageSizeAndCount

    • Output: 81 interpolated frames (5s at 16fps).

  • Group 3: Video Sampling

    • Nodes: WanVideoSamplerWanVideoDecode

    • Parameters: 20 steps, CFG=4.0, DPM++ scheduler.

  • Group 4: Upscaling

    • Nodes: UpscaleModelLoaderVHS_VideoCombine

    • Output: 4K MP4 video (CRF=19).


5. Inputs & Outputs

  • Inputs:

    • Start/end images (JPG/PNG).

    • Prompt (e.g., "A woman practicing yoga").

    • Frame count (default: 81), resolution (832x480).

  • Output:

    • Upscaled MP4 video (filename prefix xiao).


6. Notes

⚠️ VRAM: Minimum 8GB (RTX 3060+ recommended).
⚠️ Errors:

  • OOM error no attribute 'is_directml_enabled' → Reduce frames/resolution.

  • Upscale OOM → Disable ImageUpscaleWithModelBatched or switch to 2x model.
    ⚠️ Tips:

  • Start/end frames must match dimensions (e.g., 512x512).

  • Enable WanVideoTeaCache for >100 frames to save VRAM.


Demo Steps

  1. Replace LoadImage paths with your start/end frames.

  2. Modify prompts in WanVideoTextEncode.

  3. Click Queue Prompt. Output saves to ComfyUI/output.