1. Workflow Overview

This workflow leverages Alibaba’s WAN2.1 model with ReCamMaster technology to generate dynamic camera motion effects. Key features:

• Video Stylization: Transforms input videos into anime-style or custom motion effects.

• AI Camera Control: Simulates professional cinematography (e.g., zoom, pan) via ReCamMaster.

• Multimodal Input: Supports text prompts (Chinese/English), reference videos, and auto-generated camera paths.

Core Models:

• Wan2_1_kwai_recammaster_1_3B: Custom video model with motion control.

• UMT5-XXL: Multilingual text encoder for Chinese prompts.

• Florence-2-large-ft: Microsoft’s vision model for video analysis.

2. Key Nodes

Dependencies:

• Manual downloads:

  • Wan2_1_kwai_recammaster_1_3B_step20000_bf16.safetensors

  • Save to: ComfyUI/models/wan2.1/

3. Workflow Groups

Group 1: Video Input & Preprocessing

• Input: Reference video (e.g., export (2) (1).mp4).

• Steps:

  1. VHS_LoadVideo extracts frames.

  2. Florence2Run generates captions (e.g., "autumnal forest scene").

  3. ImageResizeKJ resizes to 512x768.

Group 2: Motion Effect Generation

• Core Nodes:

  • WanVideoReCamMasterDefaultCamera: Selects motion mode (default: "zoom_out").

  • WanVideoSampler: Renders frames with text prompts and camera paths.

  • VHS_VideoCombine: Exports final MP4 with motion effects.

4. Inputs & Outputs

• Inputs:

  • Video: MP4 file (480P recommended).

  • Prompts: e.g., "video of an autumnal forest scene".

  • Motion Mode: Dropdown selection (zoom/pan).

• Output:

  • MP4 video with camera effects (saved as WanVideo2_1_ReCamMaster_xxxx.mp4).

5. Notes

• VRAM: ≥12GB GPU recommended (use bf16 for lower usage).

• Debugging:

  • Missing model → Check wan2.1 folder path.

  • Motion failure → Verify WanVideoReCamMasterCameraEmbed parameters.

• Optimization:

  • Reduce output resolution (e.g., 360P) for lower VRAM.

  • Disable experimental features to speed up.