1. Workflow Overview

This workflow transforms portrait photos into hand-drawn anime-style illustrations by combining Microsoft Florence-2's visual understanding with Stable Diffusion XL's generation capabilities, producing high-quality stylized outputs.

2. Core Models

• Florence-2-base: Multimodal model for image semantic analysis

• GhostbustersXL mix_1: Primary SDXL-based generation model

• sdxl-niji style_v1.0: LORA for Japanese anime style

• xinsir_controlnet_tile_sdxl_1.0: Controls structural details

3. Key Nodes

4. Workflow Structure

Group 1: Image Preprocessing

• Input: Original portrait (1129x2000)

• Process: LoadImage → ImageResize+(scale to 1800px height)

Group 2: Semantic Analysis

• Uses Florence2Run to generate detailed captions (e.g. "1girl, solo...")

Group 3: Stylized Generation

• Pipeline: Text encoding → ControlNet → KSampler(30 steps, dpmpp_2m)

• Key Style: "niji" anime LORA (0.4 strength)

Group 4: Face Refinement

• FaceDetailer with:

  • Detector: bbox/face_yolov8m

  • Local refinement: 512x512 areas, 26 steps

5. I/O Specifications

• Input Requirements:

  • Image: Half-body portrait (>1024px recommended)

  • Prompt: Must include quality tags like "masterpiece"

• Output:

  • Final image: PNG with metadata

  • Resolution: Auto-scaled from input

6. Critical Notes

1. Required Additions:

   • Florence-2 plugin (manual GitHub install)

   • Impact Pack (for FaceDetailer)

2. Hardware: 16GB+ VRAM recommended for 1800px processing

3. Troubleshooting:

   • Download Florence-2 model to /models/florence2

   • ControlNet models must be in /models/controlnet