This premade ComfyUI workflow combines the unprecedented image quality of Flux1 dev and local video generation with the Stable Video Diffusion (SVD) model.
You must be a member of this site to download the workflow JSON file.
Table of Contents
Software
We will use ComfyUI, an alternative to AUTOMATIC1111.
Read the ComfyUI installation guide and ComfyUI beginner’s guide if you are new to ComfyUI.
Take the ComfyUI course to learn ComfyUI step-by-step.
Step-by-step guide
Step 1: Download the Flux AI model
Download the Flux1 dev FP8 checkpoint.
Put the model file in the folder ComfyUI > models > checkpoints.
Step 2: Download the SVD model
Download the SVD XT model. Put it in the ComfyUI > models > checkpoints folder.
Step 3: Update ComfyUI
ComfyUI has native support for Flux starting August 2024. You need to update your ComfyUI if you haven’t already since then.
The easiest way to update ComfyUI is through the ComfyUI Manager. Click Manager > Update All.
Make sure to reload the ComfyUI page after the update — Clicking the restart button is not enough.
Step 4: Load Flux dev workflow
Download Flux1 Video workflow below.
Drag and drop the JSON file to ComfyUI.
Press Queue Prompt to generate a video.
Usage tips
It is best to treat the video generation as a 2-step process.
- Refine the prompt to generate a good image.
- Change SVD seed to refine the video.
Select the first node in the Image to Video section, which is “SVD_img2vid_Conditioning“.
Press Ctrl-M to mute the node. You should see the node gray out like the screenshot below.
Now you can change the prompt and seed, and run the workflow without generating the video, which takes a long time.
Once you get an image you like, unmute the node by selecting it and press Ctrl-M.
Run the workflow again.
If you don’t like the video, change the seed in the Sampler node of the Image-to-Video section to generate a new video.
