This ComfyUI workflow combines three Stable Video Diffusion (SVD) videos in a text-to-video workflow. To use it, you simply need to enter each clip’s prompt.
You will learn:
- How to use a ComfyUI workflow (provided) to generate this video.
- Customization options.
You must be a member of this site to download the ComfyUI JSON workflow.
Table of Contents
Software
Stable Diffusion GUI
We will use ComfyUI, a node-based Stable Diffusion GUI. See the installation and beginner’s guide for ComfyUI if you haven’t been using it.
Step-by-step guide
Step 1: Load the ComfyUI workflow
Download the workflow JSON file below.
Drag and drop it to ComfyUI to load.
Step 2: Install missing nodes
Every time you try to run a new workflow, you may need to do some or all of the following steps.
- Install ComfyUI Manager
- Install missing nodes
- Update everything
Install ComfyUI Manager
Install ComfyUI manager if you haven’t done so already. It provides an easy way to update ComfyUI and install missing nodes.
To install this custom node, go to the custom nodes folder in the PowerShell (Windows) or Terminal (Mac) App:
cd ComfyUI/custom_nodes
Install ComfyUI by cloning the repository under the custom_nodes folder.
git clone https://github.com/ltdrdata/ComfyUI-Manager
Restart ComfyUI completely. You should see a new Manager button appearing on the menu.
If you don’t see the Manager button, check the terminal for error messages. One common issue is GIT not installed. Installing it and repeat the steps should resolve the issue.
Install missing custom nodes
To install the custom nodes that are used by the workflow but you don’t have:
- Click Manager in the Menu.
- Click Install Missing custom Nodes.
- Restart ComfyUI completely.
Update everything
You can use ComfyUI manager to update custom nodes and ComfyUI itself.
- Click Manager in the Menu.
- Click Updates All. It may take a while to be done.
- Restart the ComfyUI and refresh the ComfyUI page.
Step 3: Set the txt2img model
SDXL checkpoint
Download the Juggernaut XL model. Put it in ComfyUI > models > checkpoints.
Refresh the page and select the model in the Load Checkpoint dropdown menu.
SVD model
Download the SVD XT 1.1 model.
First, you need to fill out the agreement form here.
Then, download the model.
Put it in ComfyUI > models > checkpoints.
Press the Refresh button on the floating menu.
Select the model in the Image Only Checkpoint Loader (img2vid) dropdown menu.
Step 3: Generate video
Press Queue Prompt to generate the video. This is the MP4 video:
Customization
Prompts
Change the prompts to make your own video.
To save time, you can work on one prompt at a time to generate an image you like without generating the video.
To disable a video generation, select the SVD_img2vid_Conditioning node and press Ctrl+M (Mute) to disable it.
Change the seed of the KSampler in the txt2img section to generate a different image.
When you get an image you like, re-enable the SVD_img2vid_Conditioning node. You can do that by selecting the node and press Ctrl+M.
Video
It is normal to see a subpar video even with a good initial image. Change the video’s seed to generate a new video.
Adding video clips
You can add more video clips by duplicating the text-to-image and the image-to-video blocks.