Flux Hunyuan Text-to-Video workflow (ComfyUI)

This workflow combines an image generation model (Flux) with a video generation model (Hunyuan). Here’s how it works:

1. Generates an AI image using Flux
2. Automatically inputs the AI image into Hunyuan, the video generator
   • The AI image will be in the first frame of the video

Benefits:

1. Compared to the LTX model, both the Flux and the Hunyuan models generate higher quality videos
2. Faster to use. By merging the two separate processes of text-image and image-video, this workflow allows for you to accomplish both with just a click of a button!

You need to be a member of this site to download the ComfyUI workflow.

Sci-fi spaceship generated using the Flux Hunyuan workflow

Software

Workflow overview

1. Generate an image with the Flux.1 Dev model

2. Generate a video with the Hunyuan model

Step-by-step guide

Step 0: Update ComfyUI

Before loading the workflow, make sure your ComfyUI is up to date. The easiest way to do this is to use ComfyUI Manager.

Click the Manager button on the top toolbar.

Select Update ComfyUI.

Restart ComfyUI.

Step 1: Download the Flux AI model

Download the Flux1 dev FP8 checkpoint.

Put the model file in the folder ComfyUI > models > checkpoints.

Note: If you use my ComfyUI Colab Notebook, you don’t need to download the Flux model. Simply select the Flux1_dev model.

Step 2: Download the Hunyuan-Video model

You have some of these models if you have installed the Hunyuan Video text-to-image model.

Download hunyuan_video_image_to_video_720p_bf16.safetensors and put it in ComfyUI > models > diffusion_models.

Download clip_l.safetensors and llava_llama3_fp8_scaled.safetensors. Put them in ComfyUI > models > text_encoders.

Download hunyuan_video_vae_bf16.safetensors and put it in ComfyUI > models > vae.

Download llava_llama3_vision.safetensors and put it in ComfyUI > models > clip_vision.

Google Colab

If you use my ComfyUI Colab notebook, you don’t need to install the model files. They will be downloaded automatically.

Select the HunyuanVideo models before starting the notebook.

In the top menu, select Runtime > Change runtime type > L4 GPU. Save the settings.

Step 3: Load the workflow

Download the ComfyUI JSON workflow below.

Drag and drop the JSON file to ComfyUI.

Step 4: Install missing nodes

Click Manager > Install Missing Custom Nodes.

Install the nodes that are missing.

Restart ComfyUI.

Step 5: Revise the prompt

The prompt controls both the Flux image and the Hunyuan video. Change it to what you want to generate.

Step 6: Run the workflow

Press the Run button to generate a video.

Usage tips

It is best to treat the video generation as a 2-step process.

1. Refine the prompt to generate a good image.
2. Change the video’s seed to refine the video.

You can use the Fast Groups Muter to disable the video generation by disabling the Hunyuan Video group, as shown below.

Revise the prompt and change the seed to get a good image.

When you are happy with the image, turn the Hunyuan Video group back on.

Happy AI video generating!!!