How to use LTX Video 0.9.5 on ComfyUI

Published Categorized as Tutorial Tagged , , , , No Comments on How to use LTX Video 0.9.5 on ComfyUI

LTX Video 0.9.5 is an improved version of the LTX local video model. The model is very fast — it generates a 4-second video in 17 seconds on a consumer-grade GPU RTX4090. It’s not quite real-time, but very close.

In this article, I will cover:

  • The improvements over the previous version.
  • Text-to-video workflow
  • Image-to-video workflow
  • Fix the first and last frames in the video.

LTXV 0.9.5 Improvements

License

The good news is that the LTXV 0.9.5 version has a new Open RAIL-M license, which allows commercial use. You can host the model and use the generated videos for commercial purposes.

Text-to-video

Like the previous version, LTX Video 0.9.5 supports text-to-video. The video quality has improved.

LTX Text-to-video workflow.

Image-to-video

LTX-Video can use an image as the first frame and turn it into a video.

LTX Video 0.9.5 First frame image
First frame image.
“close up of 25yo beautiful woman face, start smiling”

Some videos from the image-to-video workflows can be quite hideous. I will give you some tips for generating good videos.

Fix the first and last frames

You can also set both the first and last frames of the video.

Text-to-image workflow

This workflow generates a 4-second video from a text description.

LTX Text-to-video workflow.

Step 0: Update ComfyUI

Before loading the workflow, make sure your ComfyUI is up-to-date. The easiest way to do this is to use ComfyUI Manager.

Click the Manager button on the top toolbar.

Select Update ComfyUI.

comfyui manager - update comfyui

Restart ComfyUI.

Step 1: Download models

Download ltx-video-2b-v0.9.5.safetensors and put it in ComfyUI > models > checkpoints.

Download t5xxl_fp16.safetensors and put it in ComfyUI > models > text_encoders.

Step 2: Load the workflow

Download the workflow below.

Drop it in ComfyUI.

Step 3: Install missing nodes

This workflow uses the Video Combine node to save the video as MP4. If you see red blocks, you don’t have the custom node that this workflow needs.

Click Manager > Install missing custom nodes and install the missing nodes.

Restart ComfyUI.

Step 4: Revise the prompt

Change the prompt to what you want to generate. LTXV works better with long and descriptive prompts. (You can use ChatGPT to expand a prompt. Put in “Expand the following video AI prompt:…”)

Step 5: Generate a video

Click the Queue button to generate the video.

queue button comfyui

Change the noise_seed value in the SamplerCustom node to generate a different video.

Image-to-video workflow

This workflow takes an input image and uses it as the first frame to generate a video. You also need to describe the video in the prompt.

“close up of 25yo beautiful woman face, start smiling”

Step 0: Update ComfyUI

Before loading the workflow, make sure your ComfyUI is up-to-date. The easiest way to do this is to use ComfyUI Manager.

Click the Manager button on the top toolbar.

Select Update ComfyUI.

comfyui manager - update comfyui

Restart ComfyUI.

Step 1: Download models

Download ltx-video-2b-v0.9.5.safetensors and put it in ComfyUI > models > checkpoints.

Download t5xxl_fp16.safetensors and put it in ComfyUI > models > text_encoders.

Step 2: Load the workflow

Download the workflow below.

Drop it in ComfyUI.

Step 3: Install missing nodes

This workflow uses the Video Combine node to save the video as MP4. If you see red blocks, you don’t have the custom node that this workflow needs.

Click Manager > Install missing custom nodes and install the missing nodes.

Restart ComfyUI.

Step 4: Upload an image

Upload an image in the Load Image node. The image will be used as the first frame of the video.

Step 5: Revise the prompt

Change the prompt to what you want to generate. The prompt should match the uploaded image and describe what will happen in the next 4 seconds.

LTXV works better with long and descriptive prompts. (You can use ChatGPT to expand a prompt. Put in “Expand the following video AI prompt:…”)

Step 6: Generate a video

Click the Queue button to generate the video.

queue button comfyui
“close up of 25yo beautiful woman face, start smiling”

Fix the first and last frames in the video

This workflow fixes the first and last frames of the video. To use it, you need two input images and a prompt.

Step 0: Update ComfyUI

Before loading the workflow, make sure your ComfyUI is up-to-date. The easiest way to do this is to use ComfyUI Manager.

Click the Manager button on the top toolbar.

Select Update ComfyUI.

comfyui manager - update comfyui

Restart ComfyUI.

Step 1: Download models

Download ltx-video-2b-v0.9.5.safetensors and put it in ComfyUI > models > checkpoints.

Download t5xxl_fp16.safetensors and put it in ComfyUI > models > text_encoders.

Step 2: Load the workflow

Download the workflow below.

Drop it in ComfyUI.

Step 3: Install missing nodes

This workflow uses the Video Combine node to save the video as MP4. If you see red blocks, you don’t have the custom node that this workflow needs.

Click Manager > Install missing custom nodes and install the missing nodes.

Restart ComfyUI.

Step 4: Upload an image

Upload the image of the first frame in the upper Load Image node and teh image of the last frame in the lower Load Image node.

Step 5: Revise the prompt

Change the prompt to what you want to generate. The prompt should match the uploaded images and describe what will happen in the next 4 seconds.

LTXV works better with long and descriptive prompts. (You can use ChatGPT to expand a prompt. Put in “Expand the following video AI prompt:…”)

Step 6: Generate a video

Click the Queue button to generate the video.

queue button comfyui

Tips

Generate a new video

Change the noise_seed in the SamplerCustom node to generate a new video.

Change video size

The default resolution is 768 x 512.

You can swap the width and height to generate a portrait video.

I don’t recommend changing the resolution. You will get lower-quality videos.

Video length

Change the length setting (measured in frames) to adjust the length of the video.

Generate more than 1 video at a time

The batch_size setting in the EmptyLTXVLatentVideo node controls how many videos are generated at a time. Change it to generate multiple videos.

Avoid difficult motions

It is too much to ask the LTXV 0.9.5 model to generate motions like a person putting on a jacket or videos with large body part movement. You will be disappointed!

Long prompts work better

The more you write in the prompt, the better the video. Use ChatGPT to expand the prompt with something like:

Expand the following AI video prompt: A very cool car transforms to a birthday cake

You may get details that do not match the input image, e.g., a blue car instead of a red one. Adjust accordingly.

Generate a few videos and pick the best one

Sometimes, the video is just bad. It is not your fault. Change the seed and generate a new one. It is fast.

Use FP8 text encoders to save space

If you use an Nvidia GPU card with RTX 4000 series or higher, you can use the text encoder in the FP8 format to save storage space. It is 5 GB instead of 10.

t5xxl_fp8_e4m3fn_scaled.safetensors

Reference

Andrew

By Andrew

Andrew is an experienced software engineer with a specialization in Machine Learning and Artificial Intelligence. He is passionate about programming, art, and education. He has a doctorate degree in engineering.

Leave a comment

Your email address will not be published. Required fields are marked *