[jszewczyk]

Hey

I have a question about many ways to load models in ComphUI like Load Diffusion Model, Load Checkpoint.

What advantage gives us weight dtype?

What’s the difference between loading Clip from Load Checkpoint and DualClipLoader?

What gives us that additional parameters?

Thanks!

 

[jszewczyk]

Why sometimes we are adding Load VAE to Sampler, if we have VAE from Flux Model?

Thanks!

• This reply was modified 1 week, 1 day ago by jszewczyk.

[Andrew]

Hi, we need three models to use a diffusion model like Stable Diffusion and Flux

Diffusion model – for denoising during sampling.
VAE – for converting the images between pixel and latent spaces.
CLIP – for encoding to text prompt for conditioning during sampling.
Some checkpoints include all three in a single checkpoint file. Even if others don’t, the Load Checkpoint node uses the default VAE and CLIP models.

In addition to the original VAE, there are improved or finetuned versions, although they are rare. You can use the Load VAE node to specify the VAE you want. Typically, the difference is minimal.

Likewise, you can load the CLIP models directly using a node. Some models, like Flux, use two text encoders. Using the Dual CLIP loader allows you to put different text prompts to different encoders. Some people swear to see a difference between the two, but this remains an under-explored area.

 

 

 

[Andrew]

Hi, we need three models to use a diffusion model like Stable Diffusion and Flux

• Diffusion model – for denoising during sampling.
• VAE – for converting the images between pixel and latent spaces.
• CLIP – for encoding to text prompt for conditioning during sampling.

Some checkpoints include all three in a single checkpoint file. Even if others don’t, the Load Checkpoint node uses the default VAE and CLIP models.

In addition to the original VAE, there are improved or finetuned versions, although they are rare. You can use the Load VAE node to specify the VAE you want. Typically, the difference is minimal.

Likewise, you can load the CLIP models directly using a node. Some models, like Flux, use two text encoders. Using the Dual CLIP loader allows you to put different text prompts to different encoders. Some people swear to see a difference between the two, but this remains an under-explored area.

 

 

 

[jszewczyk]

Generally, if I use FLUX better for doing modifications will be to add Lora Models than adding modifications by custom VAE and CLIP?

[jszewczyk]

Do you see any difference between KSampler and XLabs Sampler except the port for controlnet?

• This reply was modified 1 week ago by jszewczyk.
• This reply was modified 1 week ago by jszewczyk.

[Andrew]

LoRA should be the go-to method for modifying models. A LoRA can also modify CLIP, but the main effect is in modifying the diffusion model.

I haven’t done a comparison, but sampling is a pretty standard process. I don’t think they would do anything different.

[jszewczyk]

Thanks. Great, I tested and XLabs looks like be slower, I will use a standard sampler If I don’t need controlnet.

[Author]

Posts