What is LyCORIS and how to use them in Stable Diffusion

You have probably used LoRA models. How about LyCORIS models? They are great alternatives to LoRAs. In this guide, you will learn what they are, where to download them and how to use them.

What is a LyCORIS model?

LyCORIS stands for Lora beYond Conventional methods, Other Rank adaptation Implementations for Stable diffusion (Yeah, this is totally made up). Similar to LoRA, It is a collection of methods for making small changes to a Stable Diffusion checkpoint model.

What can you do with LyCORIS? They are used to modify a checkpoint model.

• Injecting a character: celebrity, fictional character, etc
• Modifying the style
• Modifying the clothing
• Modifying the background
• Adding an object like cars, buildings
• Adding an animal

Here’s an example with the Realistic Vision v2 model using a fashion LyCORIS Royal Gowns.

Difference between LoRA and LyCORIS

Well, that sounds like LoRA. What’s the difference?

LyCORIS and LoRA models aim to make minor adjustments to a Stable Diffusion model using a small file. Both modify the U-Net through matrix decomposition, but their approaches differ.

LoRA is the original method. LyCORIS is a collection of LoRA-like methods. They are LoCon, LoHa, LoKR, and DyLoRA.

LoRA

A LoRA model modifies the cross-attention layer of the U-Net. But the weights are stored efficiently.

Instead of storing whole matrices, it approximates a large matrix with two smaller matrices with a low rank, typically much smaller than the dimensions of the original matrix. The benefit of doing this is fewer parameters. For example, if the original matrix is 1,000 by 2,000, you need 2,000,000 parameters. But using a rank of 10, LoRA only needs 1,000 x 10 + 2,000 x 10 = 30,000 parameters. This is more than 60 times smaller.

That’s why LoRA models are so small. LoRA only stores the weight difference to the checkpoint model and only modifies the cross-attention layers of the U-Net of the checkpoint model.

LoCon (LyCORIS)

LoCon (LoRA for convolution network) is an extension of LoRA. In addition to the cross-attention layer, LoCon also modifies the convolution layers.

It is more powerful than LoRA since it modifies more parts of the model.

LoHa (LyCORIS)

LoHa (LoRA with Hadamard Product representation) is another LyCORIS method. It is inspired by FedPara, a federal learning method. LoHa has nothing to do with federal learning. It only uses its low-rank approximation method.

LoHA decomposes a large matrix into FOUR low-rank matrices. It is similar to LoRA, except it contains two LoRAs and combines them by Hadamard Product (bitwise multiplication).

It has the advantage of having fewer parameters, like LoRA, while still being theoretically more expressive.

LoKR (LyCORIS)

LoKR (LoRA with Kronecker product representation) is pretty much the same as LoHA except using Kronecker product.

DyLoRA (LyCORIS)

DyLoRA (Dynamic LoRA) is a different way of training LoRA. The benefit is the rank can be easily changed.

Seriously, what’s the difference between LoRA and LyCORIS?

LyCORIS is typically more expressive than LoRA. They can change the checkpoint model more.

This model conveniently offers a LoRA and a LyCORIS (Locon) version. The LyCORIS version captures more fine details when using the same training images.

Where to find LyCORIS models?

Civitai hosts the largest collection of Lycoris models for Stable Diffusion.

To browse Lycoris models:

1. Select Models on the top menu.
2. Click the Filter icon on the top right menu. Select LyCORIS only.

Software to use Lycoris models

AUTOMATIC1111 Web-UI is a free and popular Stable Diffusion software to use Lycoris models. You can use this GUI on Windows, Mac, or Google Colab.

AUTOMATIC1111 GUI supports LyCORIS models natively since version 1.5 (aka the SDXL update). You don’t need to use any extensions to load them. LyCORIS models are now collectively called LoRA models in the GUI.

Check out the Quick Start Guide if you are new to Stable Diffusion.

Downloading a Lycoris model

There are two ways to download a Lycoris model: (1) directly downloading from the Civitai website and (2) using the Civitai Helper extension.

Option 1: Direct download

Go to a LyCORIS model page on Civitai.

Confirm that the model Type is LYCORIS in the details section.

Download the LyCORIS model by clicking the Download button.

Put the LyCORIS file in the folder:

stable-diffusion-webui/models/Lora

Option 2: Use Civitai Helper

Alternatively, use the Civitai Helper extension to download a LyCORIS model. You will also get the cover image as a bonus.

On the Civitai Helper page, navigate to the Download Model section.

Put the URL of the model in Civitai URL.

Press Get Model info by Civitai URL.

Make a selection from these dropdown menus

• Sub-folder: /
• Model version: Select a variant you want.

Press Download Model.

Using the LyCORIS model

A LyCORIS model needs to be used with a Stable Diffusion checkpoint model. Each LyCORIS can only work with a specific type of Stable Diffusion model:

• v1.5
• v2
• SDXL

Follow these steps to use a LyCORIS model in AUTOMATIC1111.

1. Click the Extra Networks button under the Generate button.
2. Select the Lora tab. Click Refresh if you don’t see your model. Only models that are compatible with the selected Checkpoint model will show up. Make sure you have selected a compatible checkpoint model.
3. Click the LyCORIS model’s card. The phrase <lora:MODEL_NAME:1> should be added to the prompt.
4. Optionally adjust the number 1 in the Lora phrase. Increase it to enhance the effect. Decrease it to reduce.
5. Press Generate to generate images.

Some interesting LyCORIS models

Royal Gown

((Masterpiece, best quality)), ballgown,edgPreppy, a woman in a set of edgPreppy clothes, ribbons,frills, posing for a picture ,wearing edgPreppy with ballgown, Blonde Nadia with sunglasses and a choker

disfigured, deformed, ugly, nsfw

Model: Realistic Vision v2

The Royal Gown LyCORIS puts a beautiful gown on your model. Multiple options are available.

Iron Corset

4K, shiny, from the front, award winning photo, extremely detailed, amazing, fine detail, absurdres, by lee jeffries nikon d850 film stock photograph 4 kodak portra 400 camera f1.6 lens rich colors hyper realistic lifelike texture dramatic lighting unrealengine trending on artstation cinestill 800 tungsten, smiling, (War_Glam, a woman in armor ,armored, wearing War_Glam:1.2), corset, armor incrusted with several diamonds, a blonde european woman in a pink war_glam corset posing for a picturestanding up in a castle, black frills and ribbons , wearing a corset with a garter belt, pink fishnet stockings <lora:Iron_Corset:0.75>

disfigured, deformed, ugly, nsfw

Model: Realistic Vision v2

The Iron Corset LyCORIS lets you generate people wearing an iron corset.

Centaur

(centaur:1.4), white hair, red eyes, forehead, horse ,centaur,, lips, mature

disfigured, deformed, ugly, nsfw

Model: Anything v5

The Centaur LyCORIS generates a… Centaur.

Training LyCORIS

Use Kohya’s GUI to train your own LyCORIS.

Frequently asked questions

Where to put the LyCORIS model?

To use a LyCORIS model in AUTOMATIC1111, put it in stable-diffusion-webui/models/Lora.

You need AUTOMATIC1111 version 1.5 or higher.

Do I need an extension to use the LyCORIS models?

No, AUTOMATIC1111 has supported LyCORIS natively since version 1.5.

My LyCORIS model doesn’t show up in Extra Networks.

Ensure you have put it in the right location (stable-diffusion-webui/models/Lora.).

If you did, your LyCORIS might not be compatible with the checkpoint model selected (v1.5, v2 or SDXL). Check the LyCORIS model page or select a different checkpoint model.