Instant LoRA workflow (ComfyUI)

Updated Categorized as Workflow Tagged , , , , 24 Comments on Instant LoRA workflow (ComfyUI)

We can put a face in Stable Diffusion using LoRA and dreambooth checkpoint models. But both require training a new model, which can be time-consuming. What if you can inject a face instantly at sampling without training?

This ComfyUI workflow copies the face of a person from input images. It can be used like a custom LoRA and checkpoint model, but no training is required.

It is fast and convenient!

It takes input images like these (images from the LoRA training dataset):

And you can generate images with any prompt.

Become a member of this site to see this content

Already a member? Log in here.

Avatar

By Andrew

Andrew is an experienced engineer with a specialization in Machine Learning and Artificial Intelligence. He is passionate about programming, art, photography, and education. He has a Ph.D. in engineering.

24 comments

  1. Hello Andrew,

    I am sure I downloaded all the files correctly and put them into right folder. however I am still getting error “IPAdapterUnifiedLoaderFaceID
    LoRA model not found.”

    any idea why this is happening?
    thanks
    Pavel

  2. Hi all.
    Unfortunately, an error occurred while executing IPAdapterUnifiedLoaderFaceID:

    IP adapter model not found. Does anyone know a solution?

    1. It’s complaining about not finding the ip adapter model. Can you confirm you have done the step below?

      Install IPAdapter model
      Download the Face ID Plus v2 IP-adapter model. Put the file in ComfyUI > models > ipadapter folder.

      1. “E:\Comfy\ComfyUI_windows_portable\ComfyUI\models\ipadapter\ip-adapter-faceid-plusv2_sd15.bin” Yes, the model is loaded.

        1. Mmm.. I just tested the workflow and it is working correctly. Have you installed all 3 model files? You should see three files loaded in the console output (The paths can be different)
          INFO: Clip Vision model loaded from stable-diffusion-webui\models/clip_vision\CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
          INFO: IPAdapter model loaded from ComfyUI\models\ipadapter\ip-adapter-faceid_sd15.bin
          INFO: LoRA model loaded from stable-diffusion-webui\models/Lora\ip-adapter-faceid_sd15_lora.safetensors

  3. Even after doing a new install it can not find it. I installed Python 3.11.9
    C:\Users\signu>.\python_embeded\python.exe –version
    The system cannot find the path specified.

  4. I re-installed py AND I GET THIOS NOW
    C:\Users\signu>.\python_embeded\python.exe -m pip install .\insightface-0.7.3-cp311-cp311-win_amd64.whl
    The system cannot find the path specified.

    1. You may have missed this step: Open the File Explorer App. Navigate to the ComfyUI_windows_portable folder. In the address bar, type cmd and press Enter.

      You should see C:\Users\…\ComfyUI_windows_portable> before running the pip install command.

  5. I can not find the IPAdapterApplyFaceID i fine many other but not the IPAdapterApplyFaceID.
    So i am not able to get this working missing two things the IPAdapterApplyFaceID and the Insight face all other are instaled.

    1. You should be able to download the json file and drop it to comfyui. The workflow is updated to use IPAdapter unified loader. You need to use ComfyUI manager to install the missing nodes: Manager > Install missing custom nodes.

  6. ComfyUI workflow does not work. Probably because of recent changes IPAdapter. It will be great if you could update and provide a working one

      1. Got this when I tried. Error on Unified Loader Box. May be some explanation of how workflow works ?
        ======================
        Error occurred when executing IPAdapterUnifiedLoaderFaceID:

        IPAdapter model not found.

        File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\execution.py”, line 151, in recursive_execute
        output_data, output_ui = get_output_data(obj, input_data_all)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\execution.py”, line 81, in get_output_data
        return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\execution.py”, line 74, in map_node_over_list
        results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py”, line 468, in load_models
        raise Exception(“IPAdapter model not found.”)

        1. Looks like it couldn’t find the IP adapter model. It uses “ip-adapter-faceid-plusv2_sd15.bin”. Make sure you have it in your ip-adapters folder with exactly the same name.

  7. Hello all,
    I have the message: No module named ‘insightface’
    I treid to install:
    pip install insightface==0.7.3
    but problem stays.
    Any ideas?
    Thanx a lot

  8. I tried reinstalling everything. My first issue has been resolved. Now I am seeing this error.
    ————————
    Error occurred when executing InsightFaceLoader:

    module ‘google’ has no attribute ‘protobuf’

    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py”, line 535, in load_insight_face
    from insightface.app import FaceAnalysis
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\__init__.py”, line 16, in
    from . import model_zoo
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\__init__.py”, line 1, in
    from .model_zoo import get_model
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\model_zoo.py”, line 11, in
    from .arcface_onnx import *
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\arcface_onnx.py”, line 10, in
    import onnx
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnx\__init__.py”, line 75, in
    from onnx import serialization
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnx\serialization.py”, line 22, in
    _Proto = TypeVar(“_Proto”, bound=google.protobuf.message.Message)

    Please help me fix this.

  9. Hi Andrew,
    I followed the procedure as it is.
    I get the following error.
    —————————————————–
    Prompt outputs failed validation.
    IPAdapterModelLoader:
    – Value not in list: ipadapter_file: ‘None’ not in []
    ——————————————————–
    I am unable to update the ipadapter_file from null to the desired file in UI. I see that I have the file present at the correct location.
    “C:\Users\XXX\ComfyUI_windows_portable\ComfyUI\models\ipadapter\ip-adapter-faceid-plusv2_sd15.bin”

    How can I fix this issue?

  10. If I’m understanding this correctly you aren’t actually making a LORA but instead something that acts similar for photos correct? I assume this is photomaker?

    1. That’s correct, face via conditioning. Apparently this is a thing now with multiple techs to choose from. photomaker is nice but it’s not the first one. this is from a slightly older one.

Leave a comment

Your email address will not be published. Required fields are marked *