Instant LoRA workflow (ComfyUI)

1,175 views
Updated Categorized as Workflow Tagged , , , 6 Comments on Instant LoRA workflow (ComfyUI)

We can put a face in Stable Diffusion using LoRA and dreambooth checkpoint models. But both require training a new model, which can be time-consuming. What if you can inject a face instantly at sampling without training?

This ComfyUI workflow copies the face of a person from input images. It can be used like a custom LoRA and checkpoint model, but no training is required.

It is fast and convenient!

It takes input images like these (images from the LoRA training dataset):

And you can generate images with any prompt.

To read this content, become a member of this site

Already a member? Log in here.

Avatar

By Andrew

Andrew is an experienced engineer with a specialization in Machine Learning and Artificial Intelligence. He is passionate about programming, art, photography, and education. He has a Ph.D. in engineering.

6 comments

  1. I tried reinstalling everything. My first issue has been resolved. Now I am seeing this error.
    ————————
    Error occurred when executing InsightFaceLoader:

    module ‘google’ has no attribute ‘protobuf’

    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 152, in recursive_execute
    output_data, output_ui = get_output_data(obj, input_data_all)
    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 82, in get_output_data
    return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 75, in map_node_over_list
    results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
    File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py”, line 535, in load_insight_face
    from insightface.app import FaceAnalysis
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\__init__.py”, line 16, in
    from . import model_zoo
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\__init__.py”, line 1, in
    from .model_zoo import get_model
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\model_zoo.py”, line 11, in
    from .arcface_onnx import *
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\arcface_onnx.py”, line 10, in
    import onnx
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnx\__init__.py”, line 75, in
    from onnx import serialization
    File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnx\serialization.py”, line 22, in
    _Proto = TypeVar(“_Proto”, bound=google.protobuf.message.Message)

    Please help me fix this.

  2. Hi Andrew,
    I followed the procedure as it is.
    I get the following error.
    —————————————————–
    Prompt outputs failed validation.
    IPAdapterModelLoader:
    – Value not in list: ipadapter_file: ‘None’ not in []
    ——————————————————–
    I am unable to update the ipadapter_file from null to the desired file in UI. I see that I have the file present at the correct location.
    “C:\Users\XXX\ComfyUI_windows_portable\ComfyUI\models\ipadapter\ip-adapter-faceid-plusv2_sd15.bin”

    How can I fix this issue?

  3. If I’m understanding this correctly you aren’t actually making a LORA but instead something that acts similar for photos correct? I assume this is photomaker?

    1. That’s correct, face via conditioning. Apparently this is a thing now with multiple techs to choose from. photomaker is nice but it’s not the first one. this is from a slightly older one.

Leave a comment

Your email address will not be published. Required fields are marked *