We can put a face in Stable Diffusion using LoRA and dreambooth checkpoint models. But both require training a new model, which can be time-consuming. What if you can inject a face instantly at sampling without training?
This ComfyUI workflow copies the face of a person from input images. It can be used like a custom LoRA and checkpoint model, but no training is required.
It is fast and convenient!
It takes input images like these (images from the LoRA training dataset):
And you can generate images with any prompt.
Hey Andrew,
I want to create my own instagram fashion models and post them on social media. I’ve had a workflow before joining your course. It’s simple but I want to optimize it for my use case with your help: Generating my initial face -> IPAdapter (initial face) + random body (Depth) and generating those pictures. As model I use realvisV5.0 and no lora.
I want extremely photorealistic images and want to be faster as I have to search for identical body types in different positions. Would you recommend using a Lora and how would you optimize this workflow?
Really looking forwards for your answer.
Flux is the way to go with you want realism. But it won’t be fast.
A SDXL model should have a good balance between speed and quality.
Andrew, not sure if you allow this on your site, but I made a post on Civitai which may be of use to people here, especially those who are getting errors with Unified Loader.
I called it ‘IP Adapter Bundle’, all models/ViT/LoRAs in one place. If nothing else, at least it shows the tree structure, so people know exactly where each file needs to be placed (as of 2024):
https://civitai.com/models/907545/ip-adapter-bundle
I don’t mean to self-promote, this is just a small contribution from me to the AI community. In saying that, feel free to delete this comment if it breaks your T&C. 👍
I learned how to use IP Adapter thanks to you, by reading your IP Adapter tutorial, so thank you for the work you do. 👍
Hello Andrew,
I am sure I downloaded all the files correctly and put them into right folder. however I am still getting error “IPAdapterUnifiedLoaderFaceID
LoRA model not found.”
any idea why this is happening?
thanks
Pavel
Everything works great! Thank you!
Hi all.
Unfortunately, an error occurred while executing IPAdapterUnifiedLoaderFaceID:
IP adapter model not found. Does anyone know a solution?
It’s complaining about not finding the ip adapter model. Can you confirm you have done the step below?
Install IPAdapter model
Download the Face ID Plus v2 IP-adapter model. Put the file in ComfyUI > models > ipadapter folder.
“E:\Comfy\ComfyUI_windows_portable\ComfyUI\models\ipadapter\ip-adapter-faceid-plusv2_sd15.bin” Yes, the model is loaded.
Mmm.. I just tested the workflow and it is working correctly. Have you installed all 3 model files? You should see three files loaded in the console output (The paths can be different)
INFO: Clip Vision model loaded from stable-diffusion-webui\models/clip_vision\CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors
INFO: IPAdapter model loaded from ComfyUI\models\ipadapter\ip-adapter-faceid_sd15.bin
INFO: LoRA model loaded from stable-diffusion-webui\models/Lora\ip-adapter-faceid_sd15_lora.safetensors
Even after doing a new install it can not find it. I installed Python 3.11.9
C:\Users\signu>.\python_embeded\python.exe –version
The system cannot find the path specified.
I re-installed py AND I GET THIOS NOW
C:\Users\signu>.\python_embeded\python.exe -m pip install .\insightface-0.7.3-cp311-cp311-win_amd64.whl
The system cannot find the path specified.
You may have missed this step: Open the File Explorer App. Navigate to the ComfyUI_windows_portable folder. In the address bar, type cmd and press Enter.
You should see C:\Users\…\ComfyUI_windows_portable> before running the pip install command.
I can not find the IPAdapterApplyFaceID i fine many other but not the IPAdapterApplyFaceID.
So i am not able to get this working missing two things the IPAdapterApplyFaceID and the Insight face all other are instaled.
You should be able to download the json file and drop it to comfyui. The workflow is updated to use IPAdapter unified loader. You need to use ComfyUI manager to install the missing nodes: Manager > Install missing custom nodes.
i get .\python_embeded\python.exe –version
The system cannot find the path specified.
ComfyUI workflow does not work. Probably because of recent changes IPAdapter. It will be great if you could update and provide a working one
Hi, updated.
Got this when I tried. Error on Unified Loader Box. May be some explanation of how workflow works ?
======================
Error occurred when executing IPAdapterUnifiedLoaderFaceID:
IPAdapter model not found.
File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\execution.py”, line 151, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\execution.py”, line 81, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\execution.py”, line 74, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File “C:\Users\satis\OneDrive\Documents\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py”, line 468, in load_models
raise Exception(“IPAdapter model not found.”)
Looks like it couldn’t find the IP adapter model. It uses “ip-adapter-faceid-plusv2_sd15.bin”. Make sure you have it in your ip-adapters folder with exactly the same name.
Hello all,
I have the message: No module named ‘insightface’
I treid to install:
pip install insightface==0.7.3
but problem stays.
Any ideas?
Thanx a lot
If anybody has problem with insight. I found this solustion works:
Put there insightface-0.7.3-cp311-cp311-win_amd64.whl
( https://github.com/Gourieff/Assets/raw/main/Insightface/insightface-0.7.3-cp311-cp311-win_amd64.whl )
Create bat file:
@echo off
..\python_embeded\python.exe -m pip install insightface-0.7.3-cp311-cp311-win_amd64.whl
pause
I tried reinstalling everything. My first issue has been resolved. Now I am seeing this error.
————————
Error occurred when executing InsightFaceLoader:
module ‘google’ has no attribute ‘protobuf’
File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\execution.py”, line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File “C:\Users\rajes\ComfyUI_windows_portable\ComfyUI\custom_nodes\ComfyUI_IPAdapter_plus\IPAdapterPlus.py”, line 535, in load_insight_face
from insightface.app import FaceAnalysis
File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\__init__.py”, line 16, in
from . import model_zoo
File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\__init__.py”, line 1, in
from .model_zoo import get_model
File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\model_zoo.py”, line 11, in
from .arcface_onnx import *
File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\insightface\model_zoo\arcface_onnx.py”, line 10, in
import onnx
File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnx\__init__.py”, line 75, in
from onnx import serialization
File “C:\Users\rajes\ComfyUI_windows_portable\python_embeded\lib\site-packages\onnx\serialization.py”, line 22, in
_Proto = TypeVar(“_Proto”, bound=google.protobuf.message.Message)
Please help me fix this.
You can try running the bat file “update_comfyui_and_python_dependencies.bat” in the update folder.
Hi Andrew,
I followed the procedure as it is.
I get the following error.
—————————————————–
Prompt outputs failed validation.
IPAdapterModelLoader:
– Value not in list: ipadapter_file: ‘None’ not in []
——————————————————–
I am unable to update the ipadapter_file from null to the desired file in UI. I see that I have the file present at the correct location.
“C:\Users\XXX\ComfyUI_windows_portable\ComfyUI\models\ipadapter\ip-adapter-faceid-plusv2_sd15.bin”
How can I fix this issue?
Yes the location looks correct.
Try restarting comfyui completely.
If I’m understanding this correctly you aren’t actually making a LORA but instead something that acts similar for photos correct? I assume this is photomaker?
That’s correct, face via conditioning. Apparently this is a thing now with multiple techs to choose from. photomaker is nice but it’s not the first one. this is from a slightly older one.