Lora keys not loaded

[Mark-papi]

ENVIRONMENT
Windows 10
GPU 1660 Super
32 gb ram

So i tried a lora model that i made, and i try to get results from prompts but i get an warning lora keys not loaded and the image is not the desired

[comfyanonymous]

Can you paste what these lora keys are?

[ra100]

Seems I have similar issue, I've used this to train my LoRA:
https://github.com/huggingface/diffusers/blob/main/examples/text_to_image/README_sdxl.md

and using text2img from that page yields completely different results, as if loading in ComfyUI

This works fine:

from diffusers import DiffusionPipeline
import torch

model_path = "takuoko/sd-pokemon-model-lora-sdxl"
pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16)
pipe.to("cuda")
pipe.load_lora_weights(model_path)

prompt = "A pokemon with green eyes and red legs."
image = pipe(prompt, num_inference_steps=30, guidance_scale=7.5).images[0]
image.save("pokemon.png")

Can I mimic it in ComfyUI?
I'm don't have access to machine where I've run it right now, I'll post the message, missing keys later, I can share the model I trained with prompts too.

[ra100]

I'm getting these:

left over keys: dict_keys(['cond_stage_model.clip_l.transformer.text_model.embeddings.position_ids'])
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.0.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.1.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.2.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.3.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.4.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.5.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.6.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.7.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.8.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.9.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.10.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.11.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.12.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.13.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.14.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.15.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.16.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.17.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.18.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.19.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.20.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.21.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.22.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.23.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.24.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.25.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.26.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.27.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.28.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.29.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.30.self_attn.out_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.q_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.q_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.k_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.k_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.v_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.v_proj.lora_linear_layer.up.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.out_proj.lora_linear_layer.down.weight
lora key not loaded text_encoder_2.text_model.encoder.layers.31.self_attn.out_proj.lora_linear_layer.up.weight

and if it helps, here is produced pytorch_lora_weights.bin file https://drive.google.com/file/d/16-Mr7wZM2S4SJP06XX2UtFHgYC21192n/view?usp=sharing

[comfyanonymous]

Should be fixed with: c205832

[ra100]

Thank you, I've just tried it, no errors thrown and result looks as expected.

[Alx-AI]

Getting the same error:
lora key not loaded down_blocks.1.attentions.0.transformer_blocks.0.attn1.processor.to_k_lora.down.weight ...

with the following two Lora's:
The first is a 'wrong' Lora

The second is trained with this process and uploaded here

[comfyanonymous]

Make sure you update/update_comfyui.bat on the standalone or git pull and they probably will work.

[Alx-AI]

You were right, sorry.
thank you!

[ra100]

@Mark-papi is it fixed for you too?
I guess this issue could be closed.

[ra100]

Hi @comfyanonymous,
seems the issue is back, maybe because of the finetuning reimplementation using PEFT in diffusers (huggingface/diffusers#6225)

newly trained LoRAs don't work in comfy.

partial log:

lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_B.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_B.weight

Here is the full list of keys it has issues with:
keys.txt

He're the created LoRA: pytorch_lora_weights.safetensors.zip
(trigger should be "fsfrnsl man")

[Philomorph]

Hi @comfyanonymous, seems the issue is back, maybe because of the finetuning reimplementation using PEFT in diffusers (huggingface/diffusers#6225)

newly trained LoRAs don't work in comfy.

partial log:

lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.k_proj.lora_B.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_A.weight
lora key not loaded text_encoder.text_model.encoder.layers.0.self_attn.out_proj.lora_B.weight

I'm having this same issue, but I just installed ComfyUI and started using it yesterday. Was using A1111 with no problems and my LORAs were all downloaded a month or so ago - don't know when they were trained.

But then I downloaded a new LORA I hadn't tried before and the error went away and it seems to work.

[DavidPetit]

I am also having these issues. I used this notebook to train my Lora: https://github.com/huggingface/notebooks/blob/main/diffusers/SDXL_DreamBooth_LoRA_.ipynb

The Lora works fine when testing it using CLI or inside this notebook. However, in ComfyUI, I got a lot of those errors:
lora key not loaded unet.down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k.lora.down.weight

I hope it can be fixed someday because I really love using ComfyUI. :)

[Pierian-Data]

Am also getting the lora key not loaded set of messages on a new install of ComfyUI

[rainstormads]

I'm getting this error as well. Any fix to this?

[radames]

Just for reference got this issued resolved by converting my latest LoRA weights from diffusers to ComfyUI using this script https://github.com/huggingface/diffusers/blob/main/scripts/convert_diffusers_sdxl_lora_to_webui.py, maybe these dict changes could be integrated to ComfyUI?

https://github.com/huggingface/diffusers/blob/357855f8fca8d27beb0a3fde333f46db22f29391/src/diffusers/utils/state_dict_utils.py#L25-L113