Inference with flux lora

[xcltql666]

Trained a lora, but when I use it through the following code, an error occurs ValueError: Invalid LoRA checkpoint.

import torch
from diffusers import FluxPriorReduxPipeline, FluxPipeline
from diffusers.utils import load_image
import lycoris_lora
pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-Fill-dev", torch_dtype=torch.bfloat16)
pipe.enable_model_cpu_offload() #save some VRAM by offloading the model to CPU. Remove this if you have enough GPU power


pipe.load_lora_weights("/nas/shared/NLP_A100/xxx/Flux/xxx_output_dir_20241212/", weight_name="pytorch_lora_weights.safetensors")

prompt = """

"""


image = pipe(
    prompt,
    height=1024,
    width=1472,
    guidance_scale=3.5,
    num_inference_steps=50,
    max_sequence_length=512,
    generator=torch.Generator("cpu").manual_seed(0)
).images[0]


image.save("./xxx.png")

May I ask how I should use the trained lora correctly，thanks a lot.

[bghira]

you need to follow the LYCORIS document, this is not a PEFT LoRA, you can't load it with PEFT methods. it's a bit confusing, but Lycoris isn't truly supported by Diffusers directly.