[WIP] support for customizing LoRA weights through the sdapi

[wbruna]

This is still just an idea!

Since we just got support for multiple LoRAs, we could include LoRA customization on the API side, by:

• internally allowing the weights to be changed at generation time
• showing the preloaded LoRAs under /sdapi/v1/loras
• accepting just changing the weights of the preloaded LoRAs through the lora fileld at /sdapi/v1/txt2img and /sdapi/v1/img2img

I recently implemented support on my Python client script for the mainline sd-server implementation, so I have a reasonable idea about how complicated that would be. I'm also aware that the sd.cpp C API would have to be adapted to allow changing LoRA weights without reloading the models.

Do you think this would be worth implementing?

[LostRuins]

Does it have any implications on memory use or runtime file loading?

[wbruna]

For at_runtime LoRA mode, I believe it wouldn't change at all.

For immediately LoRA mode, it could mean higher memory usage: currently, the code could be unloading the weights right after applying, since they wouldn't be needed anymore (need to check the code to be sure). And to change the weights, we need them back in memory, either reloading from disk or keeping them around in RAM. Generation latency would also increase a bit, because we'd need to reapply the LoRAs (but only when the weight is changed).