v0.9.3.9 - bugfixes, better defaults

[bghira]

What's Changed

LyCORIS

• lycoris model card updates by @bghira in lycoris model card updates #820
• lycoris: resume should use less vram now by @bghira in lycoris: resume should use less vram now #835
• update lycoris to dev branch, 3.0.1dev3 by @bghira in update lycoris to dev branch, 3.0.1dev3 #828
• (DistributedDataParallel error - uninitialized parameters #644) temporarily block training on multi-gpu setup with quanto + PEFT, inform user to go with lycoris instead by @bghira in (#644) temporarily block training on multi-gpu setup with quanto + PEFT, inform user to go with lycoris instead #837

Flux

• Generate and store attention masks for T5 for flux by @AmericanPresidentJimmyCarter in Generate and store attention masks for T5 for flux #821
• backwards-compatible flux embedding cache masks by @bghira in backwards-compatible flux embedding cache masks #823
• remove clip warning by @bghira in remove clip warning #827
• fix quanto resuming by @bghira in fix quanto resuming #834
• (missing save code for Flux full finetune #864) add flux final export for full tune by @bghira in (#864) add flux final export for full tune #871
• flux: mobius-style training via augmented guidance scale by @bghira in flux: mobius-style training via augmented guidance scale #880
• track flux cfg via wandb by @bghira in track flux cfg via wandb #881

Misc features

• add schedulefree optim w/ kahan summation by @bghira in add schedulefree optim w/ kahan summation #857
• schedulefree: return to previous stable settings and add a new preset for aggressive training by @bghira in schedulefree: return to previous stable settings and add a new preset for aggressive training #862
• (EMA: support side-by-side validations with base model network #519) add side by side comparison with base model by @bghira in (#519) add side by side comparison with base model #865
• wandb gallery mode by @bghira in wandb gallery mode #872
• combine the vae cache clear logic with bucket rebuild logic by @bghira in combine the vae cache clear logic with bucket rebuild logic #879

Misc bug-fixes

• parquet add width and height columns by @frankchieng in parquet add width and height columns #825
• quanto: remove warnings about int8/fp8 confusion as it happened so long ago now; add warning about int4 by @bghira in quanto: remove warnings about int8/fp8 confusion as it happened so long ago now; add warning about int4 #826
• Fix caption_with_blip3.py on CUDA by @anhi in Fix caption_with_blip3.py on CUDA #833
• quanto + deepspeed minor fixes for multigpu training by @bghira in quanto + deepspeed minor fixes for multigpu training #839
• deepspeed sharding fixes by @bghira in deepspeed sharding #840
• fix: only run save full model on main process by @ErwannMillon in fix: only run save full model on main process #838
• (SDXL training cannot continue from latest #851) remove shard merge code on load hook by @bghira in (#851) remove shard merge code on load hook #853
• csv backend updates by @williamzhuk in csv backend updates #645
• csv fixes by @bghira in csv fixes #856
• fix validation image filename only using resolution from first img, and, unreadable/untypeable parenthesis by @bghira in fix validation image filename only using resolution from first img, and, unreadable/untypeable parenthesis #863
• sdxl: dtype inference followup fix by @bghira in sdxl: dtype inference followup fix #873

New Contributors

• @anhi made their first contribution in Fix caption_with_blip3.py on CUDA #833
• @ErwannMillon made their first contribution in fix: only run save full model on main process #838
• @williamzhuk made their first contribution in csv backend updates #645

Full Changelog: v0.9.8.3.2...v0.9.3.9

---

This discussion was created from the release v0.9.3.9 - bugfixes, better defaults.