Skip to content

If the cpu offload ratio changes, do torch compile again? #10666

@m8rr

Description

@m8rr

Custom Node Testing

Your question

ratio changes for each run and a lot of time is spent on compilation.
probably seems normal, but is there any way to improve it?

Requested to load WAN21
loaded partially; 8163.12 MB usable, 8163.12 MB loaded, 1342.18 MB offloaded, lowvram patches: 0
100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [02:57<00:00, 88.77s/it]
Requested to load WanVAE
loaded completely; 270.18 MB usable, 242.03 MB loaded, full load: True
Prompt executed in 237.19 seconds
got prompt
loaded partially; 8120.12 MB usable, 8120.12 MB loaded, 1385.18 MB offloaded, lowvram patches: 0
100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [06:11<00:00, 185.62s/it]
Requested to load WanVAE
loaded completely; 278.26 MB usable, 242.03 MB loaded, full load: True
Prompt executed in 376.35 seconds
got prompt
loaded partially; 8120.12 MB usable, 8120.11 MB loaded, 1385.19 MB offloaded, lowvram patches: 0
100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [03:00<00:00, 90.34s/it]
Requested to load WanVAE
loaded completely; 276.26 MB usable, 242.03 MB loaded, full load: True
Prompt executed in 186.48 seconds
got prompt
loaded partially; 8118.12 MB usable, 8111.17 MB loaded, 1394.12 MB offloaded, lowvram patches: 0
100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [03:01<00:00, 90.76s/it]
Requested to load WanVAE
loaded completely; 276.26 MB usable, 242.03 MB loaded, full load: True
Prompt executed in 187.43 seconds
got prompt
loaded partially; 8118.12 MB usable, 8111.17 MB loaded, 1394.12 MB offloaded, lowvram patches: 0
100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:08<00:00,  4.49s/it]
Requested to load WanVAE
loaded completely; 359.20 MB usable, 242.03 MB loaded, full load: True
Prompt executed in 14.16 seconds
got prompt
loaded partially; 8118.12 MB usable, 8111.17 MB loaded, 1394.12 MB offloaded, lowvram patches: 0
100%|████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:08<00:00,  4.48s/it]
Requested to load WanVAE
loaded completely; 359.20 MB usable, 242.03 MB loaded, full load: True
Prompt executed in 13.82 seconds

Logs

Other

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    User SupportA user needs help with something, probably not a bug.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions