[Feature] FastVideo/SFWan2.2-I2V-A14B-Preview-Diffusers acts like T2V rather than I2V.

[JoJoistheBestOne]

Motivation

Dear authors,
recently, we use the FastVideo/SFWan2.2-I2V-A14B-Preview-Diffusers to inference in a causal way.
we found that the image the model used is in a way very similar to T2V, aka:

1. the image is firstly processed to the FastVideo/SFWan2.2-I2V-A14B-Preview-Diffusers with timestamp(0) to update the kv_cache.
2. then the FastVideo/SFWan2.2-I2V-A14B-Preview-Diffusers model will generate next chunk only in T2V mode (which means it use 16 channel noise latent as input, which is different with the wan2.2 i2v using 16 channel image latents and 4 mask channel and 16 channel noised latents, totally in 36 channel to process).

Thus, we wonder was the FastVideo/SFWan2.2-I2V-A14B-Preview-Diffusers trained in T2V model instead of I2V?

And, more,
is there any I2V way also in SF(causal)?

thanks

Related resources

No response

[JerryZhou54]

You are correct. This model is initialized from Wan2.2 T2V A14B, and distilled using text-only dataset and T2V objective. We did this because 1. We noticed that autoregressive T2V model distilled via Self-Forcing can directly do I2V inference after we tried I2V inference using Self-Forcing's 1.3b checkpoint. 2. We didn't have a high-quality large-scale image-prompt dataset back then.

[JoJoistheBestOne]

You are correct. This model is initialized from Wan2.2 T2V A14B, and distilled using text-only dataset and T2V objective. We did this because 1. We noticed that autoregressive T2V model distilled via Self-Forcing can directly do I2V inference after we tried I2V inference using Self-Forcing's 1.3b checkpoint. 2. We didn't have a high-quality large-scale image-prompt dataset back then.

thanks for your reply.

for the reason#1, the i2v inference you mentioned is the reference image to the kv-cache in the first round right?

It seems has no relation to the channel (36) which is used in the Wan2.1/Wan2.2 I2V model (the ckpt weight has 36 channel, and require the input data has 4 channel mask (1 or 0), and 16 image latent (padding with zero) , and 16 channel noise latent).

thanks again~

[JerryZhou54]

yes that's correct