Problem on T2M Evaluation with New Dataset

[voidffu]

Dear author,

Many thanks to the great work. I have some confusion about how to perform t2m_eval with a new dataset. Specifically, in the _get_t2m_evaluator function, the model requires creating t2m_textencoder, t2m_moveencoder, and t2m_motionencoder, and loading the pretrained t2m_checkpoint for subsequent metric calculations.

Since my new dataset has a different motion feature dimension (nfeats), and I have used a different text_encoder during training, does this mean I cannot load the pretrained T2M checkpoint you provided in Google Drive? Would I need to customize and retrain the T2M model?

If retraining is necessary, would it be possible for you to release the relevant code for T2M training? This part wasn't detailed in the paper, and I am a bit confused. Your clarification would be a great help and I will appreciate it.

[Jatinkalal]

Hey , what exactly is t2m_textencoder? Isn't the CLIP model used as the text encoder?
Iam figuring out what it is and will appreciate your help!.

[97KangSH]

Im also confused that point, too.

[kai-lan]

The pretrained T2M evaluators were originally released here. It used word embedding (used in t2m_textencoder) instead CLIP, because back then CLIP was not so popular. The evaluators were trained on HumanML3D data.

If you have a custom datasets, you will need train your own T2M evaluators.