Inference with RANDOM num_frames less than 49

[OrangeSodahub]

Hi, thanks for your great contributions! When I test the generations I found that if I change the num_frames not equal to 49 (e.g., 16 or 17), the generated videos are totally failed. Did you have any suggestions?

Another question is, I found your model is only compatitble for CogVideoX-5b, and if it is possible I extend it to CogVideoX1.5-5b?

Any advice would be really appreciated!

[skygoo2000]

Thank you for your question!
We recommend you interpolate your video to 49 frames before putting it into the DaS.

[OrangeSodahub]

@skygoo2000 Thanks for your useful suggestion! However, do you have any thoughts why the results were totally bad, if I must use num frames less than 49?

[skygoo2000]

Sorry to hear that.
Could you kindly provide more detailed information? For instance, the video and the prompt settings.
Additionally, you can also try repeating the last frame until the video reaches a length of 49 frames.

[OrangeSodahub]

@skygoo2000 Thanks for your useful suggestions. Here is an example, I set default heigth, width = (480, 720), and number of frames is 17 (not the default 49):

When I turn to 49, the quality becomes ok. That probably is an issue related to CogVideoX, but I still wonder if you experts have some thoughts.

Repeating the last frame is a feasible solution, however not efficient.