Question about the position embedding.

[SanGilbert]

Many Thanks for the great work!
As the report says, the model use PE interpolation to better adapt to different reslotuions. But I failed to find any related code, would you kind to point out the PE Interpolation code? Beside, have you tried RoPE? RoPE may be more suitable for different resolutions.
Thanks!

[bubbliiiing]

CogVideo-Fun-5b uses the Rope embedding like CogVideoX-5b.

[SanGilbert]

Thanks, could you kind point out the related PE Interpolation code? I failed to find it.

[ShungJhon]

@SanGilbert I guess it is here
https://github.com/aigc-apps/CogVideoX-Fun/blob/1b0e754e6c25ea9024e603d5b5c45465d9cb990c/cogvideox/models/transformer3d.py#L298

[bubbliiiing]

I will close this issue first. If there is anything unclear, you can reopen it.