Potential Application to Video Generation?

[Lydiatingting]

Really impressed by your Focus paper! The CPE formulation for keyframe
selection is elegant and effective.

Quick question: Do you think the approach could be adapted for keyframe
identification in video generation tasks? Would love to hear your
thoughts if you have a moment.

Thanks for the great work!

[ziruizhu]

Hi @Lydiatingting,

Thanks a lot for reading our paper and for the very creative idea!

Yes, we do think FOCUS could be adapted to video generation, especially for long videos. In an autoregressive setup, a FOCUS-style selector could be used to pick a small set of “important” past frames as keyframes, and use them as a compact summary of the history when generating future segments.

There is also related work on video diffusion where people follow a “sparse keyframes + interpolation/editing” pipeline for long videos. Conceptually, a FOCUS-like module could help decide which frames to promote as keyframes in such systems.

We’re currently exploring related directions on the generation side. If you have more thoughts or ideas here, I’d be very happy to discuss!

Best,
Zirui