1×8×8 Compression ratio model

[MoayedHajiAli]

Hello,

I could not find in the paper any experiments related to the 1×8×8 compression ratio model. Can you please point me to the section that discuss this experiment? Also, is that a model you are planning to release?
I am mostly interested to understand the effect of temporal compression over performance.

Thank you.

[hyc9]

Hello, when processing inputs with T=1, the encoder applies a 1×8×8 compression and no additional model parameters are required. In fact, LeanVAE automatically compresses the first frame of the input with 1×8×8, while the remaining frames are compressed with 4×8×8. If you need uniform compression 1×8×8 across all video frames, you can flatten the temporal dimension T into the batch size. We did not perform separate evaluations specifically for the first frame.

You can refer to the following usage:

# Encode an image → Get latent
latent = model.encode(image)  # (B, C, H, W) → (B, d, 1, H/8, W/8), where d=4 or 16

# Decode latent representation → Reconstruct image
image = model.decode(latent, is_image=True)  # (B, d, 1, H/8, W/8) → (B, C, H, W)