Skip to content

videos larger than 2.1GB cause rerun to crash #10929

@astraw

Description

@astraw

Describe the bug

Large video files cause failures.

To Reproduce

Steps to reproduce the behavior:

  1. Acquire or generate a large (2.1GB or more) video. E.g. ffmpeg -f lavfi -i testsrc -t 36000 -pix_fmt yuv420p -c:v libx264 -g 1 large.mp4
  2. Attempt to open this with rerun: cargo run -r -- large.mp4
  3. See error:
thread 'rayon-1' panicked at 'offset overflow'
arrow-buffer-55.2.0/src/buffer/offset.rs:132

Expected behavior
Large video should display in rerun viewer.

Backtrace

arrow-buffer-55.2.0/src/buffer/offset.rs:132
stack backtrace:
   8: core::panicking::panic_fmt
   9: core::option::expect_failed
  10: <re_types::datatypes::_datatypes::blob::Blob as re_types_core::loggable::Loggable>::to_arrow_opt

Rerun version
current main branch ( now at 204025d ) and previous, including releases.

Additional context
This seems to be related to video data being stored as a re_chunk::Blob which currently is serialized using arrow's arrow::array::ListArray which takes i32 offset which are too small for blobs greater than 2.1GB.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions