Time sampling API.

[quelsolaar]

I have just rewritten the json Parser to handle numbers a more precise by not using integers, and now I'm thinking about how to expose the time math to the end user. The obvious functions i can think of are being able to query:

-Start point of a composeable.
-End point of a composeable.
-Lookup functions like: if I'm 3.43 seconds in to a composable, in what clip am i and how far in to it?

These functionalities should be accessible using double precision floats, and integer frames with frame-rates.

Can you think of other functionality that would be useful?

[ssteinbach]

Currently we have a couple of those functions, see:
https://github.com/PixarAnimationStudios/OpenTimelineIO/blob/c4916c7d6ade79be5db3061b343556576ee07257/opentimelineio/core/composition.py#L295

top_clip_at_time

And each_child takes a search range as well.

As far as getting the range of a composable, we have range_of_child and trimmed_range_of_child or (from the child) `trimmed_range_in_parent.

We're still learning the needs in this part of the API, though, open to suggestions and use cases!

[quelsolaar]

I'm working on the implementation of proper time management and one issue i have is that clips doesn't have a frame rate. If the media itself is a video file the FPS can be gotten from that, but that forces you to have the media available and readable and that can be very cumbersome. On top of that my suspicion is that many post and VFX houses deal in image file sequences and then there is no frame-rate to be had at all.

While time such as duration contains a frame rate, this doesn't correspond to the actual frame rate. Often the rate will not at all reflect the FPS. Example:

Lets imagine we have two tracks and we want the content of these to have the exact same length.The first contains a single clip of 50 frames of 24fps footage. The second track contains a sequence with two clips. the first being 20 frames of 29.97 FPS footage and the second clip is X number of frames of 30fps footage to make the sequence the same length as the first track. In this case my math makes the duration of the second clip a value of 24175 at a rate of 11988.

I digress.

The problem I'm having is that i think our APIs and tools needs functionality like "pick out the frame on time X" and right now this isn't possible without being able to open a media file formats. Even making a fool proof implementation of such a function would be impossible since OTIO is media file format agnostic and its therefor not possible to write an implementation that can handle all possible media formats.

[jminor]

In practice, this is why most NLEs force you to pick a frame rate for your project or each track. This is usually called the "edit rate" and it can be different than the rate of the underlying media. A simple example is when editing 48KHz audio at an edit rate of 24Hz to match your video. Sometimes these applications force variance in frame rate to be dealt with at ingest by resampling clips to conform to the one edit rate.

In OTIO we are less strict, to allow for timelines that come from different applications, but in general the time values in OTIO timelines, clips, etc. should be treated as the "edit rate" where creative choices about the timing are made. There is a concept of available_range on a media reference which can come from the underlying file, but we treat it as optional.

[quelsolaar]

If you have a sequence of images, do you use the available_range rate value as framerate?

[quelsolaar]

I have thought about this all night (The wonderful opportunities given by jetlag!) and I have come to the conclusion that frame rate should be defined by OTIO, not by the material. Essentially I think it comes down to that I believe that OTIO defines how footage should be used and the frame rate is an essential part of that. (Note that i am referring to the rate of the retrieval of frames from media in a clip, not the rate of the resulting sequence generated from the OTIO file)

My main arguments are:

-Ease of use. Being able to read just the OTIO file and be able to create an image sequence without accessing media is useful, and much simpler. Accessing meta data from media is hard, and assumes the media is available (this is not true for large archival systems.).

-OTIO correctly does not try to assume any knowledge of how media is stored by using URIs. This rule is broken by forcing OTIO applications to be able to parse all manner of URIs to do basic things such as computing the number of frames needed to complete a sequence, increases the burden of developers significantly.

-The medias original playback speed is not necessarily the desired output speed. 24fps movies are often cranked to 25fps for PAL territory TV formats. If output speed is defined by media this wont be possible with OTIO, something I would find broken. OTIO speed should override media speed.

-Image sequences are common and don't have FPS associated with them, so OTIO needs to define it.

-When we add Speed Ramps OTIO will need to define it anyway. Again OTIO defined speed ramps should obviusly override media speed.

-I think OTIO files should, for simplicity, never refer to the playback speed of the media. If no speed tag is given the default media FPS should be one. (One is a unusual FPS and therefor any omission is easily caught)

One can argue if the speed of a clip should be the "rate" of avalilable_range, or its own separate tag. Im not 100% sure, but I'm leaning towards having a separate value. It makes more sense to me if "available_range" indicates something optional that have been derived from the media, where the rate value may be the original rate as indicated by the media, and then separately speed is stored to indicate how the media is used. I also assume you want to be able to indicate the frame rate without knowing the available range.

[quelsolaar]

I wrote a experimental implementation of a rate value in all clips in my fork.

[jminor]

Let's move the discussion of int/int vs float rates to here: #190