Guidance on PROGRAM-DATE-TIME drift at EXT-X-DISCONTINUITY boundaries

[joneriks7]

HLS Program Date Time (PDT) Handling During Ad Insertion - Continuous Timeline vs Origin Realignment

When splicing in ad segments, it's possible to drift away from the origin PDTs due to differences in segment lengths when removing origin segments and stitching in ad segments. We're trying to determine the best approach for PROGRAM-DATE-TIME handling when returning to main content: should we carry that drift value through the playlist (making each playlist publish its own continuous PDT timeline based on the sum of EXT-INF), or is it better to realign with the origin PDTs after the ad break/discontinuity?

Original content snippet:

#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:04.000Z
#EXTINF:4.004,
http://example.com/content-032.ts
#EXT-X-CUE-OUT:12.012
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:08.004Z
#EXTINF:4.004,
http://example.com/content-033.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:12.008Z
#EXTINF:4.004,
http://example.com/content-034.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:16.012Z
#EXTINF:4.004,
http://example.com/content-035.ts
#EXT-X-CUE-IN
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:20.016Z
#EXTINF:4.004,
http://example.com/content-036.ts

Option 1: Continuous PDT based on EXT-INF sum (12.012s of origin segments replaced by 12.0s of ad segments, 12ms negative drift):

#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:04.000Z
#EXTINF:4.004,
http://example.com/content-032.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:08.004Z
#EXTINF:2.0,
ad-segment-1.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:10.004Z <-- these PDTs are extrapolated when ad segments are spliced in
#EXTINF:2.0,
ad-segment-2.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:12.004Z 
#EXTINF:2.0,
ad-segment-3.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:14.004Z 
#EXTINF:2.0,
ad-segment-4.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:16.004Z 
#EXTINF:2.0,
ad-segment-5.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:18.004Z 
#EXTINF:2.0,
ad-segment-6.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:20.004Z  <-- continuous PDT, but 12ms behind origin timeline
#EXTINF:4.004,
http://example.com/content-036.ts

Option 2: Realign to origin PDT (creates 12ms gap due to 12.012s origin → 12.0s ads):

#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:04.000Z
#EXTINF:4.004,
http://example.com/content-032.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:08.004Z <-- these PDTs are extrapolated when ad segments are spliced in
#EXTINF:2.0,
ad-segment-1.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:10.004Z 
#EXTINF:2.0,
ad-segment-2.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:12.004Z 
#EXTINF:2.0,
ad-segment-3.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:14.004Z 
#EXTINF:2.0,
ad-segment-4.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:16.004Z 
#EXTINF:2.0,
ad-segment-5.ts
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:18.004Z 
#EXTINF:2.0,
ad-segment-6.ts
#EXT-X-DISCONTINUITY
#EXT-X-PROGRAM-DATE-TIME:2019-10-21T19:31:20.016Z  <-- 12ms GAP in continuous PDT, but PDT aligns with origin timeline
#EXTINF:4.004,
http://example.com/content-036.ts

My understanding was that Option 1 is the correct approach, but I'm unsure after re-reading the spec:

│ If the first EXT-X-PROGRAM-DATE-TIME tag in a Playlist appears after one or
more Media Segment URIs, the client SHOULD extrapolate backward from that tag (
using EXTINF durations and/or media timestamps) to associate dates with those
segments. To associate a date with any other Media Segment that does not have an
EXT-X-PROGRAM-DATE-TIME tag applied to it directly, the client SHOULD
extrapolate forward from the last EXT-X-PROGRAM-DATE-TIME tag appearing before
that segment in the Playlist.

│ One exception is permitted: the Server MAY introduce small (sub-second)
overlaps to account for drift between the encoder clock and some independently
produced date/time reference. The later segment MUST only partially overlap the
preceding segment. The client MUST resolve these ambiguous date/times in favor
of the later segment.

In general, what is the correct approach?

Are there any additional things to consider for handling separate audio (EXT-X-MEDIA) and video (EXT-X-STREAM-INF) playlists? Should they be aligned or treated separately?

Is there any threshold we should consider, for example if the accumulated delta between the origin and ad segments is 200-300ms, or even >1s?

[robwalch]

the Server MAY introduce small (sub-second) overlaps to account for drift between the encoder clock and some independently produced date/time reference.

Some drift is allowed. It is more critical for alignment between renditions and variants to remain accurate, since DateRange tags must be consistent across all playlists. For these reasons, I would recommend realigning to the origin, so that differing segment durations, especially at discontinuities, do not throw off alignment.

the client SHOULD extrapolate backward

Clients won't do this if you signal PDT on each segment. What is critical is that PDT is signalled at the start of each discontinuity and on the last segment in the playlist. This should apply accurate mapping of date-times to all segments in each discontinuity domain.

Is there any threshold we should consider, for example if the accumulated delta between the origin and ad segments is 200-300ms, or even >1s?

I haven't seen any client errors due to overlap or gaps. The date times applied to the later segments will take precedence over ranges that overlap earlier segments. This has no impact on playback or playlist parsing, but should be considered when using operations that depend on mapping to a specific date, such as seeking to a date or mapping a DateRange tag.