Bug Report: AutoNexting missed Parts after Playout Gateway downtime

[ianshade]

About me

This bug report is posted on behalf of SuperFly.tv

Observed Behavior

When dealing with a sequence of auto-nexting parts, which is usually the case in a looping Playlist, Playout Gateway going online after a period of its downtime causes all the parts that would have been played during that period to be ran through as fast as possible in a sequence of quick auto-nexts. When the parts are short and the downtime was long, it may take a considerable amount of time before Core catches up, which results in an unnecessary load on Core and the Gateway. I have not verified if it also causes flooding of controlled devices with lots of commands, possibly causing visual glitches, but I think it might.

The chain of events looks like this:

• Part A is playing, set to autonext to B at time 1000
• Playout gateway goes offline
• Playout gateway comes back online at time 5000
• Playout gateway tells core A finished playback at 1000
• Core generates a new timeline with B starting at 1000, running until 2000
• Playout gateway receives timeline, confirming B started at 1000
• Playout gateway immediately tells core that B has finished playback at 2000
• Core generates a new timeline with C starting at 2000, running until 3000
• and so on until it catches up until the current time

Expected Behavior

Not auto-nexting through all the Parts that would have played during downtime.
When the Playout Gateway goes back online, the Part that is set as Next and gets auto-nexted immediately could be treated as having started now. Alternatively a recovery step could be added, to perform necessary calculations and skip directly to the Part that would be playing if downtime didn't occur.

Version

Release51 of Sofie Core and Playout Gateway

Severity / Impact

No response

[Julusian]

When the Playout Gateway goes back online, the Part that is set as Next and gets auto-nexted immediately could be treated as having started now

Care will need to be taken here. Did playout-gateway go offline because it restarted (meaning that the part will start playing once it reconnects), or was there a network issue (meaning that the part did start playing when it was supposed to)

The main problem I see here is that it keeps taking into C, D, E as fast as possible. Core should be clever enough to realise that it has just created a part instance which is then marked as having started(and finished) before it was generated, which clearly cannot be accurate. If it were to avoid doing that, then this will become limited to at most a couple of takes.

[mint-dewit]

I expect this to be fixed as of this commit: e8826c1, the change in packages/job-worker/src/playout/timeline/rundown.ts is the relevant bit.