Design the source preview feature [backend]

[leoll2]

Design how the feature to preview the source video feed could be implemented server-side.

[leoll2]

Some ideas:

• imagine the 'preview' as a special pipeline, with one source and no sinks nor intermediate processing stages
• only one pipeline may be active at any time, either a regular one or the special 'preview' pipeline
• a source can be assigned to the preview pipeline, just like any other pipeline
  • The endpoint to bind the source and enable the preview could be PATCH /api/pipelines/preview (body {"source_id": ...})
  • The same endpoint can be used to deactivate the preview mode (body {"source_id": null})
• the preview pipeline outputs to a dedicated WebRTC stream

[MarkRedeman]

imagine the 'preview' as a special pipeline, with one source and no sinks nor intermediate processing stages

What do you think of adding an "identiy" or "null" inference step? Where this step would just forward the image without any inference results? This way we might not need to a dd a special "preview" pipeline, instead we could use this inference step as a default when creating a new pipeline.
This might also solve a potential issue where multiple people (when using Geti Tune in a headless mode) want to preview different sources at the same time. (though this might not be supported by design)

[leoll2]

What do you think of adding an "identiy" or "null" inference step? Where this step would just forward the image without any inference results?

It seems more complex to implement. Regular pipelines have several other stages other than inference: data monitoring, model monitoring, data collection, dispatching, .... If the preview data follows the regular flow, then all these stages must be disabled/bypassed as well (-> additional orchestration complexity). Moreover, when the preview ends, you would also need to flush all the intermediate queues because such data is not supposed to be actually consumed by the pipeline.

multiple people (when using Geti Tune in a headless mode) want to preview different sources at the same time

This is a niche scenario, not considered in the requirements. Whether we end up supporting it or not, it shouldn't drive our design choices.

[itallix]

I've added a design document in a slightly modified MADR format. We can consider adopting this approach if it fits our needs. The simplest extension would be to add an additional thread that connects the frame queue with the RTC queue. However, there are some open questions that need to be addressed.
Please share your feedback and thoughts!