Add: implement interlace auto-detection for ST40P RX and update documentation

  add ST40/ST40p API flag and frame_info field_num/interlaced propagation

  wire gst st40p rx auto-detect property and warn when cadence is unknown

  add validation harness + tests for auto-detect; document usage and signals

Author: moleksy

doc/st40_validation_updates.md

@@ -9,6 +9,8 @@ This note captures all recent ST40/ST40p feature changes and the accompanying va
 - **Frame-info log surfacing:** When tests pass `log_frame_info=True`, the harness dumps the frame-info file and a summary directly into the test log (no artifact download needed).
 - **ST40P test mutation knobs:** The GStreamer TX plugin accepts `tx-test-mode` with helpers for `no-marker`, `seq-gap`, `bad-parity`, and `paced` (optionally with `tx-test-pkt-count` and `tx-test-pacing-ns`). These drive targeted negative/edge-path tests.
 - **Interlaced & split-mode coverage:** RX/TX can be flagged interlaced independently; mismatch fails fast. Split-mode plus marker handling is now covered in tests.
+- **Interlace auto-detect option:** RX can infer progressive vs interlaced cadence from RTP F bits (`rx-auto-detect-interlaced=true` in GStreamer, `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED` in C API). Use when cadence is unknown; RX updates `interlaced` after first field detection while still logging field_num/interlaced in frame-info.
+- **Warnings when cadence is unknown:** Pipeline RX and the GStreamer RX plugin emit a warning if neither `rx-interlaced` nor auto-detect is set, to avoid silent progressive defaults.
 - **Integration safety nets:** New noctx integration tests for ST40 interlaced flows, and expanded GStreamer validation tests across single-host and dual-host (VF/VF) paths.
 
 ## Data path (split ANC per RTP)
@@ -44,6 +46,14 @@ flowchart LR
 - **Negative-path observability:** In interlaced mode, a missing marker or sequence gap can occur on just one field. Frame-info shows the gap on that field only, which is why `test_st40i_split_mode_frame_info_logging` and the noctx `st40i_split_seq_gap_reports_loss` exist—to prove per-field accounting is accurate.
 - **Naming clarity:** The `p` in `st40p` refers to the pipeline API, not progressive-only transport. Interlaced support is a first-class, opt-in property (`tx-interlaced` / `rx-interlaced`) and is exercised in both GStreamer and C integration suites.
 
+## Interlace auto-detect: usage and expected signals
+
+- Enable on RX when cadence is unknown: `rx-auto-detect-interlaced=true` in the GStreamer RX pipeline or set `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED` in `st40p_rx_ops.flags` (C API). Keep `rx-interlaced=false` when auto-detecting.
+- TX still needs to emit F bits (set `tx_interlaced=true` in GStreamer or `interlaced=true` in TX ops) so RX can learn cadence.
+- Warnings: RX logs a warning if both `rx-interlaced=false` and auto-detect are disabled. Expect a GST_WARNING from `mtl_st40p_rx` and a pipeline warning from `st40p` if cadence is unknown.
+- Frame-info fields: `field_num` and `interlaced` are populated once F bits are observed (`0x2/0x3` for interlaced fields, `0x0` for progressive). These are visible in both C API frame_info and GStreamer frame-info dumps.
+- Expect detection log line when auto-detect flips interlaced on RX (`detected interlaced stream (F=0x2/0x3)`); downstream tests assert on these fields rather than relying on caps only.
+
 ## API and plugin changes (developer-facing)
 
 - **TX (GStreamer st40p):**
@@ -95,6 +105,8 @@ Across all single-host tests, frame-info is dumped and summarized in pytest logs
   - `st40i_split_multi_packet_roundtrip` — sends multiple ANC blocks per field (sizes 8/6/4) in split mode; expects RX accumulation and nonzero frames.
   - `st40i_split_loopback` — interlaced split-mode loopback with custom fps and framebuff; ensures split/interlace coexist.
   - `st40i_split_seq_gap_reports_loss` — crafts a manual RTP sequence gap and validates `seq_discont`/`seq_lost` are reported via the C API frame_info, mirroring the GStreamer gap test at a lower level.
+- [tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp](tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp)
+  - `st40p_rx_auto_detect_interlace` — TX flags interlaced, RX starts progressive with `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED`; asserts `interlaced`/`field_num` populate on RX frame_info.
 
 These integration tests exercise the C pipeline APIs directly (outside the GStreamer harness) to prove split-mode, parity, marker, and sequence reporting work end-to-end without additional context setup.
 
@@ -107,7 +119,8 @@ These integration tests exercise the C pipeline APIs directly (outside the GStre
    - Missing marker: `tx-test-mode=no-marker`
    - Bad parity: `tx-test-mode=bad-parity`
    - Paced burst: `tx-test-mode=paced tx-test-pkt-count=3 tx-test-pacing-ns=200000`
-4. Inspect log: look for `FrameInfo` lines followed by `FrameInfoSummary` to confirm packet counts and seq accounting.
+4. If cadence is unknown on RX, set `rx-auto-detect-interlaced=true` (GStreamer) or `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED` (C API); leave `rx-interlaced=false` so detection can flip it. Expect a warning if you omit both.
+5. Inspect log: look for `FrameInfo` lines followed by `FrameInfoSummary` to confirm packet counts and seq accounting.
 
 ## Signals to watch
 
@@ -119,7 +132,7 @@ These integration tests exercise the C pipeline APIs directly (outside the GStre
 
 - For split-mode validation, set RX `frame_info_path` (for logging) and a power-of-two `rx_rtp_ring_size`; the plugin rejects non power-of-two values.
 - Frame-info summaries rely on the regular expression in the harness; unexpected formats will fall back to raw dump only.
-- Interlaced mismatch between TX/RX fails the test early (before file compare).
+- Interlaced mismatch between TX/RX fails the test early unless RX uses auto-detect; a warning is emitted when neither interlaced nor auto-detect is set.
 
 ## Quick architecture view (tests + logging)
 
@@ -141,4 +154,8 @@ flowchart TD
 - APIs: [include/st40_api.h](include/st40_api.h), [include/st40_pipeline_api.h](include/st40_pipeline_api.h)
 - Plugin: [ecosystem/gstreamer_plugin/gst_mtl_st40p_tx.c](ecosystem/gstreamer_plugin/gst_mtl_st40p_tx.c), [ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.c](ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.c)
 - Pipeline impl: [lib/src/st2110/pipeline/st40_pipeline_tx.c](lib/src/st2110/pipeline/st40_pipeline_tx.c), [lib/src/st2110/pipeline/st40_pipeline_rx.c](lib/src/st2110/pipeline/st40_pipeline_rx.c)
-- Tests: [tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py](tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py), [tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py](tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py), [tests/integration_tests/noctx/testcases/st40i_tests.cpp](tests/integration_tests/noctx/testcases/st40i_tests.cpp)
+- Tests:
+  - [tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py](tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py)
+  - [tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py](tests/validation/tests/dual/gstreamer/anc_format/test_anc_format_dual.py)
+  - [tests/integration_tests/noctx/testcases/st40i_tests.cpp](tests/integration_tests/noctx/testcases/st40i_tests.cpp)
+  - [tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp](tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp)

doc/validation_framework.md

@@ -125,6 +125,7 @@ The `tests/` directory contains test implementations organized by scenario type:
     - Split-mode packetized ANC with frame-info logging (sequence discontinuity, packet totals, RTP marker) and ring-size validation
     - Pacing sanity via RTP sender helpers and ramdisk-backed media fixtures (configure `ramdisk.media` in `configs/test_config.yaml`)
     - Redundant ST40p/ST40i GStreamer ANC cases with per-port seq-gap scheduling (real payloads, lifted packet caps) and frame-info checks for seq discontinuity/loss logging
+    - Interlace auto-detect on RX (`rx-auto-detect-interlaced` / `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED`) with warnings if neither interlaced nor auto-detect is set; frame-info now includes `field_num` and `interlaced` for detected cadence
   - Backend-specific tests (DMA, kernel socket, etc.)
   - Integration tests (FFmpeg, GStreamer)
 

ecosystem/gstreamer_plugin/README.md

@@ -492,6 +492,7 @@ The `mtl_st40p_rx` plugin supports all pad capabilities (the data is not checked
 | timeout             | uint    | Timeout in seconds for blocking frame retrieval             | 0 to 300                    | 60            |
 | frame-info-path     | string  | Optional path to append frame info and sequence stats per frame | N/A                      | NULL          |
 | rx-interlaced       | boolean | Whether the incoming ancillary stream is interlaced         | TRUE/FALSE                  | FALSE         |
+| rx-auto-detect-interlaced | boolean | Enable RTP F-bit based interlace detection when cadence is unknown; leaves `rx-interlaced` optional and will warn if both are false. | TRUE/FALSE | FALSE |
 | output-format       | enum    | Serialization format for received ancillary data            | `raw-udw` / `rfc8331`       | `raw-udw`     |
 
 When `output-format` is set to `rfc8331`, each ancillary packet is serialized with an 8-byte
@@ -502,6 +503,7 @@ its user data words so that downstream elements receive complete RFC8331 payload
 > **Note:** `rtp-ring-size` values are validated at runtime. If the supplied number is not a power of
 > two the element fails to initialize, matching the behavior enforced inside
 > `gst_mtl_st40p_rx_start()`.
+> **Interlace auto-detect:** When cadence is unknown, set `rx-auto-detect-interlaced=true` and leave `rx-interlaced=false`. The plugin warns if both are false. Frame-info includes `field_num` and `interlaced` once F bits are observed.
 
 #### 5.2.2. Example GStreamer Pipeline for Reception
 

ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.c

@@ -128,6 +128,7 @@ enum {
   PROP_ST40P_RX_RTP_RING_SIZE,
   PROP_ST40P_RX_TIMEOUT,
   PROP_ST40P_RX_INTERLACED,
+  PROP_ST40P_RX_AUTO_DETECT_INTERLACED,
   PROP_ST40P_RX_OUTPUT_FORMAT,
   PROP_ST40P_RX_FRAME_INFO_PATH,
   PROP_MAX
@@ -346,6 +347,13 @@ static void gst_mtl_st40p_rx_class_init(Gst_Mtl_St40p_RxClass* klass) {
                            "Set to true if ancillary stream is interlaced", FALSE,
                            G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
 
+  g_object_class_install_property(
+      gobject_class, PROP_ST40P_RX_AUTO_DETECT_INTERLACED,
+      g_param_spec_boolean(
+          "rx-auto-detect-interlaced", "Auto detect interlaced cadence",
+          "Enable RTP F-bit based interlace auto-detection when cadence is unknown",
+          FALSE, G_PARAM_READWRITE | G_PARAM_STATIC_STRINGS));
+
   g_object_class_install_property(
       gobject_class, PROP_ST40P_RX_OUTPUT_FORMAT,
       g_param_spec_enum("output-format", "Output Format",
@@ -375,6 +383,7 @@ static void gst_mtl_st40p_rx_init(Gst_Mtl_St40p_Rx* src) {
   src->rtp_ring_size = DEFAULT_RTP_RING_SIZE;
   src->timeout_s = 60;
   src->interlaced = FALSE;
+  src->auto_detect_interlaced = FALSE;
   src->output_format = GST_MTL_ST40P_RX_OUTPUT_FORMAT_RAW_UDW;
   src->frame_info_path = NULL;
   src->frame_info_fp = NULL;
@@ -413,6 +422,9 @@ static void gst_mtl_st40p_rx_set_property(GObject* object, guint prop_id,
     case PROP_ST40P_RX_INTERLACED:
       src->interlaced = g_value_get_boolean(value);
       break;
+    case PROP_ST40P_RX_AUTO_DETECT_INTERLACED:
+      src->auto_detect_interlaced = g_value_get_boolean(value);
+      break;
     case PROP_ST40P_RX_OUTPUT_FORMAT:
       src->output_format = g_value_get_enum(value);
       break;
@@ -453,6 +465,9 @@ static void gst_mtl_st40p_rx_get_property(GObject* object, guint prop_id, GValue
     case PROP_ST40P_RX_INTERLACED:
       g_value_set_boolean(value, src->interlaced);
       break;
+    case PROP_ST40P_RX_AUTO_DETECT_INTERLACED:
+      g_value_set_boolean(value, src->auto_detect_interlaced);
+      break;
     case PROP_ST40P_RX_OUTPUT_FORMAT:
       g_value_set_enum(value, src->output_format);
       break;
@@ -483,6 +498,7 @@ static gboolean gst_mtl_st40p_rx_start(GstBaseSrc* basesrc) {
       src->rx_framebuff_cnt ? src->rx_framebuff_cnt : GST_MTL_DEFAULT_FRAMEBUFF_CNT;
   ops_rx.max_udw_buff_size = src->max_udw_size;
   ops_rx.flags = 0; /* Use non-blocking mode - blocking causes preroll timeout */
+  if (src->auto_detect_interlaced) ops_rx.flags |= ST40P_RX_FLAG_AUTO_DETECT_INTERLACED;
 
   GST_DEBUG_OBJECT(src, "RX START: framebuff_cnt=%d, max_udw_buff_size=%d",
                    ops_rx.framebuff_cnt, ops_rx.max_udw_buff_size);
@@ -498,6 +514,11 @@ static gboolean gst_mtl_st40p_rx_start(GstBaseSrc* basesrc) {
   GST_DEBUG_OBJECT(src, "RX START: rtp_ring_size=%d", ops_rx.rtp_ring_size);
 
   ops_rx.interlaced = src->interlaced;
+  if (!src->interlaced && !src->auto_detect_interlaced) {
+    GST_WARNING_OBJECT(src,
+                       "RX interlaced flag not set; use rx-auto-detect-interlaced=true "
+                       "if cadence is unknown");
+  }
 
   /* Optional frame info logging */
   if (src->frame_info_path && !src->frame_info_fp) {

ecosystem/gstreamer_plugin/gst_mtl_st40p_rx.h

@@ -72,6 +72,7 @@ struct _Gst_Mtl_St40p_Rx {
   guint rtp_ring_size;
   guint timeout_s;
   gboolean interlaced;
+  gboolean auto_detect_interlaced;
   gint output_format; /* GstMtlSt40pRxOutputFormat enum value */
   gchar* frame_info_path;
   FILE* frame_info_fp;

include/st40_api.h

@@ -106,6 +106,12 @@ struct st40_tx_test_config {
  * If enable the rtcp.
  */
 #define ST40_RX_FLAG_ENABLE_RTCP (MTL_BIT32(1))
+/**
+ * Flag bit in flags of struct st40_rx_ops.
+ * If set, lib will auto-detect progressive vs interlaced based on RTP F bits.
+ * The interlaced boolean in st40_rx_ops is ignored once detection completes.
+ */
+#define ST40_RX_FLAG_AUTO_DETECT_INTERLACED (MTL_BIT32(2))
 
 /**
  * Session type of st2110-40(ancillary) streaming
@@ -336,7 +342,7 @@ struct st40_tx_ops {
   enum st_fps fps;
   /** Mandatory. 7 bits payload type define in RFC3550 */
   uint8_t payload_type;
-  /** Mandatory. interlaced or not */
+  /** Mandatory unless ST40_RX_FLAG_AUTO_DETECT_INTERLACED is set. Interlaced or not. */
   bool interlaced;
 
   /** Optional. Synchronization source defined in RFC3550, if zero the session will assign

include/st40_pipeline_api.h

@@ -60,6 +60,12 @@ struct st40_frame_info {
   /** TAI timestamp measured right after the RTP packet for this frame was received */
   uint64_t receive_timestamp;
 
+  /** RTP F bits (field number) for this frame: 0b00 progressive, 0b10 first field,
+   * 0b11 second field. */
+  uint8_t field_num;
+  /** True if the frame was flagged as interlaced (F bits indicate field 1/2). */
+  bool interlaced;
+
   /** priv pointer for lib, do not touch this */
   void* priv;
 };
@@ -192,6 +198,12 @@ enum st40p_rx_flag {
    * Force the numa of the created session, both CPU and memory
    */
   ST40P_RX_FLAG_FORCE_NUMA = (MTL_BIT32(2)),
+  /**
+   * If set, lib will auto-detect progressive vs interlaced using RTP F bits. The
+   * st40p_rx_ops.interlaced field becomes optional and will be updated after
+   * detection.
+   */
+  ST40P_RX_FLAG_AUTO_DETECT_INTERLACED = (MTL_BIT32(3)),
   /** Enable the st40p_rx_get_frame block behavior to wait until a frame becomes
    available or timeout(default: 1s, use st40p_rx_set_block_timeout to customize)*/
   ST40P_RX_FLAG_BLOCK_GET = (MTL_BIT32(15)),
@@ -204,7 +216,7 @@ enum st40p_rx_flag {
 struct st40p_rx_ops {
   /** Mandatory. rx port info */
   struct st_rx_port port;
-  /** Mandatory. interlaced or not */
+  /** Mandatory unless ST40P_RX_FLAG_AUTO_DETECT_INTERLACED is set. interlaced or not */
   bool interlaced;
   /** Mandatory. the frame buffer count. */
   uint16_t framebuff_cnt;

lib/src/st2110/pipeline/st40_pipeline_rx.c

@@ -160,6 +160,8 @@ static int rx_st40p_rtp_ready(void* priv) {
     frame_info->rtp_timestamp = rtp_timestamp;
     frame_info->timestamp = rtp_timestamp;
     frame_info->epoch = 0;
+    frame_info->field_num = hdr->first_hdr_chunk.f & 0x3;
+    frame_info->interlaced = (frame_info->field_num & 0x2) ? true : false;
     ctx->inflight_frame = framebuff;
     ctx->inflight_rtp_timestamp = rtp_timestamp;
   } else {
@@ -168,6 +170,13 @@ static int rx_st40p_rtp_ready(void* priv) {
     if (!frame_info->receive_timestamp ||
         (frame_info->receive_timestamp > receive_timestamp))
       frame_info->receive_timestamp = receive_timestamp;
+
+    /* Update field metadata if a later packet carries interlace info */
+    uint8_t pkt_field = hdr->first_hdr_chunk.f & 0x3;
+    if (!frame_info->field_num || frame_info->field_num != pkt_field) {
+      frame_info->field_num = pkt_field;
+      frame_info->interlaced = (pkt_field & 0x2) ? true : false;
+    }
   }
 
   if (ctx->last_seq_valid[s_port]) {
@@ -321,6 +330,15 @@ static int rx_st40p_create_transport(struct mtl_main_impl* impl, struct st40p_rx
   ops_rx.payload_type = ops->port.payload_type;
   ops_rx.ssrc = ops->port.ssrc;
   ops_rx.interlaced = ops->interlaced;
+  if (ops->flags & ST40P_RX_FLAG_AUTO_DETECT_INTERLACED)
+    ops_rx.flags |= ST40_RX_FLAG_AUTO_DETECT_INTERLACED;
+
+  if (!ops->interlaced && !(ops->flags & ST40P_RX_FLAG_AUTO_DETECT_INTERLACED)) {
+    warn(
+        "%s(%d), rx-interlaced not set; enable ST40P_RX_FLAG_AUTO_DETECT_INTERLACED if "
+        "cadence is unknown\n",
+        __func__, ctx->idx);
+  }
 
   for (int i = 0; i < ops_rx.num_port; i++) {
     memcpy(ops_rx.ip_addr[i], ops->port.ip_addr[i], MTL_IP_ADDR_LEN);
@@ -410,6 +428,8 @@ static int rx_st40p_init_fbs(struct st40p_rx_ctx* ctx, struct st40p_rx_ops* ops)
     frame_info->seq_lost = 0;
     frame_info->rtp_marker = false;
     frame_info->receive_timestamp = 0;
+    frame_info->field_num = 0;
+    frame_info->interlaced = false;
     frame_info->priv = framebuff;
 
     dbg("%s(%d), init fb %u\n", __func__, idx, i);
@@ -588,6 +608,8 @@ int st40p_rx_put_frame(st40p_rx_handle handle, struct st40_frame_info* frame_inf
   frame_info->seq_lost = 0;
   frame_info->rtp_marker = false;
   frame_info->receive_timestamp = 0;
+  frame_info->field_num = 0;
+  frame_info->interlaced = false;
   framebuff->stat = ST40P_RX_FRAME_FREE;
   ctx->stat_put_frame++;
 

lib/src/st2110/pipeline/st40_pipeline_tx.c

@@ -157,6 +157,8 @@ static int tx_st40p_frame_done(void* priv, uint16_t frame_idx,
   frame_info->timestamp = meta->timestamp;
   frame_info->epoch = meta->epoch;
   frame_info->rtp_timestamp = meta->rtp_timestamp;
+  frame_info->interlaced = ctx->ops.interlaced;
+  frame_info->field_num = ctx->ops.interlaced ? (meta->second_field ? 0x3 : 0x2) : 0x0;
 
   mt_pthread_mutex_lock(&ctx->lock);
   if (ST40P_TX_FRAME_IN_TRANSMITTING == framebuff->stat) {
@@ -331,6 +333,8 @@ static int tx_st40p_init_fbs(struct st40p_tx_ctx* ctx, struct st40p_tx_ops* ops)
     frame_info->seq_lost = 0;
     frame_info->rtp_marker = false;
     frame_info->receive_timestamp = 0;
+    frame_info->field_num = 0;
+    frame_info->interlaced = false;
 
     /* addr will be resolved later in tx_st40p_create_transport */
     frame_info->priv = framebuff;

lib/src/st2110/st_header.h

@@ -1200,6 +1200,9 @@ struct st_rx_ancillary_session_impl {
   struct st_rx_ancillary_sessions_mgr* mgr;
   bool attached;
   struct st40_rx_ops ops;
+  bool interlace_auto;
+  bool interlace_detected;
+  bool interlace_interlaced;
   char ops_name[ST_MAX_NAME_LEN];
   struct st_rx_session_priv priv[MTL_SESSION_PORT_MAX];
   struct st_rx_ancillary_session_handle_impl* st40_handle;