Add: Enhance interlace auto-detection: update frame-info structure and handling for sequence gaps, ensuring accurate cadence re-learning and logging.

Author: moleksy

doc/st40_validation_updates.md

@@ -9,7 +9,8 @@ This note captures all recent ST40/ST40p feature changes and the accompanying va
 - **Frame-info log surfacing:** When tests pass `log_frame_info=True`, the harness dumps the frame-info file and a summary directly into the test log (no artifact download needed).
 - **ST40P test mutation knobs:** The GStreamer TX plugin accepts `tx-test-mode` with helpers for `no-marker`, `seq-gap`, `bad-parity`, and `paced` (optionally with `tx-test-pkt-count` and `tx-test-pacing-ns`). These drive targeted negative/edge-path tests.
 - **Interlaced & split-mode coverage:** RX/TX can be flagged interlaced independently; mismatch fails fast. Split-mode plus marker handling is now covered in tests.
-- **Interlace auto-detect option:** RX can infer progressive vs interlaced cadence from RTP F bits (`rx-auto-detect-interlaced=true` in GStreamer, `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED` in C API). Use when cadence is unknown; RX updates `interlaced` after first field detection while still logging field_num/interlaced in frame-info.
+- **Interlace auto-detect option:** RX can infer progressive vs interlaced cadence from RTP F bits (`rx-auto-detect-interlaced=true` in GStreamer, `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED` in C API). Use when cadence is unknown; RX updates `interlaced` after first field detection while still logging `second_field`/`interlaced` in frame-info.
+- **Auto-detect reset on discontinuity:** A sequence gap (`seq_discont>0`) clears the interlace detection state; RX re-learns cadence from subsequent F bits and continues logging `second_field`/`interlaced` per field.
 - **Warnings when cadence is unknown:** Pipeline RX and the GStreamer RX plugin emit a warning if neither `rx-interlaced` nor auto-detect is set, to avoid silent progressive defaults.
 - **Integration safety nets:** New noctx integration tests for ST40 interlaced flows, and expanded GStreamer validation tests across single-host and dual-host (VF/VF) paths.
 
@@ -51,8 +52,10 @@ flowchart LR
 - Enable on RX when cadence is unknown: `rx-auto-detect-interlaced=true` in the GStreamer RX pipeline or set `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED` in `st40p_rx_ops.flags` (C API). Keep `rx-interlaced=false` when auto-detecting.
 - TX still needs to emit F bits (set `tx_interlaced=true` in GStreamer or `interlaced=true` in TX ops) so RX can learn cadence.
 - Warnings: RX logs a warning if both `rx-interlaced=false` and auto-detect are disabled. Expect a GST_WARNING from `mtl_st40p_rx` and a pipeline warning from `st40p` if cadence is unknown.
-- Frame-info fields: `field_num` and `interlaced` are populated once F bits are observed (`0x2/0x3` for interlaced fields, `0x0` for progressive). These are visible in both C API frame_info and GStreamer frame-info dumps.
+- Frame-info fields: `second_field` (bool) and `interlaced` are populated once F bits are observed (bit1 set for interlaced; `second_field=true` when F==0x3). These are visible in both C API frame_info and GStreamer frame-info dumps.
 - Expect detection log line when auto-detect flips interlaced on RX (`detected interlaced stream (F=0x2/0x3)`); downstream tests assert on these fields rather than relying on caps only.
+- Discontinuity handling: a session-level sequence hole (e.g., `tx-test-mode=seq-gap` or real loss) clears detection state; expect `seq_discont>0` in frame-info and a subsequent “detected interlaced stream” log once new F bits arrive. This proves re-learn after reset.
+- Build verification: RX attach logs should show `rx_ancillary_session_attach(... flags 0x4 ... auto)` when `rx-auto-detect-interlaced=true` propagates; if flags stay `0x0`, rebuild the GStreamer plugin and ensure the test uses the fresh `--gst-plugin-path`.
 
 ## API and plugin changes (developer-facing)
 
@@ -106,7 +109,7 @@ Across all single-host tests, frame-info is dumped and summarized in pytest logs
   - `st40i_split_loopback` — interlaced split-mode loopback with custom fps and framebuff; ensures split/interlace coexist.
   - `st40i_split_seq_gap_reports_loss` — crafts a manual RTP sequence gap and validates `seq_discont`/`seq_lost` are reported via the C API frame_info, mirroring the GStreamer gap test at a lower level.
 - [tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp](tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp)
-  - `st40p_rx_auto_detect_interlace` — TX flags interlaced, RX starts progressive with `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED`; asserts `interlaced`/`field_num` populate on RX frame_info.
+  - `st40p_rx_auto_detect_interlace` — TX flags interlaced, RX starts progressive with `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED`; asserts `interlaced`/`second_field` populate on RX frame_info.
 
 These integration tests exercise the C pipeline APIs directly (outside the GStreamer harness) to prove split-mode, parity, marker, and sequence reporting work end-to-end without additional context setup.
 

doc/validation_framework.md

@@ -125,7 +125,7 @@ The `tests/` directory contains test implementations organized by scenario type:
     - Split-mode packetized ANC with frame-info logging (sequence discontinuity, packet totals, RTP marker) and ring-size validation
     - Pacing sanity via RTP sender helpers and ramdisk-backed media fixtures (configure `ramdisk.media` in `configs/test_config.yaml`)
     - Redundant ST40p/ST40i GStreamer ANC cases with per-port seq-gap scheduling (real payloads, lifted packet caps) and frame-info checks for seq discontinuity/loss logging
-    - Interlace auto-detect on RX (`rx-auto-detect-interlaced` / `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED`) with warnings if neither interlaced nor auto-detect is set; frame-info now includes `field_num` and `interlaced` for detected cadence
+    - Interlace auto-detect on RX (`rx-auto-detect-interlaced` / `ST40P_RX_FLAG_AUTO_DETECT_INTERLACED`) with warnings if neither interlaced nor auto-detect is set; frame-info now includes `second_field` (bool) and `interlaced` for detected cadence, and detection resets on `seq_discont` before re-learning from subsequent F bits
   - Backend-specific tests (DMA, kernel socket, etc.)
   - Integration tests (FFmpeg, GStreamer)
 

ecosystem/gstreamer_plugin/README.md

@@ -503,7 +503,7 @@ its user data words so that downstream elements receive complete RFC8331 payload
 > **Note:** `rtp-ring-size` values are validated at runtime. If the supplied number is not a power of
 > two the element fails to initialize, matching the behavior enforced inside
 > `gst_mtl_st40p_rx_start()`.
-> **Interlace auto-detect:** When cadence is unknown, set `rx-auto-detect-interlaced=true` and leave `rx-interlaced=false`. The plugin warns if both are false. Frame-info includes `field_num` and `interlaced` once F bits are observed.
+> **Interlace auto-detect:** When cadence is unknown, set `rx-auto-detect-interlaced=true` and leave `rx-interlaced=false`. The plugin warns if both are false. Sequence discontinuities reset detection; frame-info shows `seq_discont` when it happens and `interlaced` re-populates after new F bits arrive.
 
 #### 5.2.2. Example GStreamer Pipeline for Reception
 

include/st40_pipeline_api.h

@@ -60,9 +60,8 @@ struct st40_frame_info {
   /** TAI timestamp measured right after the RTP packet for this frame was received */
   uint64_t receive_timestamp;
 
-  /** RTP F bits (field number) for this frame: 0b00 progressive, 0b10 first field,
-   * 0b11 second field. */
-  uint8_t field_num;
+  /** True if this frame represents the second interlaced field (F=0b11). */
+  bool second_field;
   /** True if the frame was flagged as interlaced (F bits indicate field 1/2). */
   bool interlaced;
 

lib/src/st2110/pipeline/st40_pipeline_rx.c

@@ -160,8 +160,9 @@ static int rx_st40p_rtp_ready(void* priv) {
     frame_info->rtp_timestamp = rtp_timestamp;
     frame_info->timestamp = rtp_timestamp;
     frame_info->epoch = 0;
-    frame_info->field_num = hdr->first_hdr_chunk.f & 0x3;
-    frame_info->interlaced = (frame_info->field_num & 0x2) ? true : false;
+    uint8_t f_bits = hdr->first_hdr_chunk.f & 0x3;
+    frame_info->interlaced = (f_bits & 0x2) ? true : false;
+    frame_info->second_field = frame_info->interlaced && (f_bits & 0x1);
     ctx->inflight_frame = framebuff;
     ctx->inflight_rtp_timestamp = rtp_timestamp;
   } else {
@@ -173,9 +174,12 @@ static int rx_st40p_rtp_ready(void* priv) {
 
     /* Update field metadata if a later packet carries interlace info */
     uint8_t pkt_field = hdr->first_hdr_chunk.f & 0x3;
-    if (!frame_info->field_num || frame_info->field_num != pkt_field) {
-      frame_info->field_num = pkt_field;
-      frame_info->interlaced = (pkt_field & 0x2) ? true : false;
+    bool pkt_interlaced = (pkt_field & 0x2) ? true : false;
+    bool pkt_second_field = pkt_interlaced && (pkt_field & 0x1);
+    if ((frame_info->interlaced != pkt_interlaced) ||
+        (frame_info->second_field != pkt_second_field)) {
+      frame_info->interlaced = pkt_interlaced;
+      frame_info->second_field = pkt_second_field;
     }
   }
 
@@ -428,7 +432,7 @@ static int rx_st40p_init_fbs(struct st40p_rx_ctx* ctx, struct st40p_rx_ops* ops)
     frame_info->seq_lost = 0;
     frame_info->rtp_marker = false;
     frame_info->receive_timestamp = 0;
-    frame_info->field_num = 0;
+    frame_info->second_field = false;
     frame_info->interlaced = false;
     frame_info->priv = framebuff;
 
@@ -608,7 +612,7 @@ int st40p_rx_put_frame(st40p_rx_handle handle, struct st40_frame_info* frame_inf
   frame_info->seq_lost = 0;
   frame_info->rtp_marker = false;
   frame_info->receive_timestamp = 0;
-  frame_info->field_num = 0;
+  frame_info->second_field = false;
   frame_info->interlaced = false;
   framebuff->stat = ST40P_RX_FRAME_FREE;
   ctx->stat_put_frame++;

lib/src/st2110/pipeline/st40_pipeline_tx.c

@@ -158,7 +158,7 @@ static int tx_st40p_frame_done(void* priv, uint16_t frame_idx,
   frame_info->epoch = meta->epoch;
   frame_info->rtp_timestamp = meta->rtp_timestamp;
   frame_info->interlaced = ctx->ops.interlaced;
-  frame_info->field_num = ctx->ops.interlaced ? (meta->second_field ? 0x3 : 0x2) : 0x0;
+  frame_info->second_field = ctx->ops.interlaced ? meta->second_field : false;
 
   mt_pthread_mutex_lock(&ctx->lock);
   if (ST40P_TX_FRAME_IN_TRANSMITTING == framebuff->stat) {
@@ -333,7 +333,7 @@ static int tx_st40p_init_fbs(struct st40p_tx_ctx* ctx, struct st40p_tx_ops* ops)
     frame_info->seq_lost = 0;
     frame_info->rtp_marker = false;
     frame_info->receive_timestamp = 0;
-    frame_info->field_num = 0;
+    frame_info->second_field = false;
     frame_info->interlaced = false;
 
     /* addr will be resolved later in tx_st40p_create_transport */

lib/src/st2110/st_rx_ancillary_session.c

@@ -223,6 +223,11 @@ static int rx_ancillary_session_handle_pkt(struct mtl_main_impl* impl,
   /* hole in seq id packets going into the session check if the seq_id of the session is
    * consistent */
   if (seq_id != (uint16_t)(s->session_seq_id + 1)) {
+    if (s->interlace_auto && s->interlace_detected) {
+      s->interlace_detected = false;
+      dbg("%s(%d,%d), reset interlace detect after seq discont %u->%u\n", __func__,
+          s->idx, s_port, s->session_seq_id, seq_id);
+    }
     dbg("%s(%d,%d), session seq_id %u out of order %d\n", __func__, s->idx, s_port,
         seq_id, s->session_seq_id);
     s->stat_pkts_out_of_order++;

tests/integration_tests/noctx/testcases/st40p_auto_detect_tests.cpp

@@ -20,14 +20,14 @@ class St40pAutoDetectStrategy : public FrameTestStrategy {
 
     if (info->interlaced) {
       saw_interlaced = true;
-      last_field = info->field_num;
-      EXPECT_TRUE(last_field == 0x2 || last_field == 0x3)
-          << "Unexpected field bits: " << static_cast<int>(last_field);
+      last_second_field = info->second_field;
+      second_field_sampled = true;
     }
   }
 
   bool saw_interlaced = false;
-  uint8_t last_field = 0;
+  bool last_second_field = false;
+  bool second_field_sampled = false;
 };
 
 }  // namespace
@@ -62,6 +62,6 @@ TEST_F(NoCtxTest, st40p_rx_auto_detect_interlace) {
   ASSERT_GT(handler->rxFrames(), 0u) << "No frames received";
   EXPECT_EQ(handler->txFrames(), handler->rxFrames()) << "TX/RX frame count mismatch";
   EXPECT_TRUE(strategy->saw_interlaced) << "Auto-detect did not see interlaced F bits";
-  EXPECT_TRUE(strategy->last_field == 0x2 || strategy->last_field == 0x3)
-      << "Auto-detected field bits invalid";
+  EXPECT_TRUE(strategy->second_field_sampled)
+      << "Auto-detect did not surface field cadence metadata";
 }

tests/validation/tests/single/gstreamer/anc_format/test_anc_format.py

@@ -1606,7 +1606,7 @@ def test_st40i_interlace_flag_mismatch(
 
 
 @pytest.mark.nightly
-def test_st40p_interlace_auto_detect(
+def test_st40p_interlace_auto_detect_reset(
     hosts,
     build,
     media,
@@ -1617,8 +1617,8 @@ def test_st40p_interlace_auto_detect(
     media_file,
 ):
     """
-    Validate that RX auto-detect accepts interlaced TX even when the receiver does not
-    declare interlace upfront, relying on RTP F bits to learn cadence.
+    Validate that RX auto-detect accepts interlaced TX and, after a forced sequence gap
+    reset, re-learns cadence via RTP F bits without explicit interlace hints.
 
     .. rubric:: Pass Criteria
     - Pipeline succeeds with TX interlaced and RX auto-detect enabled.
@@ -1651,6 +1651,8 @@ def test_st40p_interlace_auto_detect(
         tx_interlaced=True,
     )
 
+    frame_info_path = _frame_info_path(os.path.dirname(output_file_path))
+
     rx_config = GstreamerApp.setup_gstreamer_st40p_rx_pipeline(
         build=build,
         nic_port_list=interfaces_list[1],
@@ -1661,12 +1663,13 @@ def test_st40p_interlace_auto_detect(
         rx_framebuff_cnt=3,
         rx_interlaced=False,
         rx_auto_detect_interlaced=True,
+        frame_info_path=frame_info_path,
     )
 
     expectation = "RX auto-detect resolves interlaced TX without explicit cadence hint"
 
     try:
-        with _test_summary("test_st40p_interlace_auto_detect", expectation):
+        with _test_summary("test_st40p_interlace_auto_detect_reset", expectation):
             assert GstreamerApp.execute_test(
                 build=build,
                 tx_command=tx_config,
@@ -1679,9 +1682,72 @@ def test_st40p_interlace_auto_detect(
                 sleep_interval=5,
                 log_frame_info=True,
             )
+
+        # Force a sequence gap to reset auto-detect state and prove cadence re-learns
+        gap_frame_info_path = _frame_info_path(os.path.dirname(output_file_path))
+        tx_config_gap = GstreamerApp.setup_gstreamer_st40p_tx_pipeline(
+            build=build,
+            nic_port_list=interfaces_list[0],
+            input_path=input_file_path,
+            tx_payload_type=113,
+            tx_queues=4,
+            tx_framebuff_cnt=3,
+            tx_fps=50,
+            tx_did=67,
+            tx_sdid=2,
+            tx_interlaced=True,
+            tx_split_anc_by_pkt=True,
+            tx_test_mode="seq-gap",
+            tx_test_pkt_count=200,
+        )
+
+        rx_config_gap = GstreamerApp.setup_gstreamer_st40p_rx_pipeline(
+            build=build,
+            nic_port_list=interfaces_list[1],
+            output_path=output_file_path,
+            rx_payload_type=113,
+            rx_queues=4,
+            timeout=15,
+            rx_framebuff_cnt=3,
+            rx_interlaced=False,
+            rx_auto_detect_interlaced=True,
+            frame_info_path=gap_frame_info_path,
+        )
+
+        media_create.remove_file(output_file_path, host=host)
+
+        reset_expectation = "RX auto-detect re-learns cadence after seq gap reset with frame-info logged"
+
+        with _test_summary(
+            "test_st40p_interlace_auto_detect_reset_gap", reset_expectation
+        ):
+            assert GstreamerApp.execute_test(
+                build=build,
+                tx_command=tx_config_gap,
+                rx_command=rx_config_gap,
+                input_file=input_file_path,
+                output_file=output_file_path,
+                test_time=test_time,
+                host=host,
+                tx_first=False,
+                sleep_interval=5,
+                log_frame_info=True,
+            )
+
+            gap_log = run(f"cat {gap_frame_info_path}", host=host)
+            gap_entries = _parse_frame_info_entries(gap_log.stdout_text or "")
+            assert (
+                gap_entries
+            ), "Seq-gap auto-detect reset produced no frame-info entries"
+            assert any(
+                entry.get("seq_discont", 0) > 0 for entry in gap_entries
+            ), "Seq-gap auto-detect reset did not log any discontinuity after gap"
     finally:
         media_create.remove_file(input_file_path, host=host)
         media_create.remove_file(output_file_path, host=host)
+        media_create.remove_file(frame_info_path, host=host)
+        if "gap_frame_info_path" in locals():
+            media_create.remove_file(gap_frame_info_path, host=host)
 
 
 @pytest.mark.nightly