Test: Implemented OOP approach for tested applications in the pytest framework #1304

awilczyns · 2025-10-28T12:23:53Z

Added parent class for common parameters and execution logic across RxTxApp, FFmpeg, and Gstreamer. Includes RxTxApp child class implementation.

tests/validation/mtl_engine/config/app_mappings.py

tests/validation/mtl_engine/config/param_mappings.py

tests/validation/mtl_engine/rxtxapp.py

tests/validation/tests/single/st20p/integrity/test_integrity_refactored.py

tests/validation/tests/single/st22p/quality/test_quality_refactored.py

tests/validation/mtl_engine/application_base.py

staszczuk · 2025-12-19T09:28:16Z

tests/validation/mtl_engine/config/app_mappings.py

+    "unicast_tx_ip": "192.168.17.101",
+    "unicast_rx_ip": "192.168.17.102",
+    "multicast_tx_ip": "192.168.17.101",
+    "multicast_rx_ip": "192.168.17.102",
+    "multicast_destination_ip": "239.168.48.9",


Please don't hardcode IP addresses. Use IP pools instead.

andremiszcz · 2025-12-22T16:05:53Z

tests/validation/mtl_engine/application_base.py

+            rx_process = self.start_process(rx_command, build, test_time, rx_host)
+            time.sleep(sleep_interval)
+            tx_process = self.start_process(tx_command, build, test_time, tx_host)
+        tx_output = self.capture_stdout(tx_process, tx_name)


there will be no stdout_text untill process is not finished. Ensure that you wait, or kill the process before calling process.stdout_text attrubute

Add parent class for common parameters and execution logic across RxTxApp, FFmpeg, and Gstreamer. Includes RxTxApp child class implementation. Signed-off-by: Wilczynski, Andrzej <[email protected]>

…OOP approach. Signed-off-by: Wilczynski, Andrzej <[email protected]>

…n main, cleanup Signed-off-by: Wilczynski, Andrzej <[email protected]>

Signed-off-by: Wilczynski, Andrzej <[email protected]>

moleksy

LGTM after changes

DawidWesierski4

im generally fine with the overall

i think it would be good to tag all of the shortcomings of the new class-oriented fix

i believe used legacy functions should be just moved into some common folder and or made better and let in the new framework
Especially the add_interfaces irks me the wrong way as its wirdly uses same logic twice?!

I think setting @verified tag would be nice

i generally think this is nitpicking and all of those comments can be ignored as long as the refactored tests are working in NIghtly safely

Great work glhf

DawidWesierski4 · 2026-01-07T12:50:25Z

tests/validation/mtl_engine/config/param_mappings.py

+    "input_file": "st20p_url",  # for input files
+    "output_file": "st20p_url",  # for output files (RX)


i guess using input_file as general will not work when working with st30 ?
but im fine with this as is
i would just TODO -> and make the url more general so taht st40 and st30 can be "handled" by this in the future

DawidWesierski4 · 2026-01-07T12:55:49Z

tests/validation/mtl_engine/config/param_mappings.py

+    "width": "width",
+    "height": "height",


thaaat can be right or can be wrong
If we are taking those from caps its width -> for rx elements its rx-width
i don't think we are enabling any of gstreamer tests with this pr mby just adding a todo to unify this before unlocking the rx side would be enough ?
Or would this work oob ?

Generally todo
I would add the #TOFIX tag above if possible

DawidWesierski4 · 2026-01-07T13:02:46Z

tests/validation/mtl_engine/rxtxapp.py

+
+        # Build command line
+        cmd_parts = [
+            "sudo",


@staszczuk Should sudo be included?
With MFD will it work correctly?
I believe we have some MFD implementation handling the permissions of the process?

DawidWesierski4 · 2026-01-07T13:13:17Z

tests/validation/mtl_engine/rxtxapp.py

+        try:
+            add_interfaces(config, nic_port_list, test_mode)
+        except Exception as e:
+            logger.warning(
+                f"Legacy add_interfaces failed ({e}); falling back to direct assignment"
+            )
+            # Minimal fallback assignment - handle single or dual interface configs
+            config["interfaces"][0]["name"] = nic_port_list[0]
+            # Set IP addresses based on test mode using ip_pools directly
+            if test_mode == "unicast":
+                config["interfaces"][0]["ip"] = ip_pools.tx[0] if ip_pools.tx else None
+                config["tx_sessions"][0]["dip"][0] = (
+                    ip_pools.rx[0] if ip_pools.rx else None
+                )
+                config["rx_sessions"][0]["ip"][0] = (
+                    ip_pools.tx[0] if ip_pools.tx else None
+                )
+            elif test_mode == "multicast":
+                config["interfaces"][0]["ip"] = ip_pools.tx[0] if ip_pools.tx else None
+                config["tx_sessions"][0]["dip"][0] = (
+                    ip_pools.rx_multicast[0] if ip_pools.rx_multicast else None
+                )
+                config["rx_sessions"][0]["ip"][0] = (
+                    ip_pools.rx_multicast[0] if ip_pools.rx_multicast else None
+                )
+            elif test_mode == "kernel":
+                config["tx_sessions"][0]["dip"][0] = "127.0.0.1"
+                config["rx_sessions"][0]["ip"][0] = "127.0.0.1"
+
+            if len(nic_port_list) > 1:
+                config["interfaces"][1]["name"] = nic_port_list[1]
+                if test_mode == "unicast":
+                    config["interfaces"][1]["ip"] = (
+                        ip_pools.rx[0] if ip_pools.rx else None
+                    )
+                elif test_mode == "multicast":
+                    config["interfaces"][1]["ip"] = (
+                        ip_pools.rx[0] if ip_pools.rx else None
+                    )
+            elif direction in ("tx", "rx"):
+                # For single-direction single-interface, remove second interface
+                if len(config["interfaces"]) > 1:
+                    config["interfaces"] = [config["interfaces"][0]]


logic in error handling is the same line for lline from the legacy,
lets just delete this altogether ? Or try to not use legacy ?
i belive this is kinda confusing in currnet form
Please and thank you

DawidWesierski4 · 2026-01-07T13:23:58Z

tests/validation/tests/single/st22p/codec/test_codec_refactored.py

+@pytest.mark.nightly
+@pytest.mark.parametrize(
+    "media_file",
+    [yuv_files_422p10le["Penguin_1080p"]],
+    indirect=["media_file"],
+    ids=["Penguin_1080p"],
+)
+@pytest.mark.parametrize("codec", ["JPEG-XS", "H264_CBR"])
+def test_codec_refactored(
+    hosts,
+    build,
+    media,
+    setup_interfaces: InterfaceSetup,
+    test_time,
+    codec,
+    test_config,
+    prepare_ramdisk,
+    pcap_capture,
+    media_file,
+):
+    media_file_info, media_file_path = media_file
+    host = list(hosts.values())[0]
+    interfaces_list = setup_interfaces.get_interfaces_list_single(
+        test_config.get("interface_type", "VF")
+    )
+
+    app = RxTxApp(app_path="./tests/tools/RxTxApp/build")
+    app.create_command(
+        session_type="st22p",
+        test_mode="multicast",
+        nic_port_list=interfaces_list,
+        width=media_file_info["width"],
+        height=media_file_info["height"],
+        framerate=f"p{media_file_info['fps']}",
+        codec=codec,
+        quality="speed",
+        pixel_format=media_file_info["file_format"],
+        input_file=media_file_path,
+        codec_threads=2,
+        test_time=test_time,
+    )
+    app.execute_test(build=build, test_time=test_time, host=host, netsniff=pcap_capture)


this is just a comment but there is no support for codec meaning
1 . we don't check if the codec is "loaded" in the logs
2. we don't have suppoort for kahawai.json, -- we can't really change it from the framework

would be good to make the #TODO or #FIXME somwhere here

Sakoram · 2026-01-07T12:51:12Z

tests/validation/mtl_engine/rxtxapp.py

+class RxTxApp(Application):
+    """RxTxApp framework implementation (unified model)."""
+
+    def get_framework_name(self) -> str:


Can we call this get_app_name()? "framework" is confusing in this context.

Sakoram · 2026-01-07T13:02:47Z

tests/validation/mtl_engine/rxtxapp.py

+        # We rebuild the legacy shell for all session types but only populate the active one.
+
+        session_type = self.universal_params.get(
+            "session_type", UNIVERSAL_PARAMS["session_type"]


is using UNIVERSAL_PARAMS["session_type"] as default value makes any sense if self.universal_params is copy of UNIVERSAL_PARAMS? This suggests that self.universal_params can be edited after copy. is this true?

ok. I read some more, and I understand that those params are changed in set_universal_params(). Please rename set_universal_params() and self.universal_params to something like set_params() and self.params. The params are no longer "universal" after set_universal_params()

Sakoram · 2026-01-07T13:07:11Z

tests/validation/mtl_engine/application_base.py

+    @abstractmethod
+    def create_command(self, **kwargs):
+        """Populate self.command (+ self.config for frameworks that need it).
+
+        Implementations MUST:
+        - call self.set_universal_params(**kwargs)
+        - set self.command (string)
+        - optionally set self.config
+        - write config file immediately if applicable
+        They MAY return (self.command, self.config) for backward compatibility with existing tests.
+        """
+        raise NotImplementedError


instead of writing all of this in comment I guess it would be better to just implement this method and call from it an abstract version of _create_rxtxapp_command_and_config. in derived classes only this method should be implemented.

Sakoram · 2026-01-07T13:17:30Z

tests/validation/mtl_engine/config/app_mappings.py

+    "YUV422RFC4175PG2BE10": "yuv422p10le",  # RFC4175 to planar 10-bit LE
+}
+
+SESSION_TYPE_MAP = {


is this ever used? _get_session_type_from_config does not use it.

Sakoram · 2026-01-07T13:18:50Z

tests/validation/mtl_engine/config/app_mappings.py

+
+
+# Default port configuration by session type
+DEFAULT_PORT_CONFIG = {


is this ever used?

Sakoram · 2026-01-07T13:19:20Z

tests/validation/mtl_engine/config/app_mappings.py

+}
+
+# Default ST22p-specific configuration
+DEFAULT_ST22P_CONFIG = {


is this ever used?

Sakoram · 2026-01-07T13:19:41Z

tests/validation/mtl_engine/config/app_mappings.py

+}
+
+# Default FFmpeg configuration
+DEFAULT_FFMPEG_CONFIG = {


is this ever used?

Sakoram · 2026-01-07T13:19:58Z

tests/validation/mtl_engine/config/app_mappings.py

+}
+
+# Default GStreamer configuration
+DEFAULT_GSTREAMER_CONFIG = {"default_session_type": "mtl_st20p"}


is this ever used?

Sakoram · 2026-01-07T13:20:23Z

tests/validation/mtl_engine/config/app_mappings.py

+APP_NAME_MAP = {"rxtxapp": "RxTxApp", "ffmpeg": "ffmpeg", "gstreamer": "gst-launch-1.0"}
+
+# Format conversion mappings
+FFMPEG_FORMAT_MAP = {


is this ever used?

Sakoram · 2026-01-07T13:23:08Z

tests/validation/mtl_engine/rxtxapp.py

+from .config.param_mappings import RXTXAPP_CMDLINE_PARAM_MAP
+from .config.universal_params import UNIVERSAL_PARAMS
+from .execute import log_fail
+from .RxTxApp import (


Do I understand correctly that the idea of extxapp.py is to replace RxTxApp.py? If we want to remove RxTxApp.py I guess we need to copy those functions

Sakoram · 2026-01-07T13:41:08Z

tests/validation/tests/single/st20p/integrity/test_integrity_refactored.py

+    [
+        yuv_files_422rfc10["Penguin_720p"],
+        yuv_files_422rfc10["Penguin_1080p"],
+        pytest.param(yuv_files_422p10le["Penguin_720p"], marks=pytest.mark.nightly),


I'm not sure how this works but isn't @pytest.mark.nightly marking all of those params as nightly and adding marks=pytest.mark.nightly redundant?

Sakoram · 2026-01-07T13:42:12Z

tests/validation/tests/single/st22p/fps/test_fps_refactored.py

+    [
+        "p23",
+        "p24",
+        pytest.param("p25", marks=pytest.mark.nightly),


I'm not sure how this works but isn't @pytest.mark.nightly marking all of those params as nightly and adding marks=pytest.mark.nightly redundant?

awilczyns force-pushed the test-framework-refactor-rxtxapp branch 13 times, most recently from c225d8a to 1db3b68 Compare October 28, 2025 18:51

awilczyns marked this pull request as ready for review October 28, 2025 19:11

awilczyns requested review from DawidWesierski4, Falron98 and Sakoram as code owners October 28, 2025 19:11

DawidWesierski4 requested changes Oct 30, 2025

View reviewed changes

awilczyns force-pushed the test-framework-refactor-rxtxapp branch 9 times, most recently from e6991e5 to 5601ccf Compare December 17, 2025 15:44

moleksy reviewed Dec 17, 2025

View reviewed changes

tests/validation/mtl_engine/application_base.py Outdated Show resolved Hide resolved

awilczyns force-pushed the test-framework-refactor-rxtxapp branch from 5601ccf to 8c89785 Compare December 17, 2025 15:51

staszczuk requested changes Dec 19, 2025

View reviewed changes

andremiszcz requested changes Dec 22, 2025

View reviewed changes

awilczyns force-pushed the test-framework-refactor-rxtxapp branch 3 times, most recently from 3b7ca6c to e53377d Compare December 23, 2025 12:51

awilczyns added 6 commits January 7, 2026 09:44

Implemented OOP approach for tested applications in the pytest framework

8f177bf

Add parent class for common parameters and execution logic across RxTxApp, FFmpeg, and Gstreamer. Includes RxTxApp child class implementation. Signed-off-by: Wilczynski, Andrzej <[email protected]>

Added refactored tests for single host, st20p category. Based on new …

1ebe157

…OOP approach. Signed-off-by: Wilczynski, Andrzej <[email protected]>

Added refactored tests for single host, st22p category. Based on new …

f4d9739

…OOP approach. Signed-off-by: Wilczynski, Andrzej <[email protected]>

Addresed review comments, adjusted to the latest changes in pytests o…

ee507e3

…n main, cleanup Signed-off-by: Wilczynski, Andrzej <[email protected]>

Changes to support remote host test execution

afbf5db

Signed-off-by: Wilczynski, Andrzej <[email protected]>

Cleanup and removed hardocded IP addresses

dc0b347

Signed-off-by: Wilczynski, Andrzej <[email protected]>

awilczyns force-pushed the test-framework-refactor-rxtxapp branch from e53377d to dc0b347 Compare January 7, 2026 08:44

awilczyns enabled auto-merge (squash) January 7, 2026 08:44

moleksy self-requested a review January 7, 2026 12:36

moleksy reviewed Jan 7, 2026

View reviewed changes

moleksy approved these changes Jan 7, 2026

View reviewed changes

DawidWesierski4 approved these changes Jan 7, 2026

View reviewed changes

Sakoram reviewed Jan 7, 2026

View reviewed changes

andremiszcz approved these changes Jan 7, 2026

View reviewed changes

Sakoram reviewed Jan 7, 2026

View reviewed changes

		"input_file": "st20p_url", # for input files
		"output_file": "st20p_url", # for output files (RX)



		# Default port configuration by session type
		DEFAULT_PORT_CONFIG = {

Test: Implemented OOP approach for tested applications in the pytest framework #1304

Are you sure you want to change the base?

Test: Implemented OOP approach for tested applications in the pytest framework #1304

Uh oh!

Conversation

awilczyns commented Oct 28, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

moleksy left a comment

Choose a reason for hiding this comment

Uh oh!

DawidWesierski4 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sakoram Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Sakoram Jan 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Sakoram Jan 7, 2026 •

edited

Loading

Sakoram Jan 7, 2026 •

edited

Loading