Skip to content

Commit 9e551f5

Browse files
hakehuangJarmouniA
authored and
jhedberg
committed
twister: harness: add display harness
display harness to validate display content Signed-off-by: Hake Huang <[email protected]> Co-authored-by: Abderrahmane JARMOUNI <[email protected]>
1 parent c1d0dfd commit 9e551f5

File tree

16 files changed

+1204
-1
lines changed

16 files changed

+1204
-1
lines changed

‎scripts/pylib/display-twister-harness/camera_shield/README.rst‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
==============
2+
Display capture Twister harness
3+
==============
4+
5+
6+
Configuration example
7+
---------------------
8+
9+
.. code-block:: console
10+
11+
case_config:
12+
device_id: 0 # Try different camera indices
13+
res_x: 1280 # x resolution
14+
res_y: 720 # y resolution
15+
fps: 30 # analysis frame pre-second
16+
run_time: 20 # Run for 20 seconds
17+
tests:
18+
timeout: 30 # second wait for prompt string
19+
prompt: "screen starts" # prompt to show the test start
20+
expect: ["tests.drivers.display.check.shield"]
21+
plugins:
22+
- name: signature
23+
module: plugins.signature_plugin
24+
class: VideoSignaturePlugin
25+
status: "enable"
26+
config:
27+
operations: "compare" # operation ('generate', 'compare')
28+
metadata:
29+
name: "tests.drivers.display.check.shield" # finger-print stored metadata
30+
platform: "frdm_mcxn947"
31+
directory: "./fingerprints" # fingerprints directory to compare with, not used in generate mode
32+
duration: 100 # number of frames to check
33+
method: "combined" #Signature method ('phash', 'dhash', 'histogram', 'combined')
34+
threshold: 0.65
35+
phash_weight: 0.35
36+
dhash_weight: 0.25
37+
histogram_weight: 0.2
38+
edge_ratio_weight: 0.1
39+
gradient_hist_weight: 0.1
40+
41+
example zephyr display tests
42+
----------------------------
43+
44+
1. Setup camera to capture display content
45+
46+
- UVC compatible camera with at least 2 megapixels (such as 1080p)
47+
- A light-blocking black curtain
48+
- A PC host where camera connect to
49+
- DUT connected to the same PC host for flashing and serial console
50+
51+
2. Generate video fingerprints
52+
53+
- build and flash the known-to-work display app to DUT
54+
e.g.
55+
```
56+
west build -b frdm_mcxn947/mcxn947/cpu0 tests/drivers/display/display_check
57+
west flash
58+
```
59+
60+
- clone code
61+
```bash
62+
git clone https://github.com/hakehuang/camera_shield
63+
```
64+
65+
66+
- follow the instructions in the repo's README.
67+
- set the signature capture mode as below in config.yaml
68+
```yaml
69+
- name: signature
70+
module: .plugins.signature_plugin
71+
class: VideoSignaturePlugin
72+
status: "enable"
73+
config:
74+
operations: "generate" # operation ('generate', 'compare')
75+
metadata:
76+
name: "tests.drivers.display.check.shield" # finger-print stored metadata
77+
platform: "frdm_mcxn947"
78+
directory: "./fingerprints" # fingerprints directory to compare with not used in generate mode
79+
```
80+
81+
- Run generate fingerprints program outside the camera_shield folder
82+
83+
Note:
84+
On Ubuntu 24.04, you may need to do ```export QT_QPA_PLATFORM=xcb``` to resolve below error
85+
86+
```bash
87+
qt.qpa.plugin: Could not find the Qt platform plugin "wayland" in "~/camera_shield/.ven/lib/python3.12/site-packages/cv2/qt/plugins"
88+
```
89+
90+
```bash
91+
python -m camera_shield.main --config camera_shield/config.yaml
92+
```
93+
94+
video fingerprint for captured screenshots will be recorded in directory './fingerprints' by default
95+
96+
- set environment variable to "DISPLAY_TEST_DIR"
97+
98+
```bash
99+
DISPLAY_TEST_DIR=~/camera_shield/
100+
```
101+
102+
3. Run test
103+
```bash
104+
# export the fingerprints path
105+
export DISPLAY_TEST_DIR=<path to "fingerprints" parent-folder>
106+
107+
# Twister hardware map file settings:
108+
# Ensure your map file has the required fixture
109+
# in the example below, you need to have "fixture_display"
110+
111+
# Ensure you have installed the required Python packages for tests in scripts/requirements-run-test.txt
112+
113+
# Run detection program
114+
scripts/twister --device-testing --hardware-map map.yml -T tests/drivers/display/display_check/
115+
116+
```
117+
118+
Notes
119+
-----
120+
121+
1. When generating the fingerprints, they will be stored in folder "name" as defined in "metadata" from ``config.yaml`` .
122+
2. The DUT testcase name shall match the value in the metadata 'name' field of the captured fingerprint's config.
123+
3. You can put multiple fingerprints in one folder, it will increase compare time,
124+
but will help to check other defects.

‎scripts/pylib/display-twister-harness/camera_shield/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Copyright 2025 NXP
2+
#
3+
# SPDX-License-Identifier: Apache-2.0

‎scripts/pylib/display-twister-harness/camera_shield/config.yaml‎

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
case_config: {device_id: 0, fps: 30, res_y: 720, res_x: 1280, run_time: 20}
2+
plugins:
3+
- class: VideoSignaturePlugin
4+
config:
5+
dhash_weight: 0.25
6+
directory: ${DISPLAY_TEST_DIR}/./fingerprints
7+
duration: 100
8+
edge_ratio_weight: 0.1
9+
gradient_hist_weight: 0.1
10+
histogram_weight: 0.2
11+
metadata: {name: tests.drivers.display.check.shield, platform: frdm_mcxn947}
12+
method: combined
13+
operations: compare
14+
phash_weight: 0.35
15+
threshold: 0.65
16+
module: .plugins.signature_plugin
17+
name: signature
18+
status: enable
19+
tests:
20+
expect: [tests.drivers.display.check.shield]
21+
prompt: screen starts
22+
timeout: 30

‎scripts/pylib/display-twister-harness/camera_shield/main.py‎

Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
# Copyright (c) 2025 NXP
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
import importlib
6+
import io
7+
import os
8+
import sys
9+
import time
10+
from string import Template
11+
12+
import cv2
13+
import yaml
14+
15+
from camera_shield.uvc_core.camera_controller import UVCCamera
16+
from camera_shield.uvc_core.plugin_base import PluginManager
17+
18+
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding='utf-8')
19+
20+
21+
class Application:
22+
def __init__(self, config_path="config.yaml"):
23+
def resolve_env_vars(yaml_dict):
24+
"""Process yaml with Template strings for safer environment variable resolution."""
25+
if isinstance(yaml_dict, dict):
26+
return {k: resolve_env_vars(v) for k, v in yaml_dict.items()}
27+
elif isinstance(yaml_dict, list):
28+
return [resolve_env_vars(i) for i in yaml_dict]
29+
elif isinstance(yaml_dict, str):
30+
# Create a template and substitute environment variables
31+
template = Template(yaml_dict)
32+
return template.safe_substitute(os.environ)
33+
else:
34+
return yaml_dict
35+
36+
self.active_plugins = {} # Initialize empty plugin dictionary
37+
with open(config_path, encoding="utf-8-sig") as f:
38+
config = yaml.safe_load(f)
39+
self.config = resolve_env_vars(config)
40+
41+
os.environ["DISPLAY"] = ":0"
42+
43+
self.case_config = {
44+
"device_id": 0,
45+
"res_x": 1280,
46+
"res_y": 720,
47+
"fps": 30,
48+
"run_time": 20,
49+
}
50+
51+
if "case_config" in self.config:
52+
self.case_config["device_id"] = self.config["case_config"].get("device_id", 0)
53+
self.case_config["res_x"] = self.config["case_config"].get("res_x", 1280)
54+
self.case_config["res_y"] = self.config["case_config"].get("res_y", 720)
55+
self.case_config["fps"] = self.config["case_config"].get("fps", 30)
56+
self.case_config["run_time"] = self.config["case_config"].get("run_time", 20)
57+
58+
self.camera = UVCCamera(self.case_config)
59+
self.plugin_manager = PluginManager()
60+
self.load_plugins()
61+
self.results = []
62+
63+
def load_plugins(self):
64+
for plugin_cfg in self.config["plugins"]:
65+
if plugin_cfg.get("status", "disable") == "disable":
66+
continue
67+
module = importlib.import_module(plugin_cfg["module"], package=__package__)
68+
plugin_class = getattr(module, plugin_cfg["class"])
69+
self.active_plugins[plugin_cfg["name"]] = plugin_class(
70+
plugin_cfg["name"], plugin_cfg.get("config", {})
71+
)
72+
self.plugin_manager.register_plugin(plugin_cfg["name"], plugin_class)
73+
74+
def handle_results(self, results, frame):
75+
for name, plugin in self.active_plugins.items():
76+
if name in results:
77+
plugin.handle_results(results[name], frame)
78+
79+
def shutdown(self):
80+
self.camera.release()
81+
for plugin in self.active_plugins.values():
82+
self.results += plugin.shutdown()
83+
84+
def run(self):
85+
try:
86+
start_time = time.time()
87+
self.camera.initialize()
88+
for name, plugin in self.active_plugins.items(): # noqa: B007
89+
plugin.initialize()
90+
while True:
91+
ret, frame = self.camera.get_frame()
92+
if not ret:
93+
continue
94+
95+
# Maintain OpenCV event loop
96+
if cv2.waitKey(1) == 27: # ESC key
97+
break
98+
99+
results = {}
100+
for name, plugin in self.active_plugins.items():
101+
results[name] = plugin.process_frame(frame)
102+
103+
self.handle_results(results, frame)
104+
self.camera.show_frame(frame)
105+
frame_delay = 1 / self.case_config["fps"]
106+
if time.time() - start_time > self.case_config["run_time"]:
107+
break
108+
time.sleep(frame_delay)
109+
110+
except KeyboardInterrupt:
111+
print("quit by key input\n")
112+
finally:
113+
self.shutdown()
114+
115+
return self.results
116+
117+
118+
if __name__ == "__main__":
119+
app = Application()
120+
app.run()

‎scripts/pylib/display-twister-harness/camera_shield/plugins/__init__.py‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
# Copyright 2025 NXP
2+
#
3+
# SPDX-License-Identifier: Apache-2.0

0 commit comments

Comments
 (0)