Skip to content

Commit 298c449

Browse files
docs: update validation framework and quick start guide with RxTxApp build instructions and troubleshooting tips
1 parent 3974112 commit 298c449

File tree

3 files changed

+173
-100
lines changed

3 files changed

+173
-100
lines changed

doc/validation_framework.md

Lines changed: 121 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,13 @@
22

33
The Media Transport Library (MTL) Validation Framework provides comprehensive testing capabilities for various aspects of the MTL, including protocol compliance, performance, and integration testing.
44

5-
> **🚀 Quick Start**: For rapid setup, see [Validation Quick Start Guide](validation_quickstart.md)
5+
## Documentation Navigation
6+
7+
🚀 **Quick Setup**: [Validation Quick Start Guide](validation_quickstart.md) - Get running in 3 steps
8+
📁 **Local README**: [tests/validation/README.md](../tests/validation/README.md) - Quick reference and test categories
9+
🔧 **Build Guide**: [build.md](build.md) - MTL build instructions
10+
11+
---
612

713
## Overview
814

@@ -39,13 +45,31 @@ A shell script for generating test frames for video testing:
3945
- Configurable color patterns and test signals
4046
- Generates files like `ParkJoy_1080p.yuv`, test patterns, and various resolution formats
4147

48+
**Prerequisites**: Requires FFmpeg with text filters enabled.
49+
4250
**Usage**:
4351
```bash
4452
cd tests/validation/common # Must be in this directory
4553
./gen_frames.sh # Generates test media files for validation
4654
# Generated files will be available for test configuration
4755
```
4856

57+
**Troubleshooting**: If you get "No such filter: 'drawtext'" errors, install a complete FFmpeg build or skip media generation.
58+
59+
#### RxTxApp Test Tool
60+
61+
**CRITICAL**: Tests require the RxTxApp tool which is not built by the main MTL build process.
62+
63+
**Build Instructions** (required before running tests):
64+
```bash
65+
cd tests/tools/RxTxApp
66+
meson setup build
67+
meson compile -C build
68+
cd ../../..
69+
```
70+
71+
**Location**: After building, RxTxApp is available at `tests/tools/RxTxApp/build/RxTxApp`
72+
4973
**Supported Formats**:
5074
- Resolutions: 3840x2160, 1920x1080, 1280x720, 640x360
5175
- Pixel formats: yuv422p, yuv422p10le
@@ -133,8 +157,23 @@ cd ..
133157
# 3. Build MTL
134158
cd Media-Transport-Library
135159
./build.sh
160+
161+
# 4. Install MTL system-wide (REQUIRED for RxTxApp)
162+
sudo ninja install -C build
163+
sudo ldconfig
164+
165+
# 5. Build required test tools (CRITICAL for validation)
166+
cd tests/tools/RxTxApp
167+
meson setup build
168+
meson compile -C build
169+
cd ../../..
136170
```
137171

172+
> **⚠️ CRITICAL**:
173+
> - The RxTxApp tool is required for validation tests but not built by the main build process
174+
> - RxTxApp requires MTL to be installed system-wide to build successfully
175+
> - You must build it separately after installing MTL
176+
138177
For complete build instructions, see [doc/build.md](build.md).
139178

140179
#### 2. Other Prerequisites
@@ -147,33 +186,20 @@ For complete build instructions, see [doc/build.md](build.md).
147186

148187
### Environment Setup
149188

150-
> **⚠️ IMPORTANT**: Run all commands in the `tests/validation/` directory
189+
> **🚀 Quick Setup**: See [Validation Quick Start Guide](validation_quickstart.md) for streamlined setup steps.
190+
191+
For detailed setup:
151192

152-
1. Create and activate a Python virtual environment:
193+
1. Create Python virtual environment in `tests/validation/`:
153194

154195
```bash
155-
cd tests/validation # Must be in this directory!
196+
cd tests/validation
156197
python3 -m venv venv
157198
source venv/bin/activate
158-
```
159-
160-
**Note**: If you're using VS Code or other development tools that auto-configure Python environments, ensure you're using the correct Python interpreter. The tests require the packages from `tests/validation/requirements.txt`.
161-
162-
2. Install required dependencies:
163-
164-
```bash
165-
# Main framework requirements (run in tests/validation/)
166199
pip install -r requirements.txt
167-
168-
# Additional integrity test components (optional but recommended)
169200
pip install -r common/integrity/requirements.txt
170201
```
171202

172-
Verify installation:
173-
```bash
174-
python -m pytest --version
175-
```
176-
177203
### Configuration
178204

179205
#### Critical Configuration Steps
@@ -265,8 +291,6 @@ sudo ./script/nicctl.sh create_vf "0000:18:00.1" # Replace with your secondary
265291

266292
> **⚠️ CRITICAL**: Tests must be run as **root user**, not regular user. MTL validation framework requires root privileges for network operations.### Basic Test Execution
267293

268-
**⚠️ CRITICAL**: All tests must be run as **root user**. Regular users will fail.
269-
270294
### Run specific test with parameters
271295

272296
**Examples of running tests with specific parameters**:
@@ -284,19 +308,9 @@ pytest --topology_config=configs/topology_config.yaml --test_config=configs/test
284308
pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml "tests/dual/st30p/st30p_format/test_st30p_format.py::test_st30p_format[pcm24]"
285309
```
286310

287-
Run all tests:
311+
> **🚀 Quick Test Execution**: See [Quick Start Guide](validation_quickstart.md#3-run-tests) for basic test commands.
288312

289-
```bash
290-
cd tests/validation
291-
source venv/bin/activate # Activate virtual environment
292-
python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml
293-
```
294-
295-
Run smoke tests:
296-
297-
```bash
298-
python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml -m smoke
299-
```
313+
For comprehensive test execution:
300314

301315
### Running Specific Tests with Parameters
302316

@@ -313,13 +327,13 @@ python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=c
313327
Run specific test modules:
314328

315329
```bash
316-
python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml tests/single/st20p/test_st20p_rx.py
330+
sudo ./venv/bin/python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml tests/single/st20p/test_st20p_rx.py
317331
```
318332

319333
Run specific test cases with parameters:
320334

321335
```bash
322-
python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml "tests/single/st20p/fps/test_fps.py::test_fps[|fps = p60|-ParkJoy_1080p]"
336+
sudo ./venv/bin/python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml "tests/single/st20p/fps/test_fps.py::test_fps[|fps = p60|-ParkJoy_1080p]"
323337
```
324338

325339
### Test Categories
@@ -429,6 +443,58 @@ pip install -r requirements.txt
429443
**Problem**: Network operation failures
430444
**Solution**: Many tests require root privileges for network operations. Run with appropriate sudo permissions.
431445

446+
#### Build and Setup Issues
447+
448+
**Problem**: `RxTxApp: command not found`
449+
**Solution**: Build the RxTxApp test tool separately:
450+
```bash
451+
cd tests/tools/RxTxApp
452+
meson setup build
453+
meson compile -C build
454+
cd ../../..
455+
```
456+
457+
**Problem**: RxTxApp build fails with "ST20P_TX_FLAG_EXACT_USER_PACING undeclared" or other header errors
458+
**Solution**: Install MTL system-wide before building RxTxApp:
459+
```bash
460+
cd /path/to/Media-Transport-Library
461+
sudo ninja install -C build
462+
sudo ldconfig
463+
# Then build RxTxApp
464+
cd tests/tools/RxTxApp
465+
rm -rf build # Clean previous failed build
466+
meson setup build
467+
meson compile -C build
468+
```
469+
470+
**Problem**: `No module named pytest` when using sudo
471+
**Solution**: Use the virtual environment python with sudo:
472+
```bash
473+
# Wrong: sudo python3 -m pytest
474+
# Correct:
475+
sudo ./venv/bin/python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml
476+
```
477+
478+
**Problem**: DSA SSH key errors: `ValueError: q must be exactly 160, 224, or 256 bits long`
479+
**Solution**: Generate new RSA SSH keys and configure SSH access:
480+
```bash
481+
# Generate RSA keys (as your regular user, not root)
482+
ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa
483+
484+
# Set up SSH access for root@localhost
485+
ssh-copy-id -i ~/.ssh/id_rsa.pub root@localhost
486+
487+
# Update topology_config.yaml to use your user's key path:
488+
# key_path: /home/your-username/.ssh/id_rsa (not /root/.ssh/id_rsa)
489+
```
490+
491+
**Problem**: FFmpeg `No such filter: 'drawtext'` when running gen_frames.sh
492+
**Solution**: Install complete FFmpeg build or skip media generation:
493+
```bash
494+
sudo apt install ffmpeg # Full installation
495+
# Or skip: some tests may work without generated media
496+
```
497+
432498
#### Media File Access
433499
**Problem**: Media files not found
434500
**Solution**: Verify that test media files are available and accessible at the path specified in `media_path`
@@ -437,6 +503,26 @@ pip install -r requirements.txt
437503
**Problem**: Tests timing out on slower systems
438504
**Solution**: Increase timeout values in test_config.yaml for slower systems
439505

506+
### Quick Reference Tables
507+
508+
#### Build Issues
509+
510+
| Problem | Solution |
511+
|---------|----------|
512+
| `RxTxApp: command not found` | Build RxTxApp: `cd tests/tools/RxTxApp && meson setup build && meson compile -C build` |
513+
| `MTL library not found` | Install MTL system-wide: `sudo ninja install -C build && sudo ldconfig` |
514+
| `DSA key error: q must be exactly 160, 224, or 256 bits` | Generate RSA keys: `ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa` |
515+
516+
#### Runtime Issues
517+
518+
| Problem | Solution |
519+
|---------|----------|
520+
| `Permission denied` | Use root user: `sudo ./venv/bin/python3 -m pytest` |
521+
| `No module named pytest` | Don't use `sudo python3`, use `sudo ./venv/bin/python3` |
522+
| `Config path errors` | Update placeholder paths in config files |
523+
| `SSH connection failed` | Ensure SSH keys are set up for root@localhost access |
524+
| `No such filter: 'drawtext'` | Install FFmpeg with text filters or skip media generation |
525+
440526
### Debugging Tests
441527
442528
Use pytest's debug features:

doc/validation_quickstart.md

Lines changed: 38 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -2,19 +2,17 @@
22

33
This quick start guide helps you get the MTL validation framework running with minimal setup. For detailed information, see the [complete validation framework documentation](validation_framework.md).
44

5-
## Prerequisites (Must Complete First!)
5+
## Prerequisites
66

7-
1. **🏗️ Build MTL** (CRITICAL - tests will fail without this):
8-
```bash
9-
cd /path/to/Media-Transport-Library
10-
./build.sh
11-
```
12-
> If this fails, see [detailed build instructions](build.md)
7+
1. **🏗️ MTL Build Complete**: MTL must be built and test tools available
8+
👉 **[Follow complete build instructions](validation_framework.md#setup-and-installation)**
139

1410
2. **📋 Basic Requirements**:
1511
- Python 3.9+
1612
- Root user access (MTL validation requires root privileges)
1713
- Network interfaces configured for testing
14+
- FFmpeg with text filters (for media generation)
15+
- Compatible SSH keys (RSA recommended, not DSA)
1816

1917
## Quick Setup (3 steps)
2018

@@ -35,9 +33,19 @@ Update two key files:
3533
```yaml
3634
# Key settings to update:
3735
username: root # Must be root for MTL operations
38-
key_path: /root/.ssh/id_rsa # Your SSH key path
36+
key_path: /home/your-username/.ssh/id_rsa # YOUR user's SSH key path (not /root/)
37+
ip_address: 127.0.0.1 # For localhost testing
38+
port: 22 # Standard SSH port
3939
```
4040
41+
> **⚠️ SSH Key Setup**:
42+
> - Use your regular user's SSH keys (e.g., `/home/gta/.ssh/id_rsa`), not root's keys
43+
> - If you get DSA key errors, generate new RSA keys:
44+
> ```bash
45+
> ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa
46+
> ssh-copy-id -i ~/.ssh/id_rsa.pub root@localhost
47+
> ```
48+
4149
**[tests/validation/configs/test_config.yaml](../tests/validation/configs/test_config.yaml)**:
4250
```yaml
4351
# Replace MTL_PATH_PLACEHOLDER with your actual paths:
@@ -46,13 +54,15 @@ mtl_path: /home/gta/Media-Transport-Library/
4654
```
4755

4856
### 3. Run Tests
49-
**Basic smoke test**:
57+
**Basic smoke test** (must run as root):
5058
```bash
5159
cd tests/validation
52-
source venv/bin/activate
53-
python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml -m smoke -v
60+
# Use full path to venv python with sudo:
61+
sudo ./venv/bin/python3 -m pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml -m smoke -v
5462
```
5563

64+
> **💡 Root Execution**: Don't use `sudo python3` (uses system python). Use `sudo ./venv/bin/python3` to use the virtual environment.
65+
5666
**Run specific test with parameters**:
5767
```bash
5868
pytest --topology_config=configs/topology_config.yaml --test_config=configs/test_config.yaml "tests/single/st20p/fps/test_fps.py::test_fps[|fps = p60|-ParkJoy_1080p]"
@@ -72,29 +82,27 @@ sudo ./script/nicctl.sh create_vf ${TEST_PF_PORT_R}
7282

7383
## Quick Troubleshooting
7484

75-
| Error | Solution |
76-
|-------|----------|
77-
| `RxTxApp: command not found` | Build MTL first with `./build.sh` |
78-
| `Permission denied` | Use root user (not regular user) |
85+
| Common Error | Quick Solution |
86+
|--------------|----------------|
87+
| `RxTxApp: command not found` | [Follow build instructions](validation_framework.md#rxtxapp-test-tool) |
88+
| `Permission denied` | Use root: `sudo ./venv/bin/python3 -m pytest` |
89+
| `No module named pytest` | Don't use `sudo python3`, use `sudo ./venv/bin/python3` |
7990
| `Config path errors` | Update placeholder paths in config files |
91+
| SSH/FFmpeg issues | See [detailed troubleshooting](validation_framework.md#troubleshooting) |
8092

8193
## Generate Test Media (Optional)
8294

83-
For video testing, generate test frames (must run from specific directory):
84-
```bash
85-
cd tests/validation/common # Must be in this directory
86-
./gen_frames.sh # Generates test media files
87-
```
88-
89-
The script supports:
90-
- Multiple resolutions (3840x2160, 1920x1080, 1280x720, 640x360)
91-
- Different pixel formats (yuv422p, yuv422p10le)
92-
- Configurable color patterns and test signals with timestamps
93-
- Various frame rates and durations
94-
- Generates files like `ParkJoy_1080p.yuv` used in test examples
95+
For video testing, you may need test media files:
96+
👉 **[See media generation instructions](validation_framework.md#gen_framessh)**
9597

9698
---
9799

98-
**Need more details?**[Complete Documentation](validation_framework.md)
99-
**Build issues?**[Build Guide](build.md)
100-
**Configuration help?**[Configuration Guide](configuration_guide.md)
100+
## Documentation Navigation
101+
102+
📖 **Complete Documentation**: [Validation Framework](validation_framework.md) - Detailed information, configuration, and advanced features
103+
🔧 **Build Issues**: [Build Guide](build.md) - MTL build instructions
104+
⚙️ **Configuration Help**: [Configuration Guide](configuration_guide.md) - Network and environment setup
105+
106+
## Summary
107+
108+
This quick start guide gets you running tests in minutes. For production use, detailed configuration, or troubleshooting complex issues, refer to the complete documentation above.

0 commit comments

Comments
 (0)