Skip to content

Commit bdb8a9a

Browse files
committed
README: Add documentation for compare-results-fstests.py script
Add comprehensive documentation for the bin/compare-results-fstests.py script, including usage examples, output modes, features, and requirements. The documentation covers: - Basic and verbose usage modes - Description of default and verbose output formats - Key features like kernel version detection and multi-profile support - Example commands with actual commit IDs - Expected commit message format requirements Generated-by: Claude AI Signed-off-by: Luis Chamberlain <[email protected]>
1 parent e25dd63 commit bdb8a9a

File tree

1 file changed

+62
-0
lines changed

1 file changed

+62
-0
lines changed

README.md

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,3 +85,65 @@ To see fstests results you can use something like this:
8585
tar -xOJf fstests/mcgrof/xfs/libvirt-qemu/20240505-0001/6.9.0-rc6.xz 6.9.0-rc6/xfs_reflink_1024/xfs/033.out.bad
8686
tar -xOJf fstests/mcgrof/xfs/libvirt-qemu/20240505-0001/6.9.0-rc6.xz 6.9.0-rc6/xfs_reflink_1024/xfs/033.dmesg
8787
```
88+
89+
# Comparing fstests results
90+
91+
The `bin/compare-results-fstests.py` script allows you to compare test results between two commits to identify regressions and fixes. The script parses commit messages containing structured fstests results and provides a detailed comparison.
92+
93+
## Usage
94+
95+
Basic usage:
96+
```bash
97+
./bin/compare-results-fstests.py <baseline-commit> <test-commit>
98+
```
99+
100+
Verbose output with detailed side-by-side comparison:
101+
```bash
102+
./bin/compare-results-fstests.py <baseline-commit> <test-commit> --verbose
103+
```
104+
105+
## Output Modes
106+
107+
### Default Mode
108+
Shows only changed test results:
109+
- Lists new failures (regressions) with `+` prefix
110+
- Lists resolved failures (fixes) with `-` prefix
111+
- Organized by test profile
112+
- Reports "No changes" if test results are identical
113+
114+
### Verbose Mode (`--verbose` or `-v`)
115+
Provides a comprehensive side-by-side comparison:
116+
- Displays all tests from both commits in a table format
117+
- Shows `[pass]` or `[fail]` status for baseline and test
118+
- Annotates regressions with `--> regression`
119+
- Annotates fixes with `--> fixed`
120+
- Includes summary statistics (total regressions, fixes, unchanged failures)
121+
122+
## Features
123+
124+
- **Automatic kernel version detection**: Extracts KERNEL: line from commit messages
125+
- **Multi-profile support**: Handles multiple filesystem test profiles (xfs, ext4, btrfs, etc.)
126+
- **Smart failure resolution**: Only marks a failure as resolved if it doesn't appear in any other test profile
127+
- **Commit summary display**: Shows abbreviated commit IDs and subject lines for easy reference
128+
129+
## Example
130+
131+
```bash
132+
# Compare two test runs
133+
./bin/compare-results-fstests.py 8ffd015db85f a1b2c3d4e5f6
134+
135+
# Get detailed verbose output
136+
./bin/compare-results-fstests.py 8ffd015db85f a1b2c3d4e5f6 -v
137+
```
138+
139+
## Requirements
140+
141+
The script expects commit messages to contain structured test results in the following format:
142+
```
143+
KERNEL: <kernel-version>
144+
145+
<profile_name>: <N> tests, ...
146+
Failures: <test1> <test2> ...
147+
148+
Totals: ...
149+
```

0 commit comments

Comments
 (0)