Skip to content

Commit fcb5530

Browse files
BHFockBHFock
authored
Merge pull request #33 from BHFock/test_docs
Update documentation for test suite
2 parents 38639bd + cd1a2c9 commit fcb5530

File tree

2 files changed

+66
-214
lines changed

2 files changed

+66
-214
lines changed

‎README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,8 @@ git cl br docs-fix
8585

8686
📘 [Design Notes](docs/design-notes.md): Technical architecture
8787

88+
📘 [Tests](tests/README.md): Test suite and shell walkthroughs
89+
8890
📘 [Why git-cl?](docs/why-git-cl.md): History and motivation
8991

9092
📘 [Paper](docs/paper.md): Design, workflow, and related work

‎tests/README.md‎

Lines changed: 64 additions & 214 deletions
Original file line numberDiff line numberDiff line change
@@ -1,259 +1,109 @@
11
# git-cl Test Suite
22

3-
## Purpose
3+
Automated integration tests for [git-cl](https://github.com/BHFock/git-cl). Each test script creates a temporary Git repository, runs git-cl commands, and verifies the results. Every test can also be exported as a standalone shell walkthrough — a step-by-step terminal session you can follow to learn how git-cl works.
44

5-
This test suite provides automated integration tests for [git-cl](https://github.com/BHFock/git-cl). It serves two goals:
6-
7-
1. **Regression testing** — verify that git-cl behaves correctly across Python versions, Git versions, and operating systems.
8-
2. **Worked examples** — each test script can be exported as a standalone shell walkthrough for user training.
9-
10-
git-cl is feature complete. These tests document and protect existing behaviour rather than driving new features.
11-
12-
13-
## Design Decisions
14-
15-
### Why Python tests with shell export?
16-
17-
The test framework is written in Python, but each test can be exported as a self-contained shell script. This gives us:
18-
19-
- **Clean test logic** — assertions, JSON inspection, and temporary repo management are natural in Python.
20-
- **Readable training material** — the exported shell scripts read as step-by-step terminal sessions a user can follow.
21-
- **No external dependencies** — the framework uses only the Python standard library.
22-
23-
### What is tested?
24-
25-
Each test script creates a temporary Git repository, runs git-cl commands against it, and verifies the results by checking:
26-
27-
- **Command output** — does `git cl add` print the expected confirmation?
28-
- **Exit codes** — does the command succeed or fail as expected?
29-
- **Internal metadata** — does `.git/cl.json` contain the right changelist entries?
30-
- **Git state** — are the correct files staged, committed, or reverted?
31-
- **Git status codes** — are files in the expected state (`[ M]`, `[A ]`, `[??]`, etc.)?
32-
- **Working directory context** — do commands behave correctly from subdirectories?
33-
34-
### What is NOT tested?
35-
36-
- Interactive terminal behaviour (colour output, terminal width)
37-
- Concurrent access or file locking under load
38-
- Platform-specific behaviour (Windows, network filesystems)
395

6+
## Usage
407

41-
## Architecture
8+
Run all tests:
429

4310
```
44-
tests/
45-
├── README.md ← this file
46-
├── test_helpers.py ← TestRepo class and shell export logic
47-
├── run_tests.py ← test runner (discovers and runs all test scripts)
48-
├── test_basic_add_status.py
49-
├── test_stage_unstage.py
50-
├── test_commit.py
51-
├── test_diff.py
52-
├── test_remove_delete.py
53-
├── test_checkout.py
54-
├── test_stash_unstash.py
55-
├── test_branch.py
56-
├── test_validation.py
57-
├── test_git_states.py
58-
├── test_subdirectory.py
59-
└── test_edge_cases.py
60-
```
61-
62-
### test_helpers.py
63-
64-
Provides the `TestRepo` class — a context manager that:
65-
66-
- Creates a fresh temporary Git repository with an initial commit
67-
- Offers helper methods for common operations (`write_file`, `run`, `load_cl_json`, ...)
68-
- Records every operation so the session can be exported as a shell script
69-
- Provides assertion methods that print pass/fail results and log themselves for export
70-
- Cleans up the temporary directory on exit
71-
72-
#### run() — command execution
73-
74-
`run()` accepts commands as a string or as a list of arguments:
75-
76-
```python
77-
# Simple commands — string is fine, parsed with shlex.split()
78-
repo.run("git cl add docs file.txt")
79-
80-
# Arguments containing spaces — use a list for precise control
81-
repo.run(["git", "cl", "commit", "docs", "-m", "Fix the bug"])
11+
./run_tests.py
8212
```
8313

84-
When a string is passed, it is split using `shlex.split()`, which handles quoted
85-
arguments correctly (e.g. `'git cl commit docs -m "Fix the bug"'` works as expected).
86-
87-
When a list is passed, it is used directly as the argument vector.
88-
89-
For shell export, list-form commands are reconstructed with proper quoting.
90-
91-
#### run_in() — subdirectory execution
92-
93-
`run_in(subdir, command)` runs a command from within a subdirectory of the
94-
repository, without changing the test process's working directory:
14+
Run a single test:
9515

96-
```python
97-
# Test that adding a file from a subdirectory normalises the path
98-
repo.run_in("src", "git cl add my-list main.py")
9916
```
100-
101-
This passes `cwd=repo_dir/subdir` to `subprocess.run`. In the shell export, it
102-
becomes a subshell: `(cd src && git cl add my-list main.py)`.
103-
104-
### Test scripts
105-
106-
Each test script is executable and follows the same structure:
107-
108-
```python
109-
#!/usr/bin/env python3
110-
from test_helpers import TestRepo
111-
112-
def run_tests(repo: TestRepo):
113-
repo.section("Setup")
114-
# ... prepare repository ...
115-
116-
repo.section("Test scenario name")
117-
output = repo.run("git cl add my-list file.txt")
118-
repo.assert_in("Added to", output, "confirms addition")
119-
120-
cl = repo.load_cl_json()
121-
repo.assert_true("my-list" in cl, "changelist exists")
122-
123-
if __name__ == "__main__":
124-
# Handles both test execution and --export
125-
...
17+
./test_basic_add_status.py
12618
```
12719

128-
### Shell export
129-
130-
Every test script supports `--export` to produce a shell walkthrough:
20+
Export a test as a shell walkthrough:
13121

13222
```
13323
./test_basic_add_status.py --export > walkthrough_add_status.sh
13424
```
13525

136-
The exported script contains:
137-
- The same git-cl commands as the Python test
138-
- `# Check:` comments where assertions were, describing expected outcomes
139-
- Section headers matching the test structure
14026

141-
In export mode, assertion output (pass/fail lines) is suppressed. Only the
142-
shell script is written to stdout.
27+
## Shell Walkthroughs
14328

144-
The exported scripts are for reading and manual line-by-line execution, not
145-
for automated testing.
29+
Every test script supports `--export` to produce a self-contained shell script. These walkthroughs mirror the test scenarios but are written for reading and manual line-by-line execution. `# Check:` comments tell you what to expect at each step.
14630

31+
Here is a shortened example from `test_basic_add_status.py --export`:
14732

148-
## Test Plan
33+
```bash
34+
#!/usr/bin/env bash
35+
# git-cl walkthrough: add and status
36+
#
37+
# Run it line by line to learn how git-cl works.
38+
# Lines starting with '# Check:' tell you what to expect.
14939

150-
### Core Commands
40+
set -euo pipefail
15141

152-
| Script | Commands under test | Key behaviours verified |
153-
|-------------------------------|----------------------------------------|------------------------------------------------------------|
154-
| `test_basic_add_status.py` | `add`, `status`, `st` | Creating changelists, assigning files, reassignment, |
155-
| | | status grouping, filtering, `--include-no-cl`, aliases |
156-
| `test_stage_unstage.py` | `stage`, `unstage` | Staging tracked files, skipping untracked files, |
157-
| | | `--delete` flag, round-trip stage/unstage, changelist |
158-
| | | preserved by default |
159-
| `test_commit.py` | `commit`, `ci` | Commit with `-m` and `-F`, `--keep` flag, changelist |
160-
| | | deleted by default, untracked files skipped, alias |
161-
| `test_diff.py` | `diff` | Single changelist diff, multiple changelist diff, |
162-
| | | `--staged` flag |
163-
| `test_remove_delete.py` | `remove`, `rm`, `delete`, `del` | Removing files from changelists, deleting changelists, |
164-
| | | `--all` flag, files untouched on disk, aliases |
165-
| `test_checkout.py` | `checkout`, `co` | Reverting files to HEAD, only affects named changelist, |
166-
| | | changelist kept by default, `--delete` flag, alias |
42+
# Create a temporary Git repository
43+
cd $(mktemp -d)
44+
git init --quiet
45+
git config user.email "test@git-cl.test"
46+
git config user.name "git-cl test"
16747

168-
### Advanced Commands
48+
# === Setup: create files with an initial commit ===
16949

170-
| Script | Commands under test | Key behaviours verified |
171-
|-------------------------------|----------------------------------------|------------------------------------------------------------|
172-
| `test_stash_unstash.py` | `stash`, `unstash` | Stash single changelist, unstash restores files and |
173-
| | | metadata, `--all` flag, selective unstash after stash all |
174-
| `test_branch.py` | `branch`, `br` | Branch from changelist, custom branch name, `--from` base, |
175-
| | | other changelists stashed, alias |
176-
177-
### Git States and Working Directory
178-
179-
| Script | Focus area | Key behaviours verified |
180-
|-------------------------------|----------------------------------------|------------------------------------------------------------|
181-
| `test_git_states.py` | Git status codes | Changelists with files in each common state: |
182-
| | | `[ M]` unstaged modification, `[M ]` staged modification, |
183-
| | | `[MM]` mixed staged/unstaged, `[A ]` newly added, |
184-
| | | `[AM]` added then modified, `[ D]` unstaged deletion, |
185-
| | | `[D ]` staged deletion, `[??]` untracked. |
186-
| | | Correct display in `git cl st`, correct behaviour with |
187-
| | | `--all` flag for uncommon codes. |
188-
| `test_subdirectory.py` | Path resolution | Running git-cl commands from the repo root, a subdirectory,|
189-
| | | and a nested subdirectory. Verifying that paths are stored |
190-
| | | as repo-root-relative in `cl.json` and displayed correctly |
191-
| | | relative to the current working directory. |
192-
193-
### Validation and Edge Cases
194-
195-
| Script | Focus area | Key behaviours verified |
196-
|-------------------------------|----------------------------------------|------------------------------------------------------------|
197-
| `test_validation.py` | Input validation | Invalid changelist names (special chars, reserved words, |
198-
| | | dots-only, too long), dangerous paths, directory traversal,|
199-
| | | non-existent files, non-existent changelists |
200-
| `test_edge_cases.py` | Boundary conditions | Empty working directory, no changelists, nested |
201-
| | | subdirectories, deleted files, file reassignment, |
202-
| | | already-staged files, rapid add/remove cycles |
203-
204-
205-
## Requirements
206-
207-
- Python 3.10+
208-
- Git
209-
- A Unix-like OS (Linux, macOS)
210-
- `git-cl` available in `$PATH`
50+
echo "hello" > file1.txt
51+
echo "world" > file2.txt
52+
git add file1.txt file2.txt
53+
git commit --quiet -m "Add initial files"
54+
echo "hello modified" > file1.txt
55+
echo "world modified" > file2.txt
21156

57+
# === Add files to a new changelist ===
21258

213-
## Usage
59+
git cl add feature1 file1.txt file2.txt
60+
# Check: output contains 'Added to 'feature1''
21461

215-
Run all tests:
62+
# === View status grouped by changelist ===
21663

217-
```
218-
./run_tests.py
64+
git cl st
65+
# Check: output contains 'feature1:'
66+
# Check: output contains 'file1.txt'
67+
# Check: output contains 'file2.txt'
21968
```
22069

221-
Run a single test:
222-
223-
```
224-
./test_basic_add_status.py
225-
```
22670

227-
Export a test as a shell walkthrough:
71+
## What's Tested
22872

229-
```
230-
./test_basic_add_status.py --export > walkthrough_add_status.sh
231-
```
73+
### Core Commands
23274

75+
| Script | Commands |
76+
|---|---|
77+
| `test_basic_add_status.py` | `add`, `status` / `st`, filtering, `--include-no-cl` |
78+
| `test_stage_unstage.py` | `stage`, `unstage`, `--delete` flag, round-trip |
79+
| `test_commit.py` | `commit` / `ci`, `-m`, `-F`, `--keep` flag |
80+
| `test_diff.py` | `diff`, multiple changelists, `--staged` |
81+
| `test_remove_delete.py` | `remove` / `rm`, `delete` / `del`, `--all` |
82+
| `test_checkout.py` | `checkout` / `co`, `--delete`, `--force` |
23383

234-
## Resolved Design Decisions
84+
### Advanced Commands
23585

236-
> Decisions made during development of the test framework.
86+
| Script | Commands |
87+
|---|---|
88+
| `test_stash_unstash.py` | `stash` / `sh`, `unstash` / `us`, `--all` |
89+
| `test_branch.py` | `branch` / `br`, custom name, `--from` base |
23790

238-
### Commit messages with spaces
91+
### States, Paths, and Validation
23992

240-
`run()` uses `shlex.split()` when given a string, which correctly handles quoted
241-
arguments. For full control, a list of arguments can be passed instead. Shell export
242-
reconstructs the command with proper quoting from the list form.
93+
| Script | Focus |
94+
|---|---|
95+
| `test_git_states.py` | All common Git status codes (`[ M]`, `[M ]`, `[MM]`, `[A ]`, `[AM]`, `[ D]`, `[D ]`, `[??]`) |
96+
| `test_subdirectory.py` | Path normalisation from subdirectories, cross-directory add, relative display |
97+
| `test_validation.py` | Invalid names, reserved words, path traversal, missing arguments |
98+
| `test_edge_cases.py` | Empty states, reassignment, duplicate files, deleted files |
24399

244-
### Subdirectory execution
245100

246-
`run_in(subdir, command)` sets the `cwd` for the subprocess without affecting the
247-
test process. Exported as `(cd subdir && command)` in shell scripts.
101+
## How the Tests Work
248102

249-
### Export noise
103+
The test framework lives in `test_helpers.py` and provides a single class: `TestRepo`. It is used as a context manager that creates a fresh temporary Git repository on entry and cleans it up on exit.
250104

251-
In `--export` mode, all assertion output is suppressed. Only the shell script is
252-
written to stdout. This means `./test_basic_add_status.py --export > walkthrough.sh`
253-
produces a clean file with no test noise mixed in.
105+
`TestRepo` offers helpers for common operations — `write_file`, `run`, `run_in` (execute from a subdirectory), `load_cl_json`, `get_staged_files`, and a set of assertion methods (`assert_in`, `assert_equal`, `assert_exit_code`, etc.) that print pass/fail results during the test run.
254106

255-
### Test runner
107+
Every operation is recorded internally. When a test script is run with `--export`, the recorded operations are replayed as a shell script instead of executing assertions. This is how the same test code serves both as an automated check and as a readable walkthrough.
256108

257-
`run_tests.py` discovers `test_*.py` files in its directory, runs each as a subprocess,
258-
collects exit codes, and prints a summary. Each test script is self-contained and can
259-
be run independently.
109+
Each `test_*.py` script follows the same pattern: import `TestRepo`, define a `run_tests(repo)` function with sections and assertions, and handle both normal execution and `--export` mode in `__main__`. Test scripts are self-contained and can be run independently. `run_tests.py` discovers and runs all of them, collecting results into a summary.

0 commit comments

Comments
 (0)