test/fix: add test suite with E2E validation and fix bugs found against live ZoneMinder#60
test/fix: add test suite with E2E validation and fix bugs found against live ZoneMinder#60nabbi wants to merge 4 commits intoZoneMinder:masterfrom
Conversation
Introduce pytest-based test infrastructure for pyzm, which previously had zero tests. Establishes a safety net ahead of upcoming refactoring efforts. - pytest config (pyproject.toml) with markers and coverage settings - JSON response fixtures mirroring real ZM API payloads - Shared conftest fixtures (mock login, suppressed logger, exit guard) - Unit tests: auth flows, API methods, request handling, helper edge cases - Integration tests: full login -> monitors -> events -> states workflows - Testing strategy documentation (docs/testing-strategy.md) Core modules coverage: api.py 91%, helpers 80-100%. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
97 tests (81 readonly, 16 write) that validate pyzm against a real ZM instance, catching response structure drift, type coercion mismatches, and flash() redirect handling that unit tests with mocked HTTP miss. Tests skip automatically when ZM_API_URL env vars are unset. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Running the E2E test suite against a real ZoneMinder instance exposed
five bugs in pyzm that unit tests (with hand-crafted fixtures) never
caught — exactly the kind of drift E2E tests are designed to surface.
Monitors.add() missing type/device parameters:
E2E test_add_monitor → ZM returned 500. Captured the response body
and found: "Field 'Device' doesn't have a default value". The real
API requires Monitor[Type] and Monitor[Device] but Monitors.add()
had no support for either. Added both as first-class parameters.
Monitors.add() enabled=False silently dropped:
E2E test_set_enabled → monitor created with enabled=False was not
actually disabled. The guard `if options.get('enabled')` is falsy
for False, so Monitor[Enabled] was never sent. Changed to
`if options.get('enabled') is not None`.
Monitor.enabled() string vs int comparison:
E2E test_set_enabled → after enabling a monitor, enabled() returned
False. The real API returns Enabled as int 1, not string "1". The
comparison `== '1'` fails for int. Fixed with str() coercion.
_make_request raises BAD_IMAGE on successful PUT:
E2E test_set_config_value → Configs.set() PUT returned 200 with
Content-Length: 0. _make_request only handled empty bodies for
DELETE, so PUT fell through to the BAD_IMAGE error path. Extended
the empty-body handler to cover PUT.
Also adjusted E2E test assertions for raw field types (Colours returns
int from real API, not string as fixtures assumed).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Removed 3 helper tests that duplicated assertions already covered by API-level tests (per project rule). Rewrote README to lead with the core philosophy: the real ZM API is the source of truth. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Hi @nabbi, thanks for this PR. I'm Claude, an AI assistant working with @pliablepixels on pyzm v2. pyzm v2 ( Sure enough, the live tests caught a real bug right away: Commit: pliablepixels@26d1ecd Just a note — this repo ( Thanks for the ideas and the effort here. |
pyzm had zero tests. This PR adds a 4-tier test suite with 73 unit/integration tests and a full E2E suite designed to run against a live ZoneMinder instance. Issues in pyzm handling of API were found and corrected as a result of building these test cases.
Why
pyzm wraps the ZoneMinder REST API, but hand-crafted JSON fixtures can silently diverge from real server responses — wrong field names, different types (int vs str), HTML flash redirects instead of JSON, broken filter URL building. Unit tests alone can't catch these. The test suite is designed around one principle: the real ZoneMinder API is the source of truth.
What's included
Commit 1 — Unit + integration tests (73 tests)
handling, and all public methods
URL building, pagination limits
Commit 2 — E2E test suite against live ZoneMinder
Commit 3 — Bug fixes found by E2E tests
Commit 4 — Remove test overlaps, update README
Running
Unit + integration (no live ZM needed)
pytest tests/unit/ tests/integration/ -v
E2E readonly
ZM_API_URL=https://zm.local/zm/api ZM_USER=admin ZM_PASSWORD=secret
pytest tests/e2e/ -m e2e_readonly
E2E write (with cleanup)
ZM_E2E_WRITE=1 pytest tests/e2e/ -m e2e_write