Testing strategy

[stalniy]

Summary

This proposal outlines when to write and run different types of tests.

The main goal is to establish a testing strategy that:

1. Helps identify issues more quickly and efficiently.
2. Provides clear guidance on when to execute specific test types.

Motivation

Currently, in the console-api project, most tests are end-to-end (E2E) and are executed during PR-level checks. While E2E tests validate the entire flow, they are difficult to debug and maintain, especially for developers unfamiliar with the project context. Running E2E tests at the PR level is not ideal because failures in external services (which we do not own and cannot fix) can block development or lead to ignoring CI failures, which is also problematic.

Testing Strategy

Unit Tests for the UI

When to use:

• For 99% of components and hooks.
• Less frequently for services, as they typically delegate work to the HTTP layer.

Why:

• Ensures components and hooks behave as expected in full isolation.

When to run:

• Locally or as part of PR checks.

Soft requirements:

• All dependencies must be mocked.
• Components, hooks, and services should be tested in full isolation.

Additional notes

Works well with Testing Library, which facilitates user-centric testing.

---

Unit Tests for the Service Layer

When to use:

• Best suited for testing pure logic.

Why:

• Ensures entities function correctly in isolation.
• Fast and easy to maintain.
• Acts as the first line of defense against failures.
• AI-assisted test generation should be feasible for this type.

When to run:

• Locally or as part of PR checks.

Soft requirements:

• All dependencies must be mocked.
• Services and classes should be tested in full isolation.

Additional notes

• Unit testing Repository patterns is often unnecessary since they primarily delegate work to a database. If an entity just delegates all tasks, unit testing its logic adds little value.

---

Integration Tests for the Service Layer and Repository Pattern

When to use:

• When testing services that rely heavily on external dependencies (e.g., database logic, Repository services).

Why:

• Validates that the entire solution functions as expected (e.g., stale-anonymous-users-cleanup.spec.ts).
• Unlike unit tests, integration tests should rely on actual fixtures in the database rather than mocks.

When to run:

• Locally or as part of PR checks.

Soft requirements:

• Mocks should be avoided; use real fixtures instead. Mock 3rd party service calls if needed
• Only for services and databases under our control.

---

Functional or API Tests

When to use:

• When testing service functionality as a black box, independent of external services.

Why:

• Ensures business-critical functionality works correctly in isolation.

When to run:

• Locally or as part of PR checks.

Soft requirements:

• External network calls must be mocked (e.g., using nock or msw).
• Do NOT mock application-level services, as they are internal implementation details at this level.

Additional notes:
These tests should only fail if functional requirements change, not due to internal refactoring.

---

End-to-End (E2E) Tests

When to use:

• When verifying that a deployed service functions correctly in a target environment (beta or production).

Why:

• Ensures business-critical functionality works as expected post-deployment.

When to run:

• ONLY after deployment to the target environment. For example, after deployment to beta → run E2E tests on beta → deploy to prod → run E2E tests on prod.

Soft requirements:

• No mocks should be used.

[ygrishajev]

Makes total sense to me! Thanks for the clear outline!