Skip to content

Commit f3675b0

Browse files
eleanorjboydeleanorjboyd
authored
Add meta-instruction files for learnings and testing feature area (#25473)
1 parent 2968abd commit f3675b0

File tree

2 files changed

+215
-0
lines changed

2 files changed

+215
-0
lines changed

‎.github/instructions/learning.instructions.md‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
---
2+
applyTo: '**'
3+
description: This document describes how to deal with learnings that you make. (meta instruction)
4+
---
5+
6+
This document describes how to deal with learnings that you make.
7+
It is a meta-instruction file.
8+
9+
Structure of learnings:
10+
11+
- Each instruction file has a "Learnings" section.
12+
- Each learning has a counter that indicates how often that learning was useful (initially 1).
13+
- Each learning has a 1 sentence description of the learning that is clear and concise.
14+
15+
Example:
16+
17+
```markdown
18+
## Learnings
19+
20+
- Prefer `const` over `let` whenever possible (1)
21+
- Avoid `any` type (3)
22+
```
23+
24+
When the user tells you "learn!", you should:
25+
26+
- extract a learning from the recent conversation
27+
_ identify the problem that you created
28+
_ identify why it was a problem
29+
_ identify how you were told to fix it/how the user fixed it
30+
_ generate only one learning (1 sentence) that helps to summarize the insight gained
31+
- then, add the reflected learning to the "Learnings" section of the most appropriate instruction file
32+
33+
Important: Whenever a learning was really useful, increase the counter!!
34+
When a learning was not useful and just caused more problems, decrease the counter.

‎.github/instructions/testing_feature_area.instructions.md‎

Lines changed: 181 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,181 @@
1+
---
2+
applyTo: 'src/client/testing/**'
3+
---
4+
5+
# Testing feature area — Discovery, Run, Debug, and Results
6+
7+
This document maps the testing support in the extension: discovery, execution (run), debugging, result reporting and how those pieces connect to the codebase. It's written for contributors and agents who need to navigate, modify, or extend test support (both `unittest` and `pytest`).
8+
9+
## Overview
10+
11+
- Purpose: expose Python tests in the VS Code Test Explorer (TestController), support discovery, run, debug, and surface rich results and outputs.
12+
- Scope: provider-agnostic orchestration + provider-specific adapters, TestController mapping, IPC with Python-side scripts, debug launch integration, and configuration management.
13+
14+
## High-level architecture
15+
16+
- Controller / UI bridge: orchestrates TestController requests and routes them to workspace adapters.
17+
- Workspace adapter: provider-agnostic coordinator that translates TestController requests to provider adapters and maps payloads back into TestItems/TestRuns.
18+
- Provider adapters: implement discovery/run/debug for `unittest` and `pytest` by launching Python scripts and wiring named-pipe IPC.
19+
- Result resolver: translates Python-side JSON/IPCPayloads into TestController updates (start/pass/fail/output/attachments).
20+
- Debug launcher: prepares debug sessions and coordinates the debugger attach flow with the Python runner.
21+
22+
## Key components (files and responsibilities)
23+
24+
- Entrypoints
25+
- `src/client/testing/testController/controller.ts` — `PythonTestController` (main orchestrator).
26+
- `src/client/testing/serviceRegistry.ts` — DI/wiring for testing services.
27+
- Workspace orchestration
28+
- `src/client/testing/testController/workspaceTestAdapter.ts` — `WorkspaceTestAdapter` (provider-agnostic entry used by controller).
29+
- Provider adapters
30+
- Unittest
31+
- `src/client/testing/testController/unittest/testDiscoveryAdapter.ts`
32+
- `src/client/testing/testController/unittest/testExecutionAdapter.ts`
33+
- Pytest
34+
- `src/client/testing/testController/pytest/pytestDiscoveryAdapter.ts`
35+
- `src/client/testing/testController/pytest/pytestExecutionAdapter.ts`
36+
- Result resolution and helpers
37+
- `src/client/testing/testController/common/resultResolver.ts` — `PythonResultResolver` (maps payload -> TestController updates).
38+
- `src/client/testing/testController/common/testItemUtilities.ts` — helpers for TestItem lifecycle.
39+
- `src/client/testing/testController/common/types.ts` — `ITestDiscoveryAdapter`, `ITestExecutionAdapter`, `ITestResultResolver`, `ITestDebugLauncher`.
40+
- `src/client/testing/testController/common/debugLauncher.ts` — debug session creation helper.
41+
- `src/client/testing/testController/common/utils.ts` — named-pipe helpers and command builders (`startDiscoveryNamedPipe`, etc.).
42+
- Configuration
43+
- `src/client/testing/common/testConfigurationManager.ts` — per-workspace test settings.
44+
- `src/client/testing/configurationFactory.ts` — configuration service factory.
45+
- Utilities & glue
46+
- `src/client/testing/utils.ts` — assorted helpers used by adapters.
47+
- Python-side scripts: `python_files/unittestadapter/*`, `python_files/pytestadapter/*` — discovery/run code executed by adapters.
48+
49+
## Python subprocess runners (what runs inside Python)
50+
51+
The adapters in the extension don't implement test discovery/run logic themselves — they spawn a Python subprocess that runs small helper scripts located under `python_files/` and stream structured events back to the extension over the named-pipe IPC. This is a central part of the feature area; changes here usually require coordinated edits in both the TypeScript adapters and the Python scripts.
52+
53+
- Unittest helpers (folder: `python_files/unittestadapter`)
54+
55+
- `discovery.py` — performs `unittest` discovery and emits discovery payloads (test suites, cases, locations) on the IPC channel.
56+
- `execution.py` / `django_test_runner.py` — run tests for `unittest` and, where applicable, Django test runners; emit run events (start, stdout/stderr, pass, fail, skip, teardown) and attachment info.
57+
- `pvsc_utils.py`, `django_handler.py` — utility helpers used by the runners for environment handling and Django-specific wiring.
58+
- The adapter TypeScript files (`testDiscoveryAdapter.ts`, `testExecutionAdapter.ts`) construct the command line, start a named-pipe listener, and spawn these Python scripts using the extension's ExecutionFactory (activated interpreter) so the scripts execute inside the user's selected environment.
59+
60+
- Pytest helpers (folder: `python_files/vscode_pytest`)
61+
62+
- `_common.py` — shared helpers for pytest runner scripts.
63+
- `run_pytest_script.py` — the primary pytest runner used for discovery and execution; emits the same structured IPC payloads the extension expects (discovery events and run events).
64+
- The `pytest` execution adapter (`pytestExecutionAdapter.ts`) and discovery adapter build the CLI to run `run_pytest_script.py`, start the pipe, and translate incoming payloads via `PythonResultResolver`.
65+
66+
- IPC contract and expectations
67+
68+
- Adapters rely on a stable JSON payload contract emitted by the Python scripts: identifiers for tests, event types (discovered, collected, started, passed, failed, skipped), timings, error traces, and optional attachments (logs, captured stdout/stderr, file links).
69+
- The extension maps these payloads to `TestItem`/`TestRun` updates via `PythonResultResolver` (`src/client/testing/testController/common/resultResolver.ts`). If you change payload shape, update the resolver and tests concurrently.
70+
71+
- How the subprocess is started
72+
- Execution adapters use the extension's `ExecutionFactory` (preferred) to get an activated interpreter and then spawn a child process that runs the helper script. The adapter will set up environment variables and command-line args (including the pipe name / run-id) so the Python runner knows where to send events and how to behave (discovery vs run vs debug).
73+
- For debug sessions a debug-specific entry argument/port is passed and `common/debugLauncher.ts` coordinates starting a VS Code debug session that will attach to the Python process.
74+
75+
## Core functionality (what to change where)
76+
77+
- Discovery
78+
- Entry: `WorkspaceTestAdapter.discoverTests` → provider discovery adapter. Adapter starts a named-pipe listener, spawns the discovery script in an activated interpreter, forwards discovery events to `PythonResultResolver` which creates/updates TestItems.
79+
- Files: `workspaceTestAdapter.ts`, `*DiscoveryAdapter.ts`, `resultResolver.ts`, `testItemUtilities.ts`.
80+
- Run / Execution
81+
- Entry: `WorkspaceTestAdapter.executeTests` → provider execution adapter. Adapter spawns runner in an activated env, runner streams run events to the pipe, `PythonResultResolver` updates a `TestRun` with start/pass/fail and attachments.
82+
- Files: `workspaceTestAdapter.ts`, `*ExecutionAdapter.ts`, `resultResolver.ts`.
83+
- Debugging
84+
- Flow: debug request flows like a run but goes through `debugLauncher.ts` to create a VS Code debug session with prepared ports/pipes. The Python runner coordinates attach/continue with the debugger.
85+
- Files: `*ExecutionAdapter.ts`, `common/debugLauncher.ts`, `common/types.ts`.
86+
- Result reporting
87+
- `resultResolver.ts` is the canonical place to change how JSON payloads map to TestController constructs (messages, durations, error traces, attachments).
88+
89+
## Typical workflows (short)
90+
91+
- Full discovery
92+
93+
1. `PythonTestController` triggers discovery -> `WorkspaceTestAdapter.discoverTests`.
94+
2. Provider discovery adapter starts pipe and launches Python discovery script.
95+
3. Discovery events -> `PythonResultResolver` -> TestController tree updated.
96+
97+
- Run tests
98+
99+
1. Controller collects TestItems -> creates `TestRun`.
100+
2. `WorkspaceTestAdapter.executeTests` delegates to execution adapter which launches the runner.
101+
3. Runner events arrive via pipe -> `PythonResultResolver` updates `TestRun`.
102+
4. On process exit the run is finalized.
103+
104+
- Debug a test
105+
1. Debug request flows to execution adapter.
106+
2. Adapter prepares ports and calls `debugLauncher` to start a VS Code debug session with the run ID.
107+
3. Runner coordinates with the debugger; `PythonResultResolver` still receives and applies run events.
108+
109+
## Tests and examples to inspect
110+
111+
- Unit/integration tests for adapters and orchestration under `src/test/` (examples):
112+
- `src/test/testing/common/testingAdapter.test.ts`
113+
- `src/test/testing/testController/workspaceTestAdapter.unit.test.ts`
114+
- `src/test/testing/testController/unittest/testExecutionAdapter.unit.test.ts`
115+
- Adapter tests demonstrate expected telemetry, debug-launch payloads and result resolution.
116+
117+
## History & evolution (brief)
118+
119+
- Migration to TestController API: the code organizes around VS Code TestController, mapping legacy adapter behaviour into TestItems/TestRuns.
120+
- Named-pipe IPC: discovery/run use named-pipe IPC to stream events from Python runner scripts (`python_files/*`) which enables richer, incremental updates and debug coordination.
121+
- Environment activation: adapters prefer the extension ExecutionFactory (activated interpreter) to run discovery and test scripts.
122+
123+
## Pointers for contributors (practical)
124+
125+
- To extend discovery output: update the Python discovery script in `python_files/*` and `resultResolver.ts` to parse new payload fields.
126+
- To change run behaviour (args/env/timouts): update the provider execution adapter (`*ExecutionAdapter.ts`) and add/update tests under `src/test/`.
127+
- To change debug flow: edit `common/debugLauncher.ts` and adapters' debug paths; update tests that assert launch argument shapes.
128+
129+
## Django support (how it works)
130+
131+
- The extension supports Django projects by delegating discovery and execution to Django-aware Python helpers under `python_files/unittestadapter`.
132+
- `python_files/unittestadapter/django_handler.py` contains helpers that invoke `manage.py` for discovery or execute Django test runners inside the project context.
133+
- `python_files/unittestadapter/django_test_runner.py` provides `CustomDiscoveryTestRunner` and `CustomExecutionTestRunner` which integrate with the extension by using the same IPC contract (they use `UnittestTestResult` and `send_post_request` to emit discovery/run payloads).
134+
- How adapters pass Django configuration:
135+
- Execution adapters set environment variables (e.g. `MANAGE_PY_PATH`) and modify `PYTHONPATH` so Django code and the custom test runner are importable inside the spawned subprocess.
136+
- For discovery the adapter may run the discovery helper which calls `manage.py test` with a custom test runner that emits discovery payloads instead of executing tests.
137+
- Practical notes for contributors:
138+
- Changes to Django discovery/execution often require edits in both `django_test_runner.py`/`django_handler.py` and the TypeScript adapters (`testDiscoveryAdapter.ts` / `testExecutionAdapter.ts`).
139+
- The Django test runner expects `TEST_RUN_PIPE` environment variable to be present to send IPC events (see `django_test_runner.py`).
140+
141+
## Settings referenced by this feature area
142+
143+
- The extension exposes several `python.testing.*` settings used by adapters and configuration code (declared in `package.json`):
144+
- `python.testing.pytestEnabled`, `python.testing.unittestEnabled` — enable/disable frameworks.
145+
- `python.testing.pytestPath`, `python.testing.pytestArgs`, `python.testing.unittestArgs` — command path and CLI arguments used when spawning helper scripts.
146+
- `python.testing.cwd` — optional working directory used when running discovery/runs.
147+
- `python.testing.autoTestDiscoverOnSaveEnabled`, `python.testing.autoTestDiscoverOnSavePattern` — control automatic discovery on save.
148+
- `python.testing.debugPort` — default port used for debug runs.
149+
- `python.testing.promptToConfigure` — whether to prompt users to configure tests when potential test folders are found.
150+
- Where to look in the code:
151+
- Settings are consumed by `src/client/testing/common/testConfigurationManager.ts`, `src/client/testing/configurationFactory.ts`, and adapters under `src/client/testing/testController/*` which read settings to build CLI args and env for subprocesses.
152+
- The setting definitions and descriptions are in `package.json` and localized strings in `package.nls.json`.
153+
154+
## Coverage support (how it works)
155+
156+
- Coverage is supported by running the Python helper scripts with coverage enabled and then collecting a coverage payload from the runner.
157+
- Pytest-side coverage logic lives in `python_files/vscode_pytest/__init__.py` (checks `COVERAGE_ENABLED`, imports `coverage`, computes per-file metrics and emits a `CoveragePayloadDict`).
158+
- Unittest adapters enable coverage by setting environment variable(s) (e.g. `COVERAGE_ENABLED`) when launching the subprocess; adapters and `resultResolver.ts` handle the coverage profile kind (`TestRunProfileKind.Coverage`).
159+
- Flow summary:
160+
1. User starts a Coverage run via Test Explorer (profile kind `Coverage`).
161+
2. Controller/adapters set `COVERAGE_ENABLED` (or equivalent) in the subprocess env and invoke the runner script.
162+
3. The Python runner collects coverage (using `coverage` or `pytest-cov`), builds a file-level coverage map, and sends a coverage payload back over the IPC.
163+
4. `PythonResultResolver` (`src/client/testing/testController/common/resultResolver.ts`) receives the coverage payload and stores `detailedCoverageMap` used by the TestController profile to show file-level coverage details.
164+
- Tests that exercise coverage flows are under `src/test/testing/*` and `python_files/tests/*` (see `testingAdapter.test.ts` and adapter unit tests that assert `COVERAGE_ENABLED` is set appropriately).
165+
166+
## Interaction with the VS Code API
167+
168+
- TestController API
169+
- The feature area is built on VS Code's TestController/TestItem/TestRun APIs (`vscode.tests.createTestController` / `tests.createTestController` in the code). The controller creates a `TestController` in `src/client/testing/testController/controller.ts` and synchronizes `TestItem` trees with discovery payloads.
170+
- `PythonResultResolver` maps incoming JSON events to VS Code API calls: `testRun.appendOutput`, `testRun.passed/failed/skipped`, `testRun.end`, and `TestItem` updates (labels, locations, children).
171+
- Debug API
172+
- Debug runs use the Debug API to start an attach/launch session. The debug launcher implementation is in `src/client/testing/testController/common/debugLauncher.ts` which constructs a debug configuration and calls the VS Code debug API to start a session (e.g. `vscode.debug.startDebugging`).
173+
- Debug adapter/resolver code in the extension's debugger modules may also be used when attaching to Django or test subprocesses.
174+
- Commands and configuration
175+
- The Test Controller wires commands that appear in the Test Explorer and editor context menus (see `package.json` contributes `commands`) and listens to configuration changes filtered by `python.testing` in `src/client/testing/main.ts`.
176+
- Execution factory & activated environments
177+
- Adapters use the extension `ExecutionFactory` to spawn subprocesses in an activated interpreter (so the user's venv/conda is used). This involves the extension's internal environment execution APIs and sometimes `envExt` helpers when the external environment extension is present.
178+
179+
## Learnings
180+
181+
- Never await `showErrorMessage()` calls in test execution adapters as it blocks the test UI thread and freezes the Test Explorer (1)

0 commit comments

Comments
 (0)