[🚫 Bug] EXTENSION_DEV_PATH appears to have no effect on run-tests

[kurmasz]

Describe the bug

EXTENSION_DEV_PATH appears to have no effect on run-tests.

I cloned vscode-extension-tester-example and tried to run the tests on the compiled source (instead of the .vsix file).

Specifically, I ran the following after cloning the repository:

• npm install
• npm run compile
• npx extester get-vscode
• npx extester get-chromedriver
• EXTENSION_DEV_PATH=$(pwd) node ../../vscode-extension-tester/packages/extester/out/cli.js run-tests './out/ui-test/*.test.js' --code_version max --code_settings settings.json

When I do this, the tests run, but several fail. If you watch carefully, you can see that the Hello World and WebView Test commands are not available.

If this is not the correct way to run the tests on the compiled project source, then please add detail to the documentation.

If this is the correct way, I think I may see the source of the bug:

CodeUtil.runTests sets process.env.EXTENSION_DEV_PATH before it would ever be read. Thus, any value ever passed from the environment simply gets overwritten.

async runTests(testFilesPattern: string[], runOptions: RunOptions = DEFAULT_RUN_OPTIONS): Promise<number> {
		if (!runOptions.offline) {
			await this.checkCodeVersion(runOptions.vscodeVersion ?? DEFAULT_RUN_OPTIONS.vscodeVersion);
		} else {
			this.availableVersions = [this.getExistingCodeVersion()];
		}
		const literalVersion =
			runOptions.vscodeVersion === undefined || runOptions.vscodeVersion === 'latest' ? this.availableVersions[0] : runOptions.vscodeVersion;

		// add chromedriver to process' path
		const finalEnv: NodeJS.ProcessEnv = {};
		Object.assign(finalEnv, process.env);
		const key = 'PATH';
		finalEnv[key] = [this.downloadFolder, process.env[key]].join(path.delimiter);

		console.log('In CodeUtil 1.  Extensions Folder: ', this.extensionsFolder);
		console.log(process.env);
		process.env = finalEnv;
		process.env.TEST_RESOURCES = this.downloadFolder;
		process.env.EXTENSIONS_FOLDER = this.extensionsFolder;
		====> process.env.EXTENSION_DEV_PATH = this.coverage ? process.cwd() : undefined;

I discovered this when I added console.log(process.env) to the line before this one

	let options = new Options().setChromeBinaryPath(codePath).addArguments(...args) as any;

and saw that EXTENSION_DEV_PATH was undefined

I also went into node_modules and commented out line 309 of codeUtil.js and the above sequence behaved as I expected.

Steps to reproduce

Specifically, I ran the following after cloning the repository:

• npm install
• npm run compile
• npx extester get-vscode
• npx extester get-chromedriver
• EXTENSION_DEV_PATH=$(pwd) node ../../vscode-extension-tester/packages/extester/out/cli.js run-tests './out/ui-test/*.test.js' --code_version max --code_settings settings.json

Logs

Operating System

macOS 15.5

Visual Studio Code

1.102.0

vscode-extension-tester

8.17.0

NodeJS

24

npm

No response

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[kurmasz]

Any updates? Is this really a bug? Or am I misunderstanding how this should work?