diff --git a/CHANGELOG.md b/CHANGELOG.md index 473c67c..4cccace 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,6 @@ +## 2.0.0 (23-Jul-2025) +* Use [Test Coverage Tool](https://openexchange.intersystems.com/package/Test-Coverage-Tool) to present coverage information (#24) + ## 0.2.3 (13-Nov-2024) * Add manual refresh to test trees, and automatic refresh when client setting changes (#20) * Fix overprompting during authentication. diff --git a/README.md b/README.md index a35a50d..ae86408 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,10 @@ # InterSystems Testing Manager -This preview extension uses VS Code's [Testing API](https://code.visualstudio.com/api/extension-guides/testing) to discover, run and debug unit test classes built with the [%UnitTest testing framework](https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=TUNT_WhatIsPercentUnitTest) of the InterSystems IRIS platforms, plus Caché-based predecessors supporting the `/api/atelier` REST service. +> **New in Version 2.0 - Test Coverage** +> +> The v2.0 release has been entered into the [InterSystems Developer Tools Contest 2025](https://openexchange.intersystems.com/contest/42). Please support it with your vote between 28th July and 3rd August. + +This extension uses VS Code's [Testing API](https://code.visualstudio.com/api/extension-guides/testing) to discover, run and debug unit test classes built with the [%UnitTest testing framework](https://docs.intersystems.com/irislatest/csp/docbook/DocBook.UI.Page.cls?KEY=TUNT_WhatIsPercentUnitTest) of the InterSystems IRIS platforms, plus Caché-based predecessors supporting the `/api/atelier` REST service. It augments the ObjectScript, InterSystems Language Server and Server Manager extensions, which are elements of the [InterSystems ObjectScript Extension Pack](https://marketplace.visualstudio.com/items?itemName=intersystems-community.objectscript-pack). @@ -16,6 +20,12 @@ _Client-side editing workspace_ _Server-side editing workspace_ +When used alongside [Test Coverage Tool](https://openexchange.intersystems.com/package/Test-Coverage-Tool) this extension presents coverage inside VS Code: + +![Code coverage example](images/README/Coverage-example.png) + +_Code coverage example showing coverage of Test Coverage Tool's own unit tests_ + In order to support topologies in which client-side-managed test classes have to be run in the namespace of a remote server, this extension uses the `/_vscode` web application on the test-running server, no matter whether local or remote. ## Server Preparations @@ -34,6 +44,8 @@ In order to support topologies in which client-side-managed test classes have to ``` > If you previously used the `%UnitTest` framework in a namespace, be aware that you are probably replacing an existing value. Consider taking a note of that in case you need to revert. +3. If you want to gather and display test coverage data, set up [Test Coverage Tool](https://openexchange.intersystems.com/package/Test-Coverage-Tool) in the namespace(s) where your tests will execute. + ## Workspace Preparations For a workspace using client-side editing, test classes are by default sought in `.cls` files under the `internal/testing/unit_tests` subfolder, using the conventional layout of one additional subfolder per package-name element. If your test classes are located elsewhere, use the `intersystems.testingManager.client.relativeTestRoot` setting to point there. @@ -50,7 +62,7 @@ A subfolder is shown for each root folder of your workspace, which may be a mult At the level of an individual test class the final expansion shows a leaf for each `TestXXX` method. -Hovering over any level of a tests tree will reveal action buttons that run all the tests from this level down. The 'Run' button does so without stopping at any breakpoints, in contrast to the 'Debug' button. At class or method level a 'Go to Test' button opens the class code and positions the cursor appropriately. At higher levels this button navigates to Explorer View. +Hovering over any level of a tests tree will reveal action buttons that run all the tests from this level down. The 'Run Test' button does so without stopping at any breakpoints, in contrast to the 'Debug Test' button. At class or method level a 'Go to Test' button opens the class code and positions the cursor appropriately. At higher levels this button navigates to Explorer View. When a test class is open in an editor tab it displays icons in the gutter at the top of the class and at the start of each test method. These show the outcome of the most recent run, if any, and can be clicked to perform testing operations. @@ -59,6 +71,9 @@ The `...` menu of the Testing panel in Test Explorer includes several useful com ## Debugging Tests After opening a test class, click in the gutter to set a VS Code breakpoint in the normal manner. Then launch the test-run with the Debug option on the context menu of the testing icons in the gutter. +## Obtaining Test Coverage Information +Use the 'Run with Coverage' option to submit your tests to [Test Coverage Tool](https://openexchange.intersystems.com/package/Test-Coverage-Tool). When the run finishes the 'TEST COVERAGE' view will appear, usually below the 'TEST EXPLORER'. Use this to discover what proportion of executable code lines were covered by the most recent coverage run. Open sources to see color markers on line numbers showing covered (green) and not covered (red) lines. Learn more in the [VS Code documentation](https://code.visualstudio.com/docs/debugtest/testing#_test-coverage). + ## Recent Testing History The %UnitTest framework persists results of runs in server-side tables. The 'Recent History' root folder lets you explore the most recent ten sets of results for each server and namespace the workspace uses. diff --git a/images/README/Coverage-example.png b/images/README/Coverage-example.png new file mode 100644 index 0000000..2ba1198 Binary files /dev/null and b/images/README/Coverage-example.png differ diff --git a/package-lock.json b/package-lock.json index 12995f5..296aabd 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "testingmanager", - "version": "0.2.3-SNAPSHOT", + "version": "2.0.0-SNAPSHOT", "lockfileVersion": 2, "requires": true, "packages": { "": { "name": "testingmanager", - "version": "0.2.3-SNAPSHOT", + "version": "2.0.0-SNAPSHOT", "license": "MIT", "dependencies": { "axios": "^0.24", @@ -644,10 +644,11 @@ } }, "node_modules/brace-expansion": { - "version": "1.1.11", - "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz", - "integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==", + "version": "1.1.12", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz", + "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==", "dev": true, + "license": "MIT", "dependencies": { "balanced-match": "^1.0.0", "concat-map": "0.0.1" @@ -3506,9 +3507,9 @@ } }, "brace-expansion": { - "version": "1.1.11", - "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.11.tgz", - "integrity": "sha512-iCuPHDFgrHX7H2vEI/5xpz07zSHB00TpugqhmYtVmMO6518mCuRMoOYFldEBl0g187ufozdaHgWKcYFb61qGiA==", + "version": "1.1.12", + "resolved": "https://registry.npmjs.org/brace-expansion/-/brace-expansion-1.1.12.tgz", + "integrity": "sha512-9T9UjW3r0UW5c1Q7GTwllptXwhvYmEzFhzMfZ9H7FQWt+uZePjZPjBP/W1ZEyZ1twGWom5/56TF4lPcqjnDHcg==", "dev": true, "requires": { "balanced-match": "^1.0.0", diff --git a/package.json b/package.json index ba61f81..f1f7051 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "testingmanager", "displayName": "InterSystems Testing Manager", - "version": "0.2.4-SNAPSHOT", + "version": "2.0.0-SNAPSHOT", "preview": true, "publisher": "intersystems-community", "description": "Manage testing on InterSystems servers.", diff --git a/src/commonRunTestsHandler.ts b/src/commonRunTestsHandler.ts index b709d48..fdf6542 100644 --- a/src/commonRunTestsHandler.ts +++ b/src/commonRunTestsHandler.ts @@ -1,9 +1,10 @@ import * as vscode from 'vscode'; import { IServerSpec } from "@intersystems-community/intersystems-servermanager"; -import { allTestRuns, extensionId, osAPI } from './extension'; +import { allTestRuns, extensionId, osAPI, OurTestItem } from './extension'; import { relativeTestRoot } from './localTests'; import logger from './logger'; import { makeRESTRequest } from './makeRESTRequest'; +import { OurFileCoverage } from './ourFileCoverage'; export async function commonRunTestsHandler(controller: vscode.TestController, resolveItemChildren: (item: vscode.TestItem) => Promise, request: vscode.TestRunRequest, cancellation: vscode.CancellationToken) { logger.debug(`commonRunTestsHandler invoked by controller id=${controller.id}`); @@ -14,14 +15,29 @@ export async function commonRunTestsHandler(controller: vscode.TestController, r // We don't yet support running only some TestXXX methods in a testclass const mapAuthorities = new Map>(); const runIndices: number[] =[]; - const queue: vscode.TestItem[] = []; + const queue: OurTestItem[] = []; + const coverageRequest = request.profile?.kind === vscode.TestRunProfileKind.Coverage; // Loop through all included tests, or all known tests, and add them to our queue if (request.include) { - request.include.forEach(test => queue.push(test)); + request.include.forEach((test: OurTestItem) => { + if (!coverageRequest || test.supportsCoverage) { + queue.push(test); + } + }); } else { // Run was launched from controller's root level - controller.items.forEach(test => queue.push(test)); + controller.items.forEach((test: OurTestItem) => { + if (!coverageRequest || test.supportsCoverage) { + queue.push(test); + } + }); + } + + if (coverageRequest && !queue.length) { + // No tests to run, but coverage requested + vscode.window.showErrorMessage("[Test Coverage Tool](https://openexchange.intersystems.com/package/Test-Coverage-Tool) not found.", ); + return; } // Process every test that was queued. Recurse down to leaves (testmethods) and build a map of their parents (classes) @@ -69,7 +85,7 @@ export async function commonRunTestsHandler(controller: vscode.TestController, r if (mapAuthorities.size === 0) { // Nothing included - vscode.window.showWarningMessage(`Empty test run`); + vscode.window.showErrorMessage("Empty test run.", { modal: true }); return; } @@ -143,6 +159,49 @@ export async function commonRunTestsHandler(controller: vscode.TestController, r console.log(error); } + // Map of uri strings checked for presence of a coverage.list file, recording the relative path of those that were found + const mapCoverageLists = new Map(); + for await (const mapInstance of mapTestClasses) { + const key = mapInstance[0]; + const pathParts = key.split('/'); + pathParts.pop(); + const sourceBaseUri = mapInstance[1].uri?.with({ path: mapInstance[1].uri.path.split('/').slice(0, -pathParts.length).join('/') }); + if (!sourceBaseUri) { + console.log(`No sourceBaseUri for key=${key}`); + continue; + } + while (pathParts.length > 1) { + const currentPath = pathParts.join('/'); + // Check for coverage.list file here + const coverageListUri = sourceBaseUri.with({ path: sourceBaseUri.path.concat(`${currentPath}/coverage.list`) }); + if (mapCoverageLists.has(coverageListUri.toString())) { + // Already checked this uri path, and therefore all its ancestors + break; + } + try { + await vscode.workspace.fs.stat(coverageListUri); + mapCoverageLists.set(coverageListUri.toString(), currentPath); + } catch (error) { + if (error.code !== vscode.FileSystemError.FileNotFound().code) { + console.log(`Error checking for ${coverageListUri.toString()}:`, error); + } + mapCoverageLists.set(coverageListUri.toString(), ''); + } + pathParts.pop(); + } + } + // Copy all coverage.list files found into the corresponding place under testRoot + for await (const [uriString, path] of mapCoverageLists) { + if (path.length > 0) { + const coverageListUri = vscode.Uri.parse(uriString, true); + try { + await vscode.workspace.fs.copy(coverageListUri, testRoot.with({ path: testRoot.path.concat(`${path}/coverage.list`) })); + } catch (error) { + console.log(`Error copying ${coverageListUri.path}:`, error); + } + } + } + // Next, copy the classes into the folder as a package hierarchy for await (const mapInstance of mapTestClasses) { const key = mapInstance[0]; @@ -186,11 +245,18 @@ export async function commonRunTestsHandler(controller: vscode.TestController, r } } + let managerClass = "%UnitTest.Manager"; + if (coverageRequest) { + managerClass = "TestCoverage.Manager"; + request.profile.loadDetailedCoverage = async (testRun, fileCoverage, token) => { + return fileCoverage instanceof OurFileCoverage ? fileCoverage.loadDetailedCoverage() : []; + }; + } const configuration = { "type": "objectscript", "request": "launch", "name": `${controller.id.split("-").pop()}Tests:${serverSpec.name}:${namespace}:${username}`, - "program": `##class(%UnitTest.Manager).RunTest("${testSpec}","${runQualifiers}")`, + "program": `##class(${managerClass}).RunTest("${testSpec}","${runQualifiers}")`, // Extra properties needed by our DebugAdapterTracker "testingRunIndex": runIndex, diff --git a/src/coverage.ts b/src/coverage.ts new file mode 100644 index 0000000..c923a15 --- /dev/null +++ b/src/coverage.ts @@ -0,0 +1,70 @@ +import * as vscode from 'vscode'; +import { makeRESTRequest } from './makeRESTRequest'; +import logger from './logger'; +import { OurTestRun } from './extension'; +import { serverSpecForUri } from './historyExplorer'; +import { OurFileCoverage } from './ourFileCoverage'; + +export async function processCoverage(serverName: string, namespace: string, run: OurTestRun): Promise { + const uri = run.debugSession?.workspaceFolder?.uri; + const coverageIndex = run.debugSession?.configuration.coverageIndex; + logger.debug(`processCoverage: serverName=${serverName}, namespace=${namespace}, uri=${uri?.toString()}, coverageIndex=${coverageIndex}`); + if (uri) { + const fileCoverageResults = await getFileCoverageResults(uri, namespace, coverageIndex || 0); + if (fileCoverageResults.length > 0) { + logger.debug(`Coverage results for run ${coverageIndex}: ${JSON.stringify(fileCoverageResults)}`); + fileCoverageResults.forEach(fileCoverage => { + run.addCoverage(fileCoverage); + }) + } else { + logger.debug(`No coverage results found for run ${coverageIndex}`); + } + } +} + +export async function getFileCoverageResults(folderUri: vscode.Uri, namespace: string, coverageIndex: number): Promise { + const serverSpec = serverSpecForUri(folderUri); + const fileCoverageResults: vscode.FileCoverage[] = []; + if (!serverSpec) { + logger.error(`No server spec found for URI: ${folderUri.toString()}`); + return fileCoverageResults; + } + const exportSettings = vscode.workspace.getConfiguration('objectscript.export', folderUri); + const response = await makeRESTRequest( + "POST", + serverSpec, + { apiVersion: 1, namespace, path: "/action/query" }, + { + query: "SELECT cu.Hash, cu.Name Name, cu.Type, abcu.ExecutableLines, CoveredLines, ExecutableMethods, CoveredMethods, RtnLine FROM TestCoverage_Data_Aggregate.ByCodeUnit abcu, TestCoverage_Data.CodeUnit cu WHERE abcu.CodeUnit = cu.Hash AND Run = ? ORDER BY Name", + parameters: [coverageIndex], + }, + ); + if (response) { + response?.data?.result?.content?.forEach(element => { + const fileType = element.Type.toLowerCase(); + let pathPrefix = '' + if (folderUri.scheme === 'file') { + pathPrefix = exportSettings.folder; + if (pathPrefix && !pathPrefix.startsWith('/')) { + pathPrefix = `/${pathPrefix}`; + } + if (exportSettings.atelier) { + pathPrefix += '/' + fileType; + } + } + const fileUri = folderUri.with({ path: folderUri.path.concat(pathPrefix, `/${element.Name.replace(/\./g, '/')}.${fileType}`) }); + logger.debug(`getFileCoverageResults element: ${JSON.stringify(element)}`); + logger.debug(`getFileCoverageResults fileUri: ${fileUri.toString()}`); + const fileCoverage = new OurFileCoverage( + coverageIndex, + element.Hash, + fileUri, + new vscode.TestCoverageCount(element.CoveredLines, element.ExecutableLines), + undefined, + new vscode.TestCoverageCount(element.CoveredMethods, element.ExecutableMethods) + ); + fileCoverageResults.push(fileCoverage); + }); + } + return fileCoverageResults; +} diff --git a/src/debugTracker.ts b/src/debugTracker.ts index e6cf100..e4e7bef 100644 --- a/src/debugTracker.ts +++ b/src/debugTracker.ts @@ -1,6 +1,7 @@ import * as vscode from 'vscode'; -import { allTestRuns, loadedTestController, localTestController, TestRun } from './extension'; +import { allTestRuns, loadedTestController, localTestController, OurTestRun } from './extension'; import { refreshHistoryRootItem } from './historyExplorer'; +import { processCoverage } from './coverage'; export class DebugTracker implements vscode.DebugAdapterTracker { @@ -8,7 +9,7 @@ export class DebugTracker implements vscode.DebugAdapterTracker { private serverName: string; private namespace: string; private testController: vscode.TestController - private run?: TestRun; + private run?: OurTestRun; private testingIdBase: string; private className?: string; private testMethodName?: string; @@ -56,6 +57,14 @@ export class DebugTracker implements vscode.DebugAdapterTracker { } const line: string = (message.body.output as string).replace(/\n/, ''); this.run.appendOutput(line + '\r\n'); + + const coverageMatch = line.match(/^(?:http|https):\/\/.*\/TestCoverage\.UI\.AggregateResultViewer\.cls\?Index=(\d+)/); + if (coverageMatch && this.run.debugSession) { + const coverageIndex = Number(coverageMatch[1]); + this.run.debugSession.configuration.coverageIndex = coverageIndex; + console.log(`Coverage index set to ${coverageIndex}`); + } + if (this.className === undefined) { const classBegin = line.match(/^ ([%\dA-Za-z][\dA-Za-z0-9\.]*) begins \.\.\./); if (classBegin) { @@ -139,10 +148,13 @@ export class DebugTracker implements vscode.DebugAdapterTracker { //console.log(`**Starting session ${this.session.name}, run.name = ${this.run?.name}`); } - onWillStopSession(): void { - //console.log(`**Stopping session ${this.session.name}`); + async onWillStopSession(): Promise { + console.log(`**Stopping session ${this.session.name}`); if (this.run) { + await processCoverage(this.serverName, this.namespace, this.run); + //console.log(`**processCoverage done`); this.run.end(); + //console.log(`**run.end() done`); refreshHistoryRootItem(this.serverName, this.namespace); } diff --git a/src/extension.ts b/src/extension.ts index a89a48a..5acd996 100644 --- a/src/extension.ts +++ b/src/extension.ts @@ -14,10 +14,15 @@ export let historyBrowserController: vscode.TestController; export let osAPI: any; export let smAPI: serverManager.ServerManagerAPI | undefined; -export interface TestRun extends vscode.TestRun { +export interface OurTestRun extends vscode.TestRun { debugSession?: vscode.DebugSession } -export const allTestRuns: (TestRun | undefined)[] = []; + +export interface OurTestItem extends vscode.TestItem { + supportsCoverage?: boolean +} + +export const allTestRuns: (OurTestRun | undefined)[] = []; async function getServerManagerAPI(): Promise { const targetExtension = vscode.extensions.getExtension("intersystems-community.servermanager"); diff --git a/src/historyExplorer.ts b/src/historyExplorer.ts index b340967..8f89f9b 100644 --- a/src/historyExplorer.ts +++ b/src/historyExplorer.ts @@ -1,6 +1,6 @@ import * as vscode from 'vscode'; import { IServerSpec } from "@intersystems-community/intersystems-servermanager"; -import { historyBrowserController, osAPI, smAPI } from './extension'; +import { historyBrowserController, osAPI, OurTestItem, smAPI } from './extension'; import logger from './logger'; import { makeRESTRequest } from './makeRESTRequest'; @@ -46,6 +46,24 @@ export async function setupHistoryExplorerController() { } } +export function serverSpecForUri(uri: vscode.Uri): IServerSpec | undefined { + const server = osAPI.serverForUri(uri); + if (server) { + return { + username: server.username, + password: server.password, + name: server.serverName, + webServer: { + host: server.host, + port: server.port, + pathPrefix: server.pathPrefix, + scheme: server.scheme + } + }; + } + return undefined; +} + export async function serverSpec(item: vscode.TestItem): Promise { const serverName = item.id.split(':')[0]; if (serverName) { @@ -54,24 +72,16 @@ export async function serverSpec(item: vscode.TestItem): Promise { - const child = controller.createTestItem( + const child: OurTestItem = controller.createTestItem( `${item.id}:${element.InstanceIndex}`, `${element.DateTime}`, portalUri.with({ query: `Index=${element.InstanceIndex}&$NAMESPACE=${namespace}` }) @@ -97,6 +107,7 @@ async function addTestInstances(item: vscode.TestItem, controller: vscode.TestCo child.sortText = (1e12 - element.InstanceIndex).toString().padStart(12, "0"); child.description = `run ${element.InstanceIndex}`; child.canResolveChildren = true; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); }); } @@ -104,7 +115,7 @@ async function addTestInstances(item: vscode.TestItem, controller: vscode.TestCo item.busy = false; } -async function addTestSuites(item: vscode.TestItem, controller: vscode.TestController) { +async function addTestSuites(item: OurTestItem, controller: vscode.TestController) { const spec = await serverSpec(item); const parts = item.id.split(':'); const namespace = parts[1]; @@ -122,8 +133,9 @@ async function addTestSuites(item: vscode.TestItem, controller: vscode.TestContr if (response) { const run = controller.createTestRun(new vscode.TestRunRequest(), `Item '${item.label}' history`, false); response?.data?.result?.content?.forEach(element => { - const child = controller.createTestItem(`${item.id}:${element.ID}`, `${element.Name}`); + const child: OurTestItem = controller.createTestItem(`${item.id}:${element.ID}`, `${element.Name}`); child.canResolveChildren = true; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); if (element.Status) { run.passed(child, element.Duration * 1000); @@ -137,7 +149,7 @@ async function addTestSuites(item: vscode.TestItem, controller: vscode.TestContr } } -async function addTestCases(item: vscode.TestItem, controller: vscode.TestController) { +async function addTestCases(item: OurTestItem, controller: vscode.TestController) { const spec = await serverSpec(item); const parts = item.id.split(':'); const namespace = parts[1]; @@ -155,8 +167,9 @@ async function addTestCases(item: vscode.TestItem, controller: vscode.TestContro if (response) { const run = controller.createTestRun(new vscode.TestRunRequest(), `Item '${item.label}' history`, false); response?.data?.result?.content?.forEach(element => { - const child = controller.createTestItem(`${item.id}:${element.ID}`, `${element.Name.split('.').pop()}`); + const child: OurTestItem = controller.createTestItem(`${item.id}:${element.ID}`, `${element.Name.split('.').pop()}`); child.canResolveChildren = true; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); if (element.Status) { run.passed(child, element.Duration * 1000); @@ -170,7 +183,7 @@ async function addTestCases(item: vscode.TestItem, controller: vscode.TestContro } } -async function addTestMethods(item: vscode.TestItem, controller: vscode.TestController) { +async function addTestMethods(item: OurTestItem, controller: vscode.TestController) { const spec = await serverSpec(item); const parts = item.id.split(':'); const namespace = parts[1]; @@ -190,8 +203,9 @@ async function addTestMethods(item: vscode.TestItem, controller: vscode.TestCont response?.data?.result?.content?.forEach(element => { const methodName: string = element.Name; // We drop the first 4 characters of the method name because they should always be "Test" - const child = controller.createTestItem(`${item.id}:${element.ID}`, `${methodName.slice(4)}`); + const child: OurTestItem = controller.createTestItem(`${item.id}:${element.ID}`, `${methodName.slice(4)}`); child.canResolveChildren = true; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); // Remember result fields so they can be reinstated when the descendant Asserts are 'run' @@ -208,7 +222,7 @@ async function addTestMethods(item: vscode.TestItem, controller: vscode.TestCont } } -async function addTestAsserts(item: vscode.TestItem, controller: vscode.TestController) { +async function addTestAsserts(item: OurTestItem, controller: vscode.TestController) { const spec = await serverSpec(item); const parts = item.id.split(':'); const namespace = parts[1]; @@ -238,10 +252,11 @@ async function addTestAsserts(item: vscode.TestItem, controller: vscode.TestCont } response?.data?.result?.content?.forEach(element => { - const child = controller.createTestItem(`${item.id}:${element.ID}`, `${element.Action}`); + const child: OurTestItem = controller.createTestItem(`${item.id}:${element.ID}`, `${element.Action}`); child.sortText = `${element.Counter.toString().padStart(element.MaxCounter.toString().length, "0")}`; child.description = element.Description; child.canResolveChildren = false; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); if (element.Status) { run.passed(child); @@ -268,8 +283,9 @@ export function replaceRootItems(controller: vscode.TestController, schemes?: st if (server.namespace) { const key = server.serverName + ":" + server.namespace.toUpperCase(); if (!rootMap.has(key)) { - const item = controller.createTestItem(key, key, folder.uri); + const item: OurTestItem = controller.createTestItem(key, key, folder.uri); item.canResolveChildren = true; + item.supportsCoverage = false; rootMap.set(key, item); } } diff --git a/src/localTests.ts b/src/localTests.ts index ac1729a..6e2d9dc 100644 --- a/src/localTests.ts +++ b/src/localTests.ts @@ -1,11 +1,12 @@ import * as vscode from 'vscode'; import { commonRunTestsHandler } from './commonRunTestsHandler'; -import { localTestController, osAPI } from './extension'; +import { localTestController, OurTestItem } from './extension'; import logger from './logger'; +import { resolveServerSpecAndNamespace, supportsCoverage } from './utils'; const isResolvedMap = new WeakMap(); -async function resolveItemChildren(item: vscode.TestItem) { +async function resolveItemChildren(item: OurTestItem) { if (item) { isResolvedMap.set(item, true); const itemUri = item.uri; @@ -15,15 +16,17 @@ async function resolveItemChildren(item: vscode.TestItem) { const contents = await vscode.workspace.fs.readDirectory(itemUri); contents.filter((entry) => entry[1] === vscode.FileType.Directory).forEach((entry) => { const name = entry[0]; - const child = localTestController.createTestItem(`${item.id}${name}.`, name, itemUri.with({path: `${itemUri.path}/${name}`})); + const child: OurTestItem = localTestController.createTestItem(`${item.id}${name}.`, name, itemUri.with({path: `${itemUri.path}/${name}`})); child.canResolveChildren = true; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); }); contents.filter((entry) => entry[1] === vscode.FileType.File).forEach((entry) => { const name = entry[0]; if (name.endsWith('.cls')) { - const child = localTestController.createTestItem(`${item.id}${name.slice(0, name.length - 4)}`, name, itemUri.with({path: `${itemUri.path}/${name}`})); + const child: OurTestItem = localTestController.createTestItem(`${item.id}${name.slice(0, name.length - 4)}`, name, itemUri.with({path: `${itemUri.path}/${name}`})); child.canResolveChildren = true; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); } }); @@ -46,9 +49,10 @@ async function resolveItemChildren(item: vscode.TestItem) { const match = lineText.match(/^Method Test(.+)\(/); if (match) { const testName = match[1]; - const child = localTestController.createTestItem(`${item.id}:Test${testName}`, testName, itemUri); + const child: OurTestItem = localTestController.createTestItem(`${item.id}:Test${testName}`, testName, itemUri); child.range = new vscode.Range(new vscode.Position(index, 0), new vscode.Position(index + 1, 0)) child.canResolveChildren = false; + child.supportsCoverage = item.supportsCoverage; item.children.add(child); if (!child.parent) { console.log(`*** BUG - child (id=${child.id}) has no parent after item.children.add(child) where item.id=${item.id}`); @@ -67,10 +71,11 @@ async function resolveItemChildren(item: vscode.TestItem) { } else { // Root items - replaceLocalRootItems(localTestController); + await replaceLocalRootItems(localTestController); if (localTestController.items.size > 0) { localTestController.createRunProfile('Run Local Tests', vscode.TestRunProfileKind.Run, runTestsHandler, true); localTestController.createRunProfile('Debug Local Tests', vscode.TestRunProfileKind.Debug, runTestsHandler); + localTestController.createRunProfile('Run Local Tests with Coverage', vscode.TestRunProfileKind.Coverage, runTestsHandler); } } } @@ -111,24 +116,25 @@ export function relativeTestRoot(folder: vscode.WorkspaceFolder): string { /* Replace root items with one item for each file-type workspace root for which a named server can be identified */ -function replaceLocalRootItems(controller: vscode.TestController) { +async function replaceLocalRootItems(controller: vscode.TestController) { const rootItems: vscode.TestItem[] = []; const rootMap = new Map(); - vscode.workspace.workspaceFolders?.forEach(folder => { + for await (const folder of vscode.workspace.workspaceFolders || []) { if (folder.uri.scheme === 'file') { - const server = osAPI.serverForUri(folder.uri); - if (server?.namespace) { - const key = server.serverName + ":" + server.namespace + ":"; + const { serverSpec, namespace } = await resolveServerSpecAndNamespace(folder.uri); + if (serverSpec && namespace) { + const key = serverSpec.name + ":" + namespace + ":"; if (!rootMap.has(key)) { const relativeRoot = relativeTestRoot(folder); - const item = controller.createTestItem(key, folder.name, folder.uri.with({path: `${folder.uri.path}/${relativeRoot}`})); + const item: OurTestItem = controller.createTestItem(key, folder.name, folder.uri.with({path: `${folder.uri.path}/${relativeRoot}`})); item.description = relativeRoot; item.canResolveChildren = true; + item.supportsCoverage = await supportsCoverage(folder); rootMap.set(key, item); } } } - }); + } rootMap.forEach(item => rootItems.push(item)); controller.items.replace(rootItems); } diff --git a/src/ourFileCoverage.ts b/src/ourFileCoverage.ts new file mode 100644 index 0000000..37119f0 --- /dev/null +++ b/src/ourFileCoverage.ts @@ -0,0 +1,121 @@ +import * as vscode from 'vscode'; +import logger from './logger'; +import { IServerSpec } from '@intersystems-community/intersystems-servermanager'; +import { makeRESTRequest } from './makeRESTRequest'; +import { osAPI } from './extension'; + +export class OurFileCoverage extends vscode.FileCoverage { + + public readonly codeUnit: string; + private coverageIndex: number; + + constructor(coverageIndex: number, codeUnit: string, uri: vscode.Uri, statementCoverage: vscode.TestCoverageCount, branchCoverage?: vscode.TestCoverageCount, declarationCoverage?: vscode.TestCoverageCount) { + super(uri, statementCoverage, branchCoverage, declarationCoverage); + this.coverageIndex = coverageIndex; + this.codeUnit = codeUnit; + } + + async loadDetailedCoverage(): Promise { + logger.debug(`loadDetailedCoverage invoked for ${this.codeUnit} (${this.uri.toString()})`); + const detailedCoverage: vscode.FileCoverageDetail[] = []; + const server = osAPI.serverForUri(this.uri); + const serverSpec: IServerSpec = { + username: server.username, + password: server.password, + name: server.serverName, + webServer: { + host: server.host, + port: server.port, + pathPrefix: server.pathPrefix, + scheme: server.scheme + } + }; + const namespace: string = server.namespace.toUpperCase(); + + // Get map of lines to methods + const mapMethods: Map = new Map(); + const lineToMethod: string[] = []; + let response = await makeRESTRequest( + "POST", + serverSpec, + { apiVersion: 1, namespace, path: "/action/query" }, + { + query: "SELECT element_key Line, LineToMethodMap Method FROM TestCoverage_Data.CodeUnit_LineToMethodMap WHERE CodeUnit = ? ORDER BY Line", + parameters: [this.codeUnit], + }, + ); + if (response) { + let previousMethod = ""; + let previousLine = 0; + response?.data?.result?.content?.forEach(element => { + const thisLine = Number(element.Line); + mapMethods.set(Number(element.Line), element.Method); + lineToMethod.fill(previousMethod, previousLine, thisLine -1); + previousMethod = element.Method; + previousLine = thisLine; + }); + } + + response = await makeRESTRequest( + "POST", + serverSpec, + { apiVersion: 1, namespace, path: "/action/query" }, + { + query: "SELECT TestCoverage_UI.fnVSCodeInt8Bitstring(cu.ExecutableLines) i8bsExecutableLines, TestCoverage_UI.fnVSCodeInt8Bitstring(cov.CoveredLines) i8bsCoveredLines FROM TestCoverage_Data.CodeUnit cu, TestCoverage_Data.Coverage cov WHERE cu.Hash = cov.Hash AND Run = ? AND cu.Hash = ? AND TestPath = 'all tests'", + parameters: [this.coverageIndex, this.codeUnit], + }, + ); + if (response) { + response?.data?.result?.content?.forEach(element => { + logger.debug(`getFileCoverageResults element: ${JSON.stringify(element)}`); + // Process the Uint8Bitstring values for executable and covered lines + const i8bsExecutableLines = element.i8bsExecutableLines; + const i8bsCoveredLines = element.i8bsCoveredLines; + for (let lineChunk = 0; lineChunk < i8bsExecutableLines.length; lineChunk++) { + const executableLines = i8bsExecutableLines.charCodeAt(lineChunk); + const coveredLines = i8bsCoveredLines.charCodeAt(lineChunk); + for (let bitIndex = 0; bitIndex < 8; bitIndex++) { + if ((executableLines & (1 << bitIndex)) !== 0) { + const lineNumber = lineChunk * 8 + bitIndex + 1; + const isCovered = (coveredLines & (1 << bitIndex)) !== 0; + const range = new vscode.Range(new vscode.Position(lineNumber - 1, 0), new vscode.Position(lineNumber - 1, Number.MAX_VALUE)); + const statementCoverage = new vscode.StatementCoverage(isCovered, range); + detailedCoverage.push(statementCoverage); + } + } + } + }); + } + + // Add declaration (method) coverage + response = await makeRESTRequest( + "POST", + serverSpec, + { apiVersion: 1, namespace, path: "/action/query" }, + { + query: "SELECT element_key StartLine, LineToMethodMap Method FROM TestCoverage_Data.CodeUnit_LineToMethodMap WHERE CodeUnit = ? ORDER BY StartLine", + parameters: [this.codeUnit], + }, + ); + if (response) { + let previousMethod = ""; + let previousStartLine = 0; + response?.data?.result?.content?.forEach(element => { + if (previousMethod && previousStartLine) { + const start = new vscode.Position(Number(previousStartLine) - 1, 0); + const end = new vscode.Position(Number(element.StartLine) - 2, Number.MAX_VALUE); + detailedCoverage.push(new vscode.DeclarationCoverage(previousMethod, true, new vscode.Range(start, end))); + } + previousMethod = element.Method; + previousStartLine = Number(element.StartLine); + }); + if (previousMethod && previousStartLine) { + const start = new vscode.Position(Number(previousStartLine) - 1, 0); + const end = new vscode.Position(Number.MAX_VALUE, Number.MAX_VALUE); + detailedCoverage.push(new vscode.DeclarationCoverage(previousMethod, true, new vscode.Range(start, end))); + } + } + return detailedCoverage; + } + +} diff --git a/src/serverTests.ts b/src/serverTests.ts index 2a39fdf..8e10fa5 100644 --- a/src/serverTests.ts +++ b/src/serverTests.ts @@ -1,11 +1,11 @@ import * as vscode from 'vscode'; -import { loadedTestController } from './extension'; +import { loadedTestController, OurTestItem } from './extension'; import { replaceRootItems, serverSpec } from './historyExplorer'; import logger from './logger'; import { makeRESTRequest } from './makeRESTRequest'; import { commonRunTestsHandler } from './commonRunTestsHandler'; -async function resolveItemChildren(item?: vscode.TestItem) { +async function resolveItemChildren(item?: OurTestItem) { if (item) { item.busy = true; const spec = await serverSpec(item); @@ -23,7 +23,7 @@ async function resolveItemChildren(item?: vscode.TestItem) { if (response) { for await (const element of response?.data?.result?.content) { const fullClassName: string = element.Name; - const tiClass = loadedTestController.createTestItem( + const tiClass: OurTestItem = loadedTestController.createTestItem( `${item.id}:${fullClassName}`, fullClassName, vscode.Uri.from({ @@ -33,6 +33,7 @@ async function resolveItemChildren(item?: vscode.TestItem) { query: item.uri?.query }) ); + tiClass.supportsCoverage = item.supportsCoverage; const symbols = await vscode.commands.executeCommand>('vscode.executeDocumentSymbolProvider', tiClass.uri); if (symbols?.length === 1 && symbols[0].kind === vscode.SymbolKind.Class) { const symbol = symbols[0]; @@ -40,12 +41,13 @@ async function resolveItemChildren(item?: vscode.TestItem) { (symbol as vscode.DocumentSymbol).children.forEach(childSymbol => { if (childSymbol.kind === vscode.SymbolKind.Method && childSymbol.name.startsWith("Test")) { const testMethodName = childSymbol.name; - const tiMethod = loadedTestController.createTestItem( + const tiMethod: OurTestItem = loadedTestController.createTestItem( `${tiClass.id}:${testMethodName}`, testMethodName.slice(4), tiClass.uri ); tiMethod.range = childSymbol.range; + tiMethod.supportsCoverage = tiClass.supportsCoverage; tiClass.children.add(tiMethod); } }); @@ -66,6 +68,7 @@ async function resolveItemChildren(item?: vscode.TestItem) { if (loadedTestController.items.size > 0) { loadedTestController.createRunProfile('Run Server Tests', vscode.TestRunProfileKind.Run, runTestsHandler, true); loadedTestController.createRunProfile('Debug Server Tests', vscode.TestRunProfileKind.Debug, runTestsHandler); + loadedTestController.createRunProfile('Run Server Tests with Coverage', vscode.TestRunProfileKind.Coverage, runTestsHandler); } } } diff --git a/src/utils.ts b/src/utils.ts new file mode 100644 index 0000000..9104cd7 --- /dev/null +++ b/src/utils.ts @@ -0,0 +1,104 @@ +import * as vscode from 'vscode'; +import logger from './logger'; +import { makeRESTRequest } from './makeRESTRequest'; +import { IServerSpec } from '@intersystems-community/intersystems-servermanager'; +import { osAPI } from './extension'; + +export async function resolveServerSpecAndNamespace(uri: vscode.Uri): Promise<{ serverSpec: IServerSpec | undefined, namespace?: string }> { + const server = await osAPI.asyncServerForUri(uri); + if (!server) { + logger.error(`No server found for URI: ${uri.toString()}`); + return { serverSpec: undefined, namespace: undefined }; + } + const serverSpec: IServerSpec = { + username: server.username, + password: server.password, + name: server.serverName, + webServer: { + host: server.host, + port: server.port, + pathPrefix: server.pathPrefix, + scheme: server.scheme + } + }; + return { serverSpec, namespace: server.namespace.toUpperCase() }; +} + +export async function supportsCoverage(folder: vscode.WorkspaceFolder): Promise { + const { serverSpec, namespace } = await resolveServerSpecAndNamespace(folder.uri); + + if (!serverSpec) { + logger.error(`No server spec found for URI: ${folder.uri.toString()}`); + return false; // No server spec means we can't check coverage support + } + if (!namespace) { + logger.error(`No namespace found for URI: ${folder.uri.toString()}`); + return false; // No server spec means we can't check coverage support + } + logger.debug(`Checking coverage support for namespace: ${namespace}`); + let response = await makeRESTRequest( + "HEAD", + serverSpec, + { apiVersion: 1, namespace, path: "/doc/TestCoverage.Data.CodeUnit.cls" } + ); + if (response?.status !== 200) { + return false; + } + response = await makeRESTRequest( + "HEAD", + serverSpec, + { apiVersion: 1, namespace, path: "/doc/TestCoverage.UI.VSCodeUtils.cls" } + ); + if (response?.status === 200) { + return true; + } + + return await createSQLUtilFunctions(serverSpec, namespace); +} + +async function createSQLUtilFunctions(serverSpec: IServerSpec, namespace: string): Promise { + logger.debug(`Creating SQL Util functions for namespace: ${namespace}`); + + const functionDDL = ` +CREATE FUNCTION fnVSCodeInt8Bitstring( + bitstring VARCHAR(32767) +) + FOR TestCoverage.UI.VSCodeUtils + RETURNS VARCHAR(32767) + LANGUAGE OBJECTSCRIPT + { + NEW output,iMod8,char,weight,i,bitvalue + SET output = "", iMod8=-1, char=0, weight=1 + FOR i=1:1:$BITCOUNT(bitstring) { + SET bitvalue = $BIT(bitstring, i) + SET iMod8 = (i-1)#8 + IF bitvalue { + SET char = char+weight + } + SET weight = weight*2 + IF iMod8 = 7 { + SET output = output_$CHAR(char) + SET char = 0, weight = 1 + SET iMod8 = -1 + } + } + if iMod8 > -1 { + SET output = output_$CHAR(char) + } + QUIT output + } + `; + const response = await makeRESTRequest( + "POST", + serverSpec, + { apiVersion: 1, namespace, path: "/action/query" }, + { + query: functionDDL + } + ); + if (!response || response.status !== 200 || response.data?.status?.errors?.length) { + vscode.window.showErrorMessage(`Failed to create SQL Util function(s) in namespace ${namespace}: ${response?.data?.status?.summary || 'Unknown error'}`, { modal: true }); + return false; + } + return true; +}