Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .github/workflows/integration-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,13 @@ on:
- main
- heretto
paths:
- 'src/heretto*.js'
- 'src/heretto*'
- '.github/workflows/integration-tests.yml'
pull_request:
branches:
- main
paths:
- 'src/heretto*.js'
- 'src/heretto*'
- '.github/workflows/integration-tests.yml'
workflow_dispatch:
# Allow manual triggering for testing
Expand Down Expand Up @@ -41,6 +41,9 @@ jobs:
- name: Install dependencies
run: npm ci

- name: Build
run: npm run build

- name: Run integration tests
env:
CI: 'true'
Expand Down
2 changes: 2 additions & 0 deletions .github/workflows/npm-test.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ jobs:
registry-url: https://registry.npmjs.org/

- run: npm ci
- run: npm run build
- run: npm test
Comment on lines 40 to 42
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workflow runs npm run build and then npm test, but npm run build currently triggers postbuild which runs tests. This results in tests running twice on every CI job. Either remove the postbuild test hook or drop the explicit npm test step here.

Copilot uses AI. Check for mistakes.

publish-npm:
Expand All @@ -53,6 +54,7 @@ jobs:
registry-url: https://registry.npmjs.org/

- run: npm ci
- run: npm run build
- run: npm publish
env:
NODE_AUTH_TOKEN: ${{secrets.npm_token}}
Expand Down
46 changes: 46 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

17 changes: 16 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,20 @@
"name": "doc-detective-resolver",
"version": "3.6.2",
"description": "Detect and resolve docs into Doc Detective tests.",
"main": "src/index.js",
"main": "dist/index.js",
"types": "dist/index.d.ts",
"exports": {
".": {
"require": "./dist/index.js",
"import": "./dist/index.mjs",
"types": "./dist/index.d.ts"
}
},
"scripts": {
"compile": "tsc && node scripts/createEsmWrapper.js",
"build": "npm run compile",
"prebuild": "rm -rf dist",
"postbuild": "npm run test",
Comment on lines +15 to +18
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

prebuild uses rm -rf dist, which will fail on Windows runners. This repo’s workflow matrix includes windows-latest, so npm run build will break CI there. Use a cross-platform delete (e.g., rimraf dist or a small Node script) instead.

Copilot uses AI. Check for mistakes.
"test": "mocha src/*.test.js --ignore src/*.integration.test.js",
"test:integration": "mocha src/*.integration.test.js --timeout 600000",
"test:all": "mocha src/*.test.js --timeout 600000",
Comment on lines +18 to 21
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With tests now importing from dist/, npm test will fail unless a build has already run. At the same time, postbuild runs npm run test, and the workflow also runs npm test after npm run build, causing tests to run twice. Consider moving compilation to pretest (or updating the test script to build first) and removing postbuild to avoid duplicate test runs.

Copilot uses AI. Check for mistakes.
Expand Down Expand Up @@ -37,13 +49,16 @@
"posthog-node": "^5.18.1"
},
"devDependencies": {
"@types/adm-zip": "^0.5.7",
"@types/node": "^22.10.5",
"body-parser": "^2.2.1",
"chai": "^6.2.2",
"express": "^5.2.1",
"mocha": "^11.7.5",
"proxyquire": "^2.1.3",
"semver": "^7.7.3",
"sinon": "^21.0.1",
"typescript": "^5.7.3",
"yaml": "^2.8.2"
}
}
21 changes: 21 additions & 0 deletions scripts/createEsmWrapper.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
const fs = require("fs").promises;
const path = require("path");

async function createEsmWrapper() {
const distDir = path.join(__dirname, "..", "dist");
await fs.mkdir(distDir, { recursive: true });

const esmContent = `// ESM wrapper for CommonJS output
import cjsModule from './index.js';
export const { detectTests, resolveTests, detectAndResolveTests } = cjsModule;
Copy link

Copilot AI Jan 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The generated dist/index.mjs only re-exports { detectTests, resolveTests, detectAndResolveTests }, but src/index.ts exports many more runtime symbols (e.g., setConfig, resolveDetectedTests, log, loadDescription, etc.). This will break ESM consumers (and TypeScript users relying on dist/index.d.ts) because the runtime exports won’t match the type declarations. Update the wrapper to export all intended public runtime exports from dist/index.js.

Suggested change
export const { detectTests, resolveTests, detectAndResolveTests } = cjsModule;
// Re-export all named exports from the CommonJS bundle
export * from './index.js';

Copilot uses AI. Check for mistakes.
export default cjsModule;
`;

await fs.writeFile(path.join(distDir, "index.mjs"), esmContent);
console.log("Created ESM wrapper at dist/index.mjs");
}

createEsmWrapper().catch((error) => {
console.error("Failed to create ESM wrapper:", error);
process.exit(1);
});
59 changes: 38 additions & 21 deletions src/arazzo.js → src/arazzo.ts
Original file line number Diff line number Diff line change
@@ -1,24 +1,40 @@
const crypto = require("crypto");
import crypto from "crypto";
import type { ArazzoDescription, ArazzoWorkflowStep, DetectedTest, Step, OpenApiDefinition } from "./types";

/**
* Doc Detective test specification created from Arazzo workflow
*/
interface ArazzoTestSpec extends DetectedTest {
id: string;
description?: string;
steps: Step[];
openApi: OpenApiDefinition[];
}

/**
* Translates an Arazzo description into a Doc Detective test specification
* @param {Object} arazzoDescription - The Arazzo description object
* @returns {Object} - The Doc Detective test specification object
* @param arazzoDescription - The Arazzo description object
* @param workflowId - The ID of the workflow to translate
* @param _inputs - Optional inputs for the workflow (currently unused)
* @returns The Doc Detective test specification object, or undefined if workflow not found
*/
function workflowToTest(arazzoDescription, workflowId, inputs) {
export function workflowToTest(
arazzoDescription: ArazzoDescription,
workflowId: string,
_inputs?: unknown
): ArazzoTestSpec | undefined {
// Initialize the Doc Detective test specification
const test = {
const test: ArazzoTestSpec = {
id: arazzoDescription.info.title || `${crypto.randomUUID()}`,
description:
arazzoDescription.info.description || arazzoDescription.info.summary,
description: arazzoDescription.info.description || arazzoDescription.info.summary,
steps: [],
openApi: [],
};

arazzoDescription.sourceDescriptions.forEach((source) => {
// Translate OpenAPI definitions to Doc Detective format
if (source.type === "openapi") {
const openApiDefinition = {
const openApiDefinition: OpenApiDefinition = {
name: source.name,
descriptionPath: source.url,
};
Expand All @@ -28,17 +44,17 @@ function workflowToTest(arazzoDescription, workflowId, inputs) {

// Find workflow by ID
const workflow = arazzoDescription.workflows.find(
(workflow) => workflow.workflowId === workflowId
(w) => w.workflowId === workflowId
);

if (!workflow) {
console.warn(`Workflow with ID ${workflowId} not found.`);
return;
return undefined;
}

// Translate each step in the workflow to a Doc Detective step
workflow.steps.forEach((workflowStep) => {
const docDetectiveStep = {
workflow.steps.forEach((workflowStep: ArazzoWorkflowStep) => {
const docDetectiveStep: Step = {
action: "httpRequest",
};

Expand All @@ -48,13 +64,13 @@ function workflowToTest(arazzoDescription, workflowId, inputs) {
} else if (workflowStep.operationPath) {
// Handle operation path references (not yet supported in Doc Detective)
console.warn(
`Operation path references arne't yet supported in Doc Detective: ${workflowStep.operationPath}`
`Operation path references aren't yet supported in Doc Detective: ${workflowStep.operationPath}`
);
return;
} else if (workflowStep.workflowId) {
// Handle workflow references (not yet supported in Doc Detective)
console.warn(
`Workflow references arne't yet supported in Doc Detective: ${workflowStep.workflowId}`
`Workflow references aren't yet supported in Doc Detective: ${workflowStep.workflowId}`
);
return;
} else {
Expand All @@ -65,14 +81,15 @@ function workflowToTest(arazzoDescription, workflowId, inputs) {

// Add parameters
if (workflowStep.parameters) {
docDetectiveStep.requestParams = {};
docDetectiveStep.requestParams = {} as Record<string, unknown>;
workflowStep.parameters.forEach((param) => {
if (param.in === "query") {
docDetectiveStep.requestParams[param.name] = param.value;
(docDetectiveStep.requestParams as Record<string, unknown>)[param.name] = param.value;
} else if (param.in === "header") {
if (!docDetectiveStep.requestHeaders)
docDetectiveStep.requestHeaders = {};
docDetectiveStep.requestHeaders[param.name] = param.value;
if (!docDetectiveStep.requestHeaders) {
docDetectiveStep.requestHeaders = {} as Record<string, unknown>;
}
(docDetectiveStep.requestHeaders as Record<string, unknown>)[param.name] = param.value;
}
// Note: path parameters would require modifying the URL, which is not handled in this simple translation
});
Expand All @@ -85,15 +102,15 @@ function workflowToTest(arazzoDescription, workflowId, inputs) {

// Translate success criteria to response validation
if (workflowStep.successCriteria) {
docDetectiveStep.responseData = {};
docDetectiveStep.responseData = {} as Record<string, unknown>;
workflowStep.successCriteria.forEach((criterion) => {
if (criterion.condition.startsWith("$statusCode")) {
docDetectiveStep.statusCodes = [
parseInt(criterion.condition.split("==")[1].trim()),
];
} else if (criterion.context === "$response.body") {
// This is a simplification; actual JSONPath translation would be more complex
docDetectiveStep.responseData[criterion.condition] = true;
(docDetectiveStep.responseData as Record<string, unknown>)[criterion.condition] = true;
}
});
}
Comment on lines 104 to 116
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Fragile status code parsing could fail on malformed conditions.

The parsing logic criterion.condition.split("==")[1].trim() assumes a specific format. If the condition doesn't contain == or has unexpected whitespace, this could throw or produce NaN.

Suggested defensive fix
         if (criterion.condition.startsWith("$statusCode")) {
-          docDetectiveStep.statusCodes = [
-            parseInt(criterion.condition.split("==")[1].trim()),
-          ];
+          const parts = criterion.condition.split("==");
+          if (parts.length >= 2) {
+            const statusCode = parseInt(parts[1].trim(), 10);
+            if (!isNaN(statusCode)) {
+              docDetectiveStep.statusCodes = [statusCode];
+            }
+          }
         } else if (criterion.context === "$response.body") {
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
if (workflowStep.successCriteria) {
docDetectiveStep.responseData = {};
docDetectiveStep.responseData = {} as Record<string, unknown>;
workflowStep.successCriteria.forEach((criterion) => {
if (criterion.condition.startsWith("$statusCode")) {
docDetectiveStep.statusCodes = [
parseInt(criterion.condition.split("==")[1].trim()),
];
} else if (criterion.context === "$response.body") {
// This is a simplification; actual JSONPath translation would be more complex
docDetectiveStep.responseData[criterion.condition] = true;
(docDetectiveStep.responseData as Record<string, unknown>)[criterion.condition] = true;
}
});
}
if (workflowStep.successCriteria) {
docDetectiveStep.responseData = {} as Record<string, unknown>;
workflowStep.successCriteria.forEach((criterion) => {
if (criterion.condition.startsWith("$statusCode")) {
const parts = criterion.condition.split("==");
if (parts.length >= 2) {
const statusCode = parseInt(parts[1].trim(), 10);
if (!isNaN(statusCode)) {
docDetectiveStep.statusCodes = [statusCode];
}
}
} else if (criterion.context === "$response.body") {
// This is a simplification; actual JSONPath translation would be more complex
(docDetectiveStep.responseData as Record<string, unknown>)[criterion.condition] = true;
}
});
}
🤖 Prompt for AI Agents
In `@src/arazzo.ts` around lines 104 - 116, The status-code extraction for
workflowStep.successCriteria is fragile: update the parsing inside the loop over
successCriteria (where criterion.condition is inspected and
docDetectiveStep.statusCodes is set) to defensively handle malformed conditions
by first checking that criterion.condition exists and contains '==' (or match
with a regex like /^\s*\$statusCode\s*==\s*(\d+)\s*$/), extract the numeric part
safely, attempt Number/parseInt and verify !isNaN before assigning to
docDetectiveStep.statusCodes, and otherwise skip the entry (or log/warn) so
malformed conditions do not throw or produce NaN.

Expand Down
10 changes: 5 additions & 5 deletions src/config.test.js
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
const assert = require("assert");
const sinon = require("sinon");
const proxyquire = require("proxyquire");
const { setConfig } = require("./config");
const { setConfig } = require("../dist/config");

before(async function () {
const { expect } = await import("chai");
Expand All @@ -24,10 +24,10 @@ describe("envMerge", function () {
replaceEnvsStub = sinon.stub().returnsArg(0);

// Setup proxyquire
setConfig = proxyquire("./config", {
setConfig = proxyquire("../dist/config", {
"doc-detective-common": { validate: validStub },
"./utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
"./openapi": { loadDescription: sinon.stub().resolves({}) }
"../dist/utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
"../dist/openapi": { loadDescription: sinon.stub().resolves({}) }
}).setConfig;
Comment on lines +27 to 31
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

Incorrect proxyquire stub paths - same issue as index.test.js.

The proxyquire paths should match how the compiled dist/config.js requires its dependencies. The compiled file uses ./utils and ./openapi, not ../dist/utils and ../dist/openapi.

🐛 Proposed fix
     setConfig = proxyquire("../dist/config", {
       "doc-detective-common": { validate: validStub },
-      "../dist/utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
-      "../dist/openapi": { loadDescription: sinon.stub().resolves({}) }
+      "./utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
+      "./openapi": { loadDescription: sinon.stub().resolves({}) }
     }).setConfig;
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
setConfig = proxyquire("../dist/config", {
"doc-detective-common": { validate: validStub },
"./utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
"./openapi": { loadDescription: sinon.stub().resolves({}) }
"../dist/utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
"../dist/openapi": { loadDescription: sinon.stub().resolves({}) }
}).setConfig;
setConfig = proxyquire("../dist/config", {
"doc-detective-common": { validate: validStub },
"./utils": { log: logStub, loadEnvs: loadEnvsStub, replaceEnvs: replaceEnvsStub },
"./openapi": { loadDescription: sinon.stub().resolves({}) }
}).setConfig;
🤖 Prompt for AI Agents
In `@src/config.test.js` around lines 27 - 31, The proxyquire call that sets
setConfig is stubbing the wrong module paths; update the proxyquire stubs in the
test to match how the compiled module requires them by replacing "../dist/utils"
and "../dist/openapi" with "./utils" and "./openapi" respectively so the
proxyquire for setConfig (assigned from require("../dist/config")) correctly
intercepts calls to log, loadEnvs, replaceEnvs and loadDescription.

});

Expand Down Expand Up @@ -404,7 +404,7 @@ function deepObjectExpect(actual, expected) {
}

describe("resolveConcurrentRunners", function () {
const { resolveConcurrentRunners } = require("./config");
const { resolveConcurrentRunners } = require("../dist/config");
const os = require("os");
let originalCpus;

Expand Down
Loading
Loading