Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
262 changes: 262 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,268 @@ const detectedTests = await detectTests({ config });
const resolvedTests = await resolveTests({ config, detectedTests });
```

## OpenAPI 3.x Support

Doc Detective Resolver includes built-in support for OpenAPI 3.x specifications, automatically transforming API operations into executable test specifications. This enables seamless integration of API documentation with automated testing workflows.

### Detection and File Support

OpenAPI files are automatically detected based on:

1. **File Extension**: Must be `.json`, `.yaml`, or `.yml`
2. **OpenAPI Version**: Must contain `openapi: "3.x.x"` field (any 3.x version)

When a file meets these criteria, it's processed as an OpenAPI specification instead of a standard Doc Detective test file.

### Transformation Process

The transformation from OpenAPI operations to Doc Detective tests follows this workflow:

#### 1. **Operation Extraction**
- All HTTP operations (GET, POST, PUT, PATCH, DELETE, etc.) are extracted from the `paths` section
- Each operation is evaluated for safety and configuration
- Non-operation fields like `parameters`, `servers`, `summary`, and `description` at the path level are skipped

#### 2. **Safety Classification**
Operations are classified as "safe" or "unsafe" based on the following rules:

**Safe Operations (automatically included):**
- Operations with `x-doc-detective` extension (any configuration makes them explicitly safe)
- GET, HEAD, OPTIONS, POST methods (considered safe by default)

**Unsafe Operations (skipped unless explicitly marked):**
- PUT, PATCH, DELETE methods without `x-doc-detective` extension
- Any operation that could modify or delete data without explicit safety confirmation

#### 3. **Test Generation**
For each safe operation, a Doc Detective test is created with:

- **Test ID**: Uses `operationId` if available, otherwise generates `{method}-{path}`
- **Description**: Uses operation `summary`, `description`, or generates from method and path
- **Main Step**: Creates an `httpRequest` step with OpenAPI configuration
- **Dependencies**: Adds `before` and `after` steps if configured

### x-doc-detective Extensions

The `x-doc-detective` extension provides fine-grained control over test generation and execution:

#### Root-Level Configuration
Applied to all operations as default values:

```yaml
openapi: 3.0.0
x-doc-detective:
server: "https://testing.example.com"
validateSchema: true
statusCodes: [200, 201]
```

#### Operation-Level Configuration
Overrides root-level settings for specific operations:

```yaml
paths:
/users/{id}:
delete:
x-doc-detective:
validateSchema: false
before: ["getUser"]
after: ["getAllUsers"]
```

#### Supported Extension Properties

| Property | Type | Description |
|----------|------|-------------|
| `server` | string | Base URL for API requests (overrides OpenAPI servers) |
| `validateSchema` | boolean | Enable/disable response schema validation |
| `mockResponse` | boolean | Use mock responses instead of real API calls |
| `statusCodes` | array | Expected HTTP status codes for the operation |
| `useExample` | boolean | Use OpenAPI examples in requests |
| `exampleKey` | string | Specific example key to use from OpenAPI examples |
| `requestHeaders` | object | Additional headers to include in requests |
| `responseHeaders` | object | Expected response headers to validate |
| `before` | array | Operations to execute before this operation |
| `after` | array | Operations to execute after this operation |

### Dependency Management

Dependencies enable complex testing workflows by chaining operations:

#### Before Dependencies
Execute prerequisite operations before the main operation:

```yaml
post:
operationId: createUser
x-doc-detective:
before: ["loginUser", "getPermissions"]
```

#### After Dependencies
Execute cleanup or verification operations after the main operation:

```yaml
delete:
operationId: deleteUser
x-doc-detective:
after: ["verifyUserDeleted", "cleanupSession"]
```

#### Dependency Resolution
Dependencies can be referenced by:
- **Operation ID**: `"getUserById"`
- **Path + Method**: `{"path": "/users/{id}", "method": "get"}`

### Configuration Inheritance

Configuration values are merged in the following priority order (highest to lowest):

1. Operation-level `x-doc-detective` configuration
2. Root-level `x-doc-detective` configuration
3. Doc Detective global configuration
4. Default values

### Transformation Examples

#### Input: Basic OpenAPI Operation
```yaml
openapi: 3.0.0
paths:
/users:
get:
operationId: getUsers
summary: "Retrieve all users"
responses:
200:
description: "List of users"
```

#### Output: Generated Doc Detective Test
```json
{
"specId": "openapi-example",
"tests": [
{
"id": "getUsers",
"description": "Retrieve all users",
"steps": [
{
"action": "httpRequest",
"openApi": {
"operationId": "getUsers"
}
}
]
}
],
"openApi": [
{
"name": "Example API",
"definition": { /* full OpenAPI spec */ }
}
]
}
```

#### Input: Complex Operation with Dependencies
```yaml
paths:
/users/{id}:
delete:
operationId: deleteUser
x-doc-detective:
server: "https://test.api.com"
before: ["getUser", "backupUser"]
after: ["verifyDeleted"]
statusCodes: [204, 404]
```

#### Output: Generated Test with Dependencies
```json
{
"id": "deleteUser",
"steps": [
{
"action": "httpRequest",
"openApi": { "operationId": "getUser" }
},
{
"action": "httpRequest",
"openApi": { "operationId": "backupUser" }
},
{
"action": "httpRequest",
"openApi": {
"operationId": "deleteUser",
"server": "https://test.api.com",
"statusCodes": [204, 404]
}
},
{
"action": "httpRequest",
"openApi": { "operationId": "verifyDeleted" }
}
]
}
```

### Requirements and Behaviors

#### Method-Specific Requirements
- **GET/HEAD/OPTIONS**: No special requirements, considered safe by default
- **POST**: Considered safe for creation operations, no additional requirements
- **PUT/PATCH/DELETE**: Require explicit `x-doc-detective` extension to be included

#### Validation Requirements
- OpenAPI specification must be valid 3.x format
- Referenced dependencies must exist in the same OpenAPI specification
- If `validateSchema: true`, operations must have complete request/response schemas

#### Server Configuration
- If no `server` specified in `x-doc-detective`, uses first server from OpenAPI `servers` array
- If no servers defined anywhere, transformation will fail with error
- Server URLs support environment variable substitution: `https://$API_HOST/v1`

### User Expectations

#### What Gets Generated
- **One test per safe operation**: Each operation becomes a separate test
- **Automatic request/response handling**: Full HTTP request steps with OpenAPI context
- **Schema validation**: Automatic request/response validation when enabled
- **Dependency orchestration**: Before/after operations executed in correct order

#### What Doesn't Get Generated
- **Unsafe operations**: PUT/PATCH/DELETE without explicit `x-doc-detective` are skipped
- **Path-level parameters**: Only operation-level configurations are processed
- **Custom test logic**: Only `httpRequest` steps are generated, no custom actions
- **Complex workflows**: Each operation is a separate test, not part of larger workflows

#### Error Handling
- **Invalid OpenAPI**: Files that don't validate as OpenAPI 3.x are skipped
- **Missing dependencies**: Referenced operations that don't exist log warnings but don't fail the transformation
- **Schema errors**: Operations with invalid schemas log warnings but are still included
- **Server resolution**: Missing server configuration causes transformation to fail

### Integration with Doc Detective

Generated tests integrate seamlessly with the Doc Detective ecosystem:

- **Schema validation**: Uses Doc Detective's OpenAPI schema validation
- **Variable substitution**: Supports Doc Detective variable replacement patterns
- **Context resolution**: Automatically detects browser context requirements
- **Result reporting**: Test results use standard Doc Detective output format

### Best Practices

1. **Use meaningful operationIds**: They become test IDs and should be descriptive
2. **Include summaries/descriptions**: They become test descriptions for better reporting
3. **Configure servers appropriately**: Use test/staging servers, not production
4. **Mark destructive operations explicitly**: Use `x-doc-detective` on PUT/PATCH/DELETE
5. **Test dependencies carefully**: Ensure `before`/`after` operations exist and are safe
6. **Use environment variables**: Keep sensitive data out of OpenAPI files
7. **Validate your OpenAPI**: Use tools to ensure your specification is valid before testing

## Contributions

Looking to help out? See our [contributions guide](https://github.com/doc-detective/doc-detective-resolver/blob/main/CONTRIBUTIONS.md) for more info. If you can't contribute code, you can still help by reporting issues, suggesting new features, improving the documentation, or sponsoring the project.
112 changes: 112 additions & 0 deletions src/openapi-integration.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
const { expect } = require("chai");
const sinon = require("sinon");
const path = require("path");
const { detectTests } = require("./index");

describe("OpenAPI Integration Tests", () => {
let sandbox;
const minimalConfig = {
input: path.resolve(__dirname, "../test/openapi-test-example.json"),
environment: {
platform: "test"
},
fileTypes: []
};

const configWithExtensions = {
input: path.resolve(__dirname, "../test/openapi-test-with-extensions.json"),
environment: {
platform: "test"
},
fileTypes: []
};

beforeEach(() => {
sandbox = sinon.createSandbox();
sandbox.stub(console, "log"); // Mute console logs for tests
});

afterEach(() => {
sandbox.restore();
});

it("should detect and parse OpenAPI files", async () => {
const result = await detectTests({ config: minimalConfig });

// Check that the OpenAPI file was processed
expect(result).to.be.an("array").that.has.lengthOf(1);

// Verify the generated spec structure
const spec = result[0];
expect(spec.specId).to.include("openapi-openapi-test-example");
expect(spec.openApi).to.be.an("array").that.has.lengthOf(1);
expect(spec.openApi[0].name).to.equal("Test API");
expect(spec.tests).to.be.an("array");

// We expect 3 tests (GET /users, POST /users, GET /users/{userId}, PUT /users/{userId} with safe override)
// DELETE is now included because it has an x-doc-detective extension
expect(spec.tests).to.have.lengthOf(5);

// Verify each test has the expected format
spec.tests.forEach(test => {
expect(test.id).to.be.a("string");
expect(test.description).to.be.a("string");
expect(test.steps).to.be.an("array").that.has.lengthOf.at.least(1);

// Each test should have an httpRequest step
const httpStep = test.steps[0];
expect(httpStep.action).to.equal("httpRequest");
expect(httpStep.openApi).to.be.an("object");
expect(httpStep.openApi.operationId).to.be.a("string");
});

// Find specific tests to verify
const getUsersTest = spec.tests.find(t => t.id === "getUsers");
const createUserTest = spec.tests.find(t => t.id === "createUser");
const getUserTest = spec.tests.find(t => t.id === "getUser");
const updateUserTest = spec.tests.find(t => t.id === "updateUser");

expect(getUsersTest).to.exist;
expect(createUserTest).to.exist;
expect(getUserTest).to.exist;
expect(updateUserTest).to.exist;
});

it("should support x-doc-detective extensions", async () => {
const result = await detectTests({ config: configWithExtensions });

// Check that the OpenAPI file was processed
expect(result).to.be.an("array").that.has.lengthOf(1);

// Verify the generated spec structure
const spec = result[0];
expect(spec.specId).to.include("openapi-openapi-test-with-extensions");
expect(spec.tests).to.be.an("array");

// We expect 3 tests (getProducts, createProduct, deleteProduct with safe override)
expect(spec.tests).to.have.lengthOf(3);

// Find specific tests to verify
const getProductsTest = spec.tests.find(t => t.id === "getProducts");
const createProductTest = spec.tests.find(t => t.id === "createProduct");
const deleteProductTest = spec.tests.find(t => t.id === "deleteProduct");

expect(getProductsTest).to.exist;
expect(createProductTest).to.exist;
expect(deleteProductTest).to.exist;

// Check that root extensions are applied
expect(getProductsTest.steps[0].openApi.server).to.equal("https://testing.example.com/v1");
expect(getProductsTest.steps[0].openApi.validateSchema).to.equal(true);

// Check that operation level overrides work
expect(createProductTest.steps[1].openApi.validateSchema).to.equal(false);

// Check for dependencies (before and after)
expect(createProductTest.steps).to.have.lengthOf(2);
expect(createProductTest.steps[0].openApi.operationId).to.equal("getProducts");

expect(deleteProductTest.steps).to.have.lengthOf(2);
expect(deleteProductTest.steps[1].openApi.operationId).to.equal("getProducts");
});
});
Loading