Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
163 changes: 163 additions & 0 deletions docs/get-started/real-world-examples/cli-installation.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
---
sidebar_label: CLI installation guide
description: Test command-line tool installation guides with version checks and functionality validation
---

# Testing a CLI installation guide

## When to use this

Use this approach when you document command-line tools that users install via package managers like npm, pip, or apt. Installation guides must have accurate commands that work across different environments.

## What this tests

- Installation command executes without errors
- Installed tool is accessible from the command line
- Version output matches expectations
- Basic functionality works after installation

## Prerequisites

- [Doc Detective installed](/docs/get-started/installation)
- Access to run shell commands on your system
- The CLI tool you're testing (or a test environment)

## Complete example

```json title="cli-installation.spec.json"
{
"tests": [
{
"steps": [
{
"description": "Install the CLI tool using npm",
"runShell": {
"command": "npm install -g doc-detective",
"exitCodes": [0]
}
},
{
"description": "Verify installation by checking the version",
"runShell": {
"command": "doc-detective --version",
"stdio": "/^\\d+\\.\\d+\\.\\d+/"
}
},
{
"description": "Test basic functionality with help command",
"runShell": {
"command": "doc-detective --help",
"stdio": "Usage:"
}
},
{
"description": "Run a simple test to validate the tool works",
"runShell": {
"command": "echo '{\"tests\": [{\"steps\": [{\"checkLink\": \"https://example.com\"}]}]}' > /tmp/test.json && doc-detective --input /tmp/test.json",
"exitCodes": [0]
}
}
]
}
]
}
```

## Expected output

When the test passes, you should see output indicating all steps completed successfully:

```json
{
"summary": {
"steps": {
"pass": 4,
"fail": 0
}
}
}
```

Each shell command executes without errors, version output matches the expected pattern, and the basic functionality test runs successfully.

## Common variations

**Test installation with version pinning:**

```json
{
"description": "Install a specific version",
"runShell": {
"command": "npm install -g [email protected]",
"exitCodes": [0]
}
}
```

**Test platform-specific installation:**

Use the [`runOn`](/docs/references/schemas/test#fields) property to run different commands based on the platform:

```json
{
"tests": [
{
"runOn": [
{
"platforms": ["linux", "mac"]
}
],
"steps": [
{
"description": "Install on Unix-like systems",
"runShell": "sudo apt-get install -y your-tool"
}
]
},
{
"runOn": [
{
"platforms": ["windows"]
}
],
"steps": [
{
"description": "Install on Windows",
"runShell": "choco install your-tool"
}
]
}
]
}
```

**Validate command output with variables:**

```json
{
"steps": [
{
"description": "Get installed version",
"runShell": {
"command": "doc-detective --version",
"stdio": "/.+/"
},
"variables": {
"INSTALLED_VERSION": "$$stdio.stdout"
}
},
{
"description": "Echo the captured version",
"runShell": {
"command": "echo Installed version: $INSTALLED_VERSION"
}
}
]
}
```

## Next steps

- Learn more about the [`runShell`](/docs/get-started/actions/runShell) action
- Explore [test contexts](/docs/get-started/config/contexts) for cross-platform testing
- Review [variables](/docs/get-started/actions/loadVariables) for capturing command output
156 changes: 156 additions & 0 deletions docs/get-started/real-world-examples/code-examples.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
---
sidebar_label: Code examples
description: Test code examples across multiple programming languages with validation
---

# Testing code examples with multiple languages

## When to use this

Use this approach when your SDK or API documentation provides code examples in multiple programming languages. Each language's examples must be syntactically correct and produce expected results.

## What this tests

- Code examples execute without errors
- Output matches documentation
- Dependencies are available
- Cross-language consistency

## Prerequisites

- [Doc Detective installed](/docs/get-started/installation)
- Programming language runtimes installed (Python, Node.js, etc.)
- Required libraries/packages for code examples

## Complete example

```json title="code-examples.spec.json"
{
"tests": [
{
"steps": [
{
"description": "Test Python example - Hello World",
"runCode": {
"language": "python",
"code": "print('Hello from Python!')",
"stdio": "Hello from Python!"
}
},
{
"description": "Test Python example - API request",
"runCode": {
"language": "python",
"code": "import json\ndata = {'status': 'success', 'message': 'API working'}\nprint(json.dumps(data))",
"stdio": "/.*success.*/"
}
},
{
"description": "Test JavaScript example - Hello World",
"runCode": {
"language": "javascript",
"code": "console.log('Hello from Node.js!');",
"stdio": "Hello from Node.js!"
}
},
{
"description": "Test JavaScript example - JSON parsing",
"runCode": {
"language": "javascript",
"code": "const data = {status: 'success', message: 'API working'}; console.log(JSON.stringify(data));",
"stdio": "/.*success.*/"
}
},
{
"description": "Test Bash example - environment check",
"runCode": {
"language": "bash",
"code": "echo \"Shell: $SHELL\" && echo \"Path configured\"",
"stdio": "/Path configured/"
}
}
]
}
]
}
```

## Expected output

When the test passes, you should see:

```json
{
"summary": {
"steps": {
"pass": 5,
"fail": 0
}
}
}
```

Each code example executes in its respective language runtime and produces the expected output.

## Common variations

**Test code with dependencies:**

```json
{
"steps": [
{
"description": "Install required package",
"runShell": {
"command": "pip install requests"
}
},
{
"description": "Test code using the package",
"runCode": {
"language": "python",
"code": "import requests\nresponse = requests.get('https://httpbin.org/status/200')\nprint(response.status_code)",
"stdio": "200"
}
}
]
}
```

**Validate error handling:**

```json
{
"description": "Test Python exception handling",
"runCode": {
"language": "python",
"code": "try:\n x = 1 / 0\nexcept ZeroDivisionError:\n print('Caught division by zero')",
"stdio": "Caught division by zero"
}
}
```

**Test multi-line code blocks:**

```json
{
"description": "Test Python function definition",
"runCode": {
"language": "python",
"code": [
"def greet(name):",
" return f'Hello, {name}!'",
"",
"result = greet('Developer')",
"print(result)"
],
"stdio": "Hello, Developer!"
}
}
```

## Next steps

- Learn more about the [`runCode`](/docs/get-started/actions/runCode) action
- Explore [`runShell`](/docs/get-started/actions/runShell) for managing dependencies
- Review [variables](/docs/get-started/actions/loadVariables) for sharing data between steps
Loading
Loading