Skip to content

Commit 2086408

Browse files
jkodroffclaude
jkodroff
and
claude
authored
Restructure integration testing documentation (#16435)
Split the monolithic integration testing page into three focused pages: - Overview page explaining integration testing concepts and options - Dedicated framework page with updated Go v3 SDK examples - Dedicated Automation API page with numbered workflow steps Changes: - Update SDK imports from pkg/v2 to pkg/v3 - Improve intro to explain black-box testing before using the term - Remove outdated ephemeral environments section - Add clear sections for "Why integration testing?" and benefits - Organize testing options with explicit benefits and constraints - Maintain SEO with proper aliases configuration - Update menu structure with parent/child relationships Fixes #16430 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude <[email protected]>
1 parent 8f62d9a commit 2086408

File tree

4 files changed

+288
-187
lines changed

4 files changed

+288
-187
lines changed

‎content/docs/iac/guides/testing/integration.md‎

Lines changed: 0 additions & 187 deletions
This file was deleted.

‎content/docs/iac/guides/testing/integration/_index.md‎

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
title_tag: "Integration Testing for Pulumi Programs"
3+
meta_desc: "Guide to integration testing of Pulumi programs with various testing frameworks and approaches."
4+
title: Integration testing
5+
h1: Integration Testing for Pulumi Programs
6+
meta_image: /images/docs/meta-images/docs-meta.png
7+
weight: 3
8+
menu:
9+
iac:
10+
name: Integration testing
11+
parent: iac-guides-testing
12+
weight: 3
13+
identifier: iac-guides-testing-integration
14+
usingpulumi:
15+
parent: testing
16+
identifier: testing-integration
17+
aliases:
18+
- /docs/guides/testing/integration/
19+
- /docs/using-pulumi/testing/integration/
20+
- /docs/iac/concepts/testing/integration/
21+
---
22+
23+
Integration testing validates your Pulumi programs by running your program and testing the results without examining internal implementation details. For example, you might test that an HTTP endpoint is available and gives the expected response to an incoming request. An integration test runs the program in combination with the Pulumi CLI or Automation API to deploy infrastructure to an ephemeral environment. Once your tests are run and the correctness of your infrastructure is verified (or refuted), you will typically destroy the infrastructure you created for the test. This approach is known as "black-box testing" because it tests the program from the outside.
24+
25+
## Why integration testing?
26+
27+
Integration testing allows you to validate that your infrastructure code works correctly in a real cloud environment. Unlike unit tests that mock cloud resources, or Policy as Code that verifies expectations for specific resources in your stack, integration tests deploy actual infrastructure to verify your code's behavior end-to-end.
28+
29+
By running a program through integration tests, you can ensure:
30+
31+
- Your project's code is syntactically well-formed and runs without errors.
32+
- Your stack's configuration and secrets work and are interpreted correctly.
33+
- Your project can be successfully deployed to your cloud provider of choice.
34+
- Resources of the desired shape are successfully created.
35+
- The infrastructure behaves as expected: for example, a health-check endpoint returns a valid HTML document, or a suite of application-level tests succeeds against the public API.
36+
- Your project can be successfully updated from its starting state to other states.
37+
- Your project can be successfully destroyed and removed from your cloud provider.
38+
39+
## Integration testing options
40+
41+
Pulumi supports several approaches to integration testing, each with different trade-offs:
42+
43+
### Integration testing framework
44+
45+
Pulumi provides a [dedicated integration testing framework](/docs/iac/guides/testing/integration/framework/) written in Go. This framework is designed to drive lifecycle operations against a Pulumi program: deploying a new stack from scratch, optionally updating it with variations, and tearing it down afterwards.
46+
47+
Benefits:
48+
49+
- Purpose-built for testing Pulumi programs
50+
- Comprehensive lifecycle testing capabilities, including the ability to test update/upgrade paths
51+
- Robust validation options for resources and runtime behavior
52+
- Can test Pulumi programs written in any supported language
53+
54+
Constraints:
55+
56+
- Tests must be written in Go
57+
58+
### Automation API
59+
60+
You can use [Automation API for integration testing](/docs/iac/guides/testing/integration/automation-api/) to programmatically run CLI actions and write tests using your preferred testing framework. This approach gives you full control over the testing process using the programming language of your choice.
61+
62+
Benefits:
63+
64+
- Write tests in your preferred language and testing framework
65+
- Full programmatic control over stack lifecycle
66+
- Easy integration with existing language-specific tooling for testing
67+
- Available in Node.js, Python, .NET, Go, and Java
68+
69+
Constraints:
70+
71+
- Not available in YAML
72+
73+
### DIY Options
74+
75+
You can also write integration tests by executing Pulumi CLI commands via shell (either by executing a shell script or by executing shell commands in your preferred language) and making assertions in any language. While Automation API makes this easier by providing language-native APIs, you can accomplish similar results with shell scripts if needed.
76+
77+
Benefits:
78+
79+
- Works with any programming language
80+
- Simple to understand and implement
81+
- No additional dependencies beyond the Pulumi CLI
82+
83+
Trade-offs:
84+
85+
- More manual error handling
86+
- Less type-safe than Automation API
87+
- More difficult to extract and validate resource properties

‎content/docs/iac/guides/testing/integration/automation-api.md‎

Lines changed: 71 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,71 @@
1+
---
2+
title_tag: "Integration Testing with Automation API"
3+
meta_desc: "Learn how to use Pulumi's Automation API to write integration tests in your preferred programming language."
4+
title: Automation API testing
5+
h1: Integration Testing with Automation API
6+
meta_image: /images/docs/meta-images/docs-meta.png
7+
weight: 2
8+
menu:
9+
iac:
10+
name: Automation API testing
11+
parent: iac-guides-testing-integration
12+
weight: 2
13+
---
14+
15+
Pulumi's [Automation API](/docs/iac/automation-api/) provides a programmatic interface for integration testing. While not a purpose-built integration testing framework, you can use Automation API in your integration tests to accomplish the same objectives of basic correctness testing, resource validation, and runtime testing. Automation API is available in all Pulumi languages except YAML and is a part of the Pulumi SDK.
16+
17+
## Testing workflow
18+
19+
To write integration tests with Automation API, follow these steps:
20+
21+
1. Create a stack using Automation API.
22+
1. Set up stack configuration and secrets.
23+
1. Deploy the stack using `up()`.
24+
1. Perform resource validation by examining the deployment state.
25+
1. Perform runtime checks by interacting with deployed infrastructure.
26+
1. Tear down the stack using `destroy()` and clean up resources.
27+
28+
## Validating resource and stack properties
29+
30+
To validate that the correct resources were created with the expected properties, you can export the stack and examine the resulting resources.
31+
32+
For example, in TypeScript:
33+
34+
```typescript
35+
export async function getDeployment(): Promise<Deployment> {
36+
const stack = await LocalWorkspace.createOrSelectStack(args);
37+
38+
return stack.exportStack();
39+
}
40+
```
41+
42+
You can then iterate through the `Deployment` object to check that the expected resources and property values are present.
43+
44+
## Validating infrastructure correctness
45+
46+
Runtime validation involves interacting with your deployed infrastructure to verify it works correctly. For example, you might:
47+
48+
- Make HTTP requests to validate an API endpoint
49+
- Query a database to verify data was created correctly
50+
- Check that files exist in object storage
51+
- Verify that VMs are responding to network requests
52+
53+
The Automation API gives you full access to stack outputs, which typically include endpoints, URLs, and other information needed to interact with your infrastructure.
54+
55+
## Example
56+
57+
The [`localProgram-tsnode-mochatests` example](https://github.com/pulumi/automation-api-examples/tree/main/nodejs/localProgram-tsnode-mochatests) demonstrates how to:
58+
59+
- Set up a stack using Automation API
60+
- Perform runtime validation checks
61+
- Tear down the stack as part of a test
62+
63+
## Additional resources
64+
65+
Integration tests written with Automation API in the Pulumi SDK repositories:
66+
67+
- [Node.js tests](https://github.com/pulumi/pulumi/blob/master/sdk/nodejs/tests/automation/localWorkspace.spec.ts)
68+
- [Go tests](https://github.com/pulumi/pulumi/blob/master/sdk/go/auto/local_workspace_test.go)
69+
- [Python tests](https://github.com/pulumi/pulumi/blob/master/sdk/python/lib/test/automation/test_local_workspace.py)
70+
- [C# tests](https://github.com/pulumi/pulumi-dotnet/blob/main/sdk/Pulumi.Automation.Tests/LocalWorkspaceTests.cs)
71+
- [Java example](https://github.com/pulumi/automation-api-examples/blob/main/java/localProgram/)

0 commit comments

Comments
 (0)