Skip to content

Commit dbd1df2

Browse files
authored
doc: add testing documentation (#1108)
1 parent 1561813 commit dbd1df2

File tree

1 file changed

+78
-0
lines changed

1 file changed

+78
-0
lines changed

docs/developer.md

Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
# Testing the CLI
2+
3+
TL;DR
4+
5+
| go test flags | Description |
6+
|---------------|------------------------------------------|
7+
| `-debug` | Enable debug mode for the test |
8+
| `-goldens` | Update the golden files of the run tests |
9+
| `-cassettes` | Update the cassettes of the run tests |
10+
11+
### Objectives of the test suite
12+
13+
- Ensure that we got no new regression when we merge new code
14+
- Avoid leaking credentials when doing integration testing with Scaleway APIs
15+
16+
### Solution
17+
18+
#### Interaction Recording: Golden & Cassettes files
19+
20+
A **cassette** file contains all the interactions of the Scaleway APIs that a CLI must produce when a given test run.
21+
A **golden** file contains the output that the CLI must produce when a given interaction recorded by a cassette occurs.
22+
23+
Having golden ensure that any change in a command output will be noticed if the behavior of the CLI change for a given interaction.
24+
Having cassette ensure that we can replay interactions without actually doing the calls.
25+
26+
Warning, if you choose to record a new cassette, you will create real resources on your organization and will be billed accordingly.
27+
Be sure to check out the resource you create in each test and remember to delete them once you need them anymore.
28+
29+
#### Metadata
30+
31+
When running a test, you might need information such as ID that you cannot know in advance (such as ID of resources).
32+
The `core.Test` uses different helpers to pass useful information around.
33+
34+
One of them is the [`core.testMetadata`](https://github.com/scaleway/scaleway-cli/blob/v2/internal/core/testing.go#L80).
35+
It is designed to store information such as ID or object describing a resource during a test.
36+
This metadata can use the `render` method to provide helpful golang templating features to have commands arguments computed dynamically.
37+
38+
#### BeforeFunc and AfterFunc
39+
40+
Usually, you might need to set up and teardown resources when you are running a test for testing a specific command.
41+
For that you can use [`BeforeFunc`](https://github.com/scaleway/scaleway-cli/blob/v2/internal/core/testing.go#L84) and [`AfterFunc`](https://github.com/scaleway/scaleway-cli/blob/v2/internal/core/testing.go#L86).
42+
Those types allow you to execute code before (`BeforeFunc`) or after (`AfterFunc`) the main command you want to test.
43+
Those functions can access the metadata to change dynamically their behavior.
44+
45+
#### Logging and debug mode
46+
47+
When you are running CLI commands, you can use the `-D` to access the user logs.
48+
You can activate the debug mode by passing the `-debug` flag when running the `go test` command.
49+
For instance:
50+
51+
```
52+
go test ./internal/namespaces/init -debug
53+
```
54+
55+
When you are developing tests, you can also use the `Logger` field that is available in the different contexts to write your own logs.
56+
57+
#### Targeting specific tests
58+
59+
The test suite is design to run quickly, but when you are recording interactions you probably don't want to record interactions for all the tests in the test suite.
60+
You can filter the test you want to run using the native filtering features of the go test command.
61+
62+
So let's suppose you would like to run the test `Test_InstallServer` in the baremetal package, you would use:
63+
64+
`go test ./internal/namespaces/baremetal/v1 -run Test_InstallServer`
65+
66+
Keep in mind that running a single file is NOT equivalent to run the test for a package.
67+
Always run the test on the whole package (here the "baremetal" package stored in the folder "./internal/namespaces/baremetal/v1") and use the `-run` to target specific tests.
68+
69+
### Adding new tests
70+
71+
We welcome contributions!
72+
If you want to contribute new tests you should have the following:
73+
74+
1. Setup your dev environment:
75+
- Install go for your platform
76+
- Install your credentials, preferably in a configuration file (run `scw init`)
77+
- Keep in mind that if you record interaction, the resource you will instantiate will be delivered and billed.
78+
- Clean up the resource you don't use once the recording is over.

0 commit comments

Comments
 (0)