|
| 1 | +# Testing the CLI |
| 2 | + |
| 3 | +TL;DR |
| 4 | + |
| 5 | +| go test flags | Description | |
| 6 | +|---------------|------------------------------------------| |
| 7 | +| `-debug` | Enable debug mode for the test | |
| 8 | +| `-goldens` | Update the golden files of the run tests | |
| 9 | +| `-cassettes` | Update the cassettes of the run tests | |
| 10 | + |
| 11 | +### Objectives of the test suite |
| 12 | + |
| 13 | +- Ensure that we got no new regression when we merge new code |
| 14 | +- Avoid leaking credentials when doing integration testing with Scaleway APIs |
| 15 | + |
| 16 | +### Solution |
| 17 | + |
| 18 | +#### Interaction Recording: Golden & Cassettes files |
| 19 | + |
| 20 | +A **cassette** file contains all the interactions of the Scaleway APIs that a CLI must produce when a given test run. |
| 21 | +A **golden** file contains the output that the CLI must produce when a given interaction recorded by a cassette occurs. |
| 22 | + |
| 23 | +Having golden ensure that any change in a command output will be noticed if the behavior of the CLI change for a given interaction. |
| 24 | +Having cassette ensure that we can replay interactions without actually doing the calls. |
| 25 | + |
| 26 | +Warning, if you choose to record a new cassette, you will create real resources on your organization and will be billed accordingly. |
| 27 | +Be sure to check out the resource you create in each test and remember to delete them once you need them anymore. |
| 28 | + |
| 29 | +#### Metadata |
| 30 | + |
| 31 | +When running a test, you might need information such as ID that you cannot know in advance (such as ID of resources). |
| 32 | +The `core.Test` uses different helpers to pass useful information around. |
| 33 | + |
| 34 | +One of them is the [`core.testMetadata`](https://github.com/scaleway/scaleway-cli/blob/v2/internal/core/testing.go#L80). |
| 35 | +It is designed to store information such as ID or object describing a resource during a test. |
| 36 | +This metadata can use the `render` method to provide helpful golang templating features to have commands arguments computed dynamically. |
| 37 | + |
| 38 | +#### BeforeFunc and AfterFunc |
| 39 | + |
| 40 | +Usually, you might need to set up and teardown resources when you are running a test for testing a specific command. |
| 41 | +For that you can use [`BeforeFunc`](https://github.com/scaleway/scaleway-cli/blob/v2/internal/core/testing.go#L84) and [`AfterFunc`](https://github.com/scaleway/scaleway-cli/blob/v2/internal/core/testing.go#L86). |
| 42 | +Those types allow you to execute code before (`BeforeFunc`) or after (`AfterFunc`) the main command you want to test. |
| 43 | +Those functions can access the metadata to change dynamically their behavior. |
| 44 | + |
| 45 | +#### Logging and debug mode |
| 46 | + |
| 47 | +When you are running CLI commands, you can use the `-D` to access the user logs. |
| 48 | +You can activate the debug mode by passing the `-debug` flag when running the `go test` command. |
| 49 | +For instance: |
| 50 | + |
| 51 | +``` |
| 52 | +go test ./internal/namespaces/init -debug |
| 53 | +``` |
| 54 | + |
| 55 | +When you are developing tests, you can also use the `Logger` field that is available in the different contexts to write your own logs. |
| 56 | + |
| 57 | +#### Targeting specific tests |
| 58 | + |
| 59 | +The test suite is design to run quickly, but when you are recording interactions you probably don't want to record interactions for all the tests in the test suite. |
| 60 | +You can filter the test you want to run using the native filtering features of the go test command. |
| 61 | + |
| 62 | +So let's suppose you would like to run the test `Test_InstallServer` in the baremetal package, you would use: |
| 63 | + |
| 64 | +`go test ./internal/namespaces/baremetal/v1 -run Test_InstallServer` |
| 65 | + |
| 66 | +Keep in mind that running a single file is NOT equivalent to run the test for a package. |
| 67 | +Always run the test on the whole package (here the "baremetal" package stored in the folder "./internal/namespaces/baremetal/v1") and use the `-run` to target specific tests. |
| 68 | + |
| 69 | +### Adding new tests |
| 70 | + |
| 71 | +We welcome contributions! |
| 72 | +If you want to contribute new tests you should have the following: |
| 73 | + |
| 74 | +1. Setup your dev environment: |
| 75 | + - Install go for your platform |
| 76 | + - Install your credentials, preferably in a configuration file (run `scw init`) |
| 77 | + - Keep in mind that if you record interaction, the resource you will instantiate will be delivered and billed. |
| 78 | + - Clean up the resource you don't use once the recording is over. |
0 commit comments