helper/resource: introduce testOutcome type #474
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
In the plugin test framework, the core test step logic can indicate error in two different ways: - Return an `error` to the plugin test framework. This is the plugin test framework's facility for test authors to make positive or negative assertions for their provider's behavior. The test author supplies a `TestStep.ExpectError` to match on the error. - Call the Go testing library's `t.Fatal`. This is a lower-level error that a test author cannot make assertions on. Core test step logic -- such as `ImportState` -- does both of these. The reason for using one or the other can be subtle and implicit. This poses a cognitive hurdle for contributing code changes. It also poses a hurdle to refactoring and breaking down large functions into smaller ones: does every smaller function need a `testing.T` parameter so that it can call `t.Fatal`? This change separates concerns by introducing a type to embed the first type of error in a result object: framework: `testOutcome`. Smaller functions can return two values: `testOutcome` and `error`, with the latter indicating a fatal error. A test step's entry point -- such as `testStepNewImportState` -- is the one place that knows that an `error` translates to `t.Fatalf` and a failed `testOutcome` is simply returned to the plugin test framework.
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In the plugin test framework, the core test step logic can indicate error in two different ways:
Return an
errorto the plugin test framework. This is the plugin test framework's facility for test authors to make positive or negative assertions for their provider's behavior. The test author supplies aTestStep.ExpectErrorto match on the error.Call the Go testing library's
t.Fatal. This is a lower-level error that a test author cannot make assertions on.Core test step logic -- such as
ImportState-- does both of these. The reason for using one or the other can be subtle and implicit. This poses a cognitive hurdle for contributing code changes. It also poses a hurdle to refactoring and breaking down large functions into smaller ones: does every smaller function need atesting.Tparameter so that it can callt.Fatal?This change separates concerns by introducing a type to embed the first type of error in a result object: framework:
testOutcome.Smaller functions can return two values:
testOutcomeanderror, with the latter indicating a fatal error.A test step's entry point -- such as
testStepNewImportState-- is the one place that knows that anerrortranslates tot.Fatalfand a failedtestOutcomeis simply returned to the plugin test framework.