Commit 46a198e
Paddy
Bring test runs back into a single process.
Historically, tests ran by starting up the Go test framework, which
would spin up a shim of Terraform (importing half of Terraform in the
process), which would drive the provider code, then the test framework
would inspect the state. This had some drawbacks, though; the shim of
Terraform wasn't quite Terraform, and could behave in different ways
sometimes. As the plugin SDK got separated from the Terraform codebase,
this drift potential became worse.
So we came up with the Binary Acceptance Testing driver, which would
start up the Go test framework, which would shell out to a Terraform
binary, which would start up a provider process to drive, and then the
test framework would inspect the state back in the test process. This
solved the issue it was addressing--we were now testing against
production versions of Terraform, guaranteeing the tests would match
what users would see--but it created some problems of its own. Our
original model ran in a single process, and as a happy accident of that,
it meant that go test could report coverage information and delve could
be used to debug tests. Our binary acceptance testing model split
everything across multiple processes, which broke coverage information
(go test couldn't see the provider code being run anymore) and delve
(delve also couldn't see the provider code). We didn't test, but
suspect, that other tooling like race detection would also break.
This led to coordinating with the Terraform core team to come up with a
new process. We would still start up the Go test framework, it would
still shell out to the Terraform binary, but instead of Terraform
spinning up a new process for the provider, a provider server would be
started in the same process as the Go test framework, and Terraform
would be told to reconnect to it. This kept us using a production
Terraform binary, so we don't need to worry about our shim drifting, but
also kept our provider and test code in the same process, where it can
be reached by delve or any of the other standard testing tooling
available.
This PR is adding support for that third approach. The pieces needed in
go-plugin and terraform-plugin-test have been merged and released. The
pieces in Terraform that are required were included in 0.12.26 and
0.13.0-beta.1. This is the final piece of the puzzle, the code that
drives the tests.
As a result of this reworking, the acctest package that was called from
TestMain in providers is no longer needed, and is being removed as part
of v2's breaking changes.
Also, as a side effect from this work, providers can be debugged even
outside of testing. To support that, the plugin.DebugServe helper was
added, though a more friendly API may be needed. Providers can include
that in their main package, with the recommendation being to switch
between plugin.DebugServe and plugin.Serve based on a flag, with the
default being plugin.Serve. This would allow production versions of
binaries to have debuggers attached to them, by starting the binary in
debug mode with the debugger attached, then having Terraform reconnect
and drive it.
There are a few differences under debug mode. Terraform performs several
graph walks per command, and usually starts a provider process at the
beginning of each graph walk and kills the process at the end of the
graph walk. Because these graph walks have no hooks, it's impossible to
support this when the server is started outside of Terraform's control.
So when in debug mode, Terraform will not care about the provider
lifecycle at all; it will not start the provider, nor will it kill the
provider process after graph walks. This means global mutable state will
be longer lived than it would be in non-debug mode. Terraform init also
will not do anything with providers that are in debug mode; it will not
attempt to fetch binaries, match versions, or find binaries on the file
system. The information provided out of band when the provider is in
debug mode is all Terraform needs to operate, so it will not bother
trying to find any other information.1 parent 79b4af5 commit 46a198e
File tree
13 files changed
+417
-114
lines changed- .circleci
- acctest
- helper/resource
- plugin
- terraform
13 files changed
+417
-114
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| 26 | + | |
| 27 | + | |
26 | 28 | | |
27 | 29 | | |
28 | 30 | | |
| |||
This file was deleted.
This file was deleted.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
7 | 7 | | |
8 | 8 | | |
9 | 9 | | |
| 10 | + | |
10 | 11 | | |
11 | 12 | | |
12 | 13 | | |
| |||
16 | 17 | | |
17 | 18 | | |
18 | 19 | | |
19 | | - | |
| 20 | + | |
| 21 | + | |
20 | 22 | | |
21 | | - | |
| 23 | + | |
22 | 24 | | |
23 | 25 | | |
24 | | - | |
| 26 | + | |
25 | 27 | | |
26 | 28 | | |
27 | | - | |
| 29 | + | |
28 | 30 | | |
29 | 31 | | |
30 | 32 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
20 | 19 | | |
21 | 20 | | |
22 | 21 | | |
| |||
25 | 24 | | |
26 | 25 | | |
27 | 26 | | |
| 27 | + | |
| 28 | + | |
28 | 29 | | |
29 | 30 | | |
30 | 31 | | |
| |||
35 | 36 | | |
36 | 37 | | |
37 | 38 | | |
| 39 | + | |
38 | 40 | | |
39 | 41 | | |
40 | 42 | | |
| |||
47 | 49 | | |
48 | 50 | | |
49 | 51 | | |
50 | | - | |
51 | 52 | | |
52 | 53 | | |
53 | 54 | | |
| |||
76 | 77 | | |
77 | 78 | | |
78 | 79 | | |
| 80 | + | |
| 81 | + | |
79 | 82 | | |
80 | 83 | | |
81 | 84 | | |
82 | 85 | | |
83 | 86 | | |
84 | | - | |
85 | | - | |
| 87 | + | |
| 88 | + | |
86 | 89 | | |
87 | 90 | | |
88 | 91 | | |
| |||
93 | 96 | | |
94 | 97 | | |
95 | 98 | | |
96 | | - | |
97 | | - | |
| 99 | + | |
| 100 | + | |
98 | 101 | | |
99 | 102 | | |
100 | 103 | | |
101 | 104 | | |
102 | | - | |
103 | | - | |
| 105 | + | |
| 106 | + | |
104 | 107 | | |
105 | 108 | | |
106 | 109 | | |
107 | 110 | | |
108 | 111 | | |
109 | 112 | | |
110 | 113 | | |
| 114 | + | |
| 115 | + | |
111 | 116 | | |
112 | 117 | | |
113 | 118 | | |
| |||
155 | 160 | | |
156 | 161 | | |
157 | 162 | | |
158 | | - | |
159 | 163 | | |
160 | 164 | | |
161 | 165 | | |
162 | | - | |
163 | | - | |
| 166 | + | |
164 | 167 | | |
165 | 168 | | |
166 | | - | |
167 | 169 | | |
168 | 170 | | |
169 | 171 | | |
| |||
229 | 231 | | |
230 | 232 | | |
231 | 233 | | |
232 | | - | |
233 | 234 | | |
234 | 235 | | |
235 | 236 | | |
| |||
251 | 252 | | |
252 | 253 | | |
253 | 254 | | |
254 | | - | |
255 | 255 | | |
256 | 256 | | |
257 | 257 | | |
258 | 258 | | |
259 | 259 | | |
260 | 260 | | |
261 | 261 | | |
262 | | - | |
263 | 262 | | |
264 | 263 | | |
265 | 264 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
0 commit comments