Skip to content

Commit ce626b6

Browse files
authored
Fix shims (#2511)
## Problem When running `go list -m` in a provider upgraded to use bridge v3.93.1 you will get this error: ``` go: github.com/hashicorp/[email protected]: invalid version: unknown revision v0.0.0 go: github.com/hashicorp/terraform-provider-tls/[email protected]: invalid version: unknown revision 000000000000 go: github.com/terraform-providers/terraform-provider-random/[email protected]: invalid version: unknown revision 000000000000 ``` Ian called this out in #2473 (comment). My response there is true in the sense that providers _do still build_ just fine -- `go build`, `go test` etc. don't touch these unused modules. The caveat I overlooked is that some IDE integrations (or more specifically any tooling built around `go list`) _will_ try to resolve all of these direct and indirect dependencies. When we consolidated modules, we moved `replace` directives for these shims out of the test module and into the root module. ## Option 0: Bring back all the modules I beg you not to do this but I do want to call it out as an option if the team would prefer it. I outline two more "idiomatic" options below. ## Option 1: Handle replacements in the consumer As a reminder, the `replace` directive only applies to the module where it's defined -- primarily to keep Go's minimal version selection algorithm simple and fast, but also to give downstream modules complete control of their builds. Hence `replace` is best suited for temporary overrides, for example to point at a local branch. Permanent `replace` directives do happen and we use them quite a lot, though. If a module truly needs a permanent `replace` to work properly (which, to be clear, should be avoided!), its consumers need to mirror the same `replace`. In other words, Option 1 is to not make any changes. Downstream projects can resolve this issue by copying these lines into `go.mod`: ``` replace github.com/hashicorp/terraform-provider-tls/shim => github.com/pulumi/pulumi-terraform-bridge/v3/pkg/pf/tests/internal/tlsshim v0.0.0-20241025174446-f865647b85ed replace github.com/hashicorp/terraform-provider-tls => github.com/hashicorp/terraform-provider-tls v1.2.1-0.20210201152640-49d7590ccf3f replace github.com/terraform-providers/terraform-provider-random/randomshim => github.com/pulumi/pulumi-terraform-bridge/v3/pkg/pf/tests/internal/randomshim v0.0.0-20241025174446-f865647b85ed ``` Clearly this is burdensome. For a real-world data point, this is the same strategy adopted by the k8s ecosystem and it has made dependency management with k8s notoriously painful. We should not do this. ## Option 2: Fork instead of abusing modules `tlsshim` and `randomshim` are their own modules because they intentionally trick the compiler into exposing internal terraform packages. This `internal` import is only legal if your module is identified as `github.com/terraform-providers/terraform-provider-random`: https://github.com/pulumi/pulumi-terraform-bridge/blob/7dfbcd601c7392fbf04bdec4c83a56b1ef115bc6/pkg/pf/tests/internal/randomshim/shim.go#L17-L24 This is hacky but it matches the guidance we gives to users, and it is _workable_ as long as the module consuming these shims doesn't have downstream consumers of its own. This is the case with most providers, but tfbridge is different because it _is_ meant to be consumed as a library. A more typical remedy for consuming private types is to fork the dependency and expose what you need. We could fork these two repositories under the `pulumi` org, but new repos come with their own overhead. Instead, I opted to simply check-in the upstream random and tls providers. I preserved the commits we were previously pinned to ([c6e90de46687](hashicorp/terraform-provider-random@c6e90de) and [afdd54107aba](hashicorp/terraform-provider-tls@afdd541), respectively) so licensing is unaffected. This way our `go.mod` will reflect a dependency on a public module that is still consumable downstream (e.g. `terraform-provider-random`), and we will use forks that expose the internals when testing. This PR implements Option 2.
1 parent 66ab90d commit ce626b6

File tree

102 files changed

+24669
-286
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

102 files changed

+24669
-286
lines changed

.gitattributes

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
11
# Fixes a false positive for gofmt on Windows.
22
# For more details,see: https://github.com/golangci/golangci-lint/issues/580
3-
*.go text eol=lf
3+
*.go text eol=lf
4+
**/vendor/** linguist-vendored

go.mod

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ require (
2727
github.com/hashicorp/terraform-plugin-mux v0.16.0
2828
github.com/hashicorp/terraform-plugin-sdk v1.7.0
2929
github.com/hashicorp/terraform-plugin-sdk/v2 v2.33.0
30-
github.com/hashicorp/terraform-provider-tls/shim v0.0.0-00010101000000-000000000000
30+
github.com/hashicorp/terraform-provider-tls v1.2.1-0.20230117062332-afdd54107aba
3131
github.com/hashicorp/terraform-svchost v0.1.1
3232
github.com/hexops/autogold/v2 v2.2.1
3333
github.com/hexops/valast v1.4.4
@@ -51,7 +51,7 @@ require (
5151
github.com/spf13/cobra v1.8.0
5252
github.com/stretchr/testify v1.9.0
5353
github.com/teekennedy/goldmark-markdown v0.3.0
54-
github.com/terraform-providers/terraform-provider-random/randomshim v0.0.0-00010101000000-000000000000
54+
github.com/terraform-providers/terraform-provider-random v1.3.2-0.20231204135814-c6e90de46687
5555
github.com/yuin/goldmark v1.7.4
5656
github.com/zclconf/go-cty v1.14.2
5757
golang.org/x/crypto v0.25.0
@@ -96,7 +96,6 @@ require (
9696
github.com/go-logr/stdr v1.2.2 // indirect
9797
github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
9898
github.com/google/s2a-go v0.1.7 // indirect
99-
github.com/hashicorp/terraform-provider-tls v0.0.0 // indirect
10099
github.com/hexops/gotextdiff v1.0.3 // indirect
101100
github.com/kylelemons/godebug v1.1.0 // indirect
102101
github.com/lucasb-eyer/go-colorful v1.2.0 // indirect
@@ -111,7 +110,6 @@ require (
111110
github.com/pulumi/appdash v0.0.0-20231130102222-75f619a67231 // indirect
112111
github.com/pulumi/esc v0.10.0 // indirect
113112
github.com/shopspring/decimal v1.3.1 // indirect
114-
github.com/terraform-providers/terraform-provider-random v1.3.2-0.20231204135814-c6e90de46687 // indirect
115113
github.com/vmihailenco/msgpack/v5 v5.4.1 // indirect
116114
github.com/vmihailenco/tagparser/v2 v2.0.0 // indirect
117115
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.49.0 // indirect
@@ -261,8 +259,6 @@ require (
261259

262260
replace github.com/hashicorp/terraform-plugin-sdk/v2 => github.com/pulumi/terraform-plugin-sdk/v2 v2.0.0-20240520223432-0c0bf0d65f10
263261

264-
replace github.com/hashicorp/terraform-provider-tls => github.com/hashicorp/terraform-provider-tls v1.2.1-0.20230117062332-afdd54107aba
262+
replace github.com/hashicorp/terraform-provider-tls => ./pkg/pf/tests/internal/tlsshim/vendor/github.com/hashicorp/terraform-provider-tls
265263

266-
replace github.com/hashicorp/terraform-provider-tls/shim => ./pkg/pf/tests/internal/tlsshim
267-
268-
replace github.com/terraform-providers/terraform-provider-random/randomshim => ./pkg/pf/tests/internal/randomshim
264+
replace github.com/terraform-providers/terraform-provider-random => ./pkg/pf/tests/internal/randomshim/vendor/github.com/hashicorp/terraform-provider-random

go.sum

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1682,8 +1682,6 @@ github.com/hashicorp/terraform-plugin-sdk v1.7.0/go.mod h1:OjgQmey5VxnPej/buEhe+
16821682
github.com/hashicorp/terraform-plugin-test v1.2.0/go.mod h1:QIJHYz8j+xJtdtLrFTlzQVC0ocr3rf/OjIpgZLK56Hs=
16831683
github.com/hashicorp/terraform-plugin-testing v1.5.1 h1:T4aQh9JAhmWo4+t1A7x+rnxAJHCDIYW9kXyo4sVO92c=
16841684
github.com/hashicorp/terraform-plugin-testing v1.5.1/go.mod h1:dg8clO6K59rZ8w9EshBmDp1CxTIPu3yA4iaDpX1h5u0=
1685-
github.com/hashicorp/terraform-provider-tls v1.2.1-0.20230117062332-afdd54107aba h1:xoUp/RqxaIf20SzMbRTbdM/ORPoJ6CBGuw9PoxyGWqk=
1686-
github.com/hashicorp/terraform-provider-tls v1.2.1-0.20230117062332-afdd54107aba/go.mod h1:TBjYP1eCMBy/8zMvovtWNHC9/HT/qaRNqN8HdxeIWaE=
16871685
github.com/hashicorp/terraform-registry-address v0.2.3 h1:2TAiKJ1A3MAkZlH1YI/aTVcLZRu7JseiXNRHbOAyoTI=
16881686
github.com/hashicorp/terraform-registry-address v0.2.3/go.mod h1:lFHA76T8jfQteVfT7caREqguFrW3c4MFSPhZB7HHgUM=
16891687
github.com/hashicorp/terraform-svchost v0.0.0-20191011084731-65d371908596/go.mod h1:kNDNcF7sN4DocDLBkQYz73HGKwN1ANB1blq4lIYLYvg=
@@ -2031,8 +2029,6 @@ github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsT
20312029
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
20322030
github.com/teekennedy/goldmark-markdown v0.3.0 h1:ik9/biVGCwGWFg8dQ3KVm2pQ/wiiG0whYiUcz9xH0W8=
20332031
github.com/teekennedy/goldmark-markdown v0.3.0/go.mod h1:kMhDz8La77A9UHvJGsxejd0QUflN9sS+QXCqnhmxmNo=
2034-
github.com/terraform-providers/terraform-provider-random v1.3.2-0.20231204135814-c6e90de46687 h1:4gT3jbJFnnGV2Liq4jziT7C4w/OOM+pkuwBGVDagRgc=
2035-
github.com/terraform-providers/terraform-provider-random v1.3.2-0.20231204135814-c6e90de46687/go.mod h1:nH2NdrX91QoiiaPJsTZ9vKoezUq5IgHJ8nijpD6lT5o=
20362032
github.com/texttheater/golang-levenshtein v1.0.1 h1:+cRNoVrfiwufQPhoMzB6N0Yf/Mqajr6t1lOv8GyGE2U=
20372033
github.com/texttheater/golang-levenshtein v1.0.1/go.mod h1:PYAKrbF5sAiq9wd+H82hs7gNaen0CplQ9uvm6+enD/8=
20382034
github.com/tidwall/gjson v1.17.0 h1:/Jocvlh98kcTfpN2+JzGQWQcqrPQwDrVEMApx/M5ZwM=

pkg/pf/Makefile

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -80,8 +80,7 @@ gen.testdata::
8080
tidy::
8181
go mod tidy
8282
cd tests && go mod tidy
83-
cd tests/internal/randomshim && go mod tidy
84-
cd tests/internal/tlsshim && go mod tidy
83+
cd dynamic && go mod tidy
8584
cd tests/testdatagen/genrandom && go mod tidy
8685

8786
.PHONY: todos

pkg/pf/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ Follow these steps to bridge a Terraform Provider to Pulumi.
2525
[exposes](https://github.com/hashicorp/terraform-provider-random/blob/main/internal/provider/provider.go#L13) a `func
2626
New() provider.Provider` call. Since this definition lives in an `internal` package it cannot easily be referenced in
2727
an external Go project, but it is still possible to reference it using Go linker tricks. See
28-
`tests/internal/randomshim/shim.go` for a full example.
28+
[pulumi-random](https://github.com/pulumi/pulumi-random/tree/48c0b3014aeaa0cb95fd6419d631cb2555ce89ac/provider/shim) for a full example.
2929

3030
2. Populate a `ProviderInfo` struct, mapping Terraform resource names to Pulumi tokens. Replace `myprovider` with your
3131
provider name.
@@ -137,7 +137,7 @@ to the Plugin Framework.
137137
138138
import (
139139
"github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema"
140-
"github.com/terraform-providers/terraform-provider-tls/internal/provider"
140+
"github.com/hashicorp/terraform-provider-tls/internal/provider"
141141
)
142142
143143
func NewProvider() *schema.Provider {

pkg/pf/tests/internal/randomshim/go.mod

Lines changed: 0 additions & 38 deletions
This file was deleted.

pkg/pf/tests/internal/randomshim/shim.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,7 @@ package randomshim
1616

1717
import (
1818
tfpf "github.com/hashicorp/terraform-plugin-framework/provider"
19-
"github.com/terraform-providers/terraform-provider-random/internal/provider"
19+
"github.com/terraform-providers/terraform-provider-random/provider"
2020
)
2121

2222
func NewProvider() tfpf.Provider {

0 commit comments

Comments
 (0)