Using the Terraform Docker Provider with a DigitalOcean droplet resource

[AlexEscalante]

I am using Terraform to create Digital Ocean droplets, and I would like to use the Docker provider to set up my app after the droplet is created, however, I can't configure the Docker provider to use the droplet:

resource "digitalocean_droplet" "dev" {
  ...
}

provider "docker" {
  host = "ssh://devops@${digitalocean_droplet.dev.ipv4_address}" # <-- this fails
}

I can't interpolate the address of the created droplet inside the provider block, I get the following error:

Error: Error initializing Docker client: unable to parse docker host

How can I get this working?

Thanks in advance for any help…

[kumavis]

My understanding is that this is not possible bc providers must be configured statically at boot.

Provider configurations in Terraform must be statically-defined because they are one object in Terraform which is needed for essentially all operations, including planning and destroying resources. In particular, a provider configuration must always outlive all of the resource instances that belong to it, and so must survive for at least one additional terraform apply than the associated resource configurations do.

• @apparentlymart
  https://discuss.hashicorp.com/t/is-anyone-aware-of-how-to-instantiate-dynamic-providers/34776

You can use expressions in the values of these configuration arguments, but can only reference values that are known before the configuration is applied. This means you can safely reference input variables, but not attributes exported by resources (with an exception for resource arguments that are specified directly in the configuration).
https://developer.hashicorp.com/terraform/language/providers/configuration#interpolation

Generally speaking, Terraform always needs to evaluate provider configurations during the planning step because providers are allowed to rely on those settings in order to create the plan, and so it typically isn't possible to have a provider configuration refer to something created only during the apply step.
https://stackoverflow.com/questions/69996346/how-to-make-terraform-provider-dependent-on-a-resource-being-created

I expected this to work and this usecase to be supported by terraform ("manage different resources and compose them together with the same config format"), i guess I'll be looking for a different tool, maybe pulumi.

a friend said:

Ugh yeah that is a failing of base terraform... Part of how we handled that was using terragrunt, which makes it easier to break those out into separate tf states and create those dependencies

[apparentlymart]

Hey there,

I don't work on Terraform at HashiCorp anymore and so the following is some potentially-stale information, but hopefully it's useful to the developers of this provider in learning more about something that could potentially solve this in future. This is new information since that comment I left in the HashiCorp "discuss" forum several years ago, although what I said there does remain broadly true at this point as far as I know.

The umbrella issue which covers this particular problem, along with other unknown-values-related problems, is hashicorp/terraform#30937. As far as I'm aware, Terraform currently has experimental support for a solution to this problem called "Deferred Actions", which includes an extension to the provider plugin protocol that allows a provider to respond to a planning requests by saying, in essence, "I don't yet have enough information to plan that". The hashicorp/kubernetes provider now handles its own variant of the problem above due to the work from hashicorp/terraform-provider-kubernetes#2510.

Unfortunately, this Docker provider is implemented with the older Terraform Plugin SDK, which was originally designed for much older versions of Terraform and is quite difficult to extend with new capabilities from modern Terraform. As far as I know, it does not currently offer any way for the provider to participate in the deferred actions mechanism. I don't know if the folks at HashiCorp are intending to extend the SDK to support deferred actions in future or if making use of this would require first migrating to the modern Plugin Framework, as documented in hashicorp/terraform-plugin-framework#1007.

And of course, as long as Terraform CLI continues to treat "deferred actions" as experimental then this feature would not be directly usable anyway, but hopefully someone will update hashicorp/terraform#30937 at some point to share some news on that.

---

There are some possible ways to make things work even without those new features in place, with some caveats.

The provider currently tries to proactively use the given server URL as part of its "configure" function, which seems to be where this error originated:

terraform-provider-docker/internal/provider/provider.go

Lines 189 to 192 in 7155ab0

	client, err := config.NewClient()
	if err != nil {
	return nil, diag.Errorf("Error initializing Docker client: %s", err)
	}

In the case described in the opening message of this discussion, where digitalocean_droplet.dev.ipv4_address doesn't yet have a known value, Terraform Core sends the provider a Configure request with host set to an unknown value of string type. Unfortunately the legacy Terraform SDK handles that situation poorly: it treats it as if the argument isn't set at all, and so I expect that once the provider's own Configure function is being called host is set either to an empty string or some default value that isn't valid, and so this error occurs.

However, if this provider doesn't actually have any need to connect to the Docker server during the planning phase then it could follow the same strategy that some other providers have taken of completely ignoring the host argument until one of the resource operations (Create, Update, Delete) actually needs to interact with the remote API. Since those functions are guaranteed to never be called in the planning phase, that neatly avoids any need for the hostname to be known during the planning phase.

However, the promised caveats:

• Data sources do expect to be able to make network requests during the planning phase, and so those would still fail if the hostname were not known yet.
• Delaying any processing of the host argument until an API call needs to be made means that an incorrect host argument would not be detected until the apply phase.

For a real example of this technique, notice that the cyrilgdn/postgresql provider waits until it needs to start a DB transaction before actually connecting to the Postgres server, which therefore avoids problems when its own host argument is populated with an IP address that can't be predicted before the apply phase because it doesn't actually need to start transactions when planning.

---

For what it's worth, Pulumi has to deal with essentially this same situation -- a provider configuration being too unknown to be able to use it during their "preview" phase -- and resolves it by skipping configuring the provider at all and then just skipping trying to preview anything belonging to that provider.

Terraform Core instead always asks the provider to configure and plan and expects the provider to notice and deal with any unknown values in the configuration. The "deferred actions" model has extended that by giving the provider some new options for exactly how it might react to unknown values. Terraform works this way because in practice most providers are actually able to plan changes without interacting with the remote API at all, because their planning is really just some validation and normalization logic implemented inside the provider itself. The Kubernetes provider is an exception because Kubernetes supports dynamic schema via its concept of "CRDs", and so the provider needs to be able to fetch the schema from the remote API in order to determine whether the desired state is valid and predict what the final result should be.

I don't know if this Docker provider does actually need to interact with the remote API in order to plan any of its resource types. If it does then you'd need to rely on deferred actions to avoid this problem. But if not[^1], I think it's already possible to solve the problem by delaying the attempt to connect to the remote API until the provider actually needs to make a request, and therefore the host value wouldn't matter at all during the planning phase (unless there's a data source that needs to be read).

---

I've shared all of this just in the hope that it's useful. As noted above, I don't actually work on Terraform anymore and so I've just described the current situation as far as I understand it from an outsider perspective. I can't offer any help with changing that situation, but hopefully the "deferred actions" mechanism will offer a more robust answer before too long.

[kumavis]

@apparentlymart thanks for the writeup/