Plugin Framework inconsistent behaviour when trying to rename a schema field nested under a collection

[VenelinMartinov]

What happened?

Resources and datasources bridged via the Plugin Framework have a very inconsistent behaviour with ProviderInfo name overrides which prevents provider authors from working around naming issues, for example for pulumi/pulumi#17826

The behaviour is different for properties in datasources versus resources and for each one, the behaviour is different with blocks versus attributes:

Scenario	Override Works	Error
ListNestedAttribute datasource	No	No
ListNestedBlock datasource	Yes	No
ListNestedAttribute resource	No	Yes
ListNestedBlock resource	Yes	No

Blocks work correctly but attributes do not. For datasources the failure is silent, while for resource we get an irrelevant error message:

testprovider_attr_res: [{collection} {}]: .Fields should be .Elem.Fields

With the following ProviderInfo override for each:

"collection": {
	Name: "collection1",
	Elem: &tfbridge.SchemaInfo{
		Fields: map[string]*tfbridge.SchemaInfo{
			"prop": {Name: "prop1"},
		},
	},
}

Note that the .Fields property is already on the .Elem, so the error message is incorrect.

Example

ListNestedAttribute datasource schema

Attributes: map[string]dschema.Attribute{
	"collection": dschema.ListNestedAttribute{
		NestedObject: dschema.NestedAttributeObject{
			Attributes: map[string]dschema.Attribute{
				"prop": dschema.StringAttribute{
					Optional: true,
				},
			},
		},
	},
}

---

ListNestedBlock datasource schema

Blocks: map[string]dschema.Block{
	"collection": dschema.ListNestedBlock{
		NestedObject: dschema.NestedBlockObject{
			Attributes: map[string]dschema.Attribute{
				"prop": dschema.StringAttribute{
					Optional: true,
				},
			},
		},
	},
}

---

ListNestedAttribute resource schema

Attributes: map[string]rschema.Attribute{
"collection": rschema.ListNestedAttribute{
	NestedObject: rschema.NestedAttributeObject{
		Attributes: map[string]rschema.Attribute{
			"prop": rschema.StringAttribute{
				Optional: true,
			},
		},
	},
}

---

ListNestedBlock resource schema

Blocks: map[string]rschema.Block{
	"collection": rschema.ListNestedBlock{
		NestedObject: rschema.NestedBlockObject{
			Attributes: map[string]rschema.Attribute{
				"prop": rschema.StringAttribute{
					Optional: true,
				},
			},
		},
	},
}

Output of pulumi about

.

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction.
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

[VenelinMartinov]

I've built tests to repro the issue in #2806

[VenelinMartinov]

Root-caused to the shim layer mess described in #2231 and #2232

Confirming this fixed the databricks issue...

[VenelinMartinov]

Databricks is getting other changes to the schema too as a result of the above fix. Something with pluralisation of datasource parameterers. Will try a more direct fix for databricks and revisit this after

[VenelinMartinov]

Databricks was unblocked with a patch, removing the P1