[Breaking change]: The System.Text.Json metadata reader now unescapes metadata property names.

[eiriktsarpalis]

Description

System.Text.Json versions older than .NET 9 would not unescape metadata property names, meaning that this JSON:

{ "$id": "1", "value" : { "$ref": "1" }}

does not have the same meaning as

{ "$id": "1", "value" : { "\u0024ref": "1" }}

in the context of reference preservation, even though the two documents are equal with different escaping. This is known to create problems when roundtripping metadata properties whose names require escaping such as:

string json = JsonSerializer.Serialize<Base>(new Derived());
Console.WriteLine(json); // {"categor\u00EDa":"derived"}
Console.WriteLine(JsonSerializer.Deserialize<Base>(json) is Derived); // False

[JsonPolymorphic(TypeDiscriminatorPropertyName = "categoría")]
[JsonDerivedType(typeof(Derived), "derived")]
public record Base;
public record Derived : Base;

Even though this is fixing a legitimate bug, it has been reported by users that we have an exception message encouraging users to use escaping in cases where metadata property validation needs to be bypassed. Therefore, this breaking change issue needs to be filed.

Version

.NET 9

Previous behavior

The code

JsonSerializerOptions options = new() { ReferenceHandler = ReferenceHandler.Preserve };
JsonSerializer.Deserialize<MyPoco>("""{"\u0024invalid" : 42 }""", options);
JsonSerializer.Deserialize<MyPoco>("""{"$invalid" : 42 }""", options);

record MyPoco;

Will succeed in the first call but throw an exception in the second call. This is because the first property wasn't being unescaped, allowing for the bypass of metadata property validation.

New behavior

Both deserialization calls fail with the error:

Unhandled exception. System.Text.Json.JsonException: Properties that start with '$' are not allowed in types that support metadata.

Type of breaking change

• Binary incompatible: Existing binaries might encounter a breaking change in behavior, such as failure to load or execute, and if so, require recompilation.
• Source incompatible: When recompiled using the new SDK or component or to target the new runtime, existing source code might require source changes to compile successfully.
• Behavioral change: Existing binaries might behave differently at run time.

Reason for change

Correctness and reliability.

Recommended action

Users should avoid using escaping to bypass metadata property validation, and should instead pick property names that do not conflict with metadata properties.

Feature area

Serialization

Affected APIs

No response

---

Associated WorkItem - 377118