Skip to content

Unstable tests due to STJ being used as a dynamic baseline #192

@AArnott

Description

@AArnott

5 serialization tests fail erratically when run as a large group in Release configurations.
In the output below, "expected" is the serialization output from STJ and "actual" is the output from the PolyType-based JSON serializer sample.

My analysis: the failures within each test are around the same 2 test types as the other failing tests. I believe STJ did some reflection over these types, and due to the highly concurrent nature of the tests, some of this scanning overlaps and interacted, causing the order of discovered members to be non-deterministic. This then gets locked in for that process, such that all other tests using those test types manifest the same unexpected order of serialized JSON properties.

PolyType.Tests.JsonTests_SourceGen.Roundtrip_Value

Standard Output: 
Expected: {"x10":true,"x11":"-10675199.02:48:05.4775808","x12":"9999-12-31T23:59:59.9999999","x13":42,"x14":"str2","x15":"str","x16":2,"x17":true,"x18":"-10675199.02:48:05.4775808","x19":"9999-12-31T23:59:59.9999999","x20":42,"x21":"str2","x22":"str","x23":2,"x24":true,"x25":"-10675199.02:48:05.4775808","x26":"9999-12-31T23:59:59.9999999","x27":42,"x28":"str2","x29":"str","x30":2,"x31":true,"x32":"-10675199.02:48:05.4775808","x33":"9999-12-31T23:59:59.9999999","x34":42,"x35":"str2","x36":"str","x37":2,"x38":true,"x39":"-10675199.02:48:05.4775808","x40":"9999-12-31T23:59:59.9999999","x41":42,"x42":"str2","RequiredProperty":"str","x01":"str","x02":2,"x03":true,"x04":"-10675199.02:48:05.4775808","x05":"9999-12-31T23:59:59.9999999","x06":42,"x07":"str2","x08":"str","x09":2,"requiredField":42}
Actual:   {"x01":"str","x02":2,"x03":true,"x04":"-10675199.02:48:05.4775808","x05":"9999-12-31T23:59:59.9999999","x06":42,"x07":"str2","x08":"str","x09":2,"x10":true,"x11":"-10675199.02:48:05.4775808","x12":"9999-12-31T23:59:59.9999999","x13":42,"x14":"str2","x15":"str","x16":2,"x17":true,"x18":"-10675199.02:48:05.4775808","x19":"9999-12-31T23:59:59.9999999","x20":42,"x21":"str2","x22":"str","x23":2,"x24":true,"x25":"-10675199.02:48:05.4775808","x26":"9999-12-31T23:59:59.9999999","x27":42,"x28":"str2","x29":"str","x30":2,"x31":true,"x32":"-10675199.02:48:05.4775808","x33":"9999-12-31T23:59:59.9999999","x34":42,"x35":"str2","x36":"str","x37":2,"x38":true,"x39":"-10675199.02:48:05.4775808","x40":"9999-12-31T23:59:59.9999999","x41":42,"x42":"str2","RequiredProperty":"str","requiredField":42}

PolyType.Tests.JsonTests_ReflectionEmit.Roundtrip_Value

Expected: "{"x10":true,"x11":"-10675199.02:48:05.4775808","x1"···
Actual:   "{"x01":"str","x02":2,"x03":true,"x04":"-10675199.0"···

PolyType.Tests.JsonTests_ReflectionEmit.Roundtrip_Value

Expected: "{"IntProperty":42,"StringProperty":"str"}"
Actual:   "{"StringProperty":"str","IntProperty":42}"

PolyType.Tests.JsonTests_ReflectionEmit.Roundtrip_Property

Expected: "{"Value":{"x10":true,"x11":"-10675199.02:48:05.477"···
Actual:   "{"Value":{"x01":"str","x02":2,"x03":true,"x04":"-1"

PolyType.Tests.JsonTests_ReflectionEmit.Roundtrip_CollectionElement

Expected: "[{"x10":true,"x11":"-10675199.02:48:05.4775808","x"···
Actual:   "[{"x01":"str","x02":2,"x03":true,"x04":"-10675199."···

PolyType.Tests.JsonTests_ReflectionEmit.Roundtrip_DictionaryEntry

Expected: "{"key1":{"x10":true,"x11":"-10675199.02:48:05.4775"···
Actual:   "{"key1":{"x01":"str","x02":2,"x03":true,"x04":"-10"···

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions