Discussion: Anonymous struct types

[kekyo]

Discussion: Anonymous struct types

Summary

Support anonymous struct types. Specifically, allow anonymous type derives from value type.

Motivation

Anonymous types are applied anonymous naming but fully structured class type.
There will contain named properties with easy syntax. Need value types almost same as it.
We know value tuple is same, it can have naming values.
But the value tuple has a problem for serialization/reflection issues (Newtonsoft.Json #1230).
We can avoid it and easy to use already known anonymous types like syntax.

Detailed design

Standard anonymous types syntax:

// It's standard anonymous types. (Derived from System.Object)
var foo = new { Bar = 123, Baz = "ABC" };

Anonymous struct types syntax:

// It's anonymous struct types. (Derived from System.ValueType)
var foo = new struct { Bar = 123, Baz = "ABC" };

And I'm thinking it'll enable by default for C# future release with apply syntax context switch likely nullable reference type:

#anonymous struct     // The pragma naming is for example.

// It's anonymous struct types.
var foo = new { Bar = 123, Baz = "ABC" };

// It's standard anonymous types.
var foo = new class { Bar = 123, Baz = "ABC" };

(And include enabler in csproj property)

It'll not make compatibility issue between objref type and value type because the specification apply only inside for a method body
(The anonymous (class|struct) types can't expose outside for the method body.)

Drawbacks

It'll cause problem in corner case if use it in explicitly objref knowledge:

// It's standard anonymous types.
var foo = new { Bar = 123, Baz = "ABC" };
var bar = foo;

Debug.Assert(object.ReferenceEquals(foo, bar));

The code fragment raise assertion error if switched to enable anonymous struct types.

Alternatives

I'll withdraw it if value tuple's problem fixes in the future.

[svick]

What are the use cases where all of the following conditions apply?

1. You need to access property names by reflection.
2. The single allocation significantly affects performance.
3. Creating a custom struct is too cumbersome.

There certainly are cases where one or two of those conditions are true, but I think all three need to be true often enough to make this a worthwhile feature.

Also, I'm fairly certain #anonymous struct is not going to happen. C# has been very hesitant to introduce dialects and while #nullable enable in C# 8.0 does that, that's for the "billion dollar mistake", not a small feature like this one.

[yaakov-h]

I don’t see how making it a struct helps in the slightest. You can’t avoid the allocation when reflection requires boxing your struct to object anyway.

[svick]

@yaakov-h You can avoid the allocation, if you don't use reflection directly, but instead use it together with Expression or Reflection.Emit to generate the code that actually processes the input.

[yaakov-h]

@svick but then you have to allocate those...

[marksmeltzer]

@yaakov-h sure, but just once :) That's the kind of thing that static constructors on static generic classes are tailor made for, or lazy initialized static fields.

[marksmeltzer]

On the flip side, if anonymous structs were added, they could always be retrofitted to replace the current value tuple. To keep things backwards compatible, the compiler could still emit the TupleElementNamesAttribute, etc.

I don't know the full history behind the value tuple implementation choices, but there's likely some reason that the current approach was taken -- using only metadata attributes to record the tuple property names rather than emitting unique structs.

If the unique structs had been implemented, the compiler could have emitted an implicit conversion operator to a standard unnamed ValueTuple. It seems like compiler generated anonymous structs could have been a viable choice at the time (I'm sure I'm missing something). The compiler could even have emitted an implicit conversion in the other direction as well to enable duck typing: as long as the definitions match, why not?

(obviously using just metadata and ValueTuple likely results in smaller DLLs -- maybe that was a large factor in the rationale)

[HaloFour]

@marksmeltzer

I don't know the full history behind the value tuple implementation choices, but there's likely some reason that the current approach was taken

To allow sharing. Otherwise, every use of a tuple within assembly would result in generating new and incompatible structs. One assembly would not be able to pass an (int,int) to another as the compiler would have emitted two completely separate and incompatible types.

It's also because of this that it would be impossible to retrofit anonymous structs onto tuples.

[marksmeltzer]

@HaloFour

Is that the only reason? Seems quite weak...

If the compiler would have emitted the implicit conversion operators to and from the raw ValueTuple<,> types as I suggested above, that could have solved that problem quite easily. (I was thinking it had to do with assemblies getting lots redundant of ad hoc types...?)

In the same manner, anonymous classes could also received implicit conversion operations to/from ValueTuple<,>.

The order the properties are defined would dictate which Item slot the property value would be assigned to.

[marksmeltzer]

The only gap is the duck typing scenario where named tuple A can currently be converted to named tuple B because they are both actually ValueTuple<,> in reality. (Since the names are just metadata.)

That problem could have been solved by allowing the compiler to automatically generate duck type conversion code to convert between anonymous named tuples and ValueTuple<,> -- e.g. the anonymous named tuples would have an attribute and the compiler would simply invoke the implicit conversion operators to go from A -> ValueTuple<,> -> B. Better yet, since the compiler would know they are semantically guaranteed to convert, it could just new up a B using A.Item1, A.Item2, etc., inline. That would probably perform better than using double conversion.

There might be some performance drawback to that conversion scenario compared to the chosen implementation, which has zero conversion cost (because there isn't any conversion). So that could also have been a reasonable justification.

[HaloFour]

@marksmeltzer

https://github.com/dotnet/csharplang/blob/master/meetings/2015/LDM-2015-02-11.md#type-equivalence

There's also the question of speakability. What do you call a generated tuple type? If you expose it publicly, it needs a predictable name, and the method of generating that name must be in the spec.

[marksmeltzer]

@HaloFour same problem applies to anonymous classes, right? And those got a naming convention.

[marksmeltzer]

I would have been perfectly fine with duck typing on anonymous named tuples as generated types -- in contrast to the LDM notes:

Unfortunately this leaves problems at runtime: now there's too much type equivalence. If I want to do a runtime test to see if an object is an (int x, int y), I would get false positives for an (int a, int b), since the names wouldn't be part of the runtime type:
...
Of course we could come up with tricks so that a tuple would carry some kind of member name info around at runtime, but not without cost. Or we could decide that the names are ephemeral, compile time only entities, kind of like parameter names, and losing them when boxing is actually a good thing.
...
A nice thing about this scheme is that it leads to very little code bloat: it only relies on framework types, and not on compiler generated types in the assembly.
...
However, we are more inclined not to allow any of these conversions. They don't seem particularly helpful, and are likely to mask something you did wrong. People can always deconstruct and reconstruct if they want to get to a different tuple type, so our default position is to be super rigid here and only match exactly the same names and types in exactly the same order.

Nice thing about making them generated types would have been implicit support for seeing the named properties in other languages or older C# compilers. Oh well.

But I would love to see named tuples retrofitted and upgraded to full fledged anonymous structs in the future, and have those become the new de facto super named tuples -- caveat: unless there was some really good runtime reason not to.

I realize that would be a fair bit of work, so I'm not expecting it to happen. But if it did, it would make life easier in many scenarios. Not the least of which would be enabling reflection based serialization frameworks.

[marksmeltzer]

@HaloFour as for what to call them, I would suggest just calling them anonymous structs. And I would suggest that they be implicitly convertible to ValueTuple<,>. Moreover, to complete the paradigm anonymous classes should also be implicitly convertible to ValueTuple<,>.

Having symmetry for both anonymous structs and anonymous classes would complete the conceptual framework to make this all coherent and intuitive.

Simply stated: anonymous types can be treated as tuples, and tuple value order will be the same as the order of the values assigned in the anonymous type instantiation.

BTW, thanks for the link!

[Thaina]

#668

[yaakov-h]

@marksmeltzer

same problem applies to anonymous classes, right? And those got a naming convention.

Anonymous classes have no visible type signature. You cannot have a method that accepts or returns an anonymous type. It can pass around an object of that type, but only as object.

There is no public/fixed naming convention because they are not visible anywhere that a name is required.

[kekyo]

Thank you for all, I was able to organize my own thoughts.

The idea's trigger was problem from Newtonsoft.Json's serialization issue.
I can write F#. So the tuple syntax liked rather than anonymous type syntax, and positively use it in C# too.
I wrote the deserialize code from json string:

// Got nothing
var (firstName, lastName) = JsonConvert.DeserializeObject<(string firstName, string lastName)>(
  @"{'firstName':'Kouji','lastName':'Matsui'}");

But it causes failure storing each properties. I understand what problem for naming extraction from tuples.
It's in TupleElementNamesAttribute and it's applied to the method.
I feel these problem can't fix or looks like very difficult issue (I read the NewtonSoft.Json issue).

I retired to use tuple in this situation, try to change anonymous type rather than tuple type:

public static T Deserialize<T>(string json, T defaultValue)
{
    try
    {
        return JsonConvert.DeserializeObject<T>(json);
    }
    catch
    {
        return defaultValue;
    }
}

static void Main(string[] args)
{
    var result = Deserialize(
        @"{'firstName':'Kouji','lastName':'Matsui'}",
        new { firstName="", lastName="" });
}

It's bit ugly but perfect work (allocation cost only used for defaulted value. but the code isn't important and isn't focused).
Then I was thinking about if the anonymous types can be value type.

We wrote a lot of anonymous types in the C# 6 or less versions. I was thinking continue:

1. The syntax easy understands and already spreads many average developers. And maybe my syntax idea is easy understandable.
2. Anonymous types can lookup property names everywhere (compared to tuple types).
3. Most useful situation with anonymous types in LINQ. Many LINQ used code contains anonymous type syntax. We know there aren't expose anoymous type metadata outside method body. I feel it makes safer and easier less allocation costs if can totally switch to value type.

Afterall. I understanded it constraining by No.2 topic. I think that this problem has not been solved and will not be solved in the future. (So I wrote alternatives section :)

[marksmeltzer]

@yaakov-h you're technically correct, but I think that's speaking past my point. If I were to declare an anonymous tuple/struct like so, (string Name, int Age), I'm saying I don't particularly care what the underlying name generated by the compiler is. Name it algorithmically using a hash code of the types and names and their ordinal positions -- that would give something approaching a unique but deterministic name -- or even pick a random number and convert it to string. (As long they are implicitly convertible, it wouldn't matter in principle, but type reuse would have some practical benefits.)

Anytime I had a method that accepted a (string Name, int Age) parameter, or a field, or a variable in the same assembly, it could reuse the same type from that assembly because the typename could be derived deterministically. My suggestion was to simply duck type the structs -- so cross assembly types would just be implicitly convertible regardless of names as long as the types and ordinal positions matched.

Value tuples could have been implemented that way. And I think that would also achieve the goal of anonymous structs. Moreover, it would be nice if anonymous classes behaved the same way and were implicitly convertible to/from ValueTuple<> with the correct types.

For instance, having anonymous structs and anonymous classes be implicitly convertible to/from ValueTuple<> would open up some really interesting "templating" scenarios using generics once the C# compiler gets smarter about inferring generic types.

In other words, although @kekyo is talking mostly about serialization, I see this as being applicable to a larger class of problems.

Again, I doubt this will happen any time soon, and probably not at all. It would be nice though.

[marksmeltzer]

@yaakov-h TLDR version: the anonymous struct / tuple definition would be a stand in for a typename name. Just as it currently is.

[HaloFour]

@marksmeltzer

You can't have a randomly generated type name. You need a predictable type name, that will always be the same, regardless of the compiler. That means that the name needs to be in the specification and the compiler needs to carefully adhere to the specification. Otherwise tuples can simply never cross assembly boundaries, thus can never be exposed from a public API.

Also, using implicit conversion between ValueTuple<...> or some other type to bridge these generated structs would incur a fairly severely performance penalty due to the number of copies. It would also make ref structs across boundaries entirely impossible.

The design of tuples was settled years ago and I severely doubt that they're willing to reconsider their decision especially given that every single one of these points was mentioned and addressed. Tuples are intentionally positional ephemeral constructs, not intended for persistence. The names exist only to aid the developer in referencing the elements.

[marksmeltzer]

@HaloFour of course a random type name would be a genuinely bad idea. I was trying to use an emphatic hyperbole to illustrate that I wouldn't care what the name was. Your point is well taken, and my first suggestion was to use a deterministically derived name.

Also, using implicit conversion between ValueTuple<...> or some other type to bridge these generated structs would incur a fairly severely performance penalty due to the number of copies.

I did mention above that if there was a significant, demonstrable runtime cost to this method, that it wouldn't work. That's an empirical question though, and I don't know how to guess at its answer without taking a stab at it. Moreover, there might be some clever trick to obtain better performance once the implementation is at least in a working state. As you say, it might not have legs at all.

Was a scenario like that ever tested and benchmarked by the CLR/C# team? If the JIT was smart about duck typing and the types were immutable, I have a strong suspicion that all of the copying could be eliminated a la readonly struct.

It would also make ref structs across boundaries entirely impossible.

There's another good point on the surface, but again I think if the JIT was smart about how the duck typing was implemented, I think it wouldn't be an issue for the C# compiler to manage.

The design of tuples was settled years ago and I severely doubt that they're willing to reconsider their decision especially given that every single one of these points was mentioned and addressed. Tuples are intentionally positional ephemeral constructs, not intended for persistence. The names exist only to aid the developer in referencing the elements.

Alas, I lamentably agree that bit completely. So here I'm describing a C# language feature that probably would require a major assist from the JIT/CLI teams, and that probably won't happen. Too many moving parts.

[kekyo]

@marksmeltzer I feel these features looks like charming for me, but has trade off for difficult topics. I predicted (but couldn't clearly) it, so my proposal isn't containing how enlarge exposing anonymous (class|struct) types outside the method.

[VanKrock]

I think better value tuple convert to anonymous struct