Discussion: "semi-constructed" object state / "Immutable" fields.

[AartBluestoke]

Problem statement:

Immutables (#421,#2543,#2443), Records, friend types, discussions about primary constructors and how to add new members to types, nullability handling, and other discussions about constant-ish alterations to the language seem to get stuck around "how to handle deferred construction" .

I believe the problem is that there is a significant difference between when the language considers and object to have been constructed (which is closely tied to memory allocation, and the completion of the "new" function call) and that developers have objects that they consider to be semi-constructed objects.

For example during serialization where reflection is used to poke 'readonly' things which are in a Constructed() but not constructed (ie, properly initialized fields) state, and the function responsible for initialization wants to be able to violate "private/readonly-ness" of an object before returning it for consumption/and sharing .

Proposal

1. An object is considered semi-constructed until the end of the function call that "New'd" it (unless we add an annotation to allow that responsibility to escape to the calling function).
2. fields can be flagged as "readonly!" (or some other syntax) which means that the field is considered Immutable. (something similer for properties is needed, can't think of syntax of the top of my head).
3. Immutable fields can only be written while the object is in a semi-constructed state.
4. only fully constructed objects can be stored in immutable fields.
5. The runtime is free to make any optimisation reasonable from the assumption that immutable fields will never change after an objects construction is completed.

Example:

public class myImmutable{
    public readonly! string value;
}

public class otherImmutable{
    public readonly! myImmutable other;
    string f(){
       return other.value;
    }
}

public class test{
static myImmutable getImmutable(){
    return new myImmutable(){value="inner"}; // allowed here because we've created it in this function call
}
static otherImmutable getOtherImmutable(myImmutable inner){
// inner.value="fail"; // unsafe code here, as it may be an attempt to alter a readonly! field; if you do this you have to trust that you are never passed a fully-constructed object ; more comments on runtime vs compiletime later

return new otherImmutable (){value=inner}; // allowed here because we've created it in this function call
}

void main(){
var myImm =getImmutable();
// myImm.Value = "fail"; Compile error here, as you know this is immutable upon return. (note, some corner cases may by design allow a hung thread to expose a semi-constructed object here, but only in a case where this object could become fully constructed at any time)
var otherimm= getOtherImmutable(myImm);

Console.Writeline(otherimm.f());
// the compiler/jit is allowed to optimise this to
Console.Writeline("inner");
}

I believe that there would be many optimisations possible through recursively discoverable immutable values at runtime by the jit (or AOT) - when an inner func is JIT'd it may be possible that an immutable return value is discovered. In which case certain inlinings not currently allowed would be.

Enforcement of semi-constructed objects:

a) compile time:

1. syntax enforcement inside the function that directly contains the new

• easy to implement, is a minor change from current "readonly" accessor rules

2. syntax enforcement inside any child functions called from the function

• is a problem equal in complexity to null-refference tracing; except recursive into child calls, and not affecting the parents.
• probably needed, as many times it will be child functions manipulating the semi-constructed object
• defer to runtime?

3. code generation

• easy to implement - it's emitting prior to a return statement one boolean set for each semi-constructed object within the function call a f(){ y= new(..);...; y.isConstructed=true;)

b) runtime:

• requires the object (header?) to have a hidden 'bool' field , and the compiler to emit an instruction to toggle that flag.
• requires writes to the immutable field to check the bool first, requiring either runtime cooperation, or a separate compiler check (which would always eval true in correctly written code, thus be subject to significant optimisation)
• ?update to reflection to make writing to an immutable field check, unless in an unsafe context
• updating the runtime to take advantage of this optimisation safely could be complex

Implementation Considerations:

• has to compile into an annotation on fields, indicating "immutability", but how to track "is this object currently semi-constructed" is the issue.
  -forwards compatability
  this proposal as described doesn't require assistance from the runtime, so is fully forwards compatable.
• backwards compatibility
  adding the first immutable field to a type is potentially a binary breaking change, if the runtime implementation introduces a new boolean field, and there isn't a bit spare somewhere in the object definition to handle it
  Thought: is it possible to have the value of boolean annotation altered by the runtime? -- this would be the first time an annotation wasn't a const value

when could an object be considered complete

if a developer want an object to be considered "complete" part the way through a function call (as mentioned by @YairHalberstadt below

1. this use is by-design, as this is the specific case where you often want this to still be mutable.
   a) i can tell this might be a bug as x isn't returned, for non-returned immutables, it is an error to mutate the readonly:
   ai) without declaring where the scope for which construction completes.
   aii) after the first time it is used as a function parameter?
   aiiI) after the last time it is used as a function parameter?
   b) perhaps an informational "late assignment of readonly field at x.readonly=42; to assist with diagnoses of bugs. (enable "pit of success, at cost of slightly annoying people who use this feature?)
2. increase the complexity of the suggestion to have some way to declare "done" on an object;
   a) construction is complete on assignment to another variable (eg, var useX();var y=x; the object is complete;
   b) construction is complete on leaving a scope that that isn't a branching or loop operator (ie, manual { } scopes, using statements, ...
   c) construction is complete upon returning from a function which uses it as a parameter? (but a factory might want to do multiple things (populate name, lookup address, apply record security...) )
3. have a language assisted statement that declares it done (in line with null suppression, use the ! operator to declare it no longer semi-constructed)
   a) x! as a statement.
   b) new! as an operator to declare that partial construction is complete.

---

Code example that got me thinking about some of the above, based on the thought "couldn't we just inline the readonly"

using System;
using System.Reflection;

public class Test
{
    private readonly string foo = "Foo";

    [Fact]
    public static void testReflection()
    {
        Test test = new Test();
        FieldInfo field = typeof(Test).GetField
            ("foo", BindingFlags.Instance | BindingFlags.NonPublic);
        field.SetValue(test, "Hello");
          Assert.Equal("Foo",test.getFoo()); // assert fail
    }        
}

this is "safe" code, but very much not actually safe for anyone who assumes that the readonly things don't change... It'd be nice for a runtime guarantee to guard that, not just a compile time declaration of intent.

[YairHalberstadt]

I think that there's some interesting ideas here.

However this is all coming from the assumption that an object is not fully constructed till after the method it's constructor is called in.

This works really well for factory methods. However another far more common pattern is to create an object and then use it in the same method:

M()
{
    var x = new ImmutableClass();
    UseX(x);
    x.readonlyField = 42; // oops, this just changed
}

How would you deal with that?

[ufcpp]

initonly?

[AartBluestoke]

@YairHalberstadt there are several answers to that depending on your point of view, some thoughts i had on directions here are that:
0) not a hard to trace bug as mutability bugs of an immutable object are confined to within a few lines of code.

1. this use is by-design, as this is the specific case where you want this to still be mutable.
   1a) i can tell this might be a bug as x isn't returned, for non-returned immutables, it is an error to mutate the readonly:
   1ai) without declaring where the scope for which construction completes.
   1aii) after the first time it is used as a function parameter?
   1aiiI) after the last time it is used as a function parameter?
   1b) perhaps an informational "late assignment of readonly field at x.readonly=42; to assist with diagnoses of bugs. (enable "pit of success, at cost of slightly annoying people who use this feature?)
2. increase the complexity of the suggestion to have some way to declare "done" on an object;
   2a) construction is complete on assignment to another variable (eg, var useX();var y=x; the object is complete;
   2b) construction is complete on leaving a scope that that isn't a branching or loop operator (ie, manual { } scopes, using statements, ...
   2c) construction is complete upon returning from a function which uses it as a parameter? (but a factory might want to do multiple things (populate name, lookup address, apply record security...) )
3. have a language assisted statement that declares it done (in line with null suppression, use the ! operator to declare it no longer semi-constructed)
   3a) x! as a statement.
   3b) new! as an operator to declare that partial construction is complete.

@ufcpp the initonly part of the records proposal only allows the semi-constructed state to extend to the end of the object initializer, which doesn't allow sophisticated function calls or logic to be applied in the construction.

[CyrusNajmabadi]

It feels like this could all be designed around the same concept c# has for ensuring all the fields of a struct are assigned before the struct instance itself is considered fully assigned.

For classes we'd naught just need to Mark the fields and props that would have to be assigned post construction anda similar analysis would be performed.

[Joe4evr]

It feels like this could all be designed around the same concept c# has for ensuring all the fields of a struct are assigned before the struct instance itself is considered fully assigned.

For classes we'd naught just need to Mark the fields and props that would have to be assigned post construction anda similar analysis would be performed.

Sounds like #2328. 😏

[AartBluestoke]

@Joe4evr , @CyrusNajmabadi those cases still have the "so when is this object considered complete" -- developers still need a way to say when they want the object to transition to being truly immutable, eg: the struct initializer proposal linked doesn't allow child function calls the possibility to mutate the late-init fields.

[ronnygunawan]

I like kotlin's lateinit better. It throws exception if the property is accessed before it is initialized, or if we initialize a readonly property twice. We can use it for something like:

class Form1 : Form {
    public lateinit string Prop { get; }

    public Form1() {
        Load += Form1_Load;
    }

    void Form1_Load(object sender, EventArg e) {
        Prop = "This property was initialized in Form1_Load";
    }
}

[HaloFour]

This is similar to some of the thinking around required initialization. I don't care for it for a multitude of reasons, mainly around the "weakening" of readonly and requiring quite a bit of additional metadata and winks/nudges to enable such support safely in the compiler. Such metadata would not play very nicely with downlevel compilers or other languages in the ecosystem. They will only see a normal property setter and assume that they can call it. Maybe modreq can be used to force all consuming compilers to have to understand the new rules, but even then it's a question of every single compiler implementing the logic correctly to the spec which I think would be error prone.

[Richiban]

@ronnygunawan The problem with Kotlin's lateinit is that it's entirely a runtime feature.

i.e. the following code produces no errors or warnings:

public class MyTest {
    lateinit var s: String
    
    public val subject = s
}

fun main() {
    var m = MyTest()
    
    println(m.subject)
}

It just fails at runtime with an exception, making it little better than null.