Feature Request: Serializable Coroutine for Game Save

[isral]

I want serializable stackless coroutine for save and load game coroutine.

• For who doesn't care about game version, just serialize it.
• For who care, user must provide state id on yield and disallow same name local variable.

// https://docs.unity3d.com/Manual/Coroutines.html
Coroutine<void> Fade(float start) {
    // state 0
    for (float ft = 1f; ft >= 0; ft -= 0.1f) {
        ...
        co_yield(0); // 0 means use previous state + 1
        // state 1
    }
    for (int ft = ...) { } // not allowed, same name
    co_yield(10); // hardcoded state allow user to add or remove state for next version
    // state 10

    co_yield(SomeEnum.SomeValue); // will be casted to int
    // state (int)SomeEnum.SomeValue

    Matrix m = co_call(0, OtherCoroutine(Param1, Param2));
    // call other stackless coroutine like await

    // if coroutine implemented as IEnumerator then this can be implemented:
    // 1 Use a manager that has IEnumerator stack that call MoveNext on top of stack
    //   every co_call to another coroutine will add IEnumerator to stack
    //   manager will stop when co_yield encountered
    // or
    // 2 foreach (var _ in OtherCoroutine(Param1, Param2)) {
    //      yield return null;
    //   }
    // Note:
    // - co_call translated to yield return OtherCoroutine
    // - co_yield translated to yield return null
    // - option 2 give clear stack on crash but slow on pause and resume
}

// Manager (option 1) example usage
class XnaLikeGame {
    CoroutineManager cm;
    void Initialize() {
        cm.Call(Fade(1.0));
    }
    void Update() {
        cm.Resume();
    }
}

// No manager (option 2) example usage
class XnaLikeGame {
    Coroutine<void> coro;
    void Initialize() {
        coro = Fade(1.0);
    }
    void Update() {
        coro.Resume();
    }
}

serialized to

<Fade>
    <__state>1</__state>
    <__callee><OtherCoroutine/></__callee> <!-- for option 2 -->
    <__result><void/></result>
    <start>1.0</start>
    <ft>0.9</ft>
    <m>...</m>
</Fade>

<CoroutineManager>
    <Stack>
        <Fade>...</Fade>
        <OtherCoroutine>...</OtherCoroutine>
    </Stack>
    <Result>...</Result>
</CoroutineManager>

• Option 3: CoroutineManager embedded on top most coroutine.
• Option 4: ES6 yield*

---

Coroutine is used for game scripting to be more intuitive rather then using a method that called every game update with switch (state) { ... }
Implementation:

• yield: https://docs.unity3d.com/Manual/Coroutines.html
• await: https://stackoverflow.com/questions/14952689
• yield serialization: https://stackoverflow.com/questions/3294224

---

I think my proposal is full of flaws, so any fix are welcome.

[PathogenDavid]

Coroutines are very much so a Unity-specific thing. It's up to Unity to provide serialization support for them, nothing in the language is prohibiting that.

If you need coroutine-like functions to survive saves, I don't recommend using Unity coroutines at all.

Personally, I think you should explicitly define the state that should survive saves and reload. Don't try to capture everything, just what actually matters. At the very least, it makes it much easier to version. (As you've suggested, serializing the automatically generated enumerable classes would be very fragile when the code changes.)

[isral]

Coroutines are very much so a Unity-specific thing. It's up to Unity to provide serialization support for them, nothing in the language is prohibiting that.

I encounter the need for coroutine before I know Unity and before I know coroutine.
I asked my friend, "How can I run a code that last more than a frame?", and he said what I need is called coroutine.

If you need coroutine-like functions to survive saves, I don't recommend using Unity coroutines at all.

I don't use Unity (see: class XnaLikeGame).
I write Unity here because it's popular and has documentation on it.

Personally, I think you should explicitly define the state that should survive saves and reload. Don't try to capture everything, just what actually matters. At the very least, it makes it much easier to version. (As you've suggested, serializing the automatically generated enumerable classes would be very fragile when the code changes.)

I find writing code on update() is getting harder when the game is getting more complex.
Coroutine lower the complexity and more intuitive.

[Thaina]

@isral In C# (and arguably many newer language like js and ts) we tends to use Task and async/await nowaday in place of Coroutine. Only too old engine try to do it that way. You should try to adopt new paradigm

[isral]

@isral In C# (and arguably many newer language like js and ts) we tends to use Task and async/await nowaday in place of Coroutine. Only too old engine try to do it that way. You should try to adopt new paradigm

Async/await is one of implementation of coroutine,
Check https://en.wikipedia.org/wiki/Coroutine , you'll see async and await words there.

I write about await twice in my proposal, in co_call, and stackoverlow link near the end.

[Thaina]

@isral That's correct, so we don't ever directly use coroutine anymore. Because it was too old. It could be abstracted into Task

I don't said you don't know about it. But I told you to accept that coroutine is not being used on the coding level anymore. It's too old. It is only a legacy of the old style framework. We better write it as Task in the same manner as we never write alloc directly anymore, we use new to do both alloc and construct in the same time. Coroutine is the same and we shouldn't add anymore support for anyone to write it directly, you should use Task in its place

[yaakov-h]

Can't you do this already with a custom AsyncTaskMethodBuilder?

[isral]

Can't you do this already with a custom AsyncTaskMethodBuilder?

I read https://blogs.msdn.microsoft.com/lucian/2016/04/20/async-workflow-2/
I don't really understand but after running the codes, looks like it produce json serialized task.
If I can move "Checkpoint.Save()" to outside of async method, maybe it will satisfy:

• For who doesn't care about game version, just serialize it.

But not:

• For who care, user must provide state id on yield and disallow same name local variable.

[YairHalberstadt]

I'm closing this as I don't believe this has anything to do with the language. Feel free to correct me if I'm wrong and I'll move this to a discussion.

[isral]

I'm closing this as I don't believe this has anything to do with the language. Feel free to correct me if I'm wrong and I'll move this to a discussion.

I'm currently creating my own language to implement this, but maybe it's too much for me.

[YairHalberstadt]

@isral

What I meant is, this seems more like a compiler/runtime issue than a language one. Do you have any suggestions as to how the language spec might change to make this possible?

[isral]

@YairHalberstadt
https://github.com/dotnet/csharplang/blob/master/spec/classes.md

An enumerable object is typically an instance of a compiler-generated enumerable class that encapsulates the code in the iterator block and implements the enumerable interfaces, but other methods of implementation are possible.

C# compiler create several private fields to make local (and temporary) variables preserved across yield.
C# compiler also creating state field with integer values (the values is up to the compiler).

In this proposal, there is a new variant of enumerable object that is user controlled.
Compiler is not allowed to make it's own private fields and states (with exception, first state (0) and last state (Int32.MaxValue or -1))
Optionally, the object also contains call stack (not CPU stack, but simulate it.)

[YairHalberstadt]

Ok, @isral I'm still not 100% sure what you're suggesting here, but I've moved this to a discussion to help try and understand this better.

I think a code sample showing what the compiler generates now, and what it should generate for your proposal to work would be helpful.

[isral]

IEnumerator Fade(float start)   // become Coroutine<void> Fade(float start)
{
    for (float ft = 1f; ft >= 0; ft -= 0.1f)
    {
        yield return null;      // become co_yield(x); with x is 0 or const int or enum value
                                // or become yield x;
                                // or become yieldAt x;
                                // or if x is 0 become yield;
    }
    // user can not create other ft because it will conflict with previous ft after promoted to field
    yield return OtherCoroutine(1, 2); // become co_call(x, OtherCoroutine(1, 2));
                                       // or become yield x call OtherCoroutine(1, 2);
                                       // or become yieldAt x call OtherCoroutine(1, 2);
                                       // or if x is 0 become yield call OtherCoroutine(1, 2);
                                       // or if x is 0 become yield OtherCoroutine(1, 2);
                                       // or if x is 0 become await OtherCoroutine(1, 2);
                                // user can return value; if Coroutine type is not Coroutine<void>
}

private sealed class <Fade>d__1 // become [Serializable] public class Fade
                                // function Fade() removed
                                // any call to function Fade() become new Fade()
    : IEnumerator<object>, IDisposable, IEnumerator // become : Coroutine<void>
{
    private int <>1__state;     // become public int __state;
    private object <>2__current;// become public IEnumerator __callee;
                                // if there is return value become public T __result;
    public float start;
    public Program <>4__this;   // either become public Program __this or removed;
    private float <ft>5__1;     // become public float ft;
                                // optionally contain coroutine stack

    private bool MoveNext()
    {
        switch (<>1__state)
        {
        default:
            return false;
        case 0:
            <>1__state = -1;
            <ft>5__1 = 1f;
            goto IL_0063;
        case 1:                 // the value 1 optionally can be changed by user to a const or enum
            <>1__state = -1;
            <ft>5__1 -= 0.1f;
            goto IL_0063;
        case 2:                 // the value 2 optionally can be changed by user to a const or enum
            {
                <>1__state = -1;
                return false;
            }
            IL_0063:
            if (<ft>5__1 >= 0f)
            {
                <>2__current = null;
                <>1__state = 1; // the value 1 optionally can be changed by user to a const or enum
                return true;
            }
            <>2__current = <>4__this.OtherCoroutine(1, 2);
            <>1__state = 2;     // the value 2 optionally can be changed by user to a const or enum
            return true;
        }
    }
...