Implemented ordering for expanded iterators

[JPPhoto]

Summary

Feature: best-effort in-order execution for nodes expanded by IterateInvocation, subject to ready state.

Why: iteration-expanded work could run in an arbitrary order when multiple iteration branches become ready at the same time, which reduces determinism and makes behavior harder to reason about.

How: compute a cached per-exec-node "iteration path" (outer to inner indices for nested iterators, respecting collector boundaries) and use it to ordered-insert ready nodes into the existing per-class ready queues. FIFO is preserved for equal iteration paths, and no blocking is introduced.

Related Issues / Discussions

QA Instructions

Create a collection, note the ordering, then iterate and pass values to a series of other nodes. Check that the outputs use the source collection's items in order.

Merge Plan

Checklist

• The PR has a short but descriptive title, suitable for a changelog
• Tests added / updated (if applicable)
• ❗Changes to a redux slice have a corresponding migration
• Documentation added / updated (if applicable)
• Updated What's New copy (if doing a release after this PR)

[lstein]

I'd like to be able to review this PR, but I've never quite understood how to use the collection/iteration functions in the workflow editor and don't think I have the expertise to identify edge cases and so forth.

If no one else is able to review, I'll come up to speed and give it a shot.

[lstein]

Following up on this, maybe you can provide an example workflow that illustrates the problems with the previous non-deterministic execution ordering and demonstrates how the PR fixes this?

[JPPhoto]

Following up on this, maybe you can provide an example workflow that illustrates the problems with the previous non-deterministic execution ordering and demonstrates how the PR fixes this?

As it's non-deterministic, I can't guarantee that any example will show anything at all. The problem is that when iteration expands, prior to this change the resulting nodes are treated like any others and there are no guarantees that the collection of expanded nodes will execute in any order at all. After this change, the first iteration's nodes should always complete before the second iteration's start, the second's before the third's, etc.

Hopefully this change accomplishes that; this section of code is getting very hard to read and understand and would likely benefit from a major refactor at some point.

[JPPhoto]

Drafting this as the collect node needs some adjustment even though execution seems to happen in order.

[JPPhoto]

Sample workflow: OrderingTestWorkflow.json

Before this PR, examining the output of the Integer Addition node:

[
    {"output_meta": null, "value": 1002, "type": "integer_output"},
    {"output_meta": null, "value": 1003, "type": "integer_output"},
    {"output_meta": null, "value": 1008, "type": "integer_output"},
    {"output_meta": null, "value": 1001, "type": "integer_output"},
    {"output_meta": null, "value": 1000, "type": "integer_output"},
    {"output_meta": null, "value": 1007, "type": "integer_output"},
    {"output_meta": null, "value": 1004, "type": "integer_output"},
    {"output_meta": null, "value": 1009, "type": "integer_output"},
    {"output_meta": null, "value": 1006, "type": "integer_output"},
    {"output_meta": null, "value": 1005, "type": "integer_output"}
]

After:

[
    {"output_meta": null, "value": 1000, "type": "integer_output"},
    {"output_meta": null, "value": 1001, "type": "integer_output"},
    {"output_meta": null, "value": 1002, "type": "integer_output"},
    {"output_meta": null, "value": 1003, "type": "integer_output"},
    {"output_meta": null, "value": 1004, "type": "integer_output"},
    {"output_meta": null, "value": 1005, "type": "integer_output"},
    {"output_meta": null, "value": 1006, "type": "integer_output"},
    {"output_meta": null, "value": 1007, "type": "integer_output"},
    {"output_meta": null, "value": 1008, "type": "integer_output"},
    {"output_meta": null, "value": 1009, "type": "integer_output"}
]

But, as I indicated above, there's something up with the Collect node.

[JPPhoto]

Collect ordering is now resolved. Try the test workflow.