fix(app): incorrect values inserted when retrying queue item

In #7688 we optimized queuing preparation logic. This inadvertently broke retrying queue items.

Previously, a `NamedTuple` was used to store the values to insert in the DB when enqueuing. This handy class provides an API similar to a dataclass, where you can instantiate it with kwargs in any order. The resultant tuple re-orders the kwargs to match the order in the class definition.

For example, consider this `NamedTuple`:
```py
class SessionQueueValueToInsert(NamedTuple):
    foo: str
    bar: str
```

When instantiating it, no matter the order of the kwargs, if you make a normal tuple out of it, the tuple values are in the same order as in the class definition:

```
t1 = SessionQueueValueToInsert(foo="foo", bar="bar")
print(tuple(t1)) # -> ('foo', 'bar')

t2 = SessionQueueValueToInsert(bar="bar", foo="foo")
print(tuple(t2)) # -> ('foo', 'bar')
```

So, in the old code, when we used the `NamedTuple`, it implicitly normalized the order of the values we insert into the DB.

In the retry logic, the values of the tuple were not ordered correctly, but the use of `NamedTuple` had secretly fixed the order for us.

In the linked PR, `NamedTuple` was dropped for a normal tuple, after profiling showed `NamedTuple` to be meaningfully slower than a normal tuple.

The implicit order normalization behaviour wasn't understood, and the order wasn't fixed when changin the retry logic to use a normal tuple instead of `NamedTuple`. This results in a bug where we incorrectly create queue items in the DB. For example, we stored the `destination` in the `field_values` column.

When such an incorrectly-created queue item is dequeued, it fails pydantic validation and causes what appears to be an endless loop of errors.

The only user-facing solution is to add this line to `invokeai.yaml` and restart the app:
```yaml
clear_queue_on_startup: true
```

On next startup, the queue is forcibly cleared before the error loop is triggered. Then the user should remove this line so their queue is persisted across app launches per usual.

The solution is simple - fix the ordering of the tuple. I also added a type annotation and comment to the tuple type alias definition.

Note: The endless error loop, as a general problem, will take some thinking to fix. The queue service methods to cancel and fail a queue item still retrieve it and parse it. And the list queue items methods parse the queue items. Bit of a catch 22, maybe the solution is to simply delete totally borked queue items and log an error.

Author: psychedelicious

invokeai/app/services/session_queue/session_queue_common.py

@@ -563,14 +563,17 @@ def calc_session_count(batch: Batch) -> int:
     str,  # session (as stringified JSON)
     str,  # session_id
     str,  # batch_id
-    str | None,  # field_values (optional, as stringified JSON)
+    str | None,  # field_values (optional, as stringified JSON)‰
     int,  # priority
     str | None,  # workflow (optional, as stringified JSON)
     str | None,  # origin (optional)
     str | None,  # destination (optional)
     int | None,  # retried_from_item_id (optional, this is always None for new items)
 ]
-"""A type alias for the tuple of values to insert into the session queue table."""
+"""A type alias for the tuple of values to insert into the session queue table.
+
+**If you change this, be sure to update the `enqueue_batch` and `retry_items_by_id` methods in the session queue service!**
+"""
 
 
 def prepare_values_to_insert(

invokeai/app/services/session_queue/session_queue_sqlite.py

@@ -27,6 +27,7 @@
     SessionQueueItemDTO,
     SessionQueueItemNotFoundError,
     SessionQueueStatus,
+    ValueToInsertTuple,
     calc_session_count,
     prepare_values_to_insert,
 )
@@ -689,7 +690,7 @@ def retry_items_by_id(self, queue_id: str, item_ids: list[int]) -> RetryItemsRes
         """Retries the given queue items"""
         try:
             cursor = self._conn.cursor()
-            values_to_insert: list[tuple] = []
+            values_to_insert: list[ValueToInsertTuple] = []
             retried_item_ids: list[int] = []
 
             for item_id in item_ids:
@@ -715,16 +716,16 @@ def retry_items_by_id(self, queue_id: str, item_ids: list[int]) -> RetryItemsRes
                     else queue_item.item_id
                 )
 
-                value_to_insert = (
+                value_to_insert: ValueToInsertTuple = (
                     queue_item.queue_id,
+                    cloned_session_json,
+                    cloned_session.id,
                     queue_item.batch_id,
-                    queue_item.destination,
                     field_values_json,
-                    queue_item.origin,
                     queue_item.priority,
                     workflow_json,
-                    cloned_session_json,
-                    cloned_session.id,
+                    queue_item.origin,
+                    queue_item.destination,
                     retried_from_item_id,
                 )
                 values_to_insert.append(value_to_insert)