-
Notifications
You must be signed in to change notification settings - Fork 280
Description
Describe the bug
Based on the following MRE:
- Schema Issue: The
_engine_data_slicedoesn't update the children table's schema to includenew_field. The children table schema remainsLTable(Struct(value: Int64))instead of becomingLTable(Struct(value: Int64, new_field: Int64)). - Field Scope: The new_field exists temporarily within the row context (
DataScope) but doesn't persist to the parent table's schema. - Lost on Return: When returning data, the
new_fieldis not part of the table schema, so the decoder falls back to the default value (0) from theNewChilddataclass.
Relevant Context: #737
To Reproduce
Create a new test file under python/cocoindex/tests and run this MRE code:
import typing
from dataclasses import dataclass
import pytest
import cocoindex
@dataclass
class Child:
value: int
@dataclass
class Parent:
children: list[Child]
@dataclass
class NewChild:
value: int
new_field: int = 0
@dataclass
class ParentWithNewField:
children: list[NewChild]
@pytest.fixture(scope="session", autouse=True)
def init_cocoindex() -> typing.Generator[None, None, None]:
cocoindex.init()
yield
@cocoindex.op.function()
def extract_value(value: int) -> int:
return value
# Transform flow that should add new_field
@cocoindex.transform_flow()
def for_each_transform(
data: cocoindex.DataSlice[Parent],
) -> cocoindex.DataSlice[ParentWithNewField]:
with data["children"].row() as child:
child["new_field"] = child["value"].transform(extract_value)
return data
def test_for_each_transform():
input_data = Parent(children=[Child(1), Child(2), Child(3)])
result = for_each_transform.eval(input_data)
print(f"Result: {result}")
expected = ParentWithNewField(
children=[
NewChild(value=1, new_field=1),
NewChild(value=2, new_field=2),
NewChild(value=3, new_field=3),
]
)
assert result == expected, f"Expected {expected}, got {result}"
# Expected ParentWithNewField(children=
# [NewChild(value=1, new_field=1),
# NewChild(value=2, new_field=2),
# NewChild(value=3, new_field=3)]),
# got ParentWithNewField(children=
# [NewChild(value=1, new_field=0),
# NewChild(value=2, new_field=0),
# NewChild(value=3, new_field=0)])Expected behavior
- Intended:
child["new_field"] = ...should modify the table schema and persist - Actual: The field assignment is temporary and doesn't affect the parent table schema