Skip to content

Projection fails when reading list-of-struct data structures #284

@zpks

Description

@zpks

The current implementation of pymongoarrow/schemas.py assings a projection of 'True' to all fields in a struct.
This does not work for structs that contain list-of-struct datatypes; this can be fixed by reworking the projection creation to use Mongo's dot notation.

Example:
read in a data field like
"a": {
"b": {
"c": [
{
"event": {
"$date": "2022-01-01T00:00:00.000Z"
},
"value": 2,
}
],
}
}

the projection to read this with a schema needs to be
{"a.b.c.event": True, "a.b.c.value": True}

The current implementation of the projection returns a projection of
{"a": {"b": {"c": {"event": True}}}, which fails as the Mongo client reads "b" as an operator.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions