Skip to content

[Bug]: TypeError: unhashable type: 'dict' in avroio.py #35462

@RRap0so

Description

@RRap0so

What happened?

When type is read from an avro, (see example below).

...
 "name": "a_name",
      "type": [
        "null",
        {
          "avro.java.string": "String",
          "type": "string"
        }
      ]
...

This will fail with the following error:

INFO 2025-06-24T08:00:07.685658Z >> ReadFromAvro(f"{input_path}.avro", as_rows=True)
INFO 2025-06-24T08:00:07.685908Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO 2025-06-24T08:00:07.686203Z File "/usr/src/app/.venv/lib/python3.12/site-packages/apache_beam/io/avroio.py", line 170, in __init__
INFO 2025-06-24T08:00:07.686475Z beam_schema = avro_schema_to_beam_schema(avro_schema)
INFO 2025-06-24T08:00:07.686678Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO 2025-06-24T08:00:07.686897Z File "/usr/src/app/.venv/lib/python3.12/site-packages/apache_beam/io/avroio.py", line 607, in avro_schema_to_beam_schema
INFO 2025-06-24T08:00:07.687135Z beam_type = avro_type_to_beam_type(avro_schema)
INFO 2025-06-24T08:00:07.687346Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO 2025-06-24T08:00:07.687559Z File "/usr/src/app/.venv/lib/python3.12/site-packages/apache_beam/io/avroio.py", line 598, in avro_type_to_beam_type
INFO 2025-06-24T08:00:07.687771Z f['name'], avro_type_to_beam_type(f['type']))
INFO 2025-06-24T08:00:07.687972Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO 2025-06-24T08:00:07.688190Z File "/usr/src/app/.venv/lib/python3.12/site-packages/apache_beam/io/avroio.py", line 576, in avro_type_to_beam_type
INFO 2025-06-24T08:00:07.688387Z return avro_union_type_to_beam_type(avro_type)
INFO 2025-06-24T08:00:07.688579Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO 2025-06-24T08:00:07.688775Z File "/usr/src/app/.venv/lib/python3.12/site-packages/apache_beam/io/avroio.py", line 563, in avro_union_type_to_beam_type
INFO 2025-06-24T08:00:07.688998Z if avro_type in AVRO_PRIMITIVES_TO_BEAM_PRIMITIVES:
INFO 2025-06-24T08:00:07.689308Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INFO 2025-06-24T08:00:07.689545Z TypeError: unhashable type: 'dict'

I was able to reproduce this in a unit test.

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

  • Component: Python SDK
  • Component: Java SDK
  • Component: Go SDK
  • Component: Typescript SDK
  • Component: IO connector
  • Component: Beam YAML
  • Component: Beam examples
  • Component: Beam playground
  • Component: Beam katas
  • Component: Website
  • Component: Infrastructure
  • Component: Spark Runner
  • Component: Flink Runner
  • Component: Samza Runner
  • Component: Twister2 Runner
  • Component: Hazelcast Jet Runner
  • Component: Google Cloud Dataflow Runner

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions