Skip to content

[BUG] Rds does not handle mysql decimal data types when precision value is 19 or higher #6339

@divbok

Description

@divbok

Describe the bug
Data Prepper RDS source does not handle MySQL DECIMAL data types with precision ≥19. When RDS exports snapshots to S3, values ≥2^63 are encoded as FIXED_LEN_BYTE_ARRAY in Parquet format (per AWS documentation). The NumericTypeHandler does not support this encoding, causing pipeline failures.

To Reproduce

  1. Create MySQL table with DECIMAL(38,0) column containing values ≥2^63
  2. Export RDS snapshot to S3
  3. Configure Data Prepper pipeline with RDS source pointing to the export
  4. Pipeline fails with error:
java.lang.IllegalArgumentException: Unsupported value type. The value is of type: class java.util.ArrayList
  at org.opensearch.dataprepper.plugins.source.rds.datatype.mysql.handler.NumericTypeHandler.handleNumericType

Expected behavior
RDS source should handle all MySQL DECIMAL types regardless of precision, converting FIXED_LEN_BYTE_ARRAY (ArrayList) to BigInteger for values that exceed INT64 range.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

  • OS: [e.g. Ubuntu 20.04 LTS]
  • Version [e.g. 22]

Additional context
Add any other context about the problem here.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

Status

Done

Relationships

None yet

Development

No branches or pull requests

Issue actions