Skip to content

Conversation

@Zouxxyy
Copy link
Contributor

@Zouxxyy Zouxxyy commented Apr 17, 2025

Purpose

This PR extracts some modifications from #5242 to the PR, thanks to zhongyujiang

The InternalRowSerializer needs to maintain the handling of null value for uncompact Decimal and timestamp types in org.apache.paimon.codegen.GenerateUtils.binaryWriterWriteNull to avoid inconsistencies in hashcode, which could lead to the inconsistencies in bucket calculation.

  def binaryWriterWriteNull(indexTerm: String, writerTerm: String, t: DataType): String =
    t.getTypeRoot match {
      // ordered by type root definition
      case DECIMAL if !Decimal.isCompact(getPrecision(t)) =>
        s"$writerTerm.writeDecimal($indexTerm, null, ${getPrecision(t)})"
      case TIMESTAMP_WITHOUT_TIME_ZONE | TIMESTAMP_WITH_LOCAL_TIME_ZONE
          if !Timestamp.isCompact(getPrecision(t)) =>
        s"$writerTerm.writeTimestamp($indexTerm, null, ${getPrecision(t)})"
      case _ =>
        s"$writerTerm.setNullAt($indexTerm)"
    }

Tests

API and Format

Documentation

Copy link
Contributor

@zhongyujiang zhongyujiang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense to me.

Copy link
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi JingsongLi merged commit cd5829f into apache:master Apr 18, 2025
20 checks passed
@Zouxxyy Zouxxyy deleted the dev/fix-ser-null branch April 18, 2025 03:40
JingsongLi pushed a commit that referenced this pull request Apr 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants