add DECIMAL type support as primary key by gpalanisamyYopp · Pull Request #3087 · GoogleCloudPlatform/DataflowTemplates

gpalanisamyYopp · 2025-12-09T20:53:13Z

No description provided.

gemini-code-assist · 2025-12-09T20:53:33Z

Summary of Changes

Hello @gpalanisamyYopp, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the sourcedb-to-spanner pipeline by adding robust support for MySQL DECIMAL data types, especially when they serve as primary keys. The changes ensure that DECIMAL columns are accurately identified, their values are precisely managed during range-based splitting, and the entire migration process for such tables is thoroughly verified through new test cases. This improves the reliability and completeness of data migration for a broader range of MySQL schema configurations.

Highlights

DECIMAL Type Support: Introduced comprehensive support for MySQL's DECIMAL data type, particularly when used as a primary key, ensuring it is correctly recognized and handled throughout the data pipeline.
Precision in BigDecimal Splitting: Enhanced the splitBigDecimal method to maintain precision during range splitting operations by correctly handling the scale of BigDecimal values when converting to and from BigInteger.
New IndexType: Added a new DECIMAL enum value to IndexType to explicitly categorize decimal primary keys.
Expanded Test Coverage: Included new unit and integration tests to validate the correct discovery, splitting, and migration of tables with DECIMAL primary keys from MySQL to Spanner.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

...e/cloud/teleport/v2/source/reader/io/jdbc/uniformsplitter/range/BoundarySplitterFactory.java

…littability checks

...google/cloud/teleport/v2/source/reader/io/jdbc/dialectadapter/mysql/MysqlDialectAdapter.java

VardhanThigle · 2025-12-14T02:25:45Z

...google/cloud/teleport/v2/source/reader/io/jdbc/dialectadapter/mysql/MysqlDialectAdapter.java

        }

+        BigDecimal decimalStepSize = null;
+        if (indexType.equals(IndexType.FLOAT) || indexType.equals(IndexType.DECIMAL)) {


I am not sure if we have FLOAT type checked in yet.
Or should I hold on this reivew till FLOAT is merged in

This PR has some overlap with the float PR. It would likely be simpler to start with the float PR, and then we will rebase this PR on top of that one.

VardhanThigle · 2025-12-14T02:27:16Z

...google/cloud/teleport/v2/source/reader/io/jdbc/dialectadapter/mysql/MysqlDialectAdapter.java

+        BigDecimal decimalStepSize = null;
+        if (indexType.equals(IndexType.FLOAT) || indexType.equals(IndexType.DECIMAL)) {
+          if (numericScale > 0) {
+            decimalStepSize = BigDecimal.ONE.scaleByPowerOfTen(-numericScale);


Do we really need to couple this with stepSize or could we handle the scale as is?

My worry here is any rounding error in this exponentiation.

The scale is explicitly required to calculate the StepSize, as the StepSize is used to calculate the smallest possible difference between two values (later on in the Splitter) and then check for inequality of two values by comparing the difference to the smallest possible difference between two values.

I don't see a way to decouple these two concepts, but I also don't expect to have rounding issues given we are using a BigDecimal and a Java built-in method to perform the exponentiation. As I understand the BigDecimal type, it is explicitly designed to handle such calculations without loss of precision.

@VardhanThigle I think I see what you mean, and I expect working with the scale directly would actually simplify things a bit (I'm thinking it should allow us to use setScale and then unscaledValue to get a corresponding BigInteger, and then split with that). I'm going to work on swapping out the decimal step size in favour of just the scale itself.

I would also say, I personally think it makes more sense for the SourceColumnIndexInfo class to have just the numeric scale, and then this gets used in various ways depending on the column type (since at the end of the day, the idea of "scale" is the same for decimal, double, and float, we just make use of it in different ways). But that can be a later improvement/refactoring.

VardhanThigle · 2025-12-15T06:21:31Z

Could we please fix the build?

nmemond · 2025-12-16T16:34:28Z

@VardhanThigle I unfortunately don't have permissions to push to the fork and the developer that owns this is out of office for the rest of the year. I've re-created the PR but up-to-date with main and with build and issues fixed in #3119. Please close out this PR.

VardhanThigle · 2026-01-12T05:51:04Z

Should this be rolled back as it has already been taken care of in other PRs.

VardhanThigle · 2026-01-12T05:57:29Z

Closing as discussed over chat. This has been taken care of in other PRs.

add DECIMAL type support as primary key

c5f8afd

gpalanisamyYopp requested a review from a team as a code owner December 9, 2025 20:53

gpalanisamyYopp requested review from bharadwaj-aditya and rohitwali December 9, 2025 20:53

pull-request-size bot added the size/M label Dec 9, 2025

gpalanisamyYopp added 5 commits December 9, 2025 13:53

fix spotless error

8e61113

Merge branch 'main' into source-to-spanner-mysql-decimal-pk

7a53eed

Update MysqlDialectAdapterTest.java

0135ab7

fix spotless

67b250d

add DECIMAL PK with precision/scale

a46ee4e

VardhanThigle added the ignore-for-release label Dec 11, 2025

VardhanThigle reviewed Dec 11, 2025

View reviewed changes

...e/cloud/teleport/v2/source/reader/io/jdbc/uniformsplitter/range/BoundarySplitterFactory.java Outdated Show resolved Hide resolved

update BigDecimal splitter to handle scale correctly

8d9d652

gpalanisamyYopp requested a review from VardhanThigle December 11, 2025 16:44

NicolasRicheYopp reviewed Dec 11, 2025

View reviewed changes

...e/cloud/teleport/v2/source/reader/io/jdbc/uniformsplitter/range/BoundarySplitterFactory.java Show resolved Hide resolved

Add BigDecimal boundary splitter with precision-aware equality and sp…

6503b26

…littability checks

pull-request-size bot added size/L and removed size/M labels Dec 12, 2025

NicolasRicheYopp reviewed Dec 12, 2025

View reviewed changes

...google/cloud/teleport/v2/source/reader/io/jdbc/dialectadapter/mysql/MysqlDialectAdapter.java Outdated Show resolved Hide resolved

gpalanisamyYopp added 2 commits December 12, 2025 14:48

Fix DECIMAL type mapping to use BigDecimal splitter

dc932d8

Support DECIMAL columns with precision-aware splitting

6a31107

VardhanThigle reviewed Dec 14, 2025

View reviewed changes

nmemond mentioned this pull request Dec 16, 2025

Spanner bulk data migration: Add support for DECIMAL as a primary key #3119

Merged

VardhanThigle closed this Jan 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add DECIMAL type support as primary key#3087

add DECIMAL type support as primary key#3087
gpalanisamyYopp wants to merge 10 commits intoGoogleCloudPlatform:mainfrom
gpalanisamyYopp:source-to-spanner-mysql-decimal-pk

gpalanisamyYopp commented Dec 9, 2025

Uh oh!

gemini-code-assist bot commented Dec 9, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

VardhanThigle Dec 14, 2025

Uh oh!

rossd-bitquill Dec 15, 2025

Uh oh!

VardhanThigle Dec 14, 2025 •

edited

Loading

Uh oh!

rossd-bitquill Dec 15, 2025

Uh oh!

nmemond Dec 15, 2025

Uh oh!

nmemond Dec 15, 2025

Uh oh!

VardhanThigle commented Dec 15, 2025

Uh oh!

nmemond commented Dec 16, 2025

Uh oh!

VardhanThigle commented Jan 12, 2026

Uh oh!

VardhanThigle commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

gpalanisamyYopp commented Dec 9, 2025

Uh oh!

gemini-code-assist bot commented Dec 9, 2025

Summary of Changes

Highlights

Footnotes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

VardhanThigle Dec 14, 2025

Choose a reason for hiding this comment

Uh oh!

rossd-bitquill Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

VardhanThigle Dec 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rossd-bitquill Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

nmemond Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

nmemond Dec 15, 2025

Choose a reason for hiding this comment

Uh oh!

VardhanThigle commented Dec 15, 2025

Uh oh!

nmemond commented Dec 16, 2025

Uh oh!

VardhanThigle commented Jan 12, 2026

Uh oh!

VardhanThigle commented Jan 12, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

VardhanThigle Dec 14, 2025 •

edited

Loading