Skip to content

Add source ddl classes#2525

Merged
Deep1998 merged 3 commits intoGoogleCloudPlatform:mainfrom
Deep1998:rev-cp-1
Jul 14, 2025
Merged

Add source ddl classes#2525
Deep1998 merged 3 commits intoGoogleCloudPlatform:mainfrom
Deep1998:rev-cp-1

Conversation

@Deep1998
Copy link
Copy Markdown
Contributor

@Deep1998 Deep1998 commented Jul 8, 2025

Create classes similar to spanner ddl but for source databases which allows storing the schema for migrations. This would be useful to remove dependency on session file for identifying source types.

@codecov
Copy link
Copy Markdown

codecov bot commented Jul 8, 2025

Codecov Report

Attention: Patch coverage is 89.54248% with 16 lines in your changes missing coverage. Please review.

Project coverage is 54.71%. Comparing base (ece5482) to head (28752ba).
Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
...anner/sourceddl/MySqlInformationSchemaScanner.java 86.48% 2 Missing and 3 partials ⚠️
...oud/teleport/v2/spanner/sourceddl/SourceTable.java 54.54% 5 Missing ⚠️
...r/sourceddl/CassandraInformationSchemaScanner.java 92.15% 2 Missing and 2 partials ⚠️
...m/google/cloud/teleport/v2/spanner/ddl/Column.java 0.00% 1 Missing ⚠️
...ud/teleport/v2/spanner/sourceddl/SourceSchema.java 75.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2525      +/-   ##
============================================
+ Coverage     49.59%   54.71%   +5.12%     
+ Complexity     4809     1855    -2954     
============================================
  Files           942      452     -490     
  Lines         57751    25142   -32609     
  Branches       6241     2594    -3647     
============================================
- Hits          28640    13757   -14883     
+ Misses        27063    10560   -16503     
+ Partials       2048      825    -1223     
Components Coverage Δ
spanner-templates 70.98% <89.54%> (+1.11%) ⬆️
spanner-import-export ∅ <ø> (∅)
spanner-live-forward-migration 79.12% <89.54%> (+0.33%) ⬆️
spanner-live-reverse-replication 77.60% <89.54%> (+0.32%) ⬆️
spanner-bulk-migration 87.92% <89.54%> (+0.03%) ⬆️
Files with missing lines Coverage Δ
.../teleport/v2/spanner/migrations/schema/Schema.java 90.47% <100.00%> (+0.31%) ⬆️
...rceddl/AbstractSourceInformationSchemaScanner.java 100.00% <100.00%> (ø)
...ud/teleport/v2/spanner/sourceddl/SourceColumn.java 100.00% <100.00%> (ø)
...eport/v2/spanner/sourceddl/SourceDatabaseType.java 100.00% <100.00%> (ø)
...m/google/cloud/teleport/v2/spanner/ddl/Column.java 57.98% <0.00%> (-0.35%) ⬇️
...ud/teleport/v2/spanner/sourceddl/SourceSchema.java 75.00% <75.00%> (ø)
...r/sourceddl/CassandraInformationSchemaScanner.java 92.15% <92.15%> (ø)
...anner/sourceddl/MySqlInformationSchemaScanner.java 86.48% <86.48%> (ø)
...oud/teleport/v2/spanner/sourceddl/SourceTable.java 54.54% <54.54%> (ø)

... and 515 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@Deep1998 Deep1998 marked this pull request as ready for review July 8, 2025 10:14
@Deep1998 Deep1998 requested a review from a team as a code owner July 8, 2025 10:14
@Deep1998 Deep1998 added the improvement Making existing code better label Jul 8, 2025
Copy link
Copy Markdown
Contributor

@bharadwaj-aditya bharadwaj-aditya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of these abstract classes seem like shell implementations. They can be interfaces as well to simplify the structure.

Please add some test cases with edge cases , null handling, missing fields etc.

@Deep1998
Copy link
Copy Markdown
Contributor Author

Some of these abstract classes seem like shell implementations. They can be interfaces as well to simplify the structure.

Please add some test cases with edge cases , null handling, missing fields etc.

Thanks, added more test cases and removed the AbstractSourceInformationSchemaScanner for simplicity. The data model classes (SourceSchema, SourceTable, SourceColumn) are abstract classes because they use AutoValue, which allows us to avoid a lot of boiler plate code, so those I have kept as is.

Copy link
Copy Markdown
Contributor

@bharadwaj-aditya bharadwaj-aditya left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Deep1998 Deep1998 merged commit 6ab22b3 into GoogleCloudPlatform:main Jul 14, 2025
16 of 17 checks passed
MnkyGns pushed a commit to MnkyGns/DataflowTemplates that referenced this pull request Feb 12, 2026
* Add source ddl classes

* Update soruceddl with columna and table getter

* Resolve comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Making existing code better size/XXL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants