Skip to content

Conversation

@narrowizard
Copy link
Collaborator

Problem

#8676

Root cause:

  • Each column rename operation called HasColumn() twice (checking old and new column names)
  • Each HasColumn() call queries information_schema for all column metadata
  • Each RenameColumn() triggered a deferred SELECT to clear PostgreSQL's query plan cache
  • Total: ~270 information_schema queries + 135 cache clearing SELECTs

Solution

Optimized the migration by:

  1. Cache column metadata: Fetch all column names once per table using GetColumns() instead of calling HasColumn() for each column
  2. Batch cache clearing: Clear PostgreSQL cached plan once per table instead of after each rename
  3. Handle dialect differences:
    - PostgreSQL: Execute separate ALTER TABLE ... RENAME COLUMN statements (required by syntax)
    - MySQL: Use single ALTER TABLE with multiple CHANGE COLUMN clauses

Performance Impact

Metric Before After Improvement
Execution time 22s 4s 5.5x faster
information_schema queries ~270 3 90x reduction
Cache clearing SELECTs 135 3 45x reduction

Testing

Performance comparison

Database Before (s) After (s)
PostgreSQL image image
MySQL image image

Schema validation

mysql

image image image

Postgres

image image image

…data

Reduce PostgreSQL migration time from 22s to 4s by:

1. Cache column metadata per table instead of querying information_schema
   for each column check (270 queries → 3 queries)
2. Batch PostgreSQL cache clearing (135 SELECTs → 3 SELECTs)
3. Handle PostgreSQL and MySQL syntax differences appropriately

The key optimization is fetching all column names once via GetColumns()
and using a map for O(1) lookups instead of calling HasColumn() twice
per column rename operation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <[email protected]>
@dosubot dosubot bot added size:L This PR changes 100-499 lines, ignoring generated files. component/plugins This issue or PR relates to plugins improvement priority/high This issue is very important labels Jan 8, 2026
@narrowizard narrowizard requested review from d4x1 and warren830 January 8, 2026 03:29
@narrowizard narrowizard self-assigned this Jan 8, 2026
@d4x1 d4x1 merged commit 529a71b into main Jan 8, 2026
14 checks passed
@d4x1 d4x1 deleted the fix-tapd-migration branch January 8, 2026 04:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

component/plugins This issue or PR relates to plugins improvement priority/high This issue is very important size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants