From b757a7b438881b8035d5588836ffc4c00f1f13bb Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Tue, 1 Jul 2025 05:42:59 +0000 Subject: [PATCH 1/3] feat: Add comprehensive documentation for @pgsql/transform package MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Add transform package to main README packages table - Document core use case: single source of truth deparser - Explain multi-version AST transformation (PG13→PG17) - Highlight key limitation: only PG17-parseable SQL supported - Add comprehensive usage examples and architecture overview - Fix packages/transform/README.md with correct package documentation - Include installation, quick start, and integration examples Co-Authored-By: Dan Lynch --- README.md | 123 ++++++++++++++++++++++++++++++++++- packages/transform/README.md | 109 +++++++++++++++++++++++++++++-- 2 files changed, 226 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index f084f1cb..4d85296a 100644 --- a/README.md +++ b/README.md @@ -132,6 +132,7 @@ await deparse(stmt); | [**@pgsql/cli**](./packages/pgsql-cli) | Unified CLI for all PostgreSQL AST operations | • Parse SQL to AST
• Deparse AST to SQL
• Generate TypeScript from protobuf
• Single tool for all operations | | [**@pgsql/utils**](./packages/utils) | Type-safe AST node creation utilities | • Programmatic AST construction
• Runtime Schema
• Seamless integration with types | | [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | • Generate TypeScript interfaces from protobuf
• Create enum mappings and utilities
• AST helper generation | +| [**@pgsql/transform**](./packages/transform) | Multi-version PostgreSQL AST transformer | • Transform ASTs between PostgreSQL versions (13→17)
• Single source of truth deparser pipeline
• Backward compatibility for legacy SQL | ## 🛠️ Development @@ -217,6 +218,126 @@ console.log(await deparse(query)); // SELECT name, email FROM users WHERE age > 18 ``` +## AST Transformation Between PostgreSQL Versions + +The `@pgsql/transform` package enables transformation of PostgreSQL ASTs between different PostgreSQL versions, providing a crucial component for building a single source of truth deparser that works with legacy SQL code. + +### Core Use Case + +The transform package serves as the backbone for maintaining backward compatibility in the deparser pipeline. It allows you to: + +- **Transform legacy ASTs**: Convert ASTs from older PostgreSQL versions (13, 14, 15, 16) to the latest version (17) +- **Build unified deparsers**: Create a single deparser that can handle SQL from multiple PostgreSQL versions +- **Maintain compatibility**: Support legacy codebases while leveraging the latest PostgreSQL parser features + +### Key Limitation + +The transform package only supports ASTs that derive from SQL parseable by PostgreSQL 17. This means: + +- ✅ **Supported**: SQL code from PG13-16 that is still valid in PG17 +- ❌ **Not supported**: Deprecated PG13 syntax that was removed in later versions +- ❌ **Not supported**: SQL that cannot be parsed by the PG17 parser + +This limitation is by design - it ensures that all transformed ASTs can be reliably deparsed using the latest PostgreSQL grammar. + +### Installation + +```bash +npm install @pgsql/transform +``` + +### Basic Usage + +#### Multi-Version Transformer + +```typescript +import { ASTTransformer } from '@pgsql/transform'; + +const transformer = new ASTTransformer(); + +// Transform from any version to PG17 +const pg17Ast = transformer.transformToLatest(pg13Ast, 13); + +// Transform between specific versions +const pg15Ast = transformer.transform(pg13Ast, 13, 15); + +// Convenience methods for common transformations +const pg17FromPg13 = transformer.transform13To17(pg13Ast); +const pg17FromPg14 = transformer.transform14To17(pg14Ast); +``` + +#### Direct Transformers + +For better performance when you know the source and target versions: + +```typescript +import { + PG13ToPG17Transformer, + PG14ToPG17Transformer, + PG15ToPG17Transformer, + PG16ToPG17Transformer +} from '@pgsql/transform'; + +// Direct transformation from PG13 to PG17 +const pg13to17 = new PG13ToPG17Transformer(); +const pg17Ast = pg13to17.transform(pg13Ast); + +// Works with both individual nodes and ParseResult objects +const transformedParseResult = pg13to17.transform(pg13ParseResult); +``` + +#### Integration with Parser and Deparser + +```typescript +import { parse } from '@pgsql/parser'; // Multi-version parser +import { deparse } from 'pgsql-deparser'; +import { PG13ToPG17Transformer } from '@pgsql/transform'; + +// Parse with older PostgreSQL version +const pg13Ast = await parse('SELECT * FROM users', { version: 13 }); + +// Transform to latest version for deparser +const transformer = new PG13ToPG17Transformer(); +const pg17Ast = transformer.transform(pg13Ast); + +// Deparse using latest grammar +const sql = await deparse(pg17Ast); +console.log(sql); // SELECT * FROM users +``` + +### Transformation Chain + +The package supports both incremental and direct transformations: + +**Incremental Chain**: PG13 → PG14 → PG15 → PG16 → PG17 +- Each step handles version-specific changes +- Useful for debugging transformation issues +- Allows intermediate version targeting + +**Direct Transformers**: PG13 → PG17, PG14 → PG17, etc. +- Optimized single-step transformations +- Better performance for common use cases +- Internally chains the incremental transformers + +### Version Support + +| Source Version | Target Versions | Transformer Class | +|----------------|-----------------|-------------------| +| PostgreSQL 13 | 14, 15, 16, 17 | `V13ToV14Transformer`, `PG13ToPG17Transformer` | +| PostgreSQL 14 | 15, 16, 17 | `V14ToV15Transformer`, `PG14ToPG17Transformer` | +| PostgreSQL 15 | 16, 17 | `V15ToV16Transformer`, `PG15ToPG17Transformer` | +| PostgreSQL 16 | 17 | `V16ToV17Transformer`, `PG16ToPG17Transformer` | + +### Architecture + +The transform package is designed to work seamlessly with the broader pgsql-parser ecosystem: + +1. **Parse** legacy SQL using version-specific parsers +2. **Transform** the resulting AST to the latest version +3. **Deparse** using the unified, latest-version deparser + +This architecture enables a single source of truth for SQL generation while maintaining support for legacy codebases. + ## Related * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. @@ -232,4 +353,4 @@ console.log(await deparse(query)); AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. -No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. \ No newline at end of file +No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. diff --git a/packages/transform/README.md b/packages/transform/README.md index 1cadcf96..4d0074f2 100644 --- a/packages/transform/README.md +++ b/packages/transform/README.md @@ -8,13 +8,112 @@ - - - +

-`@pgsql/types` is a TypeScript library providing type definitions for PostgreSQL AST nodes, primarily used in conjunction with [`pgsql-parser`](https://github.com/launchql/pgsql-parser). It offers a comprehensive and type-safe way to interact with the AST nodes generated by PostgreSQL query parsing. +`@pgsql/transform` is a TypeScript library for transforming PostgreSQL ASTs between different PostgreSQL versions. It serves as a crucial component for building a single source of truth deparser that can handle SQL from multiple PostgreSQL versions while maintaining backward compatibility. + +## 🎯 Core Purpose + +The transform package enables you to: + +- **Transform legacy ASTs**: Convert ASTs from PostgreSQL versions 13-16 to version 17 +- **Build unified deparsers**: Create a single deparser pipeline that works with multiple PostgreSQL versions +- **Maintain backward compatibility**: Support legacy codebases while leveraging the latest PostgreSQL features + +## ⚠️ Key Limitation + +This package only supports ASTs derived from SQL that is parseable by PostgreSQL 17. This means: + +- ✅ **Supported**: SQL from PG13-16 that remains valid in PG17 +- ❌ **Not supported**: Deprecated syntax from older versions that was removed +- ❌ **Not supported**: SQL that cannot be parsed by the PG17 parser + +This design ensures all transformed ASTs can be reliably deparsed using the latest PostgreSQL grammar. + +## 📦 Installation + +```bash +npm install @pgsql/transform +``` + +## 🚀 Quick Start + +### Multi-Version Transformer + +```typescript +import { ASTTransformer } from '@pgsql/transform'; + +const transformer = new ASTTransformer(); + +// Transform any version to PG17 +const pg17Ast = transformer.transformToLatest(pg13Ast, 13); + +// Transform between specific versions +const pg15Ast = transformer.transform(pg13Ast, 13, 15); + +// Convenience methods +const result = transformer.transform13To17(pg13Ast); +``` + +### Direct Transformers + +For better performance when you know source and target versions: + +```typescript +import { PG13ToPG17Transformer } from '@pgsql/transform'; + +const transformer = new PG13ToPG17Transformer(); +const pg17Ast = transformer.transform(pg13Ast); +``` + +### Integration Example + +```typescript +import { parse } from '@pgsql/parser'; +import { deparse } from 'pgsql-deparser'; +import { PG13ToPG17Transformer } from '@pgsql/transform'; + +// Parse with older version +const pg13Ast = await parse('SELECT * FROM users', { version: 13 }); + +// Transform to latest +const transformer = new PG13ToPG17Transformer(); +const pg17Ast = transformer.transform(pg13Ast); + +// Deparse with latest grammar +const sql = await deparse(pg17Ast); +``` + +## 🔄 Transformation Chain + +**Incremental**: PG13 → PG14 → PG15 → PG16 → PG17 +- Step-by-step version upgrades +- Useful for debugging transformation issues + +**Direct**: PG13 → PG17, PG14 → PG17, etc. +- Single-step transformations +- Optimized for performance + +## 📋 Supported Transformations + +| From | To | Transformer | +|------|----|-----------| +| PG13 | PG14, PG15, PG16, PG17 | `V13ToV14Transformer`, `PG13ToPG17Transformer` | +| PG14 | PG15, PG16, PG17 | `V14ToV15Transformer`, `PG14ToPG17Transformer` | +| PG15 | PG16, PG17 | `V15ToV16Transformer`, `PG15ToPG17Transformer` | +| PG16 | PG17 | `V16ToV17Transformer`, `PG16ToPG17Transformer` | + +## 🏗️ Architecture + +The transform package fits into the broader pgsql-parser ecosystem: + +1. **Parse** legacy SQL with version-specific parsers +2. **Transform** ASTs to the latest version +3. **Deparse** using the unified, latest-version deparser + +This enables a single source of truth for SQL generation while supporting legacy codebases. ## Related @@ -31,4 +130,4 @@ AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. -No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. \ No newline at end of file +No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. From d16f118abe30ee9cc4b2455ee04ebd2f6de90d52 Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Tue, 1 Jul 2025 06:17:18 +0000 Subject: [PATCH 2/3] refactor: Simplify main README.md transform documentation - Remove detailed transform documentation from main README - Keep only the packages table entry for @pgsql/transform - Detailed documentation remains in packages/transform/README.md - Main README now has concise highlight as requested Co-Authored-By: Dan Lynch --- README.md | 120 ------------------------------------------------------ 1 file changed, 120 deletions(-) diff --git a/README.md b/README.md index 4d85296a..6abbc4fc 100644 --- a/README.md +++ b/README.md @@ -218,126 +218,6 @@ console.log(await deparse(query)); // SELECT name, email FROM users WHERE age > 18 ``` -## AST Transformation Between PostgreSQL Versions - -The `@pgsql/transform` package enables transformation of PostgreSQL ASTs between different PostgreSQL versions, providing a crucial component for building a single source of truth deparser that works with legacy SQL code. - -### Core Use Case - -The transform package serves as the backbone for maintaining backward compatibility in the deparser pipeline. It allows you to: - -- **Transform legacy ASTs**: Convert ASTs from older PostgreSQL versions (13, 14, 15, 16) to the latest version (17) -- **Build unified deparsers**: Create a single deparser that can handle SQL from multiple PostgreSQL versions -- **Maintain compatibility**: Support legacy codebases while leveraging the latest PostgreSQL parser features - -### Key Limitation - -The transform package only supports ASTs that derive from SQL parseable by PostgreSQL 17. This means: - -- ✅ **Supported**: SQL code from PG13-16 that is still valid in PG17 -- ❌ **Not supported**: Deprecated PG13 syntax that was removed in later versions -- ❌ **Not supported**: SQL that cannot be parsed by the PG17 parser - -This limitation is by design - it ensures that all transformed ASTs can be reliably deparsed using the latest PostgreSQL grammar. - -### Installation - -```bash -npm install @pgsql/transform -``` - -### Basic Usage - -#### Multi-Version Transformer - -```typescript -import { ASTTransformer } from '@pgsql/transform'; - -const transformer = new ASTTransformer(); - -// Transform from any version to PG17 -const pg17Ast = transformer.transformToLatest(pg13Ast, 13); - -// Transform between specific versions -const pg15Ast = transformer.transform(pg13Ast, 13, 15); - -// Convenience methods for common transformations -const pg17FromPg13 = transformer.transform13To17(pg13Ast); -const pg17FromPg14 = transformer.transform14To17(pg14Ast); -``` - -#### Direct Transformers - -For better performance when you know the source and target versions: - -```typescript -import { - PG13ToPG17Transformer, - PG14ToPG17Transformer, - PG15ToPG17Transformer, - PG16ToPG17Transformer -} from '@pgsql/transform'; - -// Direct transformation from PG13 to PG17 -const pg13to17 = new PG13ToPG17Transformer(); -const pg17Ast = pg13to17.transform(pg13Ast); - -// Works with both individual nodes and ParseResult objects -const transformedParseResult = pg13to17.transform(pg13ParseResult); -``` - -#### Integration with Parser and Deparser - -```typescript -import { parse } from '@pgsql/parser'; // Multi-version parser -import { deparse } from 'pgsql-deparser'; -import { PG13ToPG17Transformer } from '@pgsql/transform'; - -// Parse with older PostgreSQL version -const pg13Ast = await parse('SELECT * FROM users', { version: 13 }); - -// Transform to latest version for deparser -const transformer = new PG13ToPG17Transformer(); -const pg17Ast = transformer.transform(pg13Ast); - -// Deparse using latest grammar -const sql = await deparse(pg17Ast); -console.log(sql); // SELECT * FROM users -``` - -### Transformation Chain - -The package supports both incremental and direct transformations: - -**Incremental Chain**: PG13 → PG14 → PG15 → PG16 → PG17 -- Each step handles version-specific changes -- Useful for debugging transformation issues -- Allows intermediate version targeting - -**Direct Transformers**: PG13 → PG17, PG14 → PG17, etc. -- Optimized single-step transformations -- Better performance for common use cases -- Internally chains the incremental transformers - -### Version Support - -| Source Version | Target Versions | Transformer Class | -|----------------|-----------------|-------------------| -| PostgreSQL 13 | 14, 15, 16, 17 | `V13ToV14Transformer`, `PG13ToPG17Transformer` | -| PostgreSQL 14 | 15, 16, 17 | `V14ToV15Transformer`, `PG14ToPG17Transformer` | -| PostgreSQL 15 | 16, 17 | `V15ToV16Transformer`, `PG15ToPG17Transformer` | -| PostgreSQL 16 | 17 | `V16ToV17Transformer`, `PG16ToPG17Transformer` | - -### Architecture - -The transform package is designed to work seamlessly with the broader pgsql-parser ecosystem: - -1. **Parse** legacy SQL using version-specific parsers -2. **Transform** the resulting AST to the latest version -3. **Deparse** using the unified, latest-version deparser - -This architecture enables a single source of truth for SQL generation while maintaining support for legacy codebases. - ## Related * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. From 35aa7c924d087334e9d4a2ce2569fe04da89bf01 Mon Sep 17 00:00:00 2001 From: Dan Lynch Date: Mon, 30 Jun 2025 23:22:34 -0700 Subject: [PATCH 3/3] readme --- packages/transform/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/packages/transform/README.md b/packages/transform/README.md index 4d0074f2..31de6f2e 100644 --- a/packages/transform/README.md +++ b/packages/transform/README.md @@ -14,7 +14,7 @@ `@pgsql/transform` is a TypeScript library for transforming PostgreSQL ASTs between different PostgreSQL versions. It serves as a crucial component for building a single source of truth deparser that can handle SQL from multiple PostgreSQL versions while maintaining backward compatibility. -## 🎯 Core Purpose +## Transforming ASTs Between PG Versions The transform package enables you to: @@ -22,7 +22,7 @@ The transform package enables you to: - **Build unified deparsers**: Create a single deparser pipeline that works with multiple PostgreSQL versions - **Maintain backward compatibility**: Support legacy codebases while leveraging the latest PostgreSQL features -## ⚠️ Key Limitation +## Key Limitation This package only supports ASTs derived from SQL that is parseable by PostgreSQL 17. This means: @@ -32,7 +32,7 @@ This package only supports ASTs derived from SQL that is parseable by PostgreSQL This design ensures all transformed ASTs can be reliably deparsed using the latest PostgreSQL grammar. -## 📦 Installation +## Installation ```bash npm install @pgsql/transform