diff --git a/CLI.md b/CLI.md new file mode 100644 index 00000000..49943b14 --- /dev/null +++ b/CLI.md @@ -0,0 +1,195 @@ +# CLI Consolidation Plan + +## Overview + +Consolidate all CLI functionality from `@launchql/proto-cli`, `pgsql-parser`, and add deparser capabilities into a unified `@pgsql/cli` package. + +## Current State + +### 1. `@launchql/proto-cli` (packages/cli) +- **Binary**: `pg-proto-parser` +- **Commands**: + - `codegen`: Generate TypeScript from protobuf files + - `protogen`: Download and process proto files + - `runtime-schema`: Generate runtime schema for AST nodes +- **Dependencies**: Uses `inquirerer` for interactive CLI, `minimist` for arg parsing + +### 2. `pgsql-parser` (packages/parser) +- **Binary**: `pgsql-parser` +- **Functionality**: Parse SQL files and output JSON AST +- **Options**: + - `--pl`: Parse PL/pgSQL functions only +- **Dependencies**: Uses `minimist` directly in cli.js + +### 3. `deparser` (packages/deparser) +- **No CLI**: Currently no CLI interface +- **Functionality**: Convert AST back to SQL + +## New CLI Structure: `@pgsql/cli` + +### Package Changes + +1. **Rename**: `@launchql/proto-cli` β†’ `@pgsql/cli` +2. **Binary**: `pgsql` (shorter, cleaner) +3. **Remove minimist** from parser and deparser packages +4. **Centralize** all CLI logic in the cli package + +### Command Structure + +``` +pgsql [options] + +Commands: + parse Parse SQL to AST + deparse Convert AST to SQL + proto-gen Generate TypeScript from protobuf (formerly codegen) + proto-fetch Download and process proto files (formerly protogen) + runtime-schema Generate runtime schema for AST nodes + +Global Options: + -h, --help Show help + -v, --version Show version +``` + +### Command Details + +#### 1. `pgsql parse` +```bash +pgsql parse [options] + +Options: + -o, --output Output to file instead of stdout + -f, --format Output format: json, pretty (default: pretty) + --pl Parse as PL/pgSQL function only + --clean Clean the AST tree (remove location info) + -h, --help Show help +``` + +#### 2. `pgsql deparse` +```bash +pgsql deparse [options] + +Options: + -o, --output Output to file instead of stdout + -i, --input Input JSON file (or use stdin) + --format SQL formatting options + -h, --help Show help +``` + +#### 3. `pgsql proto-gen` +```bash +pgsql proto-gen --inFile --outDir [options] + +Options: + --inFile Input .proto file (required) + --outDir Output directory (required) + --enums Generate TypeScript enums + --enums-json Generate JSON enum mappings + --types Generate TypeScript interfaces + --utils Generate utility functions + --ast-helpers Generate AST helper methods + --wrapped-helpers Generate wrapped AST helpers + --optional Make all fields optional + --keep-case Keep original field casing + --remove-undefined Remove UNDEFINED enum at position 0 + -h, --help Show help +``` + +#### 4. `pgsql proto-fetch` +```bash +pgsql proto-fetch [options] + +Options: + --url Proto file URL to download + --inFile Where to save the proto file + --outFile Generated JS output file + --replace-pkg Original package name to replace + --with-pkg New package name + -h, --help Show help +``` + +#### 5. `pgsql runtime-schema` +```bash +pgsql runtime-schema --inFile --outDir [options] + +Options: + --inFile Input .proto file (required) + --outDir Output directory (required) + --format Output format: json, typescript (default: json) + --filename Output filename (default: runtime-schema) + -h, --help Show help +``` + +## Implementation Steps + +### Phase 1: Setup New Package +1. Copy `packages/cli` to new location (keep history) +2. Update package.json: + - Name: `@pgsql/cli` + - Binary: `pgsql` + - Update dependencies +3. Remove `minimist` dependency from parser/deparser packages + +### Phase 2: Implement Core Commands +1. Create new command structure without `inquirerer` for non-interactive mode +2. Implement `parse` command: + - Move logic from parser/src/cli.js + - Add output options + - Add format options +3. Implement `deparse` command: + - Import deparser functionality + - Add file I/O handling + - Add formatting options + +### Phase 3: Refactor Proto Commands +1. Rename `codegen` β†’ `proto-gen` +2. Rename `protogen` β†’ `proto-fetch` +3. Update command options to use consistent naming +4. Add new options for wrapped helpers + +### Phase 4: Improve Help System +1. Create detailed help for each command +2. Add examples to help text +3. Create man page style documentation + +### Phase 5: Testing & Documentation +1. Add comprehensive tests for all commands +2. Update README with new command structure +3. Create migration guide from old CLIs +4. Add shell completion scripts + +## Benefits + +1. **Single Entry Point**: One CLI tool for all PostgreSQL AST operations +2. **Consistent Interface**: Unified command structure and options +3. **Better Discoverability**: Clear command hierarchy with good help +4. **Reduced Dependencies**: Remove minimist from individual packages +5. **Extensibility**: Easy to add new commands in the future + +## Migration Guide + +### For `pgsql-parser` users: +```bash +# Old +pgsql-parser file.sql + +# New +pgsql parse file.sql +``` + +### For `pg-proto-parser` users: +```bash +# Old +pg-proto-parser codegen --inFile pg_query.proto --outDir out + +# New +pgsql proto-gen --inFile pg_query.proto --outDir out +``` + +## Future Enhancements + +1. **Interactive Mode**: Keep inquirerer for interactive command selection +2. **Config Files**: Support `.pgsqlrc` configuration files +3. **Plugins**: Allow extending with custom commands +4. **Watch Mode**: Auto-regenerate on file changes +5. **Batch Processing**: Process multiple files at once \ No newline at end of file diff --git a/README.md b/README.md index 9fa490cc..fb117eeb 100644 --- a/README.md +++ b/README.md @@ -4,6 +4,7 @@

+

@@ -11,97 +12,70 @@ -

-The real PostgreSQL parser for Node.js, `pgsql-parser` provides symmetric parsing and deparsing of SQL statements using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It allows you to parse SQL queries into AST and modify or reconstruct SQL queries from the AST. - -## Installation +## PostgreSQL AST Tools -```sh -npm install pgsql-parser -``` -## Key Features +A comprehensive monorepo for PostgreSQL Abstract Syntax Tree (AST) parsing, manipulation, and code generation. This collection of packages provides everything you need to work with PostgreSQL at the AST level, from parsing SQL queries to generating type-safe TypeScript definitions. -- **True PostgreSQL Parsing:** Utilizes the real PostgreSQL source code for accurate parsing. -- **Symmetric Parsing and Deparsing:** Convert SQL to AST and back, enabling query manipulation. -- **AST Manipulation:** Easily modify parts of a SQL statement through the AST. -- **WebAssembly Powered:** Cross-platform compatibility without native dependencies. +## πŸ“¦ Packages Overview -## API +| Package | Description | Key Features | +|---------|-------------|--------------| +| [**pgsql-parser**](./packages/parser) | The real PostgreSQL parser for Node.js | β€’ Uses actual PostgreSQL C parser via WebAssembly
β€’ Symmetric parsing and deparsing
β€’ Battle-tested with 23,000+ SQL statements | +| [**pgsql-deparser**](./packages/deparser) | Lightning-fast SQL generation from AST | β€’ Pure TypeScript, zero dependencies
β€’ No WebAssembly overhead
β€’ Perfect for AST-to-SQL conversion only | +| [**@pgsql/cli**](./packages/pgsql-cli) | Unified CLI for all PostgreSQL AST operations | β€’ Parse SQL to AST
β€’ Deparse AST to SQL
β€’ Generate TypeScript from protobuf
β€’ Single tool for all operations | +| [**@pgsql/utils**](./packages/utils) | Type-safe AST node creation utilities | β€’ Programmatic AST construction
β€’ Enum value conversions
β€’ Seamless integration with types | +| [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | β€’ Generate TypeScript interfaces from protobuf
β€’ Create enum mappings and utilities
β€’ AST helper generation | -The package exports both async and sync methods. Async methods handle initialization automatically, while sync methods require explicit initialization. +## πŸš€ Quick Start -⚠️ We recommend using `@pgsql/deparser` instead of `deparse` from `pgsql-parser`. The deparser package is more complete, supports sub-expressions, and doesn't require the WebAssembly module, making it lighter and more flexible for most use cases. It will soon be deprecated, in a minor version bump. +### Installation -### Async Methods (Recommended) - -```typescript -import { parse, deparse, parseFunction } from 'pgsql-parser'; - -// Parse SQL to AST -const stmts = await parse('SELECT * FROM test_table'); - -// Deparse AST back to SQL -const sql = await deparse(stmts); - -// Parse PL/pgSQL functions -const funcAst = await parseFunction(` - CREATE FUNCTION get_count() RETURNS integer AS $$ - BEGIN - RETURN (SELECT COUNT(*) FROM users); - END; - $$ LANGUAGE plpgsql; -`); -``` +Choose the packages you need: -### Sync Methods +```bash +# For parsing SQL to AST and back +npm install pgsql-parser -Sync methods require explicit initialization using `loadModule()`: +# For only converting AST to SQL (lighter weight) +npm install pgsql-deparser -```typescript -import { loadModule, parseSync, deparseSync } from 'pgsql-parser'; +# For the unified CLI tool +npm install -g @pgsql/cli -// Initialize first (required for sync methods) -await loadModule(); +# For programmatic AST construction +npm install @pgsql/utils -// Now safe to use sync methods -const stmts = parseSync('SELECT * FROM test_table'); -const sql = deparseSync(stmts); +# For protobuf parsing and code generation +npm install pg-proto-parser ``` -**Note:** We recommend using async methods as they handle initialization automatically. Use sync methods only when necessary, and always call `loadModule()` first. - -## Parser Example - -Rewrite part of a SQL query: - -```js -import { parse, deparse } from 'pgsql-parser'; - -const stmts = await parse('SELECT * FROM test_table'); - -// Assuming the structure of stmts is known and matches the expected type -stmts[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table'; +### Basic Usage -console.log(await deparse(stmts)); +#### Parse SQL to AST +```typescript +import { parse } from 'pgsql-parser'; -// SELECT * FROM "another_table" +const ast = await parse('SELECT * FROM users WHERE id = 1'); +console.log(JSON.stringify(ast, null, 2)); +// {"version":170004,"stmts":[{"stmt":{"SelectStmt":{"targetList":[{"ResTarget": ... ,"op":"SETOP_NONE"}}}]} ``` -## Deparser Example - -The `pgsql-deparser` module serializes ASTs to SQL in pure TypeScript, avoiding the full parser's native dependencies. It's useful when only SQL string conversion from ASTs is needed, and is written in pure TypeScript for easy cross-environment deployment. +#### Convert AST back to SQL +```typescript +import { deparse } from 'pgsql-deparser'; -Here's how you can use the deparser in your TypeScript code, using [`@pgsql/utils`](https://github.com/launchql/pgsql-parser/tree/main/packages/utils) to create an AST for `deparse`: +const sql = deparse(ast); +console.log(sql); // SELECT * FROM users WHERE id = 1 +``` -```ts +#### Build AST Programmatically +```typescript import * as t from '@pgsql/utils'; import { RangeVar, SelectStmt } from '@pgsql/types'; -import { deparseSync as deparse } from 'pgsql-deparser'; -// This could have been obtained from any JSON or AST, not necessarily @pgsql/utils const stmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ t.nodes.resTarget({ @@ -120,66 +94,133 @@ const stmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ limitOption: 'LIMIT_OPTION_DEFAULT', op: 'SETOP_NONE' }); +``` -// Modify the AST if needed -(stmt.SelectStmt.fromClause[0] as {RangeVar: RangeVar}).RangeVar.relname = 'another_table'; +#### Use the CLI +```bash +# Parse SQL file +pgsql parse query.sql -// Deparse the modified AST back to a SQL string -console.log(deparse(stmt)); +# Convert AST to SQL +pgsql deparse ast.json -// Output: SELECT * FROM another_table +# Generate TypeScript from protobuf +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums ``` -## CLI +### Package Relationships + +- **pgsql-parser** provides full parsing and deparsing capabilities using the actual PostgreSQL parser +- **pgsql-deparser** offers a lightweight alternative for just converting AST to SQL +- **@pgsql/utils** helps construct ASTs programmatically with type safety +- **pg-proto-parser** generates TypeScript definitions from PostgreSQL protobuf files +- **@pgsql/cli** unifies all functionality into a single command-line tool +## πŸ› οΈ Development + +This project uses Yarn workspaces and Lerna for monorepo management. + +### Setup +```bash +# Install dependencies +yarn install + +# Build all packages +yarn build + +# Run tests +yarn test ``` -npm install -g pgsql-parser + +### Building Individual Packages +```bash +cd packages/parser +npm run build ``` -### usage +## Documentation -```sh -pgsql-parser -``` +Each package has its own detailed README: + +- [`pgsql-parser`](./packages/parser/README.md) +- [`pgsql-deparser`](./packages/deparser/README.md) +- [`@pgsql/cli`](./packages/pgsql-cli/README.md) +- [`@pgsql/utils`](./packages/utils/README.md) +- [`pg-proto-parser`](./packages/proto-parser/README.md) -## Versions +## Examples -As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates. +### Transform a Query -Our latest is built with `17-latest` branch from libpg_query +```typescript +import { parse } from 'pgsql-parser'; +import { deparse } from 'pgsql-deparser'; -| PostgreSQL Major Version | libpg_query | Status | npm tag | -|--------------------------|-------------|---------------------|---------| -| 17 | 17-latest | Active Development | `latest` | -| 16 | (n/a) | Not supported | -| 15 | (n/a) | Not supported | -| 14 | (n/a) | Not supported | -| 13 | 13-latest | Only Critical Fixes | `13.16.0` | -| 12 | (n/a) | Not supported | -| 11 | (n/a) | Not supported | -| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) | +// Parse the original query +const ast = await parse('SELECT * FROM users WHERE active = true'); +// Modify the table name +ast[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'customers'; -## Related +// Generate the modified SQL +const newSql = deparse(ast); +console.log(newSql); // SELECT * FROM customers WHERE active = TRUE +``` -* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. -* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. -* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser` -* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes. -* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. -* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. -* [pg-proto-parser](https://github.com/launchql/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums. -* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries. +### Build a Query Programmatically -## Credits +```typescript +import ast from '@pgsql/utils'; +import { deparse as deparseSync } from 'pgsql-deparser'; + +const query: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ + targetList: [ + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'name' })] + }) + }), + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'email' })] + }) + }) + ], + fromClause: [ + t.nodes.rangeVar({ + relname: 'users', + inh: true, + relpersistence: 'p' + }) + ], + whereClause: t.nodes.aExpr({ + kind: 'AEXPR_OP', + name: [t.nodes.string({ sval: '>' })], + lexpr: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'age' })] + }), + rexpr: t.nodes.aConst({ + ival: t.ast.integer({ ival: 18 }) + }) + }), + limitOption: 'LIMIT_OPTION_DEFAULT', + op: 'SETOP_NONE' +}); -Thanks [@lfittl](https://github.com/lfittl) for building the core `libpg_query` suite: +console.log(deparse(query)); +// SELECT name, email FROM users WHERE age > 18 +``` -* [libpg_query](https://github.com/pganalyze/libpg_query) -* [pg_query](https://github.com/lfittl/pg_query) -* [pg_query.go](https://github.com/lfittl/pg_query.go) +## Related -Thanks to [@zhm](https://github.com/zhm) for the [OG Parser](https://github.com/zhm/pg-query-parser/blob/master/LICENSE.md) that started it all. +* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. +* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. +* [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. +* [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. +* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. +* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums. +* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries. ## Disclaimer diff --git a/packages/cli/README.md b/packages/cli/README.md deleted file mode 100644 index 938027ea..00000000 --- a/packages/cli/README.md +++ /dev/null @@ -1,136 +0,0 @@ -# @launchql/proto-cli - -

- -

- -

- - - - -

- -`@launchql/proto-cli` is a command-line interface to facilitate the parsing of [pganalyze/libpg_query](https://github.com/pganalyze/libpg_query) PostgreSQL query protobufs into structured TypeScript definitions and utilities. Designed to work with [launchql/pgsql-parser](https://github.com/launchql/pgsql-parser) for maintainable upgrades. - -## Installation - -```bash -npm install -g @launchql/proto-cli -``` - -## Getting Started - -First, download the latest protobuf definition from `libpg_query`: - -```bash -wget https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto -``` - - -Run the CLI to parse the protobuf file and generate TypeScript outputs: - -```bash -pg-proto-parser --inFile pg_query.proto --outDir out -``` - - -## Features - -- Converts PostgreSQL protobuf files to TypeScript definitions. -- Automatically generates enums, interfaces, and utility functions from protobufs. -- Supports custom output directories for generated files. - - -## Usage - -After installation, you can run the `pg-proto-parser` command for code generation as follows: - -### codegen - -```bash -pg-proto-parser codegen --inFile --outDir \ - [--enums] [--enumsJSON] [--typeUnion] \ - [--header] [--types] [--utils] \ - [--case] [--optional] [--removeUndefined] -``` - -#### Options for codegen - -| Option | Description | Default Value | -|---------------------|---------------------------------------------------------------------------------------------------------------------------|---------------| -| `--inFile` | Path to the `.proto` file to be parsed. | *Required* | -| `--outDir` | Directory to save the generated TypeScript files. | *Required* | -| `--enums` | Generate TypeScript enum types for PostgreSQL enums. | `false` | -| `--enumsJSON` | Generate JSON files mapping enum names to values. | `false` | -| `--typeUnion` | Generate TypeScript unions from enum types. | `false` | -| `--header` | Include a header in the generated TypeScript files to aid in integration. | `false` | -| `--types` | Generate TypeScript interfaces for protobuf messages. | `false` | -| `--utils` | Generate TypeScript utility functions for enums. | `false` | -| `--case` | Keep field casing as defined in the protobuf file. If false, fields will be converted to camelCase. | `false` | -| `--optional` | Generate TypeScript interfaces with optional fields. | `false` | -| `--removeUndefined` | Remove the 'UNDEFINED' enum entry if it exists. | `false` | -| `--help`, `-h` | Show this help message and exit. | | -| `--version`, `-v` | Show the version number and exit. | | - -To explicitly set a boolean option to false, prepend the option with `--no-`. For example, to disable JSON enum mapping, use `--no-enumsJSON`. - -### protogen - -You can also generate code using the `protogen` command: - -```bash -pg-proto-parser protogen --protoUrl \ - --inFile \ - --outFile \ - --originalPackageName \ - --newPackageName -``` - -#### Options for protogen - -| Option | Description | Default Value | -|-------------------------|-------------------------------------------------------------------------------------|---------------| -| `--protoUrl` | Full URL to download the proto file (optional). | | -| `--inFile` | Path where the proto file will be saved or path to an existing proto file. | *Required* | -| `--outFile` | Path where the generated JavaScript file will be saved. | *Required* | -| `--originalPackageName` | Original package name to be replaced in the JS file. | protobufjs/minimal | -| `--newPackageName` | New package name to replace in the JS file. | @launchql/protobufjs/minimal | -| `--help`, `-h` | Show this help message. | | -| `--version`, `-v` | Show the version number. | | - -### runtime-schema - -Generate runtime schema for PostgreSQL AST nodes: - -```bash -pg-proto-parser runtime-schema --inFile --outDir \ - [--format ] [--filename ] -``` - -#### Options for runtime-schema - -| Option | Description | Default Value | -|---------------------|-------------------------------------------------------------------------------------|---------------| -| `--inFile` | Path to the `.proto` file to be parsed. | *Required* | -| `--outDir` | Directory to save the generated runtime schema files. | *Required* | -| `--format` | Output format for runtime schema ('json' or 'typescript'). | `json` | -| `--filename` | Filename for the runtime schema file (without extension). | `runtime-schema` | -| `--help`, `-h` | Show this help message and exit. | | -| `--version`, `-v` | Show the version number and exit. | | - - -## Related - -* [launchql/pgsql-parser](https://github.com/launchql/pgsql-parser): A node.js PostgreSQL parser/deparser that interprets and converts PostgresSQL syntax. -* [launchql/libpg-query-node](https://github.com/launchql/libpg-query-node): Node.js bindings for the libpg_query library, allowing parsing of PostgreSQL queries into parse trees. -* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes. -* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. -* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. - -## Disclaimer - -AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED β€œAS IS”, AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. - -No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. - diff --git a/packages/cli/src/commands/codegen/cli.ts b/packages/cli/src/commands/codegen/cli.ts deleted file mode 100644 index 4ddff205..00000000 --- a/packages/cli/src/commands/codegen/cli.ts +++ /dev/null @@ -1,70 +0,0 @@ -import { readAndParsePackageJson } from '../../package'; -import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; -import o from 'nested-obj'; - -export const help = (): void => { - console.log(` - Usage: - - pg-proto-parser codegen --inFile --outDir - [--enums] [--enumsJSON] [--typeUnion] - [--header] [--types] [--utils] - [--case] [--optional] [--removeUndefined] - - Options: - - --help, -h Show this help message and exit. - --inFile Path to the .proto file to be parsed. - --outDir Directory to save the generated TypeScript files. - --enums Generate TypeScript enum types for PostgreSQL enums. - --enumsJSON Generate JSON files mapping enum names to values. - --typeUnion Generate TypeScript unions from enum types. - --header Include a header in the generated TypeScript files. - --types Generate TypeScript interfaces for protobuf messages. - --utils Generate TypeScript utility functions for enums. - --case Keep field casing as defined in the protobuf file. - --optional Generate TypeScript interfaces with optional fields. - --removeUndefined Remove the 'UNDEFINED' enum entry if it exists. - --version, -v Show the version number and exit. - `); -} - -export default async (argv: any) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - if (argv.version || argv.v) { - console.log(`Version: ${readAndParsePackageJson().version}`); - process.exit(0); - } - - if (!argv.inFile || !argv.outDir) { - console.log('Input Error: inFile and outDir are required!'); - help(); - process.exit(1); - } - - const options: PgProtoParserOptions = getOptionsWithDefaults({ - outDir: argv.outDir - }); - - // Setting nested options using objectPath - o.set(options, 'enums.enabled', argv.enums); - o.set(options, 'enums.enumsAsTypeUnion', argv.typeUnion); - o.set(options, 'enums.json.enabled', argv.enumsJSON); - o.set(options, 'enums.removeUndefinedAt0', argv.removeUndefined); - o.set(options, 'includeHeader', argv.header); - o.set(options, 'parser.keepCase', argv.case); - o.set(options, 'types.enabled', argv.types); - o.set(options, 'types.optionalFields', argv.optional); - o.set(options, 'utils.astHelpers.enabled', argv.utils); - - const parser = new PgProtoParser(argv.inFile, options); - // Generate TypeScript and JSON files - await parser.write(); - - return argv; -}; diff --git a/packages/cli/src/commands/codegen/index.ts b/packages/cli/src/commands/codegen/index.ts deleted file mode 100644 index 3295e773..00000000 --- a/packages/cli/src/commands/codegen/index.ts +++ /dev/null @@ -1,90 +0,0 @@ -import { CLIOptions, Inquirerer } from 'inquirerer' -import { ParsedArgs } from 'minimist'; -import cli, { help } from './cli'; -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - argv = await prompter.prompt(argv, [ - { - type: 'text', - name: 'inFile', - message: 'provide inFile (./path/to/proto.proto):' - }, - { - type: 'text', - name: 'outDir', - message: 'provide outDir (./outputDir):' - }, - { - type: 'confirm', - name: 'enums', - message: 'Enable enums?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'typeUnion', - message: 'Use enums as type union?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'enumsJSON', - message: 'Enable JSON for enums?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'removeUndefined', - message: 'Remove undefined at index 0?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'header', - message: 'Include header in output?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'case', - message: 'Keep case in parser?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'types', - message: 'Enable types?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'optional', - message: 'Are optional fields enabled?', - default: false, - useDefault: true - }, - { - type: 'confirm', - name: 'utils', - message: 'Enable AST helpers in utils?', - default: false, - useDefault: true - } - ]); - - argv = await cli(argv); - - return argv; -}; \ No newline at end of file diff --git a/packages/cli/src/commands/index.ts b/packages/cli/src/commands/index.ts deleted file mode 100644 index 47bdfa50..00000000 --- a/packages/cli/src/commands/index.ts +++ /dev/null @@ -1,72 +0,0 @@ -import { CLIOptions, Inquirerer, Question } from 'inquirerer' -import { ParsedArgs } from 'minimist'; - -import { commands as codegen } from './codegen'; -import { commands as protogen } from './protogen'; -import { commands as runtimeSchema } from './runtime-schema'; - -import { help as codegenHelp } from './codegen/cli'; -import { help as protogenHelp } from './protogen/cli'; -import { help as runtimeSchemaHelp } from './runtime-schema/cli'; - -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - let command; - - if (argv._.length > 0) { - command = argv._.shift(); - } - - if (command) { - argv.command = command; - } - - if (argv.help || argv.h) { - codegenHelp(); - protogenHelp(); - runtimeSchemaHelp(); - process.exit(0); - } - - - const questions: Question[] = [ - { - type: 'autocomplete', - name: 'command', - message: 'choose a command', - options: [ - 'protogen', - 'codegen', - 'runtime-schema' - ] - } - ]; - - ({ command } = await prompter.prompt(argv, questions)); - - // recursive... - delete argv.command; - - // @ts-ignore - prompter.exit = () => {}; - - switch (command) { - case 'protogen': - argv = await protogen(argv, prompter, _options); - break; - - case 'codegen': - argv = await codegen(argv, prompter, _options); - break; - - case 'runtime-schema': - argv = await runtimeSchema(argv, prompter, _options); - break; - - default: - console.log(`No recognized command provided or no command given: ${command}`); - break; - } - - return argv; - -}; diff --git a/packages/cli/src/commands/protogen/index.ts b/packages/cli/src/commands/protogen/index.ts deleted file mode 100644 index ab36af52..00000000 --- a/packages/cli/src/commands/protogen/index.ts +++ /dev/null @@ -1,55 +0,0 @@ -import { CLIOptions, Inquirerer } from 'inquirerer'; -import { ParsedArgs } from 'minimist'; -import cli, { CommandOptions, help } from './cli'; -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - argv = await prompter.prompt(argv, [ - // use false, TODO: add optional flag on questions allowSkip: boolean - // @ts-ignore - { - type: 'text', - name: 'protoUrl', - message: 'Enter the URL to the proto file (optional):', - required: false, - default: false, - useDefault: true - }, - { - type: 'text', - name: 'inFile', - message: 'Provide the path where the proto file will be saved or the path to an existing proto file:', - required: true - }, - { - type: 'text', - name: 'outFile', - message: 'Provide the path where the generated JavaScript file will be saved:', - required: true - }, - { - type: 'text', - name: 'originalPackageName', - message: 'Enter the original package name to be replaced in the JS file:', - required: true, - default: 'protobufjs/minimal', - useDefault: true - }, - { - type: 'text', - name: 'newPackageName', - message: 'Enter the new package name to replace in the JS file:', - required: true, - default: '@launchql/protobufjs/minimal', - useDefault: true - } - ]); - - argv = await cli(argv as CommandOptions); - - return argv; -}; diff --git a/packages/cli/src/commands/runtime-schema/cli.ts b/packages/cli/src/commands/runtime-schema/cli.ts deleted file mode 100644 index 227e2149..00000000 --- a/packages/cli/src/commands/runtime-schema/cli.ts +++ /dev/null @@ -1,62 +0,0 @@ -import { readAndParsePackageJson } from '../../package'; -import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; -import o from 'nested-obj'; - -export const help = (): void => { - console.log(` - Usage: - - pg-proto-parser runtime-schema --inFile --outDir - [--format ] [--filename ] - - Options: - - --help, -h Show this help message and exit. - --inFile Path to the .proto file to be parsed. - --outDir Directory to save the generated runtime schema files. - --format Output format for runtime schema ('json' or 'typescript'). - --filename Filename for the runtime schema file (without extension). - --version, -v Show the version number and exit. - `); -} - -export default async (argv: any) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - if (argv.version || argv.v) { - console.log(`Version: ${readAndParsePackageJson().version}`); - process.exit(0); - } - - if (!argv.inFile || !argv.outDir) { - console.log('Input Error: inFile and outDir are required!'); - help(); - process.exit(1); - } - - const options: PgProtoParserOptions = getOptionsWithDefaults({ - outDir: argv.outDir - }); - - o.set(options, 'runtimeSchema.enabled', true); - o.set(options, 'runtimeSchema.format', argv.format || 'json'); - o.set(options, 'runtimeSchema.filename', argv.filename || 'runtime-schema'); - - o.set(options, 'types.enabled', false); - o.set(options, 'enums.enabled', false); - o.set(options, 'utils.astHelpers.enabled', false); - o.set(options, 'utils.enums.enabled', false); - - console.log(`Generating runtime schema from ${argv.inFile}...`); - const parser = new PgProtoParser(argv.inFile, options); - await parser.write(); - - const extension = argv.format === 'typescript' ? 'ts' : 'json'; - console.log(`Runtime schema generated successfully in ${argv.outDir}/${argv.filename || 'runtime-schema'}.${extension}`); - - return argv; -}; diff --git a/packages/cli/src/commands/runtime-schema/index.ts b/packages/cli/src/commands/runtime-schema/index.ts deleted file mode 100644 index 0641cb54..00000000 --- a/packages/cli/src/commands/runtime-schema/index.ts +++ /dev/null @@ -1,43 +0,0 @@ -import { CLIOptions, Inquirerer } from 'inquirerer' -import { ParsedArgs } from 'minimist'; -import cli, { help } from './cli'; - -export const commands = async (argv: Partial, prompter: Inquirerer, _options: CLIOptions) => { - - if (argv.help || argv.h) { - help(); - process.exit(0); - } - - argv = await prompter.prompt(argv, [ - { - type: 'text', - name: 'inFile', - message: 'provide inFile (./path/to/proto.proto):' - }, - { - type: 'text', - name: 'outDir', - message: 'provide outDir (./outputDir):' - }, - { - type: 'autocomplete', - name: 'format', - message: 'Choose output format:', - options: ['json', 'typescript'], - default: 'json', - useDefault: true - }, - { - type: 'text', - name: 'filename', - message: 'Filename for runtime schema (without extension):', - default: 'runtime-schema', - useDefault: true - } - ]); - - argv = await cli(argv); - - return argv; -}; diff --git a/packages/cli/src/index.ts b/packages/cli/src/index.ts deleted file mode 100644 index 40b221c2..00000000 --- a/packages/cli/src/index.ts +++ /dev/null @@ -1,22 +0,0 @@ -#!/usr/bin/env node -import { CLI, CLIOptions } from 'inquirerer'; - -import { commands } from './commands' - -export const options: Partial = { - minimistOpts: { - alias: { - v: 'version', - h: 'help' - } - } - }; - -const app = new CLI(commands, options); - -app.run().then(()=> { - // all done! -}).catch(error => { - console.error(error); - process.exit(1); -}) diff --git a/packages/deparser/README.md b/packages/deparser/README.md index 98b1f9e6..2d044d1c 100644 --- a/packages/deparser/README.md +++ b/packages/deparser/README.md @@ -73,30 +73,13 @@ console.log(deparse(stmt)); `pgsql-deparser` is particularly useful in development environments where native dependencies are problematic or in applications where only the deparser functionality is required. Its independence from the full `pgsql-parser` package allows for more focused and lightweight SQL generation tasks. -## Versions - -As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates. - -Our latest is built with `17-latest` branch from libpg_query - -| PostgreSQL Major Version | libpg_query | Status | npm tag | -|--------------------------|-------------|---------------------|---------| -| 17 | 17-latest | Active Development | `latest` | -| 16 | (n/a) | Not supported | -| 15 | (n/a) | Not supported | -| 14 | (n/a) | Not supported | -| 13 | 13-latest | Only Critical Fixes | `13.16.0` | -| 12 | (n/a) | Not supported | -| 11 | (n/a) | Not supported | -| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) | - ## Credits Built on the excellent work of several contributors: * **[Dan Lynch](https://github.com/pyramation)** β€” official maintainer since 2018 and architect of the current implementation * **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) β€” the core PostgreSQL parser that powers this project -* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability +* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM and multiple PG runtimes for better interoperability * **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings * **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser @@ -104,6 +87,7 @@ Built on the excellent work of several contributors: * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/parser/README.md b/packages/parser/README.md index 11428456..5705f4cc 100644 --- a/packages/parser/README.md +++ b/packages/parser/README.md @@ -122,42 +122,13 @@ console.log(deparse(stmt)); // Output: SELECT * FROM another_table ``` -## CLI - -``` -npm install -g pgsql-parser -``` - -### usage - -```sh -pgsql-parser -``` - -## Versions - -As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates. - -Our latest is built with `17-latest` branch from libpg_query - -| PostgreSQL Major Version | libpg_query | Status | npm tag | -|--------------------------|-------------|---------------------|---------| -| 17 | 17-latest | Active Development | `latest` | -| 16 | (n/a) | Not supported | -| 15 | (n/a) | Not supported | -| 14 | (n/a) | Not supported | -| 13 | 13-latest | Only Critical Fixes | `13.16.0` | -| 12 | (n/a) | Not supported | -| 11 | (n/a) | Not supported | -| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) | - ## Credits Built on the excellent work of several contributors: * **[Dan Lynch](https://github.com/pyramation)** β€” official maintainer since 2018 and architect of the current implementation * **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) β€” the core PostgreSQL parser that powers this project -* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability +* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM and multiple PG runtimes for better interoperability * **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings * **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser @@ -165,6 +136,7 @@ Built on the excellent work of several contributors: * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/parser/package.json b/packages/parser/package.json index 89fea750..11107686 100644 --- a/packages/parser/package.json +++ b/packages/parser/package.json @@ -19,9 +19,7 @@ "bugs": { "url": "https://github.com/launchql/pgsql-parser/issues" }, - "bin": { - "pgsql-parser": "main/cli.js" - }, + "scripts": { "copy": "copyfiles -f ../../LICENSE README.md package.json dist", "clean": "rimraf dist", @@ -44,7 +42,6 @@ "dependencies": { "@pgsql/types": "^17.6.1", "libpg-query": "17.5.2", - "pgsql-deparser": "^17.6.2", - "minimist": "^1.2.6" + "pgsql-deparser": "^17.6.2" } } diff --git a/packages/parser/src/cli.js b/packages/parser/src/cli.js deleted file mode 100644 index aa56c4e2..00000000 --- a/packages/parser/src/cli.js +++ /dev/null @@ -1,25 +0,0 @@ -#!/usr/bin/env node -import { resolve, join } from 'path'; -import { readFileSync } from 'fs'; -import { parse, parseFunction } from './index'; -import { cleanTreeWithStmt } from './utils'; -const argv = require('minimist')(process.argv.slice(2)); -const args = argv._; -if (args.length !== 1) { - console.log(argv); - console.warn('Usage: pgsql-parser '); - process.exit(1); -} -const pth = args[0].startsWith('/') - ? args[0] - : resolve(join(process.cwd(), args[0])); -const content = readFileSync(pth, 'utf-8'); -let query; -if (argv.pl) { - // plpgsql ONLY - query = parseFunction(content); -} else { - query = parse(content); -} - -process.stdout.write(JSON.stringify(cleanTreeWithStmt(query), null, 2)); diff --git a/packages/parser/src/index.ts b/packages/parser/src/index.ts index 056fce41..b577b7b1 100644 --- a/packages/parser/src/index.ts +++ b/packages/parser/src/index.ts @@ -3,5 +3,3 @@ export { parseSync as parseSync, loadModule as loadModule } from 'libpg-query'; - -export { deparse, deparseSync } from 'pgsql-deparser'; diff --git a/packages/cli/CHANGELOG.md b/packages/pgsql-cli/CHANGELOG.md similarity index 100% rename from packages/cli/CHANGELOG.md rename to packages/pgsql-cli/CHANGELOG.md diff --git a/packages/pgsql-cli/README.md b/packages/pgsql-cli/README.md new file mode 100644 index 00000000..a6253af3 --- /dev/null +++ b/packages/pgsql-cli/README.md @@ -0,0 +1,268 @@ +# @pgsql/cli + +

+ +

+ +

+ + + + +

+ +`@pgsql/cli` is a unified command-line interface for PostgreSQL AST operations, including parsing SQL to AST, deparsing AST back to SQL, and generating TypeScript definitions from PostgreSQL protobufs. It consolidates functionality from multiple packages into a single, easy-to-use CLI tool. + +## Installation + +```bash +npm install -g @pgsql/cli +``` + +## Features + +- **Parse SQL to AST**: Convert PostgreSQL queries into Abstract Syntax Trees +- **Deparse AST to SQL**: Convert AST back into SQL queries +- **Generate TypeScript from Protobuf**: Create type-safe TypeScript definitions from PostgreSQL protobuf files +- **Download and Process Proto Files**: Fetch proto files from URLs and generate JavaScript +- **Runtime Schema Generation**: Generate runtime schemas for AST nodes +- **Unified Interface**: Single CLI tool for all PostgreSQL AST operations + +## Quick Start + +```bash +# Parse SQL to AST +pgsql parse query.sql + +# Deparse AST back to SQL +pgsql deparse ast.json + +# Generate TypeScript from protobuf +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums + +# Download and process proto file +pgsql proto-fetch --url https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto --inFile pg_query.proto --outFile pg_query.js +``` + +## Commands + +### `pgsql parse` + +Parse SQL files into Abstract Syntax Trees (AST). + +```bash +pgsql parse [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|------------| +| `-o, --output` | Output to file instead of stdout | | +| `-f, --format` | Output format: `json`, `pretty` | `pretty` | +| `--pl` | Parse as PL/pgSQL function only | `false` | +| `--clean` | Clean the AST tree (remove location info) | `false` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Parse SQL and output to console +pgsql parse query.sql + +# Parse SQL and save to file +pgsql parse query.sql -o ast.json + +# Parse PL/pgSQL function +pgsql parse function.sql --pl + +# Parse and output compact JSON +pgsql parse query.sql --format json +``` + +### `pgsql deparse` + +Convert AST back to SQL. + +```bash +pgsql deparse [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|------------| +| `-i, --input` | Input JSON file (or use stdin) | | +| `-o, --output` | Output to file instead of stdout | | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Deparse from file +pgsql deparse -i ast.json + +# Deparse from stdin +cat ast.json | pgsql deparse + +# Parse and deparse in one line +pgsql parse query.sql | pgsql deparse + +# Deparse to file +pgsql deparse -i ast.json -o query.sql +``` + +### `pgsql proto-gen` + +Generate TypeScript definitions from PostgreSQL protobuf files. + +```bash +pgsql proto-gen --inFile --outDir [options] +``` + +#### Options + +| Option | Description | Default | +|-----------------------|------------------------------------------------------|------------| +| `--inFile` | Input .proto file | *Required* | +| `--outDir` | Output directory | *Required* | +| `--enums` | Generate TypeScript enums | `false` | +| `--enums-json` | Generate JSON enum mappings | `false` | +| `--types` | Generate TypeScript interfaces | `false` | +| `--utils` | Generate utility functions | `false` | +| `--ast-helpers` | Generate AST helper methods | `false` | +| `--wrapped-helpers` | Generate wrapped AST helpers | `false` | +| `--optional` | Make all fields optional | `false` | +| `--keep-case` | Keep original field casing | `false` | +| `--remove-undefined` | Remove UNDEFINED enum at position 0 | `false` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Generate types and enums +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums + +# Generate everything +pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums --utils --ast-helpers + +# Generate with optional fields +pgsql proto-gen --inFile pg_query.proto --outDir out --types --optional +``` + +### `pgsql proto-fetch` + +Download and process proto files. + +```bash +pgsql proto-fetch [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|--------------------------------| +| `--url` | Proto file URL to download | | +| `--inFile` | Where to save the proto file | *Required* | +| `--outFile` | Generated JS output file | *Required* | +| `--replace-pkg` | Original package name to replace | `protobufjs/minimal` | +| `--with-pkg` | New package name | `@launchql/protobufjs/minimal` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Download and process proto file +pgsql proto-fetch \ + --url https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto \ + --inFile pg_query.proto \ + --outFile pg_query.js + +# Process existing proto file +pgsql proto-fetch \ + --inFile pg_query.proto \ + --outFile pg_query.js \ + --replace-pkg "protobufjs/minimal" \ + --with-pkg "@custom/protobufjs" +``` + +### `pgsql runtime-schema` + +Generate runtime schema for AST nodes. + +```bash +pgsql runtime-schema --inFile --outDir [options] +``` + +#### Options + +| Option | Description | Default | +|---------------------|------------------------------------------------------|-------------------| +| `--inFile` | Input .proto file | *Required* | +| `--outDir` | Output directory | *Required* | +| `--format` | Output format: `json`, `typescript` | `json` | +| `--filename` | Output filename (without extension) | `runtime-schema` | +| `-h, --help` | Show help | | + +#### Examples + +```bash +# Generate JSON schema +pgsql runtime-schema --inFile pg_query.proto --outDir out + +# Generate TypeScript schema +pgsql runtime-schema --inFile pg_query.proto --outDir out --format typescript + +# Custom filename +pgsql runtime-schema --inFile pg_query.proto --outDir out --filename ast-schema +``` + +## Migration Guide + +### Migrating from `pgsql-parser` CLI + +If you were using the `pgsql-parser` command-line tool: + +```bash +# Old +pgsql-parser file.sql +pgsql-parser file.sql --pl + +# New +pgsql parse file.sql +pgsql parse file.sql --pl +``` + +### Migrating from `pg-proto-parser` + +If you were using the `pg-proto-parser` command-line tool: + +```bash +# Old +pg-proto-parser codegen --inFile pg_query.proto --outDir out + +# New +pgsql proto-gen --inFile pg_query.proto --outDir out +``` + +The command options remain largely the same, with some improvements: +- `codegen` β†’ `proto-gen` +- `protogen` β†’ `proto-fetch` +- Boolean flags now use kebab-case (e.g., `--enumsJSON` β†’ `--enums-json`) + +## Related + +* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. +* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. +* [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. +* [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. +* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. +* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums. +* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries. + +## Disclaimer + +AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND. + +No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value. \ No newline at end of file diff --git a/packages/cli/jest.config.js b/packages/pgsql-cli/jest.config.js similarity index 100% rename from packages/cli/jest.config.js rename to packages/pgsql-cli/jest.config.js diff --git a/packages/cli/package.json b/packages/pgsql-cli/package.json similarity index 83% rename from packages/cli/package.json rename to packages/pgsql-cli/package.json index 52fe794d..abcf576d 100644 --- a/packages/cli/package.json +++ b/packages/pgsql-cli/package.json @@ -1,12 +1,12 @@ { - "name": "@launchql/proto-cli", + "name": "@pgsql/cli", "version": "1.29.2", - "description": "The LaunchQL PG Proto parser", + "description": "Unified CLI for PostgreSQL AST parsing, deparsing, and code generation", "author": "Dan Lynch ", "main": "index.js", "module": "esm/index.js", "types": "index.d.ts", - "homepage": "https://github.com/launchql/pgsql-parser/tree/master/packages/cli#readme", + "homepage": "https://github.com/launchql/pgsql-parser/tree/master/packages/pgsql-cli#readme", "license": "SEE LICENSE IN LICENSE", "publishConfig": { "access": "public" @@ -19,7 +19,7 @@ "url": "https://github.com/launchql/pgsql-parser/issues" }, "bin": { - "pg-proto-parser": "index.js" + "pgsql": "index.js" }, "scripts": { "copy": "copyfiles -f ../../LICENSE README.md package.json dist", @@ -49,10 +49,11 @@ "@launchql/protobufjs-cli": "1.1.5", "chalk": "^4.1.0", "glob": "8.0.3", - "inquirerer": "1.9.0", "minimist": "1.2.8", "mkdirp": "3.0.1", "nested-obj": "^0.0.1", - "pg-proto-parser": "^1.28.2" + "pg-proto-parser": "^1.28.2", + "pgsql-parser": "^17.5.2", + "pgsql-deparser": "^17.6.2" } } diff --git a/packages/pgsql-cli/src/commands/deparse.ts b/packages/pgsql-cli/src/commands/deparse.ts new file mode 100644 index 00000000..0a8a9451 --- /dev/null +++ b/packages/pgsql-cli/src/commands/deparse.ts @@ -0,0 +1,60 @@ +import { readFileSync, writeFileSync } from 'fs'; +import { resolve, join } from 'path'; +import { deparse } from 'pgsql-deparser'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function deparseCommand(argv: any) { + if (argv.help) { + showHelp('deparse'); + process.exit(0); + } + + try { + let ast; + + // Read AST from file or stdin + if (argv.input || argv.i) { + const inputFile = argv.input || argv.i; + const filePath = inputFile.startsWith('/') + ? inputFile + : resolve(join(process.cwd(), inputFile)); + + const content = readFileSync(filePath, 'utf-8'); + ast = JSON.parse(content); + } else if (!process.stdin.isTTY) { + // Read from stdin + const chunks = []; + for await (const chunk of process.stdin) { + chunks.push(chunk); + } + const content = Buffer.concat(chunks).toString(); + ast = JSON.parse(content); + } else { + console.error(chalk.red('Error: No input provided. Use -i or pipe input via stdin')); + showHelp('deparse'); + process.exit(1); + } + + // Deparse AST to SQL + const sql = await deparse(ast); + + // Write output + if (argv.output || argv.o) { + const outputFile = argv.output || argv.o; + writeFileSync(outputFile, sql); + console.log(chalk.green(`βœ“ SQL written to ${outputFile}`)); + } else { + process.stdout.write(sql); + } + } catch (error: any) { + if (error.code === 'ENOENT') { + console.error(chalk.red(`Error: File not found: ${argv.input || argv.i}`)); + } else if (error instanceof SyntaxError) { + console.error(chalk.red('Error: Invalid JSON input')); + } else { + console.error(chalk.red('Deparse error:'), error.message || error); + } + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/pgsql-cli/src/commands/parse.ts b/packages/pgsql-cli/src/commands/parse.ts new file mode 100644 index 00000000..6f5c3b93 --- /dev/null +++ b/packages/pgsql-cli/src/commands/parse.ts @@ -0,0 +1,60 @@ +import { readFileSync, writeFileSync } from 'fs'; +import { resolve, join } from 'path'; +import { parse } from 'pgsql-parser'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function parseCommand(argv: any) { + if (argv.help) { + showHelp('parse'); + process.exit(0); + } + + const sqlFile = argv._[1]; + + if (!sqlFile) { + console.error(chalk.red('Error: SQL file required')); + showHelp('parse'); + process.exit(1); + } + + try { + // Resolve file path + const filePath = sqlFile.startsWith('/') + ? sqlFile + : resolve(join(process.cwd(), sqlFile)); + + // Read SQL content + const content = readFileSync(filePath, 'utf-8'); + + // Parse SQL + const ast = await parse(content); + + // Clean AST if requested + if (argv.clean) { + // For now, we'll skip the clean functionality until we can import it properly + console.warn(chalk.yellow('Warning: --clean flag is not yet implemented')); + } + + // Format output + const format = argv.format || 'pretty'; + const output = format === 'json' + ? JSON.stringify(ast) + : JSON.stringify(ast, null, 2); + + // Write output + if (argv.output) { + writeFileSync(argv.output, output); + console.log(chalk.green(`βœ“ AST written to ${argv.output}`)); + } else { + process.stdout.write(output); + } + } catch (error: any) { + if (error.code === 'ENOENT') { + console.error(chalk.red(`Error: File not found: ${sqlFile}`)); + } else { + console.error(chalk.red('Parse error:'), error.message || error); + } + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/pgsql-cli/src/commands/proto-fetch.ts b/packages/pgsql-cli/src/commands/proto-fetch.ts new file mode 100644 index 00000000..a647c509 --- /dev/null +++ b/packages/pgsql-cli/src/commands/proto-fetch.ts @@ -0,0 +1,42 @@ +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; +import { downloadProtoFile, generateProtoJS, replaceTextInProtoJS } from './proto-fetch/helpers'; + +export async function protoFetchCommand(argv: any) { + if (argv.help) { + showHelp('proto-fetch'); + process.exit(0); + } + + const url = argv.url; + const inFile = argv.inFile; + const outFile = argv.outFile; + const replacePkg = argv['replace-pkg'] || 'protobufjs/minimal'; + const withPkg = argv['with-pkg'] || '@launchql/protobufjs/minimal'; + + if (!inFile || !outFile) { + console.error(chalk.red('Error: --inFile and --outFile are required')); + showHelp('proto-fetch'); + process.exit(1); + } + + try { + if (url) { + console.log(chalk.blue(`Downloading proto file from ${url}...`)); + await downloadProtoFile(url, inFile); + console.log(chalk.green(`βœ“ Proto file saved to ${inFile}`)); + } + + console.log(chalk.blue('Generating JavaScript from proto file...')); + await generateProtoJS(inFile, outFile); + + console.log(chalk.blue(`Replacing package references...`)); + await replaceTextInProtoJS(outFile, replacePkg, withPkg); + + console.log(chalk.green(`βœ“ All operations completed successfully`)); + console.log(chalk.green(`βœ“ Generated file: ${outFile}`)); + } catch (error: any) { + console.error(chalk.red('Proto fetch error:'), error.message || error); + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/cli/src/commands/protogen/cli.ts b/packages/pgsql-cli/src/commands/proto-fetch/cli.ts similarity index 100% rename from packages/cli/src/commands/protogen/cli.ts rename to packages/pgsql-cli/src/commands/proto-fetch/cli.ts diff --git a/packages/cli/src/commands/protogen/helpers.ts b/packages/pgsql-cli/src/commands/proto-fetch/helpers.ts similarity index 100% rename from packages/cli/src/commands/protogen/helpers.ts rename to packages/pgsql-cli/src/commands/proto-fetch/helpers.ts diff --git a/packages/pgsql-cli/src/commands/proto-gen.ts b/packages/pgsql-cli/src/commands/proto-gen.ts new file mode 100644 index 00000000..2cb00b22 --- /dev/null +++ b/packages/pgsql-cli/src/commands/proto-gen.ts @@ -0,0 +1,49 @@ +import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; +import o from 'nested-obj'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function protoGenCommand(argv: any) { + if (argv.help) { + showHelp('proto-gen'); + process.exit(0); + } + + if (!argv.inFile || !argv.outDir) { + console.error(chalk.red('Error: --inFile and --outDir are required')); + showHelp('proto-gen'); + process.exit(1); + } + + try { + const options: PgProtoParserOptions = getOptionsWithDefaults({ + outDir: argv.outDir + }); + + // Set options based on flags + if (argv.enums) o.set(options, 'enums.enabled', true); + if (argv['enums-json']) o.set(options, 'enums.json.enabled', true); + if (argv.types) o.set(options, 'types.enabled', true); + if (argv.utils) o.set(options, 'utils.enums.enabled', true); + if (argv['ast-helpers']) o.set(options, 'utils.astHelpers.enabled', true); + if (argv['wrapped-helpers']) o.set(options, 'utils.wrappedAstHelpers.enabled', true); + if (argv.optional) o.set(options, 'types.optionalFields', true); + if (argv['keep-case']) o.set(options, 'parser.keepCase', true); + if (argv['remove-undefined']) o.set(options, 'enums.removeUndefinedAt0', true); + + // Additional options that might be useful + if (argv['type-union']) o.set(options, 'enums.enumsAsTypeUnion', true); + if (argv.header) o.set(options, 'includeHeader', true); + + console.log(chalk.blue('Parsing protobuf file...')); + const parser = new PgProtoParser(argv.inFile, options); + + console.log(chalk.blue('Generating TypeScript files...')); + await parser.write(); + + console.log(chalk.green(`βœ“ Files generated in ${argv.outDir}`)); + } catch (error: any) { + console.error(chalk.red('Proto generation error:'), error.message || error); + process.exit(1); + } +} \ No newline at end of file diff --git a/packages/pgsql-cli/src/commands/runtime-schema.ts b/packages/pgsql-cli/src/commands/runtime-schema.ts new file mode 100644 index 00000000..e8c92edd --- /dev/null +++ b/packages/pgsql-cli/src/commands/runtime-schema.ts @@ -0,0 +1,74 @@ +import { PgProtoParser, PgProtoParserOptions, getOptionsWithDefaults } from 'pg-proto-parser'; +import { writeFileSync } from 'fs'; +import { join } from 'path'; +import chalk from 'chalk'; +import { showHelp } from '../utils/help'; + +export async function runtimeSchemaCommand(argv: any) { + if (argv.help) { + showHelp('runtime-schema'); + process.exit(0); + } + + if (!argv.inFile || !argv.outDir) { + console.error(chalk.red('Error: --inFile and --outDir are required')); + showHelp('runtime-schema'); + process.exit(1); + } + + const format = argv.format || 'json'; + const filename = argv.filename || 'runtime-schema'; + + try { + console.log(chalk.blue('Parsing protobuf file...')); + + const options: PgProtoParserOptions = getOptionsWithDefaults({ + outDir: argv.outDir + }); + + const parser = new PgProtoParser(argv.inFile, options); + + // Generate runtime schema + console.log(chalk.blue(`Generating runtime schema in ${format} format...`)); + console.log(chalk.yellow('Warning: Runtime schema generation is not yet fully implemented')); + + // Generate placeholder schema based on format + let content: string; + let fileExt: string; + + if (format === 'typescript') { + // Generate TypeScript runtime schema placeholder + content = `// Runtime schema for PostgreSQL AST nodes +// Generated from: ${argv.inFile} + +export interface RuntimeSchema { + // TODO: Implement runtime schema generation + nodes: Record; +} + +export const runtimeSchema: RuntimeSchema = { + nodes: {} +}; +`; + fileExt = '.ts'; + } else { + // Generate JSON runtime schema placeholder + content = JSON.stringify({ + generated: new Date().toISOString(), + source: argv.inFile, + nodes: {} + }, null, 2); + fileExt = '.json'; + } + + // Write file + const outputPath = join(argv.outDir, `${filename}${fileExt}`); + writeFileSync(outputPath, content); + + console.log(chalk.green(`βœ“ Runtime schema generated: ${outputPath}`)); + } catch (error: any) { + console.error(chalk.red('Runtime schema error:'), error.message || error); + process.exit(1); + } +} + diff --git a/packages/pgsql-cli/src/index.ts b/packages/pgsql-cli/src/index.ts new file mode 100644 index 00000000..0cd713a7 --- /dev/null +++ b/packages/pgsql-cli/src/index.ts @@ -0,0 +1,69 @@ +#!/usr/bin/env node +import minimist from 'minimist'; +import chalk from 'chalk'; +import { readFileSync } from 'fs'; +import { resolve } from 'path'; + +import { parseCommand } from './commands/parse'; +import { deparseCommand } from './commands/deparse'; +import { protoGenCommand } from './commands/proto-gen'; +import { protoFetchCommand } from './commands/proto-fetch'; +import { runtimeSchemaCommand } from './commands/runtime-schema'; +import { showHelp, showVersion } from './utils/help'; + +const argv = minimist(process.argv.slice(2), { + alias: { + h: 'help', + v: 'version', + o: 'output', + f: 'format' + } +}); + +const command = argv._[0]; + +async function main() { + try { + if (argv.version) { + showVersion(); + process.exit(0); + } + + if (!command || argv.help) { + showHelp(command); + process.exit(0); + } + + switch (command) { + case 'parse': + await parseCommand(argv); + break; + + case 'deparse': + await deparseCommand(argv); + break; + + case 'proto-gen': + await protoGenCommand(argv); + break; + + case 'proto-fetch': + await protoFetchCommand(argv); + break; + + case 'runtime-schema': + await runtimeSchemaCommand(argv); + break; + + default: + console.error(chalk.red(`Unknown command: ${command}`)); + showHelp(); + process.exit(1); + } + } catch (error: any) { + console.error(chalk.red('Error:'), error.message || error); + process.exit(1); + } +} + +main(); diff --git a/packages/cli/src/package.ts b/packages/pgsql-cli/src/package.ts similarity index 100% rename from packages/cli/src/package.ts rename to packages/pgsql-cli/src/package.ts diff --git a/packages/pgsql-cli/src/utils/help.ts b/packages/pgsql-cli/src/utils/help.ts new file mode 100644 index 00000000..f1902454 --- /dev/null +++ b/packages/pgsql-cli/src/utils/help.ts @@ -0,0 +1,198 @@ +import chalk from 'chalk'; +import { readFileSync } from 'fs'; +import { resolve, join } from 'path'; + +export function showVersion() { + // Try to find package.json in various locations + const possiblePaths = [ + join(process.cwd(), 'package.json'), + join(process.argv[1], '../../package.json'), + join(process.argv[1], '../../../package.json') + ]; + + for (const path of possiblePaths) { + try { + const packageJson = JSON.parse(readFileSync(path, 'utf-8')); + if (packageJson.name === '@pgsql/cli') { + console.log(packageJson.version); + return; + } + } catch { + // Continue to next path + } + } + + // Fallback: just show the version we know + console.log('1.29.2'); +} + +export function showHelp(command?: string) { + if (command) { + switch (command) { + case 'parse': + showParseHelp(); + break; + case 'deparse': + showDeparseHelp(); + break; + case 'proto-gen': + showProtoGenHelp(); + break; + case 'proto-fetch': + showProtoFetchHelp(); + break; + case 'runtime-schema': + showRuntimeSchemaHelp(); + break; + default: + showGeneralHelp(); + } + } else { + showGeneralHelp(); + } +} + +function showGeneralHelp() { + console.log(` +${chalk.bold('pgsql')} - Unified CLI for PostgreSQL AST operations + +${chalk.yellow('Usage:')} + pgsql [options] + +${chalk.yellow('Commands:')} + ${chalk.green('parse')} Parse SQL to AST + ${chalk.green('deparse')} Convert AST to SQL + ${chalk.green('proto-gen')} Generate TypeScript from protobuf + ${chalk.green('proto-fetch')} Download and process proto files + ${chalk.green('runtime-schema')} Generate runtime schema for AST nodes + +${chalk.yellow('Global Options:')} + -h, --help Show help + -v, --version Show version + +${chalk.yellow('Examples:')} + pgsql parse query.sql + pgsql deparse ast.json + pgsql proto-gen --inFile pg_query.proto --outDir out + pgsql proto-fetch --url https://example.com/pg_query.proto --inFile pg_query.proto + +Run 'pgsql --help' for detailed help on a specific command. +`); +} + +function showParseHelp() { + console.log(` +${chalk.bold('pgsql parse')} - Parse SQL to AST + +${chalk.yellow('Usage:')} + pgsql parse [options] + +${chalk.yellow('Options:')} + -o, --output Output to file instead of stdout + -f, --format Output format: json, pretty (default: pretty) + --clean Clean the AST tree (remove location info) + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql parse query.sql + pgsql parse query.sql -o ast.json + pgsql parse query.sql --format json --clean +`); +} + +function showDeparseHelp() { + console.log(` +${chalk.bold('pgsql deparse')} - Convert AST to SQL + +${chalk.yellow('Usage:')} + pgsql deparse [options] + +${chalk.yellow('Options:')} + -i, --input Input JSON file (or use stdin) + -o, --output Output to file instead of stdout + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql deparse -i ast.json + pgsql deparse -i ast.json -o query.sql + cat ast.json | pgsql deparse + pgsql parse query.sql | pgsql deparse +`); +} + +function showProtoGenHelp() { + console.log(` +${chalk.bold('pgsql proto-gen')} - Generate TypeScript from protobuf + +${chalk.yellow('Usage:')} + pgsql proto-gen --inFile --outDir [options] + +${chalk.yellow('Required Options:')} + --inFile Input .proto file + --outDir Output directory + +${chalk.yellow('Optional Flags:')} + --enums Generate TypeScript enums + --enums-json Generate JSON enum mappings + --types Generate TypeScript interfaces + --utils Generate utility functions + --ast-helpers Generate AST helper methods + --wrapped-helpers Generate wrapped AST helpers + --optional Make all fields optional + --keep-case Keep original field casing + --remove-undefined Remove UNDEFINED enum at position 0 + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums + pgsql proto-gen --inFile pg_query.proto --outDir out --types --utils --ast-helpers + pgsql proto-gen --inFile pg_query.proto --outDir out --types --optional --keep-case +`); +} + +function showProtoFetchHelp() { + console.log(` +${chalk.bold('pgsql proto-fetch')} - Download and process proto files + +${chalk.yellow('Usage:')} + pgsql proto-fetch [options] + +${chalk.yellow('Options:')} + --url Proto file URL to download + --inFile Where to save the proto file + --outFile Generated JS output file + --replace-pkg Original package name to replace + --with-pkg New package name + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql proto-fetch --url https://raw.githubusercontent.com/pganalyze/libpg_query/16-latest/protobuf/pg_query.proto \\ + --inFile pg_query.proto \\ + --outFile pg_query.js \\ + --replace-pkg "protobufjs/minimal" \\ + --with-pkg "@launchql/protobufjs/minimal" +`); +} + +function showRuntimeSchemaHelp() { + console.log(` +${chalk.bold('pgsql runtime-schema')} - Generate runtime schema for AST nodes + +${chalk.yellow('Usage:')} + pgsql runtime-schema --inFile --outDir [options] + +${chalk.yellow('Required Options:')} + --inFile Input .proto file + --outDir Output directory + +${chalk.yellow('Optional Options:')} + --format Output format: json, typescript (default: json) + --filename Output filename (default: runtime-schema) + -h, --help Show help + +${chalk.yellow('Examples:')} + pgsql runtime-schema --inFile pg_query.proto --outDir out + pgsql runtime-schema --inFile pg_query.proto --outDir out --format typescript + pgsql runtime-schema --inFile pg_query.proto --outDir out --filename ast-schema +`); +} \ No newline at end of file diff --git a/packages/cli/tsconfig.esm.json b/packages/pgsql-cli/tsconfig.esm.json similarity index 100% rename from packages/cli/tsconfig.esm.json rename to packages/pgsql-cli/tsconfig.esm.json diff --git a/packages/cli/tsconfig.json b/packages/pgsql-cli/tsconfig.json similarity index 100% rename from packages/cli/tsconfig.json rename to packages/pgsql-cli/tsconfig.json diff --git a/packages/proto-parser/README.md b/packages/proto-parser/README.md index 8ca3a028..f216f414 100644 --- a/packages/proto-parser/README.md +++ b/packages/proto-parser/README.md @@ -319,6 +319,7 @@ ast.selectStmt({ * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/transform/README.md b/packages/transform/README.md index c5af6ec2..1cadcf96 100644 --- a/packages/transform/README.md +++ b/packages/transform/README.md @@ -20,6 +20,7 @@ * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/utils/README.md b/packages/utils/README.md index e6961f50..2f620015 100644 --- a/packages/utils/README.md +++ b/packages/utils/README.md @@ -59,17 +59,20 @@ With the AST helper methods, creating complex SQL ASTs becomes straightforward a Explore the PostgreSQL Abstract Syntax Tree (AST) as JSON objects with ease using `@pgsql/utils`. Below is an example of how you can generate a JSON AST using TypeScript: ```ts -import ast from '@pgsql/utils'; -const selectStmt = ast.selectStmt({ +import * as t from '@pgsql/utils'; +import { SelectStmt } from '@pgsql/types'; +import { deparseSync as deparse } from 'pgsql-deparser'; + +const selectStmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.aStar()] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.aStar()] }) }) ], fromClause: [ - ast.rangeVar({ + t.nodes.rangeVar({ relname: 'some_amazing_table', inh: true, relpersistence: 'p' @@ -80,47 +83,45 @@ const selectStmt = ast.selectStmt({ }); console.log(selectStmt); // Output: { "SelectStmt": { "targetList": [ { "ResTarget": { "val": { "ColumnRef": { "fields": [ { "A_Star": {} } ] } } } } ], "fromClause": [ { "RangeVar": { "relname": "some_amazing_table", "inh": true, "relpersistence": "p" } } ], "limitOption": "LIMIT_OPTION_DEFAULT", "op": "SETOP_NONE" } } +console.log(deparse(stmt)) +// Output: SELECT * FROM some_amazing_table ``` #### Select Statement ```ts -import ast, { CreateStmt, ColumnDef } from '@pgsql/utils'; -import { deparse } from 'pgsql-deparser'; +import * as t from '@pgsql/utils'; +import { RangeVar, SelectStmt } from '@pgsql/types'; +import { deparseSync as deparse } from 'pgsql-deparser'; -const selectStmt: SelectStmt = ast.selectStmt({ +const query: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ targetList: [ - ast.resTarget({ - val: ast.columnRef({ - fields: [ast.aStar()] + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'name' })] + }) + }), + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'email' })] }) }) ], fromClause: [ - ast.rangeVar({ - schemaname: 'myschema', - relname: 'mytable', + t.nodes.rangeVar({ + relname: 'users', inh: true, relpersistence: 'p' }) ], - whereClause: ast.aExpr({ + whereClause: t.nodes.aExpr({ kind: 'AEXPR_OP', - name: [ast.string({ str: '=' })], - lexpr: ast.columnRef({ - fields: [ast.string({ str: 'a' })] + name: [t.nodes.string({ sval: '>' })], + lexpr: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'age' })] }), - rexpr: ast.typeCast({ - arg: ast.aConst({ - val: ast.string({ str: 't' }) - }), - typeName: ast.typeName({ - names: [ - ast.string({ str: 'pg_catalog' }), - ast.string({ str: 'bool' }) - ], - typemod: -1 - }) + rexpr: t.nodes.aConst({ + ival: t.ast.integer({ ival: 18 }) }) }), limitOption: 'LIMIT_OPTION_DEFAULT', @@ -128,7 +129,7 @@ const selectStmt: SelectStmt = ast.selectStmt({ }); deparse(createStmt, {}); -// SELECT * FROM myschema.mytable WHERE a = TRUE +// SELECT name, email FROM users WHERE age > 18 ``` #### Creating Table Schemas Dynamically @@ -139,26 +140,26 @@ const schema = { "tableName": "users", "columns": [ { "name": "id", "type": "int", "constraints": ["PRIMARY KEY"] }, - { "name": "username", "type": "string" }, - { "name": "email", "type": "string", "constraints": ["UNIQUE"] }, + { "name": "username", "type": "text" }, + { "name": "email", "type": "text", "constraints": ["UNIQUE"] }, { "name": "created_at", "type": "timestamp", "constraints": ["NOT NULL"] } ] }; // Construct the CREATE TABLE statement -const createStmt = ast.createStmt({ - relation: ast.rangeVar({ +const createStmt = t.nodes.createStmt({ + relation: t.ast.rangeVar({ relname: schema.tableName, inh: true, relpersistence: 'p' - }).RangeVar as RangeVar, // special case due to PG AST - tableElts: schema.columns.map(column => ast.columnDef({ + }), + tableElts: schema.columns.map(column => t.nodes.columnDef({ colname: column.name, - typeName: ast.typeName({ - names: [ast.string({ str: column.type })] + typeName: t.ast.typeName({ + names: [t.nodes.string({ sval: column.type })] }), constraints: column.constraints?.map(constraint => - ast.constraint({ + t.nodes.constraint({ contype: constraint === "PRIMARY KEY" ? "CONSTR_PRIMARY" : constraint === "UNIQUE" ? "CONSTR_UNIQUE" : "CONSTR_NOTNULL" }) ) @@ -166,7 +167,7 @@ const createStmt = ast.createStmt({ }); // `deparse` function converts AST to SQL string -const sql = deparse(createStmt, {}); +const sql = deparse(createStmt); console.log(sql); // OUTPUT: @@ -179,40 +180,11 @@ console.log(sql); // ) ``` -### Enum Value Conversion - -`@pgsql/utils` provides the `getEnumValue` function to convert between the string and integer representations of enum values. - -Here are a couple of examples demonstrating how to use `@pgsql/utils` in real applications: - -#### Example 1: Converting Enum Name to Integer - -Suppose you are working with the A_Expr_Kind enum and you have the name of an enum value. You can get its integer representation like this: - -```ts -import { getEnumValue } from '@pgsql/utils'; - -const enumName = 'AEXPR_OP'; -const enumValue = getEnumValue('A_Expr_Kind', enumName); - -console.log(enumValue); // Outputs the integer value corresponding to 'AEXPR_OP' -``` - -#### Example 2: Converting Integer to Enum Name - -```ts -import { getEnumValue } from '@pgsql/utils'; - -const intValue = 1; -const enumName = getEnumValue('SortByDir', intValue); - -console.log(enumName); // Outputs 'SORTBY_ASC' if 1 corresponds to 'SORTBY_ASC' -``` - ## Related * [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration. * [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`. +* [@pgsql/parser](https://www.npmjs.com/package/@pgsql/parser): Multi-version PostgreSQL parser with dynamic version selection at runtime, supporting PostgreSQL 15, 16, and 17 in a single package. * [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs. * [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications. * [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs. diff --git a/packages/utils/__test__/utils.test.ts b/packages/utils/__test__/utils.test.ts index 1120ec86..e3be8e49 100644 --- a/packages/utils/__test__/utils.test.ts +++ b/packages/utils/__test__/utils.test.ts @@ -71,6 +71,44 @@ it('SelectStmt with WHERE clause', () => { expect(deparse(selectStmt, {})).toEqual(`SELECT * FROM myschema.mytable WHERE a = CAST('t' AS boolean)`); }); +it('queries', () => { + const query: { SelectStmt: SelectStmt } = t.nodes.selectStmt({ + targetList: [ + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'name' })] + }) + }), + t.nodes.resTarget({ + val: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'email' })] + }) + }) + ], + fromClause: [ + t.nodes.rangeVar({ + relname: 'users', + inh: true, + relpersistence: 'p' + }) + ], + whereClause: t.nodes.aExpr({ + kind: 'AEXPR_OP', + name: [t.nodes.string({ sval: '>' })], + lexpr: t.nodes.columnRef({ + fields: [t.nodes.string({ sval: 'age' })] + }), + rexpr: t.nodes.aConst({ + ival: t.ast.integer({ ival: 18 }) + }) + }), + limitOption: 'LIMIT_OPTION_DEFAULT', + op: 'SETOP_NONE' + }); + + expect(deparse(query, {})).toEqual(`SELECT name, email FROM users WHERE age > 18`); + +}); it('dynamic creation of tables', () => { // Example JSON schema const schema = { @@ -104,6 +142,6 @@ it('dynamic creation of tables', () => { }); // `deparse` function converts AST to SQL string - const sql = deparse(createStmt, {}); + const sql = deparse(createStmt); expect(sql).toMatchSnapshot(); }) \ No newline at end of file diff --git a/yarn.lock b/yarn.lock index 40c5b366..7ce20fb6 100644 --- a/yarn.lock +++ b/yarn.lock @@ -1325,9 +1325,9 @@ integrity sha512-+Fj43pSMwJs4KRrH/938Uf+uAELIgVBmQzg/q1YG10djyfA3TnrU8N8XzqCh/okZdszqBQTZf96idMfE5lnwTA== "@sinclair/typebox@^0.34.0": - version "0.34.35" - resolved "https://registry.yarnpkg.com/@sinclair/typebox/-/typebox-0.34.35.tgz#185c57551d5edf9a2f6e9d012822b06f942cfbfc" - integrity sha512-C6ypdODf2VZkgRT6sFM8E1F8vR+HcffniX0Kp8MsU8PIfrlXbNCBz0jzj17GjdmjTx1OtZzdH8+iALL21UjF5A== + version "0.34.36" + resolved "https://registry.yarnpkg.com/@sinclair/typebox/-/typebox-0.34.36.tgz#cbf53a007afa34e5f442b75888f9719f2ab463bb" + integrity sha512-JFHFhF6MqqRE49JDAGX/EPlHwxIukrKMhNwlMoB/wIJBkvu3+ciO335yDYPP3soI01FkhVXWnyNPKEl+EsC4Zw== "@sinonjs/commons@^3.0.0": version "3.0.1" @@ -2500,7 +2500,7 @@ deep-is@^0.1.3, deep-is@~0.1.3: resolved "https://registry.yarnpkg.com/deep-is/-/deep-is-0.1.4.tgz#a6f2dce612fadd2ef1f519b73551f17e85199831" integrity sha512-oIPzksmTg4/MriiaYGO+okXDT7ztn/w3Eptv/+gSIdMdKsJo0u4CfYNFJPy+4SKMuCqGw2wxnA+URMg3t8a/bQ== -deepmerge@4.3.1, deepmerge@^4.2.2, deepmerge@^4.3.1: +deepmerge@4.3.1, deepmerge@^4.2.2: version "4.3.1" resolved "https://registry.yarnpkg.com/deepmerge/-/deepmerge-4.3.1.tgz#44b5f2147cd3b00d4b56137685966f26fd25dd4a" integrity sha512-3sUqbMEc77XqpdNO7FRyRog+eW3ph+GYCbj+rK+uYyRMuwsVy0rMiVtPn+QJlKFvWP/1PYpapqYn0Me2knFn+A== @@ -3674,16 +3674,6 @@ inquirer@^8.2.4: through "^2.3.6" wrap-ansi "^6.0.1" -inquirerer@1.9.0: - version "1.9.0" - resolved "https://registry.yarnpkg.com/inquirerer/-/inquirerer-1.9.0.tgz#108071773a28ea5b950271572ac3051f34e0c92e" - integrity sha512-/LAn/F70YvRQZWz9r1q1seoO2oRMiSCSK8xKHGlkNebSibx5FppUKZLEjXgkCy1tgccas933q/Y7qNccFxrYkw== - dependencies: - chalk "^4.1.0" - deepmerge "^4.3.1" - js-yaml "^4.1.0" - minimist "^1.2.8" - ip-address@^9.0.5: version "9.0.5" resolved "https://registry.yarnpkg.com/ip-address/-/ip-address-9.0.5.tgz#117a960819b08780c3bd1f14ef3c1cc1d3f3ea5a"