Skip to content

Commit 7ffcccf

Browse files
pyramationpyramation
committed
docs: create comprehensive navigational README for monorepo
- Added detailed package overview table with descriptions and key features - Included quick start guide with installation and basic usage examples - Added architecture overview and package relationships - Included use cases and practical examples - Added contributing guidelines and development workflow - Organized related projects section - Removed references to non-existent @pgsql/types and @pgsql/enums packages - Excluded transform package as it's not production-ready
1 parent ca70cea commit 7ffcccf

File tree

4 files changed

+205
-164
lines changed

4 files changed

+205
-164
lines changed

‎README.md‎

Lines changed: 196 additions & 110 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# pgsql-parser
1+
# PostgreSQL AST Tools
22

33
<p align="center" width="100%">
44
<img height="120" src="https://github.com/launchql/pgsql-parser/assets/545047/6440fa7d-918b-4a3b-8d1b-755d85de8bea" />
@@ -11,177 +11,263 @@
1111
<a href="https://www.npmjs.com/package/pgsql-parser"><img height="20" src="https://img.shields.io/npm/dt/pgsql-parser"></a>
1212
<a href="https://www.npmjs.com/package/pgsql-parser"><img height="20" src="https://img.shields.io/npm/dw/pgsql-parser"/></a>
1313
<a href="https://github.com/launchql/pgsql-parser/blob/main/LICENSE-MIT"><img height="20" src="https://img.shields.io/badge/license-MIT-blue.svg"/></a>
14-
<a href="https://www.npmjs.com/package/pgsql-parser"><img height="20" src="https://img.shields.io/github/package-json/v/launchql/pgsql-parser?filename=packages%2Fparser%2Fpackage.json"/></a>
1514
</p>
1615

17-
The real PostgreSQL parser for Node.js, `pgsql-parser` provides symmetric parsing and deparsing of SQL statements using the actual [PostgreSQL parser](https://github.com/pganalyze/libpg_query). It allows you to parse SQL queries into AST and modify or reconstruct SQL queries from the AST.
16+
A comprehensive monorepo for PostgreSQL Abstract Syntax Tree (AST) parsing, manipulation, and code generation. This collection of packages provides everything you need to work with PostgreSQL at the AST level, from parsing SQL queries to generating type-safe TypeScript definitions.
1817

19-
## Installation
18+
## 📦 Packages Overview
2019

21-
```sh
22-
npm install pgsql-parser
23-
```
20+
| Package | Description | Key Features |
21+
|---------|-------------|--------------|
22+
| [**pgsql-parser**](./packages/parser) | The real PostgreSQL parser for Node.js | • Uses actual PostgreSQL C parser via WebAssembly<br>• Symmetric parsing and deparsing<br>• Battle-tested with 23,000+ SQL statements |
23+
| [**pgsql-deparser**](./packages/deparser) | Lightning-fast SQL generation from AST | • Pure TypeScript, zero dependencies<br>• No WebAssembly overhead<br>• Perfect for AST-to-SQL conversion only |
24+
| [**@pgsql/cli**](./packages/pgsql-cli) | Unified CLI for all PostgreSQL AST operations | • Parse SQL to AST<br>• Deparse AST to SQL<br>• Generate TypeScript from protobuf<br>• Single tool for all operations |
25+
| [**@pgsql/utils**](./packages/utils) | Type-safe AST node creation utilities | • Programmatic AST construction<br>• Enum value conversions<br>• Seamless integration with types |
26+
| [**pg-proto-parser**](./packages/proto-parser) | PostgreSQL protobuf parser and code generator | • Generate TypeScript interfaces from protobuf<br>• Create enum mappings and utilities<br>• AST helper generation |
2427

25-
## Key Features
28+
## 🚀 Quick Start
2629

27-
- **True PostgreSQL Parsing:** Utilizes the real PostgreSQL source code for accurate parsing.
28-
- **Symmetric Parsing and Deparsing:** Convert SQL to AST and back, enabling query manipulation.
29-
- **AST Manipulation:** Easily modify parts of a SQL statement through the AST.
30-
- **WebAssembly Powered:** Cross-platform compatibility without native dependencies.
30+
### Installation
3131

32-
## API
32+
Choose the packages you need:
33+
34+
```bash
35+
# For parsing SQL to AST and back
36+
npm install pgsql-parser
3337

34-
The package exports both async and sync methods. Async methods handle initialization automatically, while sync methods require explicit initialization.
38+
# For only converting AST to SQL (lighter weight)
39+
npm install pgsql-deparser
3540

36-
⚠️ We recommend using `@pgsql/deparser` instead of `deparse` from `pgsql-parser`. The deparser package is more complete, supports sub-expressions, and doesn't require the WebAssembly module, making it lighter and more flexible for most use cases. It will soon be deprecated, in a minor version bump.
41+
# For the unified CLI tool
42+
npm install -g @pgsql/cli
43+
44+
# For programmatic AST construction
45+
npm install @pgsql/utils
46+
47+
# For protobuf parsing and code generation
48+
npm install pg-proto-parser
49+
```
3750

38-
### Async Methods (Recommended)
51+
### Basic Usage
3952

53+
#### Parse SQL to AST
4054
```typescript
41-
import { parse, deparse, parseFunction } from 'pgsql-parser';
42-
43-
// Parse SQL to AST
44-
const stmts = await parse('SELECT * FROM test_table');
45-
46-
// Deparse AST back to SQL
47-
const sql = await deparse(stmts);
48-
49-
// Parse PL/pgSQL functions
50-
const funcAst = await parseFunction(`
51-
CREATE FUNCTION get_count() RETURNS integer AS $$
52-
BEGIN
53-
RETURN (SELECT COUNT(*) FROM users);
54-
END;
55-
$$ LANGUAGE plpgsql;
56-
`);
55+
import { parse } from 'pgsql-parser';
56+
57+
const ast = await parse('SELECT * FROM users WHERE id = 1');
58+
console.log(JSON.stringify(ast, null, 2));
5759
```
5860

59-
### Sync Methods
61+
#### Convert AST back to SQL
62+
```typescript
63+
import { deparse } from 'pgsql-deparser';
6064

61-
Sync methods require explicit initialization using `loadModule()`:
65+
const sql = deparse(ast);
66+
console.log(sql); // SELECT * FROM users WHERE id = 1
67+
```
6268

69+
#### Build AST Programmatically
6370
```typescript
64-
import { loadModule, parseSync, deparseSync } from 'pgsql-parser';
71+
import ast from '@pgsql/utils';
72+
73+
const selectStmt = ast.selectStmt({
74+
targetList: [
75+
ast.resTarget({
76+
val: ast.columnRef({
77+
fields: [ast.aStar()]
78+
})
79+
})
80+
],
81+
fromClause: [
82+
ast.rangeVar({
83+
relname: 'users',
84+
inh: true,
85+
relpersistence: 'p'
86+
})
87+
],
88+
limitOption: 'LIMIT_OPTION_DEFAULT',
89+
op: 'SETOP_NONE'
90+
});
91+
```
92+
93+
#### Use the CLI
94+
```bash
95+
# Parse SQL file
96+
pgsql parse query.sql
6597

66-
// Initialize first (required for sync methods)
67-
await loadModule();
98+
# Convert AST to SQL
99+
pgsql deparse ast.json
68100

69-
// Now safe to use sync methods
70-
const stmts = parseSync('SELECT * FROM test_table');
71-
const sql = deparseSync(stmts);
101+
# Generate TypeScript from protobuf
102+
pgsql proto-gen --inFile pg_query.proto --outDir out --types --enums
72103
```
73104

74-
**Note:** We recommend using async methods as they handle initialization automatically. Use sync methods only when necessary, and always call `loadModule()` first.
105+
## 🏗️ Architecture
75106

76-
## Parser Example
107+
This monorepo is organized to provide modular, focused tools that work together seamlessly:
77108

78-
Rewrite part of a SQL query:
109+
```
110+
pgsql-parser/
111+
├── packages/
112+
│ ├── parser/ # Core parser with WebAssembly
113+
│ ├── deparser/ # Pure TypeScript deparser
114+
│ ├── pgsql-cli/ # Unified CLI tool
115+
│ ├── utils/ # AST construction utilities
116+
│ ├── proto-parser/ # Protobuf code generation
117+
│ └── transform/ # (Not production-ready yet)
118+
└── ...
119+
```
120+
121+
### Package Relationships
122+
123+
- **pgsql-parser** provides full parsing and deparsing capabilities using the actual PostgreSQL parser
124+
- **pgsql-deparser** offers a lightweight alternative for just converting AST to SQL
125+
- **@pgsql/utils** helps construct ASTs programmatically with type safety
126+
- **pg-proto-parser** generates TypeScript definitions from PostgreSQL protobuf files
127+
- **@pgsql/cli** unifies all functionality into a single command-line tool
79128

80-
```js
81-
import { parse, deparse } from 'pgsql-parser';
129+
## 🛠️ Development
82130

83-
const stmts = await parse('SELECT * FROM test_table');
131+
This project uses Yarn workspaces and Lerna for monorepo management.
84132

85-
// Assuming the structure of stmts is known and matches the expected type
86-
stmts[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'another_table';
133+
### Setup
134+
```bash
135+
# Install dependencies
136+
yarn install
87137

88-
console.log(await deparse(stmts));
138+
# Build all packages
139+
yarn build
89140

90-
// SELECT * FROM "another_table"
141+
# Run tests
142+
yarn test
91143
```
92144

93-
## Deparser Example
145+
### Building Individual Packages
146+
```bash
147+
cd packages/parser
148+
npm run build
149+
```
150+
151+
## 📚 Documentation
152+
153+
Each package has its own detailed README:
94154

95-
The `pgsql-deparser` module serializes ASTs to SQL in pure TypeScript, avoiding the full parser's native dependencies. It's useful when only SQL string conversion from ASTs is needed, and is written in pure TypeScript for easy cross-environment deployment.
155+
- [pgsql-parser Documentation](./packages/parser/README.md)
156+
- [pgsql-deparser Documentation](./packages/deparser/README.md)
157+
- [@pgsql/cli Documentation](./packages/pgsql-cli/README.md)
158+
- [@pgsql/utils Documentation](./packages/utils/README.md)
159+
- [pg-proto-parser Documentation](./packages/proto-parser/README.md)
96160

97-
Here's how you can use the deparser in your TypeScript code, using [`@pgsql/utils`](https://github.com/launchql/pgsql-parser/tree/main/packages/utils) to create an AST for `deparse`:
161+
## 🎯 Use Cases
98162

99-
```ts
100-
import * as t from '@pgsql/utils';
101-
import { RangeVar, SelectStmt } from '@pgsql/types';
102-
import { deparseSync as deparse } from 'pgsql-deparser';
163+
- **SQL Query Analysis**: Parse queries to understand their structure and components
164+
- **Query Transformation**: Modify queries programmatically at the AST level
165+
- **SQL Generation**: Build complex queries programmatically with type safety
166+
- **Code Generation**: Generate TypeScript types from PostgreSQL schemas
167+
- **Query Validation**: Validate SQL syntax using the real PostgreSQL parser
168+
- **Database Tooling**: Build developer tools that understand PostgreSQL deeply
103169

104-
// This could have been obtained from any JSON or AST, not necessarily @pgsql/utils
105-
const stmt: { SelectStmt: SelectStmt } = t.nodes.selectStmt({
170+
## 💡 Examples
171+
172+
### Transform a Query
173+
```typescript
174+
import { parse } from 'pgsql-parser';
175+
import { deparse } from 'pgsql-deparser';
176+
177+
// Parse the original query
178+
const ast = await parse('SELECT * FROM users WHERE active = true');
179+
180+
// Modify the table name
181+
ast[0].RawStmt.stmt.SelectStmt.fromClause[0].RangeVar.relname = 'customers';
182+
183+
// Generate the modified SQL
184+
const newSql = deparse(ast);
185+
console.log(newSql); // SELECT * FROM customers WHERE active = TRUE
186+
```
187+
188+
### Build a Query Programmatically
189+
```typescript
190+
import ast from '@pgsql/utils';
191+
import { deparse } from 'pgsql-deparser';
192+
193+
const query = ast.selectStmt({
106194
targetList: [
107-
t.nodes.resTarget({
108-
val: t.nodes.columnRef({
109-
fields: [t.nodes.aStar()]
195+
ast.resTarget({
196+
val: ast.columnRef({
197+
fields: [ast.string({ str: 'name' })]
198+
})
199+
}),
200+
ast.resTarget({
201+
val: ast.columnRef({
202+
fields: [ast.string({ str: 'email' })]
110203
})
111204
})
112205
],
113206
fromClause: [
114-
t.nodes.rangeVar({
115-
relname: 'some_table',
207+
ast.rangeVar({
208+
relname: 'users',
116209
inh: true,
117210
relpersistence: 'p'
118211
})
119212
],
213+
whereClause: ast.aExpr({
214+
kind: 'AEXPR_OP',
215+
name: [ast.string({ str: '>' })],
216+
lexpr: ast.columnRef({
217+
fields: [ast.string({ str: 'age' })]
218+
}),
219+
rexpr: ast.aConst({
220+
val: ast.integer({ ival: 18 })
221+
})
222+
}),
120223
limitOption: 'LIMIT_OPTION_DEFAULT',
121224
op: 'SETOP_NONE'
122225
});
123226

124-
// Modify the AST if needed
125-
(stmt.SelectStmt.fromClause[0] as {RangeVar: RangeVar}).RangeVar.relname = 'another_table';
126-
127-
// Deparse the modified AST back to a SQL string
128-
console.log(deparse(stmt));
129-
130-
// Output: SELECT * FROM another_table
131-
```
132-
133-
## CLI
134-
135-
```
136-
npm install -g pgsql-parser
227+
console.log(deparse(query));
228+
// SELECT name, email FROM users WHERE age > 18
137229
```
138230

139-
### usage
231+
## 🤝 Contributing
140232

141-
```sh
142-
pgsql-parser <sqlfile>
143-
```
233+
We welcome contributions! Please see our [Contributing Guide](CONTRIBUTING.md) for details.
144234

145-
## Versions
235+
### Development Workflow
236+
1. Fork the repository
237+
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
238+
3. Commit your changes (`git commit -m 'Add some amazing feature'`)
239+
4. Push to the branch (`git push origin feature/amazing-feature`)
240+
5. Open a Pull Request
146241

147-
As of PG 13, PG majors versions maintained will have a matching dedicated major npm version. Only the latest Postgres stable release receives active updates.
242+
## 📄 License
148243

149-
Our latest is built with `17-latest` branch from libpg_query
244+
This project is licensed under the MIT License - see the [LICENSE](LICENSE-MIT) file for details.
150245

151-
| PostgreSQL Major Version | libpg_query | Status | npm tag |
152-
|--------------------------|-------------|---------------------|---------|
153-
| 17 | 17-latest | Active Development | `latest` |
154-
| 16 | (n/a) | Not supported |
155-
| 15 | (n/a) | Not supported |
156-
| 14 | (n/a) | Not supported |
157-
| 13 | 13-latest | Only Critical Fixes | `13.16.0` |
158-
| 12 | (n/a) | Not supported |
159-
| 11 | (n/a) | Not supported |
160-
| 10 | 10-latest | Not supported | `@1.3.1` ([tree](https://github.com/launchql/pgsql-parser/tree/39b7b1adc8914253226e286a48105785219a81ca)) |
246+
## 🙏 Credits
161247

248+
Built on the excellent work of several contributors:
162249

163-
## Related
250+
* **[Dan Lynch](https://github.com/pyramation)** — official maintainer since 2018 and architect of the current implementation
251+
* **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) — the core PostgreSQL parser that powers this project
252+
* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability
253+
* **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings
254+
* **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser
164255

165-
* [pgsql-parser](https://github.com/launchql/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
166-
* [pgsql-deparser](https://github.com/launchql/pgsql-parser/tree/main/packages/deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
167-
* [pgsql-enums](https://github.com/launchql/pgsql-parser/tree/main/packages/pgsql-enums): A utility package offering easy access to PostgreSQL enumeration types in JSON format, aiding in string and integer conversions of enums used within ASTs to compliment `pgsql-parser`
168-
* [@pgsql/enums](https://github.com/launchql/pgsql-parser/tree/main/packages/enums): Provides PostgreSQL AST enums in TypeScript, enhancing type safety and usability in projects interacting with PostgreSQL AST nodes.
169-
* [@pgsql/types](https://github.com/launchql/pgsql-parser/tree/main/packages/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
170-
* [@pgsql/utils](https://github.com/launchql/pgsql-parser/tree/main/packages/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
171-
* [pg-proto-parser](https://github.com/launchql/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums.
172-
* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries.
256+
## 🔗 Related Projects
173257

174-
## Credits
258+
### Core Packages (in this monorepo)
259+
* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js
260+
* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): Lightning-fast SQL generation from AST
261+
* [@pgsql/cli](https://www.npmjs.com/package/@pgsql/cli): Unified CLI for PostgreSQL AST operations
262+
* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): Type-safe AST construction utilities
263+
* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): PostgreSQL protobuf parser and code generator
175264

176-
Thanks [@lfittl](https://github.com/lfittl) for building the core `libpg_query` suite:
177265

178-
* [libpg_query](https://github.com/pganalyze/libpg_query)
179-
* [pg_query](https://github.com/lfittl/pg_query)
180-
* [pg_query.go](https://github.com/lfittl/pg_query.go)
181266

182-
Thanks to [@zhm](https://github.com/zhm) for the [OG Parser](https://github.com/zhm/pg-query-parser/blob/master/LICENSE.md) that started it all.
267+
### External Dependencies
268+
* [libpg-query](https://github.com/launchql/libpg-query-node): The PostgreSQL parser exposed for Node.js
183269

184-
## Disclaimer
270+
## ⚖️ Disclaimer
185271

186272
AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND.
187273

0 commit comments

Comments
 (0)