Skip to content

Commit a3712a2

Browse files
authored
Merge pull request #125 from launchql/feat/parser
Feat/parser
2 parents 0e148cb + 72ba897 commit a3712a2

File tree

23 files changed

+1518
-15
lines changed

23 files changed

+1518
-15
lines changed

.github/workflows/ci.yml

Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,3 +124,140 @@ jobs:
124124
- name: Run Tests 🔍
125125
run: pnpm run test
126126
working-directory: ${{ matrix.package.path }}
127+
128+
build-parser:
129+
name: Build Parser Package 📦
130+
needs: build-wasm
131+
runs-on: ubuntu-latest
132+
steps:
133+
- name: Checkout Repository 📥
134+
uses: actions/checkout@v4
135+
136+
- name: Setup Node.js 🌐
137+
uses: actions/setup-node@v4
138+
with:
139+
node-version: '20.x'
140+
141+
- name: Setup pnpm 📦
142+
uses: pnpm/action-setup@v2
143+
with:
144+
version: 8.15.1
145+
146+
- name: Get pnpm store directory 📁
147+
shell: bash
148+
run: |
149+
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
150+
151+
- name: Setup pnpm cache 🗄️
152+
uses: actions/cache@v3
153+
with:
154+
path: ${{ env.STORE_PATH }}
155+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
156+
restore-keys: |
157+
${{ runner.os }}-pnpm-store-
158+
159+
- name: Install Dependencies 🧶
160+
run: pnpm install
161+
162+
- name: Install Parser Dependencies 📦
163+
run: pnpm install
164+
working-directory: parser
165+
166+
- name: Download v15 WASM Artifacts 📥
167+
uses: actions/download-artifact@v4
168+
with:
169+
name: wasm-artifacts-v15
170+
path: versions/15/wasm/
171+
172+
- name: Download v16 WASM Artifacts 📥
173+
uses: actions/download-artifact@v4
174+
with:
175+
name: wasm-artifacts-v16
176+
path: versions/16/wasm/
177+
178+
- name: Download v17 WASM Artifacts 📥
179+
uses: actions/download-artifact@v4
180+
with:
181+
name: wasm-artifacts-v17
182+
path: versions/17/wasm/
183+
184+
- name: Build Parser 🏗
185+
run: pnpm run build
186+
working-directory: parser
187+
188+
- name: Upload Parser Artifacts 📦
189+
uses: actions/upload-artifact@v4
190+
with:
191+
name: parser-artifacts
192+
path: parser/wasm/
193+
retention-days: 1
194+
195+
test-parser:
196+
name: Test Parser on ${{ matrix.os }} ${{ matrix.os == 'ubuntu-latest' && '🐧' || matrix.os == 'macos-latest' && '🍎' || '🪟' }}
197+
needs: build-parser
198+
strategy:
199+
matrix:
200+
os: [ubuntu-latest, macos-latest, windows-latest]
201+
fail-fast: false
202+
runs-on: ${{ matrix.os }}
203+
steps:
204+
- name: Checkout Repository 📥
205+
uses: actions/checkout@v4
206+
207+
- name: Setup Node.js 🌐
208+
uses: actions/setup-node@v4
209+
with:
210+
node-version: '20.x'
211+
212+
- name: Setup pnpm 📦
213+
uses: pnpm/action-setup@v2
214+
with:
215+
version: 8.15.1
216+
217+
- name: Get pnpm store directory 📁
218+
shell: bash
219+
run: |
220+
echo "STORE_PATH=$(pnpm store path --silent)" >> $GITHUB_ENV
221+
222+
- name: Setup pnpm cache 🗄️
223+
uses: actions/cache@v3
224+
with:
225+
path: ${{ env.STORE_PATH }}
226+
key: ${{ runner.os }}-pnpm-store-${{ hashFiles('**/pnpm-lock.yaml') }}
227+
restore-keys: |
228+
${{ runner.os }}-pnpm-store-
229+
230+
- name: Install Dependencies 🧶
231+
run: pnpm install
232+
233+
- name: Install Parser Dependencies 📦
234+
run: pnpm install
235+
working-directory: parser
236+
237+
- name: Download v15 WASM Artifacts 📥
238+
uses: actions/download-artifact@v4
239+
with:
240+
name: wasm-artifacts-v15
241+
path: versions/15/wasm/
242+
243+
- name: Download v16 WASM Artifacts 📥
244+
uses: actions/download-artifact@v4
245+
with:
246+
name: wasm-artifacts-v16
247+
path: versions/16/wasm/
248+
249+
- name: Download v17 WASM Artifacts 📥
250+
uses: actions/download-artifact@v4
251+
with:
252+
name: wasm-artifacts-v17
253+
path: versions/17/wasm/
254+
255+
- name: Download Parser Artifacts 📥
256+
uses: actions/download-artifact@v4
257+
with:
258+
name: parser-artifacts
259+
path: parser/wasm/
260+
261+
- name: Run Parser Tests 🔍
262+
run: pnpm run test
263+
working-directory: parser

README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# libpg-query
22

33
<p align="center" width="100%">
4-
<img src="https://github.com/launchql/libpg-query-node/assets/545047/5fd420cc-cdc6-4211-9b0f-0eca8321ba72" alt="webincubator" width="100">
4+
<img src="https://github.com/launchql/libpg-query-node/assets/545047/5fd420cc-cdc6-4211-9b0f-0eca8321ba72" alt="hyperweb.io" width="100">
55
</p>
66

77
<p align="center" width="100%">
@@ -30,7 +30,7 @@ Built to power [pgsql-parser](https://github.com/pyramation/pgsql-parser), this
3030
* 🌐 **Node.js & Browser Support** – Consistent behavior in any JS environment
3131
* 📦 **No Native Builds Required** – No compilation, no system-specific dependencies
3232
* 🧠 **Spec-Accurate Parsing** – Produces faithful, standards-compliant ASTs
33-
* 🚀 **Production-Grade** – Millions of downloads powering 1000s of projects
33+
* 🚀 **Production-Grade** – Millions of downloads and trusted by countless projects and top teams
3434

3535
## 🚀 For Round-trip Codegen
3636

full/README.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# @libpg-query/parser
22

33
<p align="center" width="100%">
4-
<img src="https://github.com/launchql/libpg-query-node/assets/545047/5fd420cc-cdc6-4211-9b0f-0eca8321ba72" alt="webincubator" width="100">
4+
<img src="https://github.com/launchql/libpg-query-node/assets/545047/5fd420cc-cdc6-4211-9b0f-0eca8321ba72" alt="hyperweb.io" width="100">
55
</p>
66

77
<p align="center" width="100%">
@@ -30,7 +30,7 @@ Built to power [pgsql-parser](https://github.com/pyramation/pgsql-parser), this
3030
* 🌐 **Node.js & Browser Support** – Consistent behavior in any JS environment
3131
* 📦 **No Native Builds Required** – No compilation, no system-specific dependencies
3232
* 🧠 **Spec-Accurate Parsing** – Produces faithful, standards-compliant ASTs
33-
* 🚀 **Production-Grade** – Millions of downloads powering 1000s of projects
33+
* 🚀 **Production-Grade** – Millions of downloads and trusted by countless projects and top teams
3434

3535
## 🚀 For Round-trip Codegen
3636

package.json

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,10 @@
1919
"publish:types": "node scripts/publish-types.js",
2020
"publish:enums": "node scripts/publish-enums.js",
2121
"publish:versions": "node scripts/publish-versions.js",
22-
"update:versions-types": "node scripts/update-versions-types.js"
22+
"update:versions-types": "node scripts/update-versions-types.js",
23+
"build:parser": "pnpm --filter @pgsql/parser build",
24+
"test:parser": "pnpm --filter @pgsql/parser test",
25+
"publish:parser": "pnpm --filter @pgsql/parser publish"
2326
},
2427
"devDependencies": {
2528
"@types/node": "^20.0.0",

parser/.gitignore

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
node_modules/
2+
dist/
3+
dist-esm/
4+
*.log
5+
.DS_Store
6+
src/wasm/*.wasm
7+
src/wasm/*.js
8+
src/types/15/
9+
src/types/16/
10+
src/types/17/

parser/Makefile.shared

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
# Shared Makefile for building PostgreSQL parser WASM
2+
VERSION ?= 17
3+
TAG_15 = 15-4.2.4
4+
TAG_16 = 16-5.2.0
5+
TAG_17 = 17-6.1.0
6+
TAG = $(TAG_$(VERSION))
7+
8+
# Emscripten flags
9+
EMCC_CFLAGS = -O3 -flto -s WASM=1 -s TOTAL_MEMORY=16777216 \
10+
-s EXPORTED_FUNCTIONS="['_parse_sql']" \
11+
-s EXPORTED_RUNTIME_METHODS="['ccall','cwrap']" \
12+
-s MODULARIZE=1 \
13+
-s EXPORT_NAME="LibPGQuery$(VERSION)"
14+
15+
.PHONY: all
16+
all: clean download build-wasm
17+
18+
.PHONY: download
19+
download:
20+
@echo "Downloading libpg_query $(TAG)..."
21+
curl -L "https://github.com/pganalyze/libpg_query/archive/refs/tags/$(TAG).tar.gz" | tar -xz
22+
mv libpg_query-$(TAG) libpg_query
23+
24+
.PHONY: build-wasm
25+
build-wasm:
26+
@echo "Building WASM for PostgreSQL $(VERSION)..."
27+
cd libpg_query && make build_shared
28+
emcc $(EMCC_CFLAGS) \
29+
-I./libpg_query \
30+
./wasm_wrapper.c \
31+
./libpg_query/libpg_query.a \
32+
-o libpg-query.js
33+
34+
.PHONY: clean
35+
clean:
36+
rm -rf libpg_query libpg-query.js libpg-query.wasm
37+
38+
.PHONY: info
39+
info:
40+
@echo "Building PostgreSQL $(VERSION) parser"
41+
@echo "Tag: $(TAG)"
42+
@echo "Export name: LibPGQuery$(VERSION)"

parser/README.md

Lines changed: 130 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,130 @@
1+
# @pgsql/parser
2+
3+
<p align="center" width="100%">
4+
<img src="https://github.com/launchql/libpg-query-node/assets/545047/5fd420cc-cdc6-4211-9b0f-0eca8321ba72" alt="hyperweb.io" width="100">
5+
</p>
6+
7+
<p align="center" width="100%">
8+
<a href="https://github.com/launchql/libpg-query/blob/main/LICENSE-MIT"><img height="20" src="https://img.shields.io/badge/license-MIT-blue.svg"/></a>
9+
<a href="https://www.npmjs.com/package/libpg-query"><img height="20" src="https://img.shields.io/github/package-json/v/launchql/libpg-query-node?filename=versions%2F17%2Fpackage.json"/></a><br />
10+
<a href="https://github.com/launchql/libpg-query-node/actions/workflows/ci.yml"><img height="20" src="https://github.com/launchql/libpg-query-node/actions/workflows/ci.yml/badge.svg" /></a>
11+
<a href="https://github.com/launchql/libpg-query-node/actions/workflows/ci.yml"><img height="20" src="https://img.shields.io/badge/macOS-available-333333?logo=apple&logoColor=white" /></a>
12+
<a href="https://github.com/launchql/libpg-query-node/actions/workflows/ci.yml"><img height="20" src="https://img.shields.io/badge/Windows-available-333333?logo=windows&logoColor=white" /></a>
13+
<a href="https://github.com/launchql/libpg-query-node/actions/workflows/ci.yml"><img height="20" src="https://img.shields.io/badge/Linux-available-333333?logo=linux&logoColor=white" /></a>
14+
</p>
15+
16+
Multi-version PostgreSQL parser with dynamic version selection. This package provides a unified interface to parse PostgreSQL queries using different parser versions (15, 16, 17).
17+
18+
## Installation
19+
20+
```bash
21+
npm install @pgsql/parser
22+
```
23+
24+
## Usage
25+
26+
### Dynamic Version Selection
27+
28+
```javascript
29+
import { parse, PgParser } from '@pgsql/parser';
30+
31+
// Parse with default version (17)
32+
const result = await parse('SELECT 1+1 as sum');
33+
console.log(result);
34+
// { version: 17, result: { version: 170004, stmts: [...] } }
35+
36+
// Parse with specific version
37+
const result15 = await parse('SELECT 1+1 as sum', 15);
38+
console.log(result15);
39+
// { version: 15, result: { version: 150007, stmts: [...] } }
40+
41+
// Using PgParser class
42+
const parser = new PgParser(16);
43+
const result16 = await parser.parse('SELECT * FROM users');
44+
```
45+
46+
### Static Version Imports
47+
48+
For better tree-shaking and when you know which version you need:
49+
50+
```javascript
51+
// Import specific version
52+
import * as pg17 from '@pgsql/parser/v17';
53+
54+
await pg17.loadModule();
55+
const result = await pg17.parse('SELECT 1');
56+
console.log(result);
57+
// { version: 170004, stmts: [...] }
58+
```
59+
60+
### Error Handling
61+
62+
The parser returns errors in a consistent format:
63+
64+
```javascript
65+
const result = await parse('INVALID SQL');
66+
if (result.error) {
67+
console.error(result.error);
68+
// { type: 'syntax', message: 'syntax error at or near "INVALID"', position: 0 }
69+
}
70+
```
71+
72+
## API
73+
74+
### `parse(query: string, version?: 15 | 16 | 17): Promise<ParseResult>`
75+
76+
Parse a SQL query with the specified PostgreSQL version.
77+
78+
- `query`: The SQL query string to parse
79+
- `version`: PostgreSQL version (15, 16, or 17). Defaults to 17.
80+
81+
Returns a promise that resolves to:
82+
- On success: `{ version: number, result: AST }`
83+
- On error: `{ version: number, error: { type: string, message: string, position: number } }`
84+
85+
### `PgParser`
86+
87+
Class for creating a parser instance with a specific version.
88+
89+
```javascript
90+
const parser = new PgParser(version);
91+
await parser.parse(query);
92+
parser.parseSync(query); // Only available after first parse()
93+
```
94+
95+
## Version Exports
96+
97+
- `@pgsql/parser/v15` - PostgreSQL 15 parser
98+
- `@pgsql/parser/v16` - PostgreSQL 16 parser
99+
- `@pgsql/parser/v17` - PostgreSQL 17 parser
100+
101+
Each version export provides:
102+
- `loadModule()`: Initialize the WASM module
103+
- `parse(query)`: Parse a query (async)
104+
- `parseSync(query)`: Parse a query (sync, requires loadModule first)
105+
106+
## Credits
107+
108+
Built on the excellent work of several contributors:
109+
110+
* **[Dan Lynch](https://github.com/pyramation)** — official maintainer since 2018 and architect of the current implementation
111+
* **[Lukas Fittl](https://github.com/lfittl)** for [libpg_query](https://github.com/pganalyze/libpg_query) — the core PostgreSQL parser that powers this project
112+
* **[Greg Richardson](https://github.com/gregnr)** for AST guidance and pushing the transition to WASM for better interoperability
113+
* **[Ethan Resnick](https://github.com/ethanresnick)** for the original Node.js N-API bindings
114+
* **[Zac McCormick](https://github.com/zhm)** for the foundational [node-pg-query-native](https://github.com/zhm/node-pg-query-native) parser
115+
116+
## Related
117+
118+
* [pgsql-parser](https://www.npmjs.com/package/pgsql-parser): The real PostgreSQL parser for Node.js, providing symmetric parsing and deparsing of SQL statements with actual PostgreSQL parser integration.
119+
* [pgsql-deparser](https://www.npmjs.com/package/pgsql-deparser): A streamlined tool designed for converting PostgreSQL ASTs back into SQL queries, focusing solely on deparser functionality to complement `pgsql-parser`.
120+
* [@pgsql/types](https://www.npmjs.com/package/@pgsql/types): Offers TypeScript type definitions for PostgreSQL AST nodes, facilitating type-safe construction, analysis, and manipulation of ASTs.
121+
* [@pgsql/enums](https://www.npmjs.com/package/@pgsql/enums): Provides TypeScript enum definitions for PostgreSQL constants, enabling type-safe usage of PostgreSQL enums and constants in your applications.
122+
* [@pgsql/utils](https://www.npmjs.com/package/@pgsql/utils): A comprehensive utility library for PostgreSQL, offering type-safe AST node creation and enum value conversions, simplifying the construction and manipulation of PostgreSQL ASTs.
123+
* [pg-proto-parser](https://www.npmjs.com/package/pg-proto-parser): A TypeScript tool that parses PostgreSQL Protocol Buffers definitions to generate TypeScript interfaces, utility functions, and JSON mappings for enums.
124+
* [libpg-query](https://github.com/launchql/libpg-query-node): The real PostgreSQL parser exposed for Node.js, used primarily in `pgsql-parser` for parsing and deparsing SQL queries.
125+
126+
## Disclaimer
127+
128+
AS DESCRIBED IN THE LICENSES, THE SOFTWARE IS PROVIDED "AS IS", AT YOUR OWN RISK, AND WITHOUT WARRANTIES OF ANY KIND.
129+
130+
No developer or entity involved in creating Software will be liable for any claims or damages whatsoever associated with your use, inability to use, or your interaction with other users of the Software code or Software CLI, including any direct, indirect, incidental, special, exemplary, punitive or consequential damages, or loss of profits, cryptocurrencies, tokens, or anything else of value.

0 commit comments

Comments
 (0)