Skip to content

Commit 1d95cee

Browse files
IMax153cmwhited
IMax153
and
cmwhited
authored
Add Arrow Flight SQL client (#1)
* start work on arrow flight sql client * get basic version of arrow flight ipc to json working * fix import extension * remove old readers directory * refactor schema identifiers * remove tsx * remove go from flake * remove arrow-flight-ipc package * cleanup remaining identifiers * cleanup and enhance internal arrow-flight-ipc modules * test(harness): setup vitest test harness for generated arrow FlightData and RecordBatch for testing (#3) * test(harness): add vitest deps. setup vitest config. validate with test * test(harness): small fixes * test(harness): build out arrow flight message and recordbatch builder test harness impl * rebase --------- Co-authored-by: Maxwell Brown <[email protected]> * exclude internal from publish config exports * add check workflow to CI * add clean script and fix issues with JSONC and TypeScript * fix bug in flatbuffer writer * fix test sharding --------- Co-authored-by: Chris Whited <[email protected]>
1 parent 49fa9ee commit 1d95cee

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+15621
-273
lines changed

‎.github/actions/setup/action.yml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
name: Setup
2+
description: Perform standard setup and install dependencies using pnpm.
3+
inputs:
4+
node-version:
5+
description: The version of Node.js to install
6+
required: false
7+
deno-version:
8+
description: The version of Deno to install
9+
required: false
10+
bun-version:
11+
description: The version of Bun to install
12+
required: false
13+
14+
runs:
15+
using: composite
16+
steps:
17+
- name: Install pnpm
18+
uses: pnpm/action-setup@v3
19+
- name: Install node
20+
uses: actions/setup-node@v4
21+
if: ${{ inputs.node-version != '' }}
22+
with:
23+
cache: pnpm
24+
node-version: ${{ inputs.node-version }}
25+
- name: Install deno
26+
uses: denoland/setup-deno@v2
27+
if: ${{ inputs.deno-version != '' }}
28+
with:
29+
deno-version: ${{ inputs.deno-version }}
30+
- name: Install bun
31+
uses: oven-sh/setup-bun@v2
32+
if: ${{ inputs.bun-version != '' }}
33+
with:
34+
bun-version: ${{ inputs.bun-version }}
35+
- name: Install dependencies
36+
shell: bash
37+
run: pnpm install

‎.github/workflows/check.yml‎

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,106 @@
1+
name: Check
2+
3+
on:
4+
workflow_dispatch:
5+
pull_request:
6+
branches: [main]
7+
push:
8+
branches: [main]
9+
10+
concurrency:
11+
group: ${{ github.workflow }}-${{ github.ref }}
12+
cancel-in-progress: true
13+
14+
permissions: {}
15+
16+
jobs:
17+
lint:
18+
name: Lint
19+
runs-on: ubuntu-latest
20+
permissions:
21+
contents: read
22+
timeout-minutes: 10
23+
steps:
24+
- uses: actions/checkout@v4
25+
- name: Install dependencies
26+
uses: ./.github/actions/setup
27+
with:
28+
node-version: 24.10.0
29+
- run: pnpm lint
30+
31+
types:
32+
name: Types
33+
runs-on: ubuntu-latest
34+
permissions:
35+
contents: read
36+
timeout-minutes: 10
37+
steps:
38+
- uses: actions/checkout@v4
39+
- name: Install dependencies
40+
uses: ./.github/actions/setup
41+
with:
42+
node-version: 24.10.0
43+
- run: pnpm check
44+
45+
# TODO: discuss with Sebastian whether we should add Deno / Bun configuration / checks
46+
# types-deno:
47+
# name: Types on Deno
48+
# runs-on: ubuntu-latest
49+
# permissions:
50+
# contents: read
51+
# timeout-minutes: 10
52+
# steps:
53+
# - uses: actions/checkout@v4
54+
# - name: Install dependencies
55+
# uses: ./.github/actions/setup
56+
# with:
57+
# deno-version: v2.5.x
58+
# - run: deno check .
59+
60+
test:
61+
name: Test
62+
runs-on: ubuntu-latest
63+
permissions:
64+
contents: read
65+
timeout-minutes: 10
66+
strategy:
67+
fail-fast: false
68+
matrix:
69+
shard: [1/1]
70+
runtime: [Node] # TODO: Deno
71+
steps:
72+
- uses: actions/checkout@v4
73+
74+
- name: Install dependencies
75+
if: matrix.runtime == 'Node'
76+
uses: ./.github/actions/setup
77+
with:
78+
node-version: 24.10.0
79+
- name: Test
80+
if: matrix.runtime == 'Node'
81+
run: pnpm test --shard ${{ matrix.shard }}
82+
83+
- name: Install dependencies
84+
if: matrix.runtime == 'Deno'
85+
uses: ./.github/actions/setup
86+
with:
87+
deno-version: v2.5.x
88+
- name: Test
89+
if: matrix.runtime == 'Deno'
90+
run: deno task test --shard ${{ matrix.shard }}
91+
92+
# TODO: discuss with Sebastian whether we want circularity checks
93+
# circular:
94+
# name: Circular Dependencies
95+
# runs-on: ubuntu-latest
96+
# permissions:
97+
# contents: read
98+
# timeout-minutes: 10
99+
# steps:
100+
# - uses: actions/checkout@v4
101+
# - name: Install dependencies
102+
# uses: ./.github/actions/setup
103+
# with:
104+
# node-version: 24.10.0
105+
# - name: Check for circular dependencies
106+
# run: pnpm circular

‎.gitignore‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,4 +14,6 @@ node_modules/
1414
.DS_Store
1515

1616
# Scratchpad Files
17+
scratchpad/**/*.md
1718
scratchpad/**/*.ts
19+
!scratchpad/index.ts

‎eslint.config.js‎

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,7 @@ export default defineConfig(
2020
{
2121
plugins: {
2222
"simple-import-sort": simpleImportSort,
23-
"sort-destructure-keys": sortDestructureKeys,
24-
"unused-imports": unusedImports
23+
"sort-destructure-keys": sortDestructureKeys
2524
},
2625

2726
languageOptions: {
@@ -71,7 +70,6 @@ export default defineConfig(
7170
"import-x/order": "off",
7271
"simple-import-sort/imports": "off",
7372
"sort-destructure-keys/sort-destructure-keys": "error",
74-
"unused-imports/no-unused-imports": "error",
7573
"deprecation/deprecation": "off",
7674

7775
"@typescript-eslint/array-type": [
@@ -130,5 +128,28 @@ export default defineConfig(
130128
rules: {
131129
"no-console": "error"
132130
}
131+
},
132+
{
133+
files: ["scratchpad/eslint/**/*"],
134+
plugins: {
135+
"unused-imports": unusedImports
136+
},
137+
rules: {
138+
"unused-imports/no-unused-imports": "error",
139+
"@effect/dprint": [
140+
"error",
141+
{
142+
config: {
143+
indentWidth: 2,
144+
lineWidth: 80,
145+
semiColons: "asi",
146+
quoteStyle: "alwaysDouble",
147+
trailingCommas: "never",
148+
operatorPosition: "maintain",
149+
"arrowFunction.useParentheses": "force"
150+
}
151+
}
152+
]
153+
}
133154
}
134155
)

‎package.json‎

Lines changed: 16 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,25 +1,36 @@
11
{
22
"private": true,
33
"type": "module",
4-
"packageManager": "pnpm@10.24.0+sha512.01ff8ae71b4419903b65c60fb2dc9d34cf8bb6e06d03bde112ef38f7a34d6904c424ba66bea5cdcf12890230bf39f9580473140ed9c946fef328b6e5238a345a",
4+
"packageManager": "pnpm@10.25.0+sha512.5e82639027af37cf832061bcc6d639c219634488e0f2baebe785028a793de7b525ffcd3f7ff574f5e9860654e098fe852ba8ac5dd5cefe1767d23a020a92f501",
55
"scripts": {
66
"check": "tspc -b tsconfig.json",
7+
"clean": "node scripts/clean.mjs",
78
"lint": "eslint \"**/{src,test,examples,dtslint}/**/*.{ts,mjs}\"",
8-
"lint-fix": "pnpm lint --fix"
9+
"lint-fix": "pnpm lint --fix",
10+
"test": "vitest"
911
},
1012
"devDependencies": {
1113
"@effect/eslint-plugin": "^0.3.2",
12-
"@effect/language-service": "^0.57.1",
14+
"@effect/language-service": "^0.62.0",
15+
"@effect/vitest": "^0.27.0",
1316
"@eslint/js": "^9.39.1",
14-
"@types/node": "^24.10.1",
17+
"@types/node": "^25.0.0",
18+
"@vitest/coverage-v8": "^4.0.15",
19+
"@vitest/ui": "^4.0.15",
20+
"effect": "^3.19.11",
1521
"eslint": "^9.39.1",
1622
"eslint-import-resolver-typescript": "^4.4.4",
1723
"eslint-plugin-import-x": "^4.16.1",
1824
"eslint-plugin-simple-import-sort": "^12.1.1",
1925
"eslint-plugin-sort-destructure-keys": "^2.0.0",
2026
"eslint-plugin-unused-imports": "^4.3.0",
27+
"glob": "^13.0.0",
28+
"globals": "^16.5.0",
2129
"ts-patch": "^3.3.0",
2230
"typescript": "^5.9.3",
23-
"typescript-eslint": "^8.48.0"
31+
"typescript-eslint": "^8.49.0",
32+
"vite-tsconfig-paths": "^5.1.4",
33+
"vitest": "^4.0.15",
34+
"vitest-mock-express": "^2.2.0"
2435
}
2536
}

‎packages/amp/README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Amp TypeScript SDK
2+
3+
## Known Issues
4+
5+
**Enabling TypeScript's `erasableSyntaxOnly` Feature**
6+
7+
For some completely absurd reason, the `@bufbuild/protoc-gen-es` plugin does not support generating enum values as anything other than TypeScript enums at this time, so we cannot yet enable this flag.
8+
9+
See the [related `protobuf-es` issue](https://github.com/bufbuild/protobuf-es/issues/1139) for more information.

‎packages/amp/buf.gen.yaml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
version: v2
2+
3+
inputs:
4+
- git_repo: https://github.com/apache/arrow
5+
subdir: format
6+
7+
plugins:
8+
- local: protoc-gen-es
9+
out: src/Protobuf
10+
opt:
11+
- target=ts
12+
- import_extension=ts

‎packages/amp/package.json‎

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,10 +10,12 @@
1010
"url": "https://github.com/edgeandnode/amp-typescript",
1111
"directory": "packages/amp"
1212
},
13+
"sideEffects": [],
1314
"exports": {
1415
"./package.json": "./package.json",
1516
".": "./src/index.ts",
16-
"./*": "./src/*.ts"
17+
"./*": "./src/*.ts",
18+
"./internal/*.ts": null
1719
},
1820
"files": [
1921
"src/**/*.ts",
@@ -22,10 +24,27 @@
2224
"dist/**/*.d.ts",
2325
"dist/**/*.d.ts.map"
2426
],
27+
"publishConfig": {
28+
"provenance": true,
29+
"exports": {
30+
"./package.json": "./package.json",
31+
".": "./dist/index.js",
32+
"./*": "./dist/*.js",
33+
"./internal/*.ts": null
34+
}
35+
},
2536
"peerDependencies": {
26-
"effect": "^3.19.8"
37+
"@bufbuild/protobuf": "^2.10.1",
38+
"@connectrpc/connect": "^2.1.1",
39+
"@connectrpc/connect-node": "^2.1.1",
40+
"effect": "^3.19.11"
2741
},
2842
"devDependencies": {
29-
"effect": "^3.19.8"
43+
"@bufbuild/buf": "^1.61.0",
44+
"@bufbuild/protobuf": "^2.10.1",
45+
"@bufbuild/protoc-gen-es": "^2.10.1",
46+
"@connectrpc/connect": "^2.1.1",
47+
"@connectrpc/connect-node": "^2.1.1",
48+
"effect": "^3.19.11"
3049
}
3150
}

0 commit comments

Comments
 (0)