Skip to content

Commit a6411b2

Browse files
committed
[SPARK-51855] Support Spark SQL REPL
### What changes were proposed in this pull request? This PR aims to support `Spark SQL REPL` via `Spark Connect for Swift` library. ### Why are the changes needed? This is helpful for the users in two ways. - Provide an interactive environment to use `Spark Connect for Swift` library for the production, development and testing. - Provide a new simple and complete built-in example. ### Does this PR introduce _any_ user-facing change? No, this is a new `executable` target of this project in addition to the existing `library` target. ### How was this patch tested? Manual review by following new `README.md` content. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #77 from dongjoon-hyun/SPARK-51855. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent f8f7ebe commit a6411b2

File tree

3 files changed

+126
-0
lines changed

3 files changed

+126
-0
lines changed

Package.swift

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,10 @@ let package = Package(
4949
.product(name: "FlatBuffers", package: "flatbuffers"),
5050
]
5151
),
52+
.executableTarget(
53+
name: "SparkSQLRepl",
54+
dependencies: ["SparkConnect"]
55+
),
5256
.testTarget(
5357
name: "SparkConnectTests",
5458
dependencies: ["SparkConnect"],

README.md

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -106,3 +106,73 @@ SELECT * FROM t
106106

107107
You can find this example in the following repository.
108108
- https://github.com/dongjoon-hyun/spark-connect-swift-app
109+
110+
## How to use `Spark SQL REPL` via `Spark Connect for Swift`
111+
112+
This project also provides `Spark SQL REPL`. You can run it directly from this repository.
113+
114+
```
115+
$ swift run
116+
...
117+
Build of product 'SQLRepl' complete! (2.33s)
118+
Connected to Apache Spark 4.0.0 Server
119+
spark-sql (default)> SHOW DATABASES;
120+
+---------+
121+
|namespace|
122+
+---------+
123+
| default|
124+
+---------+
125+
126+
Time taken: 30 ms
127+
spark-sql (default)> CREATE DATABASE db1;
128+
++
129+
||
130+
++
131+
++
132+
133+
Time taken: 31 ms
134+
spark-sql (default)> USE db1;
135+
++
136+
||
137+
++
138+
++
139+
140+
Time taken: 27 ms
141+
spark-sql (db1)> CREATE TABLE t1 AS SELECT * FROM RANGE(10);
142+
++
143+
||
144+
++
145+
++
146+
147+
Time taken: 99 ms
148+
spark-sql (db1)> SELECT * FROM t1;
149+
+---+
150+
| id|
151+
+---+
152+
| 1|
153+
| 5|
154+
| 3|
155+
| 0|
156+
| 6|
157+
| 9|
158+
| 4|
159+
| 8|
160+
| 7|
161+
| 2|
162+
+---+
163+
164+
Time taken: 80 ms
165+
spark-sql (db1)> USE default;
166+
++
167+
||
168+
++
169+
++
170+
171+
Time taken: 26 ms
172+
spark-sql (default)> DROP DATABASE db1 CASCADE;
173+
++
174+
||
175+
++
176+
++
177+
spark-sql (default)> exit;
178+
```

Sources/SparkSQLRepl/main.swift

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
//
2+
// Licensed to the Apache Software Foundation (ASF) under one
3+
// or more contributor license agreements. See the NOTICE file
4+
// distributed with this work for additional information
5+
// regarding copyright ownership. The ASF licenses this file
6+
// to you under the Apache License, Version 2.0 (the
7+
// "License"); you may not use this file except in compliance
8+
// with the License. You may obtain a copy of the License at
9+
//
10+
// http://www.apache.org/licenses/LICENSE-2.0
11+
//
12+
// Unless required by applicable law or agreed to in writing,
13+
// software distributed under the License is distributed on an
14+
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
// KIND, either express or implied. See the License for the
16+
// specific language governing permissions and limitations
17+
// under the License.
18+
//
19+
20+
import Foundation
21+
import SparkConnect
22+
23+
let statement = /([^;]*);/
24+
25+
let spark = try await SparkSession.builder.getOrCreate()
26+
print("Connected to Apache Spark \(await spark.version) Server")
27+
28+
var isRunning = true
29+
while isRunning {
30+
print("spark-sql (\(try await spark.catalog.currentDatabase()))> ", terminator: "")
31+
guard let input = readLine() else {
32+
isRunning = false
33+
break
34+
}
35+
36+
let matches = input.matches(of: statement)
37+
for match in matches {
38+
switch match.1 {
39+
case "exit":
40+
isRunning = false
41+
break
42+
default:
43+
do {
44+
try await spark.time(spark.sql(String(match.1)).show)
45+
} catch {
46+
print("Error: \(error)")
47+
}
48+
}
49+
}
50+
}
51+
52+
await spark.stop()

0 commit comments

Comments
 (0)