Skip to content

[MINOR] Make README.md up-to-date #83

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

[MINOR] Make README.md up-to-date #83

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 24 additions & 13 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,24 +9,28 @@ This is an experimental Swift library to show how to connect to a remote Apache
So far, this library project is tracking the upstream changes like the [Apache Spark](https://spark.apache.org) 4.0.0 RC4 release and [Apache Arrow](https://arrow.apache.org) project's Swift-support.

## Requirement

- [Apache Spark 4.0.0 RC4 (April 2025)](https://dist.apache.org/repos/dist/dev/spark/v4.0.0-rc4-bin/)
- [Swift 6.0 (2024)](https://swift.org)
- [Swift 6.0 (2024) or 6.1 (2025)](https://swift.org)
- [gRPC Swift 2.1 (March 2025)](https://github.com/grpc/grpc-swift/releases/tag/2.1.2)
- [gRPC Swift Protobuf 1.1 (March 2025)](https://github.com/grpc/grpc-swift-protobuf/releases/tag/1.1.0)
- [gRPC Swift NIO Transport 1.0 (March 2025)](https://github.com/grpc/grpc-swift-nio-transport/releases/tag/1.0.2)
- [gRPC Swift Protobuf 1.2 (April 2025)](https://github.com/grpc/grpc-swift-protobuf/releases/tag/1.2.0)
- [gRPC Swift NIO Transport 1.0 (March 2025)](https://github.com/grpc/grpc-swift-nio-transport/releases/tag/1.0.3)
- [FlatBuffers v25.2.10 (February 2025)](https://github.com/google/flatbuffers/releases/tag/v25.2.10)
- [Apache Arrow Swift](https://github.com/apache/arrow/tree/main/swift)

## How to use in your apps

Create a Swift project.
```
$ mkdir SparkConnectSwiftApp
$ cd SparkConnectSwiftApp
$ swift package init --name SparkConnectSwiftApp --type executable

```bash
mkdir SparkConnectSwiftApp
cd SparkConnectSwiftApp
swift package init --name SparkConnectSwiftApp --type executable
```

Add `SparkConnect` package to the dependency like the following
```

```bash
$ cat Package.swift
import PackageDescription

Expand All @@ -49,7 +53,7 @@ let package = Package(

Use `SparkSession` of `SparkConnect` module in Swift.

```
```bash
$ cat Sources/main.swift

import SparkConnect
Expand Down Expand Up @@ -78,7 +82,7 @@ await spark.stop()

Run your Swift application.

```
```bash
$ swift run
...
Connected to Apache Spark 4.0.0 Server
Expand All @@ -105,16 +109,17 @@ SELECT * FROM t
```

You can find this example in the following repository.
- https://github.com/dongjoon-hyun/spark-connect-swift-app

- <https://github.com/dongjoon-hyun/spark-connect-swift-app>

## How to use `Spark SQL REPL` via `Spark Connect for Swift`

This project also provides `Spark SQL REPL`. You can run it directly from this repository.

```
```bash
$ swift run
...
Build of product 'SQLRepl' complete! (2.33s)
Build of product 'SparkSQLRepl' complete! (2.33s)
Connected to Apache Spark 4.0.0 Server
spark-sql (default)> SHOW DATABASES;
+---------+
Expand Down Expand Up @@ -176,3 +181,9 @@ spark-sql (default)> DROP DATABASE db1 CASCADE;
++
spark-sql (default)> exit;
```

You can use `SPARK_REMOTE` to specify the [Spark Connect connection string](https://spark.apache.org/docs/latest/spark-connect-overview.html#set-sparkremote-environment-variable) in order to provide more options.

```bash
SPARK_REMOTE=sc://localhost swift run
```
Loading