Skip to content

Commit 0183dfb

Browse files
williamhyundongjoon-hyun
authored andcommitted
[SPARK-52098] Support JDBC in DataFrameReader and Writer
### What changes were proposed in this pull request? This PR aims to Support JDBC in `DataFrameReader` and `DataFrameWriter`. ### Why are the changes needed? To support various databases via JDBC protocol. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #141 from williamhyun/jdbc. Authored-by: William Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 0dc9461 commit 0183dfb

File tree

2 files changed

+34
-0
lines changed

2 files changed

+34
-0
lines changed

Sources/SparkConnect/DataFrameReader.swift

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -258,4 +258,21 @@ public actor DataFrameReader: Sendable {
258258
self.source = "parquet"
259259
return load(paths)
260260
}
261+
262+
/// Construct a `DataFrame` representing the database table accessible via JDBC URL url named
263+
/// table and connection properties.
264+
/// - Parameters:
265+
/// - url: The JDBC URL of the form `jdbc:subprotocol:subname` to connect to.
266+
/// - table: The JDBC table that should be read from or written into.
267+
/// - properties: A string-string dictionary for connection properties.
268+
/// - Returns: A `DataFrame`.
269+
public func jdbc(_ url: String, _ table: String, _ properties: [String: String] = [:]) -> DataFrame {
270+
for (key, value) in properties {
271+
self.extraOptions[key] = value
272+
}
273+
self.extraOptions["url"] = url
274+
self.extraOptions["dbtable"] = table
275+
self.source = "jdbc"
276+
return load()
277+
}
261278
}

Sources/SparkConnect/DataFrameWriter.swift

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,4 +228,21 @@ public actor DataFrameWriter: Sendable {
228228
self.source = "text"
229229
return try await save(path)
230230
}
231+
232+
/// Saves the content of the `DataFrame` to an external database table via JDBC. In the case the
233+
/// table already exists in the external database, behavior of this function depends on the save
234+
/// mode, specified by the `mode` function (default to throwing an exception).
235+
/// - Parameters:
236+
/// - url: The JDBC URL of the form `jdbc:subprotocol:subname` to connect to.
237+
/// - table: Name of the table in the external database.
238+
/// - properties:JDBC database connection arguments, a list of arbitrary string tag/value.
239+
public func jdbc(_ url: String, _ table: String, _ properties: [String: String] = [:]) async throws {
240+
for (key, value) in properties {
241+
self.extraOptions[key] = value
242+
}
243+
self.extraOptions["url"] = url
244+
self.extraOptions["dbtable"] = table
245+
self.source = "jdbc"
246+
return try await save()
247+
}
231248
}

0 commit comments

Comments
 (0)