Skip to content

Commit 21f0512

Browse files
committed
[SPARK-50987][DOCS] Make spark-connect-overview.mds version strings up-to-date
### What changes were proposed in this pull request? This PR aims to make `spark-connect-overview.md`'s version strings up-to-date. ### Why are the changes needed? - https://apache.github.io/spark/spark-connect-overview.html **BEFORE** <img width="477" alt="Screenshot 2025-01-24 at 11 17 03 PM" src="https://github.com/user-attachments/assets/4ee91119-e116-4573-8446-32bf18342ac5" /> **AFTER** <img width="318" alt="Screenshot 2025-01-24 at 11 17 22 PM" src="https://github.com/user-attachments/assets/9e3f9061-6623-440f-8031-5ee85666675c" /> **BEFORE** <img width="546" alt="Screenshot 2025-01-24 at 11 17 58 PM" src="https://github.com/user-attachments/assets/dc3ac80b-a5fc-4ea2-bf1d-4025a6ae204f" /> **AFTER** <img width="552" alt="Screenshot 2025-01-24 at 11 18 35 PM" src="https://github.com/user-attachments/assets/8c5fe8cf-a8b1-4933-a593-3037f356c81a" /> **BEFORE** <img width="679" alt="Screenshot 2025-01-24 at 11 21 33 PM" src="https://github.com/user-attachments/assets/d4d69efe-2fb4-43ea-be13-0d1bbe251b2c" /> **AFTER** <img width="674" alt="Screenshot 2025-01-24 at 11 22 29 PM" src="https://github.com/user-attachments/assets/09a413fe-3659-4bba-b37c-609f2d6f16ba" /> ### Does this PR introduce _any_ user-facing change? This keeps the document up-to-date for Apache Spark 3.5.5/4.0.0+. ### How was this patch tested? Manual review. ### Was this patch authored or co-authored using generative AI tooling? No Closes #49665 from dongjoon-hyun/SPARK-50987. Authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]>
1 parent 60dd497 commit 21f0512

File tree

1 file changed

+6
-6
lines changed

1 file changed

+6
-6
lines changed

docs/spark-connect-overview.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -284,11 +284,11 @@ The connection may also be programmatically created using _SparkSession#builder_
284284

285285
<div data-lang="python" markdown="1">
286286

287-
First, install PySpark with `pip install pyspark[connect]==3.5.0` or if building a packaged PySpark application/library,
287+
First, install PySpark with `pip install pyspark[connect]=={{site.SPARK_VERSION_SHORT}}` or if building a packaged PySpark application/library,
288288
add it your setup.py file as:
289289
{% highlight python %}
290290
install_requires=[
291-
'pyspark[connect]==3.5.0'
291+
'pyspark[connect]=={{site.SPARK_VERSION_SHORT}}'
292292
]
293293
{% endhighlight %}
294294

@@ -335,7 +335,7 @@ Lines with a: 72, lines with b: 39
335335
To use Spark Connect as part of a Scala application/project, we first need to include the right dependencies.
336336
Using the `sbt` build system as an example, we add the following dependencies to the `build.sbt` file:
337337
{% highlight sbt %}
338-
libraryDependencies += "org.apache.spark" %% "spark-connect-client-jvm" % "3.5.0"
338+
libraryDependencies += "org.apache.spark" %% "spark-connect-client-jvm" % "{{site.SPARK_VERSION_SHORT}}"
339339
{% endhighlight %}
340340

341341
When writing your own code, include the `remote` function with a reference to
@@ -380,9 +380,9 @@ HTTP/2 interface allows for the use of authenticating proxies, which makes
380380
it possible to secure Spark Connect without having to implement authentication
381381
logic in Spark directly.
382382

383-
# What is supported in Spark 3.4
383+
# What is supported
384384

385-
**PySpark**: In Spark 3.4, Spark Connect supports most PySpark APIs, including
385+
**PySpark**: Since Spark 3.4, Spark Connect supports most PySpark APIs, including
386386
[DataFrame](api/python/reference/pyspark.sql/dataframe.html),
387387
[Functions](api/python/reference/pyspark.sql/functions.html), and
388388
[Column](api/python/reference/pyspark.sql/column.html). However,
@@ -393,7 +393,7 @@ supported in the [API reference](api/python/reference/index.html) documentation.
393393
Supported APIs are labeled "Supports Spark Connect" so you can check whether the
394394
APIs you are using are available before migrating existing code to Spark Connect.
395395

396-
**Scala**: In Spark 3.5, Spark Connect supports most Scala APIs, including
396+
**Scala**: Since Spark 3.5, Spark Connect supports most Scala APIs, including
397397
[Dataset](api/scala/org/apache/spark/sql/Dataset.html),
398398
[functions](api/scala/org/apache/spark/sql/functions$.html),
399399
[Column](api/scala/org/apache/spark/sql/Column.html),

0 commit comments

Comments
 (0)