Skip to content

Commit be2b0d3

Browse files
kokoro-teamkokoro-team
committed
Release 0.36.5.
1 parent d7de904 commit be2b0d3

File tree

2 files changed

+26
-26
lines changed

2 files changed

+26
-26
lines changed

‎CHANGES.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Release Notes
22

3-
## Next
3+
## 0.36.5 - 2025-02-13
44
Issue #1254: fix getting partitioning fields for pseudo columns
55

66
## 0.36.4 - 2024-07-24

‎README.md‎

Lines changed: 25 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -57,14 +57,14 @@ The latest version of the connector is publicly available in the following links
5757

5858
| version | Link |
5959
|------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
60-
| Spark 3.5 | `gs://spark-lib/bigquery/spark-3.5-bigquery-0.36.4.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.5-bigquery-0.36.4.jar)) |
61-
| Spark 3.4 | `gs://spark-lib/bigquery/spark-3.4-bigquery-0.36.4.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.4-bigquery-0.36.4.jar)) |
62-
| Spark 3.3 | `gs://spark-lib/bigquery/spark-3.3-bigquery-0.36.4.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.3-bigquery-0.36.4.jar)) |
63-
| Spark 3.2 | `gs://spark-lib/bigquery/spark-3.2-bigquery-0.36.4.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.2-bigquery-0.36.4.jar)) |
64-
| Spark 3.1 | `gs://spark-lib/bigquery/spark-3.1-bigquery-0.36.4.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.1-bigquery-0.36.4.jar)) |
65-
| Spark 2.4 | `gs://spark-lib/bigquery/spark-2.4-bigquery-0.36.4.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-2.4-bigquery-0.36.4.jar)) |
66-
| Scala 2.13 | `gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.13-0.36.4.jar` ([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-bigquery-with-dependencies_2.13-0.36.4.jar)) |
67-
| Scala 2.12 | `gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.36.4.jar` ([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.36.4.jar)) |
60+
| Spark 3.5 | `gs://spark-lib/bigquery/spark-3.5-bigquery-0.36.5.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.5-bigquery-0.36.5.jar)) |
61+
| Spark 3.4 | `gs://spark-lib/bigquery/spark-3.4-bigquery-0.36.5.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.4-bigquery-0.36.5.jar)) |
62+
| Spark 3.3 | `gs://spark-lib/bigquery/spark-3.3-bigquery-0.36.5.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.3-bigquery-0.36.5.jar)) |
63+
| Spark 3.2 | `gs://spark-lib/bigquery/spark-3.2-bigquery-0.36.5.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.2-bigquery-0.36.5.jar)) |
64+
| Spark 3.1 | `gs://spark-lib/bigquery/spark-3.1-bigquery-0.36.5.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-3.1-bigquery-0.36.5.jar)) |
65+
| Spark 2.4 | `gs://spark-lib/bigquery/spark-2.4-bigquery-0.36.5.jar`([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-2.4-bigquery-0.36.5.jar)) |
66+
| Scala 2.13 | `gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.13-0.36.5.jar` ([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-bigquery-with-dependencies_2.13-0.36.5.jar)) |
67+
| Scala 2.12 | `gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.36.5.jar` ([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.36.5.jar)) |
6868
| Scala 2.11 | `gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.11-0.29.0.jar` ([HTTP link](https://storage.googleapis.com/spark-lib/bigquery/spark-bigquery-with-dependencies_2.11-0.29.0.jar)) |
6969

7070
The first four versions are Java based connectors targeting Spark 2.4/3.1/3.2/3.3 of all Scala versions built on the new
@@ -107,14 +107,14 @@ repository. It can be used using the `--packages` option or the
107107

108108
| version | Connector Artifact |
109109
|------------|------------------------------------------------------------------------------------|
110-
| Spark 3.5 | `com.google.cloud.spark:spark-3.5-bigquery:0.36.4` |
111-
| Spark 3.4 | `com.google.cloud.spark:spark-3.4-bigquery:0.36.4` |
112-
| Spark 3.3 | `com.google.cloud.spark:spark-3.3-bigquery:0.36.4` |
113-
| Spark 3.2 | `com.google.cloud.spark:spark-3.2-bigquery:0.36.4` |
114-
| Spark 3.1 | `com.google.cloud.spark:spark-3.1-bigquery:0.36.4` |
115-
| Spark 2.4 | `com.google.cloud.spark:spark-2.4-bigquery:0.36.4` |
116-
| Scala 2.13 | `com.google.cloud.spark:spark-bigquery-with-dependencies_2.13:0.36.4` |
117-
| Scala 2.12 | `com.google.cloud.spark:spark-bigquery-with-dependencies_2.12:0.36.4` |
110+
| Spark 3.5 | `com.google.cloud.spark:spark-3.5-bigquery:0.36.5` |
111+
| Spark 3.4 | `com.google.cloud.spark:spark-3.4-bigquery:0.36.5` |
112+
| Spark 3.3 | `com.google.cloud.spark:spark-3.3-bigquery:0.36.5` |
113+
| Spark 3.2 | `com.google.cloud.spark:spark-3.2-bigquery:0.36.5` |
114+
| Spark 3.1 | `com.google.cloud.spark:spark-3.1-bigquery:0.36.5` |
115+
| Spark 2.4 | `com.google.cloud.spark:spark-2.4-bigquery:0.36.5` |
116+
| Scala 2.13 | `com.google.cloud.spark:spark-bigquery-with-dependencies_2.13:0.36.5` |
117+
| Scala 2.12 | `com.google.cloud.spark:spark-bigquery-with-dependencies_2.12:0.36.5` |
118118
| Scala 2.11 | `com.google.cloud.spark:spark-bigquery-with-dependencies_2.11:0.29.0` |
119119

120120
### Specifying the Spark BigQuery connector version in a Dataproc cluster
@@ -124,8 +124,8 @@ Using the standard `--jars` or `--packages` (or alternatively, the `spark.jars`/
124124

125125
To use another version than the built-in one, please do one of the following:
126126

127-
* For Dataproc clusters, using image 2.1 and above, add the following flag on cluster creation to upgrade the version `--metadata SPARK_BQ_CONNECTOR_VERSION=0.36.4`, or `--metadata SPARK_BQ_CONNECTOR_URL=gs://spark-lib/bigquery/spark-3.3-bigquery-0.36.4.jar` to create the cluster with a different jar. The URL can point to any valid connector JAR for the cluster's Spark version.
128-
* For Dataproc serverless batches, add the following property on batch creation to upgrade the version: `--properties dataproc.sparkBqConnector.version=0.36.4`, or `--properties dataproc.sparkBqConnector.uri=gs://spark-lib/bigquery/spark-3.3-bigquery-0.36.4.jar` to create the batch with a different jar. The URL can point to any valid connector JAR for the runtime's Spark version.
127+
* For Dataproc clusters, using image 2.1 and above, add the following flag on cluster creation to upgrade the version `--metadata SPARK_BQ_CONNECTOR_VERSION=0.36.5`, or `--metadata SPARK_BQ_CONNECTOR_URL=gs://spark-lib/bigquery/spark-3.3-bigquery-0.36.5.jar` to create the cluster with a different jar. The URL can point to any valid connector JAR for the cluster's Spark version.
128+
* For Dataproc serverless batches, add the following property on batch creation to upgrade the version: `--properties dataproc.sparkBqConnector.version=0.36.5`, or `--properties dataproc.sparkBqConnector.uri=gs://spark-lib/bigquery/spark-3.3-bigquery-0.36.5.jar` to create the batch with a different jar. The URL can point to any valid connector JAR for the runtime's Spark version.
129129

130130
## Hello World Example
131131

@@ -135,7 +135,7 @@ You can run a simple PySpark wordcount against the API without compilation by ru
135135

136136
```
137137
gcloud dataproc jobs submit pyspark --cluster "$MY_CLUSTER" \
138-
--jars gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.36.4.jar \
138+
--jars gs://spark-lib/bigquery/spark-bigquery-with-dependencies_2.12-0.36.5.jar \
139139
examples/python/shakespeare.py
140140
```
141141

@@ -1161,7 +1161,7 @@ using the following code:
11611161
```python
11621162
from pyspark.sql import SparkSession
11631163
spark = SparkSession.builder \
1164-
.config("spark.jars.packages", "com.google.cloud.spark:spark-bigquery-with-dependencies_2.12:0.36.4") \
1164+
.config("spark.jars.packages", "com.google.cloud.spark:spark-bigquery-with-dependencies_2.12:0.36.5") \
11651165
.getOrCreate()
11661166
df = spark.read.format("bigquery") \
11671167
.load("dataset.table")
@@ -1170,15 +1170,15 @@ df = spark.read.format("bigquery") \
11701170
**Scala:**
11711171
```scala
11721172
val spark = SparkSession.builder
1173-
.config("spark.jars.packages", "com.google.cloud.spark:spark-bigquery-with-dependencies_2.12:0.36.4")
1173+
.config("spark.jars.packages", "com.google.cloud.spark:spark-bigquery-with-dependencies_2.12:0.36.5")
11741174
.getOrCreate()
11751175
val df = spark.read.format("bigquery")
11761176
.load("dataset.table")
11771177
```
11781178

11791179
In case Spark cluster is using Scala 2.12 (it's optional for Spark 2.4.x,
11801180
mandatory in 3.0.x), then the relevant package is
1181-
com.google.cloud.spark:spark-bigquery-with-dependencies_**2.12**:0.36.4. In
1181+
com.google.cloud.spark:spark-bigquery-with-dependencies_**2.12**:0.36.5. In
11821182
order to know which Scala version is used, please run the following code:
11831183

11841184
**Python:**
@@ -1202,14 +1202,14 @@ To include the connector in your project:
12021202
<dependency>
12031203
<groupId>com.google.cloud.spark</groupId>
12041204
<artifactId>spark-bigquery-with-dependencies_${scala.version}</artifactId>
1205-
<version>0.36.4</version>
1205+
<version>0.36.5</version>
12061206
</dependency>
12071207
```
12081208

12091209
### SBT
12101210

12111211
```sbt
1212-
libraryDependencies += "com.google.cloud.spark" %% "spark-bigquery-with-dependencies" % "0.36.4"
1212+
libraryDependencies += "com.google.cloud.spark" %% "spark-bigquery-with-dependencies" % "0.36.5"
12131213
```
12141214

12151215
### Connector metrics and how to view them
@@ -1254,7 +1254,7 @@ word-break:break-word
12541254
</table>
12551255

12561256

1257-
**Note:** To use the metrics in the Spark UI page, you need to make sure the `spark-bigquery-metrics-0.36.4.jar` is the class path before starting the history-server and the connector version is `spark-3.2` or above.
1257+
**Note:** To use the metrics in the Spark UI page, you need to make sure the `spark-bigquery-metrics-0.36.5.jar` is the class path before starting the history-server and the connector version is `spark-3.2` or above.
12581258

12591259
## FAQ
12601260

0 commit comments

Comments
 (0)