Skip to content

Commit c37535c

Browse files
committed
Release 0.4.0.
1 parent e1bdf7c commit c37535c

File tree

7 files changed

+48
-21
lines changed

7 files changed

+48
-21
lines changed

CHANGES.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Release Notes
22

3-
## Next
3+
## 0.4.0 - 2025-03-10
44

55
* PR #52: feat: Scala Version Upgrade (2.12 -> 2.13).
66

README.md

Lines changed: 38 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -13,37 +13,63 @@ You can access the connector in two different ways:
1313
1. From
1414
our [Maven Central repository](https://repo1.maven.org/maven2/com/google/cloud/spark/bigtable).
1515
2. Through a public GCS bucket, located
16-
at `gs://spark-lib/bigtable/spark-bigtable_2.12-<version>.jar`.
16+
at `gs://spark-lib/bigtable/spark-bigtable_2.13-<version>.jar` or `gs://spark-lib/bigtable/spark-bigtable_2.12-<version>.jar`.
1717

1818
In Java and Scala applications, you can use different dependency management
1919
tools (e.g., Maven, sbt, or Gradle) to access the
20-
connector `com.google.cloud.spark.bigtable:spark-bigtable_2.12:<version>`(
21-
current `<version>` is `0.3.0`) and package it inside your application JAR using
22-
libraries such as Maven Shade Plugin. For PySpark applications, you can use
23-
the `--jars` flag to pass the GCS address of the connector when submitting it.
20+
connector `com.google.cloud.spark.bigtable:spark-bigtable_2.13:<version>` or
21+
`com.google.cloud.spark.bigtable:spark-bigtable_2.12:<version>` (current
22+
`<version>` is `0.4.0`) and package it inside your application JAR
23+
using libraries such as Maven Shade Plugin. For PySpark applications, you can
24+
use the `--jars` flag to pass the GCS address of the connector when submitting
25+
it.
2426

2527
For Maven, you can add the following snippet to your `pom.xml` file:
2628

2729
```xml
30+
<!-- If you are using scala 2.13 -->
31+
<dependency>
32+
<groupId>com.google.cloud.spark.bigtable</groupId>
33+
<artifactId>spark-bigtable_2.13</artifactId>
34+
<version>0.4.0</version>
35+
</dependency>
36+
```
37+
38+
```xml
39+
<!-- If you are using scala 2.12 -->
2840
<dependency>
2941
<groupId>com.google.cloud.spark.bigtable</groupId>
3042
<artifactId>spark-bigtable_2.12</artifactId>
31-
<version>0.3.0</version>
43+
<version>0.4.0</version>
3244
</dependency>
3345
```
3446

3547
For sbt, you can add the following to your `build.sbt` file:
3648

3749
```
38-
libraryDependencies += "com.google.cloud.spark.bigtable" % "spark-bigtable_2.12" % "0.3.0"
50+
// for scala 2.13
51+
libraryDependencies += "com.google.cloud.spark.bigtable" % "spark-bigtable_2.13" % "0.4.0"
52+
```
53+
54+
```
55+
// for scala 2.12
56+
libraryDependencies += "com.google.cloud.spark.bigtable" % "spark-bigtable_2.12" % "0.4.0"
3957
```
4058

4159
Finally, you can add the following to your `build.gradle` file when using
4260
Gradle:
4361

4462
```
63+
// for scala 2.13
64+
dependencies {
65+
implementation group: 'com.google.cloud.bigtable', name: 'spark-bigtable_2.13', version: '0.4.0'
66+
}
67+
```
68+
69+
```
70+
// for scala 2.12
4571
dependencies {
46-
implementation group: 'com.google.cloud.bigtable', name: 'spark-bigtable_2.12', version: '0.3.0'
72+
implementation group: 'com.google.cloud.bigtable', name: 'spark-bigtable_2.12', version: '0.4.0'
4773
}
4874
```
4975

@@ -92,10 +118,11 @@ You need the following depending on the environments you choose to use:
92118

93119
### Supported Spark versions
94120

95-
The connector supports the following Spark versions **with Scala 2.12**:
121+
The connector supports the following Spark versions:
96122

97123
| Scala version | Spark versions | Spark Application Languages |
98124
|---------------|-----------------------------------|---------------------------------------------------------|
125+
| 2.13 | 3.1.x, 3.2.x, 3.4.x, 3.5.x | Java, Scala, PySpark (`.py` files or Jupyter notebooks) |
99126
| 2.12 | 2.4.8, 3.1.x, 3.2.x, 3.4.x, 3.5.x | Java, Scala, PySpark (`.py` files or Jupyter notebooks) |
100127

101128
## Main features
@@ -306,7 +333,7 @@ objects to the following function:
306333

307334
```scala
308335
bigtableRDD.writeRDD(
309-
rdd: RDD[RowMutationEntry],
336+
rdd: RDD[RowMutationEntry],
310337
tableId: String,
311338
bigtableSparkConf: BigtableSparkConf
312339
)
@@ -326,7 +353,7 @@ corresponding to the options you want to set in your workflow. You can use the
326353
`BigtableSparkConfBuilder` class to create an instance of this class:
327354

328355
```scala
329-
val bigtableSparkConf: BigtableSparkConf =
356+
val bigtableSparkConf: BigtableSparkConf =
330357
new BigtableSparkConfBuilder()
331358
.setProjectId(someProjectId)
332359
.setInstanceId(someInstanceId)

pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
<groupId>com.google.cloud.spark.bigtable</groupId>
2222
<artifactId>spark-bigtable-connector</artifactId>
2323
<packaging>pom</packaging>
24-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
24+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
2525
<name>Spark Bigtable Connector Build Parent</name>
2626
<description>Parent project for all the Spark Bigtable Connector artifacts</description>
2727
<url>https://github.com/GoogleCloudDataproc/spark-bigtable-connector</url>

spark-bigtable-core-it/pom.xml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,14 @@
2121
<parent>
2222
<groupId>com.google.cloud.spark.bigtable</groupId>
2323
<artifactId>spark-bigtable-connector</artifactId>
24-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
24+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
2525
<relativePath>../</relativePath>
2626
</parent>
2727

2828
<groupId>com.google.cloud.spark.bigtable</groupId>
2929
<artifactId>spark-bigtable-core-it</artifactId>
3030
<name>Google Bigtable - Spark Connector Integration Tests</name>
31-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
31+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
3232

3333
<dependencies>
3434
<dependency>
@@ -52,7 +52,7 @@
5252
<dependency>
5353
<groupId>com.google.cloud.spark.bigtable</groupId>
5454
<artifactId>${connector.artifact.id}</artifactId>
55-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
55+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
5656
</dependency>
5757

5858
<dependency>

spark-bigtable-core/src/main/scala/com/google/cloud/spark/bigtable/BigtableDefaultSource.scala

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ import org.apache.spark.sql.{DataFrame, SQLContext, SaveMode, Row => SparkRow}
3131
import org.apache.yetus.audience.InterfaceAudience
3232

3333
object UserAgentInformation {
34-
val CONNECTOR_VERSION = "0.3.0" // ${NEXT_VERSION_FLAG}
34+
val CONNECTOR_VERSION = "0.4.0" // ${NEXT_VERSION_FLAG}
3535
val DATA_SOURCE_VERSION = "V1"
3636
val DATAFRAME_TEXT = "DF/" + DATA_SOURCE_VERSION
3737
val RDD_TEXT = "RDD/"

spark-bigtable_2.12/pom.xml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,14 @@
2121
<parent>
2222
<groupId>com.google.cloud.spark.bigtable</groupId>
2323
<artifactId>spark-bigtable-connector</artifactId>
24-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
24+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
2525
<relativePath>../</relativePath>
2626
</parent>
2727

2828
<groupId>com.google.cloud.spark.bigtable</groupId>
2929
<artifactId>spark-bigtable_2.12</artifactId>
3030
<name>Google Bigtable - Apache Spark Connector</name>
31-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
31+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
3232

3333
<properties>
3434
<scala.version>2.12.18</scala.version>

spark-bigtable_2.13/pom.xml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,14 +21,14 @@
2121
<parent>
2222
<groupId>com.google.cloud.spark.bigtable</groupId>
2323
<artifactId>spark-bigtable-connector</artifactId>
24-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
24+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
2525
<relativePath>../</relativePath>
2626
</parent>
2727

2828
<groupId>com.google.cloud.spark.bigtable</groupId>
2929
<artifactId>spark-bigtable_2.13</artifactId>
3030
<name>Google Bigtable - Apache Spark Connector for Scala 2.13</name>
31-
<version>0.3.0</version> <!-- ${NEXT_VERSION_FLAG} -->
31+
<version>0.4.0</version> <!-- ${NEXT_VERSION_FLAG} -->
3232

3333
<properties>
3434
<scala.version>2.13.14</scala.version>

0 commit comments

Comments
 (0)