Skip to content

Commit e374a97

Browse files
authored
Merge pull request #187353 from FabianMeiswinkel/users/fabianm/avoidstaleversionssparkoverview
Removing info for Spark overview sites that will be always stale/outdated
2 parents c75187a + 0df297c commit e374a97

File tree

2 files changed

+13
-77
lines changed

2 files changed

+13
-77
lines changed

articles/cosmos-db/sql/sql-api-sdk-java-spark-v3.md

Lines changed: 12 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -40,34 +40,23 @@ If you have any feedback or ideas on how to improve your experience create an is
4040

4141
## Documentation links
4242

43-
* [Getting started](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-spark_3_2-12/docs/quick-start.md)
44-
* [Catalog API](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-spark_3_2-12/docs/catalog-api.md)
45-
* [Configuration Parameter Reference](https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/cosmos/azure-cosmos-spark_3_2-12/docs/configuration-reference.md)
43+
* [Getting started](https://aka.ms/azure-cosmos-spark-3-quickstart)
44+
* [Catalog API](https://aka.ms/azure-cosmos-spark-3-catalog-api)
45+
* [Configuration Parameter Reference](https://aka.ms/azure-cosmos-spark-3-config)
46+
* [End-to-end sample notebook "New York City Taxi data"](https://aka.ms/azure-cosmos-spark-3-sample-nyc-taxi-data)
47+
* [Migration from Spark 2.4 to Spark 3.*](https://aka.ms/azure-cosmos-spark-3-migration)
4648

4749
## Version compatibility
50+
* [Version compatibility for Spark 3.1](https://aka.ms/azure-cosmos-spark-3-1-version-compatibility)
51+
* [Version compatibility for Spark 3.2](https://aka.ms/azure-cosmos-spark-3-2-version-compatibility)
4852

49-
| Connector | Minimum Spark version | Minimum Java version | Supported Scala versions | Supported Databricks runtimes |
50-
| ------------- | --------------------- | -------------------- | ----------------------- | ----------------------------- |
51-
| 4.4.0 | 3.1.1 | 8 | 2.12 | 8.\*, 9.\* |
52-
| 4.3.1 | 3.1.1 | 8 | 2.12 | 8.\*, 9.\* |
53-
| 4.3.0 | 3.1.1 | 8 | 2.12 | 8.\*, 9.\* |
54-
| 4.2.0 | 3.1.1 | 8 | 2.12 | 8.\* |
55-
| 4.1.0 | 3.1.1 | 8 | 2.12 | 8.\* |
56-
| 4.0.0 | 3.1.1 | 8 | 2.12 | 8.\* |
57-
| 4.0.0-beta.3 | 3.1.1 | 8 | 2.12 | 8.\* |
58-
| 4.0.0-beta.2 | 3.1.1 | 8 | 2.12 | 8.\* |
59-
| 4.0.0-beta.1 | 3.1.1 | 8 | 2.12 | 8.\* |
53+
## Release notes
54+
* [Release notes for Spark 3.1](https://aka.ms/azure-cosmos-spark-3-1-changelog)
55+
* [Release notes for Spark 3.2](https://aka.ms/azure-cosmos-spark-3-2-changelog)
6056

6157
## Download
62-
63-
You can use the maven coordinate of the jar to auto install the Spark Connector to your Databricks Runtime 8 from Maven:
64-
`com.azure.cosmos.spark:azure-cosmos-spark_3-1_2-12:4.4.0`
65-
66-
You can also integrate against Cosmos DB Spark Connector in your SBT project:
67-
68-
```scala
69-
libraryDependencies += "com.azure.cosmos.spark" % "azure-cosmos-spark_3-1_2-12" % "4.4.0"
70-
```
58+
* [Download of Cosmos DB Spark connectro for Spark 3.1](https://aka.ms/azure-cosmos-spark-3-1-changelog)
59+
* [Download of Cosmos DB Spark connectro for Spark 3.2](https://aka.ms/azure-cosmos-spark-3-2-download)
7160

7261
Azure Cosmos DB Spark connector is available on [Maven Central Repo](https://search.maven.org/search?q=g:com.azure.cosmos.spark).
7362

articles/cosmos-db/sql/sql-api-sdk-java-spark.md

Lines changed: 1 addition & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -60,60 +60,7 @@ You can use the connector with [Azure Databricks](https://azure.microsoft.com/se
6060
|**Get started** | [Accelerate big data analytics by using the Apache Spark to Azure Cosmos DB connector](./create-sql-api-spark.md) <br> [Use Apache Spark Structured Streaming with Apache Kafka and Azure Cosmos DB](../../hdinsight/apache-kafka-spark-structured-streaming-cosmosdb.md?toc=/azure/cosmos-db/toc.json&bc=/azure/cosmos-db/breadcrumb/toc.json) |
6161

6262
## Release history
63-
64-
### 3.3.0
65-
#### New features
66-
- Adds a new config option, `changefeedstartfromdatetime`, which can be used to specify the start time for when the changefeed should be processed. For more information, see [Config options](https://github.com/Azure/azure-cosmosdb-spark/wiki/Configuration-references).
67-
68-
### 3.2.0
69-
#### Key bug fixes
70-
- Fixes a regression that caused excessive memory consumption on the executors for large result sets (for example, with millions of rows), ultimately resulting in the error `java.lang.OutOfMemoryError: GC overhead limit exceeded`.
71-
72-
### 3.1.1
73-
#### Key bug fixes
74-
* Fixes a streaming checkpoint edge case in which the `ID` contains the pipe character (|) with the `ChangeFeedMaxPagesPerBatch` config applied.
75-
76-
### 3.1.0
77-
#### New features
78-
* Adds support for bulk updates when nested partition keys are used.
79-
* Adds support for Decimal and Float data types during writes to Azure Cosmos DB.
80-
* Adds support for Timestamp types when they're using Long (Unix epoch) as a value.
81-
82-
### 3.0.8
83-
#### Key bug fixes
84-
* Fixes typecast exception that occurs when the `WriteThroughputBudget` config is used.
85-
86-
### 3.0.7
87-
#### New features
88-
* Adds error information for bulk failures to exception and log.
89-
90-
### 3.0.6
91-
#### Key bug fixes
92-
* Fixes streaming checkpoint issues.
93-
94-
### 3.0.5
95-
#### Key bug fixes
96-
* To reduce noise, fixes log level of a message left unintentionally with level ERROR.
97-
98-
### 3.0.4
99-
#### Key bug fixes
100-
* Fixes a bug in structured streaming during partition splits. The bug could result in some missing change feed records or Null exceptions for checkpoint writes.
101-
102-
### 3.0.3
103-
#### Key bug fixes
104-
* Fixes a bug that causes a custom schema provided for readStream to be ignored.
105-
106-
### 3.0.2
107-
#### Key bug fixes
108-
* Fixes a regression (unshaded JAR includes all shaded dependencies) that increases build time by 50 percent.
109-
110-
### 3.0.1
111-
#### Key bug fixes
112-
* Fixes a dependency problem that causes Direct Transport over TCP to fail with RequestTimeoutException.
113-
114-
### 3.0.0
115-
#### New features
116-
* Improves connection management and connection pooling to reduce the number of metadata calls.
63+
* [Release notes](https://github.com/Azure/azure-cosmosdb-spark/blob/2.4/CHANGELOG.md)]
11764

11865
## FAQ
11966
[!INCLUDE [cosmos-db-sdk-faq](../includes/cosmos-db-sdk-faq.md)]

0 commit comments

Comments
 (0)