Skip to content

Commit 4b69b59

Browse files
authored
Merge pull request #227072 from ekote/hyperspace-not-supported-in33
hyperspace supported in 2.4 3.1 and 3.2
2 parents 20e88cc + fb9a819 commit 4b69b59

File tree

3 files changed

+13
-10
lines changed

3 files changed

+13
-10
lines changed

articles/synapse-analytics/spark/apache-spark-performance-hyperspace.md

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -2,13 +2,13 @@
22
title: Hyperspace indexes for Apache Spark
33
description: Performance optimization for Apache Spark using Hyperspace indexes
44
services: synapse-analytics
5-
author: juluczni
6-
ms.author: juluczni
5+
author: ekote
6+
ms.author: eskot
77
ms.service: synapse-analytics
88
ms.topic: conceptual
99
ms.subservice: spark
10-
ms.date: 02/15/2022
11-
ms.reviewer: euang
10+
ms.date: 02/10/2023
11+
ms.reviewer: eskot
1212
zone_pivot_groups: programming-languages-spark-all-minus-sql-r
1313
---
1414

@@ -29,6 +29,9 @@ This document is also available in notebook form, for [Python](https://github.co
2929

3030
## Setup
3131

32+
>[!Note]
33+
> Hyperspace is supported in Azure Synapse Runtime for Apache Spark 2.4 (EOLA), Azure Synapse Runtime for Apache Spark 3.1 (EOLA), and Azure Synapse Runtime for Apache Spark 3.2 (GA). However, it should be noted that Hyperspace is not supported in Azure Synapse Runtime for Apache Spark 3.3.
34+
3235
To begin with, start a new Spark session. Since this document is a tutorial merely to illustrate what Hyperspace can offer, you will make a configuration change that allows us to highlight what Hyperspace is doing on small datasets.
3336

3437
By default, Spark uses broadcast join to optimize join queries when the data size for one side of join is small (which is the case for the sample data we use in this tutorial). Therefore, we disable broadcast joins so that later when we run join queries, Spark uses sort-merge join. This is mainly to show how Hyperspace indexes would be used at scale for accelerating join queries.

articles/synapse-analytics/spark/data-sources/apache-spark-kusto-connector.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,7 @@ author: midesa
1212
---
1313

1414
# Azure Data Explorer (Kusto) connector for Apache Spark
15-
The Azure Data Explorer (Kusto) connector for Apache Spark is designed to efficiently transfer data between Kusto clusters and Spark. This connector is available in Python, Java, and .NET. It is built in to the Azure Synapse Apache Spark 2.4 runtime.
15+
The Azure Data Explorer (Kusto) connector for Apache Spark is designed to efficiently transfer data between Kusto clusters and Spark. This connector is available in Python, Java, and .NET. It is built in to the Azure Synapse Apache Spark 2.4 runtime (EOLA).
1616

1717
## Authentication
1818
When using Azure Synapse Notebooks or Apache Spark job definitions, the authentication between systems is made seamless with the linked service. The Token Service connects with Azure Active Directory to obtain security tokens for use when accessing the Kusto cluster.
@@ -23,9 +23,9 @@ For Azure Synapse Pipelines, the authentication will use the service principal n
2323
- [Connect to Azure Data Explorer](../../quickstart-connect-azure-data-explorer.md): You will need to set up a Linked Service to connect to an existing Kusto cluster.
2424

2525
## Limitations
26-
- The Azure Data Explorer (Kusto) connector is currently only supported on the Azure Synapse Apache Spark 2.4 runtime.
26+
- The Azure Data Explorer (Kusto) connector is currently only supported on the Azure Synapse Apache Spark 2.4 runtime (EOLA).
2727
- The Azure Data Explorer linked service can only be configured with the Service Principal Name.
28-
- Within Azure Synapse Notebooks or Apache Spark Job Definitions, the Azure Data Explorer connector will use AAD pass-through to connect to the Kusto Cluster.
28+
- Within Azure Synapse Notebooks or Apache Spark Job Definitions, the Azure Data Explorer connector will use Azure AD pass-through to connect to the Kusto Cluster.
2929

3030

3131
## Use the Azure Data Explorer (Kusto) connector

articles/synapse-analytics/toc.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -724,11 +724,11 @@ items:
724724
href: ./spark/apache-spark-version-support.md
725725
- name: Azure Synapse Apache Spark 3.3 runtime (Preview)
726726
href: ./spark/apache-spark-33-runtime.md
727-
- name: Azure Synapse Apache Spark 3.2 runtime
727+
- name: Azure Synapse Apache Spark 3.2 runtime (GA)
728728
href: ./spark/apache-spark-32-runtime.md
729-
- name: Azure Synapse Apache Spark 3.1 runtime
729+
- name: Azure Synapse Apache Spark 3.1 runtime (EOLA)
730730
href: ./spark/apache-spark-3-runtime.md
731-
- name: Azure Synapse Apache Spark 2.4 runtime
731+
- name: Azure Synapse Apache Spark 2.4 runtime (EOLA)
732732
href: ./spark/apache-spark-24-runtime.md
733733
- name: Lifecycle and Supportability
734734
href: ./spark/runtime-for-apache-spark-lifecycle-and-supportability.md

0 commit comments

Comments
 (0)