You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/synapse-analytics/machine-learning/tutorial-text-analytics-use-mmlspark.md
+23-14Lines changed: 23 additions & 14 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,14 +4,15 @@ description: Learn how to use text analytics in Azure Synapse Analytics.
4
4
ms.service: azure-synapse-analytics
5
5
ms.subservice: machine-learning
6
6
ms.topic: tutorial
7
-
ms.date: 11/02/2021
7
+
ms.date: 11/19/2024
8
8
author: ruixinxu
9
9
ms.author: ruxu
10
+
# customer intent: As a Synapse Analytics user, I want to be able to analyze my text using Azure AI services.
10
11
---
11
12
12
13
# Tutorial: Text Analytics with Azure AI services
13
14
14
-
[Text Analytics](/azure/ai-services/language-service/) is an [Azure AI services](/azure/ai-services/) that enables you to perform text mining and text analysis with Natural Language Processing (NLP) features. In this tutorial, you'll learn how to use [Text Analytics](/azure/ai-services/language-service/) to analyze unstructured text on Azure Synapse Analytics.
15
+
In this tutorial, you learn how to use [Text Analytics](/azure/ai-services/language-service/)to analyze unstructured text on Azure Synapse Analytics. [Text Analytics](/azure/ai-services/language-service/)is an [Azure AI services](/azure/ai-services/) that enables you to perform text mining and text analysis with Natural Language Processing (NLP) features.
15
16
16
17
This tutorial demonstrates using text analytics with [SynapseML](https://github.com/microsoft/SynapseML) to:
17
18
@@ -29,34 +30,35 @@ If you don't have an Azure subscription, [create a free account before you begin
29
30
30
31
-[Azure Synapse Analytics workspace](../get-started-create-workspace.md) with an Azure Data Lake Storage Gen2 storage account configured as the default storage. You need to be the *Storage Blob Data Contributor* of the Data Lake Storage Gen2 file system that you work with.
31
32
- Spark pool in your Azure Synapse Analytics workspace. For details, see [Create a Spark pool in Azure Synapse](../quickstart-create-sql-pool-studio.md).
32
-
- Pre-configuration steps described in the tutorial [Configure Azure AI services in Azure Synapse](tutorial-configure-cognitive-services-synapse.md).
33
-
33
+
- Preconfiguration steps described in the tutorial [Configure Azure AI services in Azure Synapse](tutorial-configure-cognitive-services-synapse.md).
34
34
35
35
## Get started
36
-
Open Synapse Studio and create a new notebook. To get started, import [SynapseML](https://github.com/microsoft/SynapseML).
36
+
37
+
Open Synapse Studio and create a new notebook. To get started, import [SynapseML](https://github.com/microsoft/SynapseML).
37
38
38
39
```python
39
40
import synapse.ml
40
-
from synapse.ml.cognitiveimport*
41
+
from synapse.ml.servicesimport*
41
42
from pyspark.sql.functions import col
42
43
```
43
44
44
45
## Configure text analytics
45
46
46
-
Use the linked text analytics you configured in the [pre-configuration steps](tutorial-configure-cognitive-services-synapse.md) .
47
+
Use the linked text analytics you configured in the [preconfiguration steps](tutorial-configure-cognitive-services-synapse.md).
47
48
48
49
```python
49
-
ai_service_name="<Your linked service for text analytics>"
50
+
linked_service_name="<Your linked service for text analytics>"
50
51
```
51
52
52
53
## Text Sentiment
53
-
The Text Sentiment Analysis provides a way for detecting the sentiment labels (such as "negative", "neutral" and "positive") and confidence scores at the sentence and document-level. See the [Supported languages in Text Analytics API](/azure/ai-services/language-service/language-detection/overview?tabs=sentiment-analysis) for the list of enabled languages.
54
+
55
+
The Text Sentiment Analysis provides a way for detecting the sentiment labels (such as "negative", "neutral", and "positive") and confidence scores at the sentence and document-level. See the [Supported languages in Text Analytics API](/azure/ai-services/language-service/language-detection/overview?tabs=sentiment-analysis) for the list of enabled languages.
54
56
55
57
```python
56
58
57
59
# Create a dataframe that's tied to it's column names
58
60
df = spark.createDataFrame([
59
-
("I am so happy today, its sunny!", "en-US"),
61
+
("I am so happy today, it's sunny!", "en-US"),
60
62
("I am frustrated by this rush hour traffic", "en-US"),
61
63
("The Azure AI services on spark aint bad", "en-US"),
62
64
], ["text", "language"])
@@ -77,13 +79,14 @@ display(results
77
79
.select("text", "sentiment"))
78
80
79
81
```
82
+
80
83
### Expected results
81
84
82
85
|text|sentiment|
83
86
|---|---|
84
-
|I am so happy today, its sunny!|positive|
85
-
|I am frustrated by this rush hour traffic|negative|
86
-
|The Azure AI services on spark aint bad|positive|
87
+
|I'm so happy today, it's sunny!|positive|
88
+
|I'm frustrated by this rush hour traffic|negative|

191
196
192
197
---
193
198
194
199
## Personally Identifiable Information (PII) V3.1
200
+
195
201
The PII feature is part of NER and it can identify and redact sensitive entities in text that are associated with an individual person such as: phone number, email address, mailing address, passport number. See the [Supported languages in Text Analytics API](/azure/ai-services/language-service/language-detection/overview?tabs=pii) for the list of enabled languages.

214
222
215
223
---
216
224
217
225
## Clean up resources
226
+
218
227
To ensure the Spark instance is shut down, end any connected sessions(notebooks). The pool shuts down when the **idle time** specified in the Apache Spark pool is reached. You can also select **stop session** from the status bar at the upper right of the notebook.
219
228
220
229

221
230
222
-
## Next steps
231
+
## Related content
223
232
224
233
*[Check out Synapse sample notebooks](https://github.com/Azure-Samples/Synapse/tree/main/MachineLearning)
0 commit comments