You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
SearchQueriesPerSecond shows the average of the search queries per second (QPS) for the search service. It's common for queries to execute in milliseconds, so only queries that measure as seconds appear in a metric like QPS. The minimum is the lowest value for search queries per second that was registered during that minute. Maximum is the highest value. Average is the aggregate across the entire minute.
25
+
#### Search queries per second
26
+
27
+
This metric shows the average of the search queries per second (QPS) for the search service. It's common for queries to execute in milliseconds, so only queries that measure as seconds appear in a metric like QPS. The minimum is the lowest value for search queries per second that was registered during that minute. Maximum is the highest value. Average is the aggregate across the entire minute.
28
+
29
+
| Aggregation type | Description |
30
+
|------------------|-------------|
31
+
| Average | The average number of seconds within a minute during which query execution occurred.|
32
+
| Count | The number of metrics emitted to the log within the one-minute interval. |
33
+
| Maximum | The highest number of search queries per second registered during a minute. |
34
+
| Minimum | The lowest number of search queries per second registered during a minute. |
35
+
| Sum | The sum of all queries executed within the minute. |
26
36
27
37
For example, within one minute, you might have a pattern like this: one second of high load that is the maximum for SearchQueriesPerSecond, followed by 58 seconds of average load, and finally one second with only one query, which is the minimum.
28
38
39
+
Another example: if a node emits 100 metrics, where the value of each metric is 40, then "Count" is 100, "Sum" is 4000, "Average" is 40, and "Max" is 40.
40
+
41
+
#### Search latency
42
+
43
+
Search latency indicates how long a query takes to complete.
44
+
45
+
| Aggregation type | Latency |
46
+
|------------------|---------|
47
+
| Average | Average query duration in milliseconds. |
48
+
| Count | The number of metrics emitted to the log within the one-minute interval. |
49
+
| Maximum | Longest running query in the sample. |
50
+
| Minimum | Shortest running query in the sample. |
51
+
| Total | Total execution time of all queries in the sample, executing within the interval (one minute). |
52
+
53
+
#### Throttled search queries percentage
54
+
55
+
This metric refers to queries that are dropped instead of processed. Throttling occurs when the number of requests in execution exceed capacity. You might see an increase in throttled requests when a replica is taken out of rotation or during indexing. Both query and indexing requests are handled by the same set of resources.
56
+
57
+
The service determines whether to drop requests based on resource consumption. The percentage of resources consumed across memory, CPU, and disk IO are averaged over a period of time. If this percentage exceeds a threshold, all requests to the index are throttled until the volume of requests is reduced.
58
+
59
+
Depending on your client, a throttled request is indicated in these ways:
60
+
61
+
+ A service returns an error `"You are sending too many requests. Please try again later."`
62
+
+ A service returns a 503 error code indicating the service is currently unavailable.
63
+
+ If you're using the portal (for example, Search Explorer), the query is dropped silently and you need to select **Search** again.
64
+
65
+
To confirm throttled queries, use **Throttled search queries** metric. You can explore metrics in the portal or create an alert metric as described in this article. For queries that were dropped within the sampling interval, use *Total* to get the percentage of queries that didn't execute.
66
+
67
+
| Aggregation type | Throttling |
68
+
|------------------|-----------|
69
+
| Average | Percentage of queries dropped within the interval. |
70
+
| Count | The number of metrics emitted to the log within the one-minute interval. |
71
+
| Maximum | Percentage of queries dropped within the interval.|
72
+
| Minimum | Percentage of queries dropped within the interval. |
73
+
| Total | Percentage of queries dropped within the interval. |
74
+
75
+
For **Throttled Search Queries Percentage**, minimum, maximum, average and total, all have the same value: the percentage of search queries that were throttled, from the total number of search queries during one minute.
Azure AI Search has the following dimensions associated with the metrics that capture a count of documents or skills that were executed, "Document processed count" and "Skill execution invocation count".
79
+
Azure AI Search has dimensions associated with the following metrics that capture a count of documents or skills that were executed.
80
+
81
+
| Metric name | Description | Dimensions | Sample use cases |
82
+
|---|---|---|---|
83
+
|**Document processed count**| Shows the number of indexer processed documents. | Data source name, failed, index name, indexer name, skillset name | Can be referenced as a rough measure of throughput (number of documents processed by indexer over time) <br> - Set up to alert on failed documents |
84
+
|**Skill execution invocation count**| Shows the number of skill invocations. | Data source name, failed, index name, indexer name, skill name, skill type, skillset name | Reference to ensure skills are invoked as expected by comparing relative invocation numbers between skills and number of skill invocations to the number of documents. <br> - Set up to alert on failed skill invocations |
32
85
33
-
| Dimension Name| Description |
86
+
| Dimension name| Description |
34
87
| -------------- | ----------- |
35
88
|**DataSourceName**| A named data source connection used during indexer execution. Valid values are one of the [supported data source types](search-indexer-overview.md#supported-data-sources). |
36
89
|**Failed**| Indicates whether the instance failed. |
@@ -43,7 +96,7 @@ Azure AI Search has the following dimensions associated with the metrics that ca
In Azure AI Search, platform metrics measure query performance, indexing volume, and skillset invocation. For a list of available metrics for Azure AI Search, see [Azure AI Search monitoring data reference](monitor-azure-cognitive-search-data-reference.md#metrics).
27
27
28
+
To learn how to analyze query and index performance, see [Analyze performance in Azure AI Search](search-performance-analysis.md).
For the available resource log categories, their associated Log Analytics tables, and the logs schemas for Azure AI Search, see [Azure AI Search monitoring data reference](monitor-azure-cognitive-search-data-reference.md#resource-logs).
Copy file name to clipboardExpand all lines: articles/search/search-howto-monitor-indexers.md
+2-7Lines changed: 2 additions & 7 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -51,14 +51,9 @@ For more information about investigating indexer errors and warnings, see [Index
51
51
52
52
## Monitor with Azure Monitoring Metrics
53
53
54
-
Azure AI Search is a monitored resource in Azure Monitor, which means that you can use [Metrics Explorer](../azure-monitor/essentials/data-platform-metrics.md#metrics-explorer) to see basic metrics about the number of indexer-processed documents and skill invocations. These metrics can be used to monitor indexer progress and [set up alerts](../azure-monitor/alerts/alerts-metric-overview.md).
54
+
Azure AI Search is a monitored resource in Azure Monitor, which means that you can use [Metrics Explorer](../azure-monitor/essentials/data-platform-metrics.md#metrics-explorer) to see basic metrics about the number of indexer-processed documents and skill invocations. These metrics can be used to monitor indexer progress and [set up alerts](../azure-monitor/alerts/alerts-metric-overview.md).
55
55
56
-
Metric views can be filtered or split up by a set of predefined dimensions.
57
-
58
-
| Metric Name | Description | Dimensions | Sample use cases |
59
-
|---|---|---|---|
60
-
| Document processed count | Shows the number of indexer processed documents. | Data source name, failed, index name, indexer name, skillset name | <br> - Can be referenced as a rough measure of throughput (number of documents processed by indexer over time) <br> - Set up to alert on failed documents |
61
-
| Skill execution invocation count | Shows the number of skill invocations. | Data source name, failed, index name, indexer name, skill name, skill type, skillset name | <br> - Reference to ensure skills are invoked as expected by comparing relative invocation numbers between skills and number of skill invocations to the number of documents. <br> - Set up to alert on failed skill invocations |
56
+
Metric views can be filtered or split up by a set of predefined dimensions. To learn about the dimensions associated with the metrics *Document processed count* and *Skill execution invocation count*, see [Metric dimensions](monitor-azure-cognitive-search-data-reference.md#metric-dimensions).
62
57
63
58
The following screenshot shows the number of documents processed by indexers within a service over an hour, split up by indexer name.
Copy file name to clipboardExpand all lines: articles/search/search-manage.md
+1-8Lines changed: 1 addition & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -85,14 +85,7 @@ By default, a search service accepts authenticated and authorized requests over
85
85
86
86
We recommend Log Analytics Workspace for durable storage so that you can run system queries in the portal.
87
87
88
-
Internally, Microsoft collects telemetry data about your service and the platform. It's stored internally in Microsoft data centers and made globally available to Microsoft support engineers when you open a support ticket.
89
-
90
-
| Monitoring data | Retention |
91
-
|-----------------|-----------|
92
-
| Activity logs | 90 days on a rolling schedule |
93
-
| Platform metrics | 93 days on a rolling schedule, except that portal visualization is limited to a 30 day window |
94
-
| Resource logs | User-managed |
95
-
| Telemetry | One and a half years |
88
+
Internally, Microsoft collects telemetry data about your service and the platform. To learn more about data retention, see [Retention of metrics](/azure/azure-monitor/essentials/data-platform-metrics#retention-of-metrics).
96
89
97
90
> [!NOTE]
98
91
> See the ["Data residency"](search-security-overview.md#data-residency) section of the security overview article for more information about data location and privacy.
Copy file name to clipboardExpand all lines: articles/search/search-monitor-queries.md
+5-45Lines changed: 5 additions & 45 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -32,63 +32,23 @@ Conditions that maximize the integrity of data measurement include:
32
32
33
33
Volume is measured as **Search Queries Per Second** (QPS), a built-in metric that can be reported as an average, count, minimum, or maximum values of queries that execute within a one-minute window. One-minute intervals (TimeGrain = "PT1M") for metrics is fixed within the system.
34
34
35
-
It's common for queries to execute in milliseconds, so only queries that measure as seconds appear in metrics.
36
-
37
-
| Aggregation Type | Description |
38
-
|------------------|-------------|
39
-
| Average | The average number of seconds within a minute during which query execution occurred.|
40
-
| Count | The number of metrics emitted to the log within the one-minute interval. |
41
-
| Maximum | The highest number of search queries per second registered during a minute. |
42
-
| Minimum | The lowest number of search queries per second registered during a minute. |
43
-
| Sum | The sum of all queries executed within the minute. |
44
-
45
-
For example, within one minute, you might have a pattern like this: one second of high load that is the maximum for SearchQueriesPerSecond, followed by 58 seconds of average load, and finally one second with only one query, which is the minimum.
46
-
47
-
Another example: if a node emits 100 metrics, where the value of each metric is 40, then "Count" is 100, "Sum" is 4000, "Average" is 40, and "Max" is 40.
35
+
To learn more about the SearchQueriesPerSecond metric, see [Search queries per second](monitor-azure-cognitive-search-data-reference.md#search-queries-per-second).
48
36
49
37
## Query performance
50
38
51
-
Service-wide, query performance is measured as search latency (how long a query takes to complete) and throttled queries that were dropped as a result of resource contention.
39
+
Service-wide, query performance is measured as *search latency*and *throttled queries*.
52
40
53
41
### Search latency
54
42
55
-
| Aggregation Type | Latency |
56
-
|------------------|---------|
57
-
| Average | Average query duration in milliseconds. |
58
-
| Count | The number of metrics emitted to the log within the one-minute interval. |
59
-
| Maximum | Longest running query in the sample. |
60
-
| Minimum | Shortest running query in the sample. |
61
-
| Total | Total execution time of all queries in the sample, executing within the interval (one minute). |
43
+
Search latency indicates how long a query takes to complete. To learn more about the SearchLatency metric, see [Search latency](monitor-azure-cognitive-search-data-reference.md#search-latency).
62
44
63
45
Consider the following example of **Search Latency** metrics: 86 queries were sampled, with an average duration of 23.26 milliseconds. A minimum of 0 indicates some queries were dropped. The longest running query took 1000 milliseconds to complete. Total execution time was 2 seconds.
Throttled queries refers to queries that are dropped instead of processed. In most cases, throttling is a normal part of running the service. It isn't necessarily an indication that there's something wrong.
70
-
71
-
Throttling occurs when the number of requests in execution exceed capacity. You might see an increase in throttled requests when a replica is taken out of rotation or during indexing. Both query and indexing requests are handled by the same set of resources.
72
-
73
-
The service determines whether to drop requests based on resource consumption. The percentage of resources consumed across memory, CPU, and disk IO are averaged over a period of time. If this percentage exceeds a threshold, all requests to the index are throttled until the volume of requests is reduced.
74
-
75
-
Depending on your client, a throttled request is indicated in these ways:
76
-
77
-
+ A service returns an error `"You are sending too many requests. Please try again later."`
78
-
+ A service returns a 503 error code indicating the service is currently unavailable.
79
-
+ If you're using the portal (for example, Search Explorer), the query is dropped silently and you need to select **Search** again.
80
-
81
-
To confirm throttled queries, use **Throttled search queries** metric. You can explore metrics in the portal or create an alert metric as described in this article. For queries that were dropped within the sampling interval, use *Total* to get the percentage of queries that didn't execute.
82
-
83
-
| Aggregation Type | Throttling |
84
-
|------------------|-----------|
85
-
| Average | Percentage of queries dropped within the interval. |
86
-
| Count | The number of metrics emitted to the log within the one-minute interval. |
87
-
| Maximum | Percentage of queries dropped within the interval.|
88
-
| Minimum | Percentage of queries dropped within the interval. |
89
-
| Total | Percentage of queries dropped within the interval. |
90
-
91
-
For **Throttled Search Queries Percentage**, minimum, maximum, average and total, all have the same value: the percentage of search queries that were throttled, from the total number of search queries during one minute.
51
+
Throttled queries refers to queries that are dropped instead of processed. In most cases, throttling is a normal part of running the service. It isn't necessarily an indication that there's something wrong. To learn more about the ThrottledSearchQueriesPercentage metric, see [Throttled search queries percentage](monitor-azure-cognitive-search-data-reference.md#throttled-search-queries-percentage).
92
52
93
53
In the following screenshot, the first number is the count (or number of metrics sent to the log). Other aggregations, which appear at the top or when hovering over the metric, include average, maximum, and total. In this sample, no requests were dropped.
94
54
@@ -155,7 +115,7 @@ Add the duration column to get the numbers for all queries, not just those that
155
115
156
116
## Create a metric alert
157
117
158
-
A metric alert establishes a threshold for sending a notification or triggering a corrective action that you define in advance. You can create alerts related to query execution, but you can also create them for resource health, search service configuration changes, skill execution, and document processing (indexing).
118
+
A [metric alert](/azure/azure-monitor/alerts/alerts-types#metric-alerts) establishes a threshold for sending a notification or triggering a corrective action that you define in advance. You can create alerts related to query execution, but you can also create them for resource health, search service configuration changes, skill execution, and document processing (indexing).
159
119
160
120
All thresholds are user-defined, so you should have an idea of what activity level should trigger the alert.
0 commit comments