Skip to content

Commit 8520671

Browse files
Merge pull request #233150 from seesharprun/cosmos-revamp-monitor-resource-logs
Cosmos DB | Revamp Azure CLI instructions to enable full-text query in logs.
2 parents 68bdfd9 + 23628db commit 8520671

File tree

1 file changed

+28
-12
lines changed

1 file changed

+28
-12
lines changed

articles/cosmos-db/monitor-resource-logs.md

Lines changed: 28 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -5,16 +5,16 @@ author: seesharprun
55
ms.author: sidandrews
66
ms.reviewer: esarroyo
77
ms.service: cosmos-db
8-
ms.custom: ignite-2022
98
ms.topic: how-to
10-
ms.date: 11/08/2022
9+
ms.date: 04/23/2023
10+
ms.custom: ignite-2022
1111
---
1212

1313
# Monitor Azure Cosmos DB data by using diagnostic settings in Azure
1414

1515
[!INCLUDE[NoSQL, MongoDB, Cassandra, Gremlin, Table](includes/appliesto-nosql-mongodb-cassandra-gremlin-table.md)]
1616

17-
Diagnostic settings in Azure are used to collect resource logs. Azure resource Logs are emitted by a resource and provide rich, frequent data about the operation of that resource. These logs are captured per request and they're also referred to as "data plane logs". Some examples of the data plane operations include delete, insert, and readFeed. The content of these logs varies by resource type.
17+
Diagnostic settings in Azure are used to collect resource logs. Resources emit Azure resource Logs and provide rich, frequent data about the operation of that resource. These logs are captured per request and they're also referred to as "data plane logs". Some examples of the data plane operations include delete, insert, and readFeed. The content of these logs varies by resource type.
1818

1919
Platform metrics and the Activity logs are collected automatically, whereas you must create a diagnostic setting to collect resource logs or forward them outside of Azure Monitor. You can turn on diagnostic setting for Azure Cosmos DB accounts and send resource logs to the following sources:
2020

@@ -42,10 +42,10 @@ Platform metrics and the Activity logs are collected automatically, whereas you
4242
| --- | --- | --- | --- |
4343
| **DataPlaneRequests** | All APIs | Logs back-end requests as data plane operations, which are requests executed to create, update, delete or retrieve data within the account. | `Requestcharge`, `statusCode`, `clientIPaddress`, `partitionID`, `resourceTokenPermissionId` `resourceTokenPermissionMode` |
4444
| **MongoRequests** | Mongo | Logs user-initiated requests from the front end to serve requests to Azure Cosmos DB for MongoDB. When you enable this category, make sure to disable DataPlaneRequests. | `Requestcharge`, `opCode`, `retryCount`, `piiCommandText` |
45-
| **CassandraRequests** | Cassandra | Logs user-initiated requests from the front end to serve requests to Azure Cosmos DB for Cassandra. When you enable this category, make sure to disable DataPlaneRequests. | `operationName`, `requestCharge`, `piiCommandText` |
45+
| **Ca/ssandraRequests** | Cassandra | Logs user-initiated requests from the front end to serve requests to Azure Cosmos DB for Cassandra. When you enable this category, make sure to disable DataPlaneRequests. | `operationName`, `requestCharge`, `piiCommandText` |
4646
| **GremlinRequests** | Gremlin | Logs user-initiated requests from the front end to serve requests to Azure Cosmos DB for Gremlin. When you enable this category, make sure to disable DataPlaneRequests. | `operationName`, `requestCharge`, `piiCommandText`, `retriedDueToRateLimiting` |
4747
| **QueryRuntimeStatistics** | NoSQL | This table details query operations executed against an API for NoSQL account. By default, the query text and its parameters are obfuscated to avoid logging personal data with full text query logging available by request. | `databasename`, `partitionkeyrangeid`, `querytext` |
48-
| **PartitionKeyStatistics** | All APIs | Logs the statistics of logical partition keys by representing the estimated storage size (KB) of the partition keys. This table is useful when troubleshooting storage skews. This PartitionKeyStatistics log is only emitted if the following conditions are true: 1. At least 1% of the documents in the physical partition have same logical partition key. 2. Out of all the keys in the physical partition, the top three keys with largest storage size are captured by the PartitionKeyStatistics log. </li></ul> If the previous conditions aren't met, the partition key statistics data isn't available. It's okay if the above conditions aren't met for your account, which typically indicates you have no logical partition storage skew. **Note**: The estimated size of the partition keys is calculated using a sampling approach that assumes the documents in the physical partition are roughly the same size. If the document sizes aren't uniform in the physical partition, the estimated partition key size may not be accurate. | `subscriptionId`, `regionName`, `partitionKey`, `sizeKB` |
48+
| **PartitionKeyStatistics** | All APIs | Logs the statistics of logical partition keys by representing the estimated storage size (KB) of the partition keys. This table is useful when troubleshooting storage skews. This PartitionKeyStatistics log is only emitted if the following conditions are true: 1. At least 1% of the documents in the physical partition have same logical partition key. 2. Out of all the keys in the physical partition, the PartitionKeyStatistics log captures the top three keys with largest storage size. </li></ul> If the previous conditions aren't met, the partition key statistics data isn't available. It's okay if the above conditions aren't met for your account, which typically indicates you have no logical partition storage skew. **Note**: The estimated size of the partition keys is calculated using a sampling approach that assumes the documents in the physical partition are roughly the same size. If the document sizes aren't uniform in the physical partition, the estimated partition key size may not be accurate. | `subscriptionId`, `regionName`, `partitionKey`, `sizeKB` |
4949
| **PartitionKeyRUConsumption** | API for NoSQL | Logs the aggregated per-second RU/s consumption of partition keys. This table is useful for troubleshooting hot partitions. Currently, Azure Cosmos DB reports partition keys for API for NoSQL accounts only and for point read/write and stored procedure operations. | `subscriptionId`, `regionName`, `partitionKey`, `requestCharge`, `partitionKeyRangeId` |
5050
| **ControlPlaneRequests** | All APIs | Logs details on control plane operations, which include, creating an account, adding or removing a region, updating account replication settings etc. | `operationName`, `httpstatusCode`, `httpMethod`, `region` |
5151
| **TableApiRequests** | API for Table | Logs user-initiated requests from the front end to serve requests to Azure Cosmos DB for Table. When you enable this category, make sure to disable DataPlaneRequests. | `operationName`, `requestCharge`, `piiCommandText` |
@@ -191,15 +191,15 @@ Use the [Azure Monitor REST API](/rest/api/monitor/diagnosticsettings/createorup
191191
> [!NOTE]
192192
> Enabling this feature may result in additional logging costs, for pricing details visit [Azure Monitor pricing](https://azure.microsoft.com/pricing/details/monitor/). It is recommended to disable this feature after troubleshooting.
193193
194-
Azure Cosmos DB provides advanced logging for detailed troubleshooting. By enabling full-text query, you’ll be able to view the deobfuscated query for all requests within your Azure Cosmos DB account. You’ll also give permission for Azure Cosmos DB to access and surface this data in your logs.
194+
Azure Cosmos DB provides advanced logging for detailed troubleshooting. By enabling full-text query, you're able to view the deobfuscated query for all requests within your Azure Cosmos DB account. You also give permission for Azure Cosmos DB to access and surface this data in your logs.
195195
196196
### [Azure portal](#tab/azure-portal)
197197
198198
1. To enable this feature, navigate to the `Features` page in your Azure Cosmos DB account.
199199
200200
:::image type="content" source="media/monitor/full-text-query-features.png" lightbox="media/monitor/full-text-query-features.png" alt-text="Screenshot of navigation to the Features page.":::
201201
202-
2. Select `Enable`, this setting will then be applied within the next few minutes. All newly ingested logs will have the full-text or PIICommand text for each request.
202+
2. Select `Enable`. This setting is applied within a few minutes. All newly ingested logs have the full-text or PIICommand text for each request.
203203
204204
:::image type="content" source="media/monitor/select-enable-full-text.png" alt-text="Screenshot of full-text being enabled.":::
205205
@@ -238,13 +238,24 @@ Azure Cosmos DB provides advanced logging for detailed troubleshooting. By enabl
238238
)
239239
```
240240
241-
1. Check if full-text query is already enabled by querying the resource using the REST API and [`az rest`](/cli/azure/reference-index#az-rest) with an HTTP `GET` verb.
241+
1. Query the resource using the REST API and [`az rest`](/cli/azure/reference-index#az-rest) with an HTTP `GET` verb to check if full-text query is already enabled.
242242
243243
```azurecli
244244
az rest \
245245
--method GET \
246246
--uri "https://management.azure.com/$uri/?api-version=2021-05-01-preview" \
247-
--query "{AccountName:name, FullTextQueryEnabled:properties.diagnosticLogSettings.enableFullTextQuery}"
247+
--query "{accountName:name,fullTextQuery:{state:properties.diagnosticLogSettings.enableFullTextQuery}}"
248+
```
249+
250+
If full-text query isn't enabled, the output would be similar to this example.
251+
252+
```json
253+
{
254+
"accountName": "<account-name>",
255+
"fullTextQuery": {
256+
"state": "None"
257+
}
258+
}
248259
```
249260
250261
1. If full-text query isn't already enabled, enable it using `az rest` again with an HTTP `PATCH` verb and a JSON payload.
@@ -256,21 +267,26 @@ Azure Cosmos DB provides advanced logging for detailed troubleshooting. By enabl
256267
--body '{"properties": {"diagnosticLogSettings": {"enableFullTextQuery": "True"}}}'
257268
```
258269
270+
> [!NOTE]
271+
> If you are using Azure CLI within a PowerShell prompt, you will need to escape the double-quotes using a backslash (`\`) character.
272+
259273
1. Wait a few minutes for the operation to complete. Check the status of full-text query by using `az rest` again.
260274
261275
```azurecli
262276
az rest \
263277
--method GET \
264278
--uri "https://management.azure.com/$uri/?api-version=2021-05-01-preview" \
265-
--query "{AccountName:name, FullTextQueryEnabled:properties.diagnosticLogSettings.enableFullTextQuery}"
279+
--query "{accountName:name,fullTextQuery:{state:properties.diagnosticLogSettings.enableFullTextQuery}}"
266280
```
267281
268282
The output should be similar to this example.
269283
270284
```json
271285
{
272-
"AccountName": "<account-name>",
273-
"FullTextQueryEnabled": "True"
286+
"accountName": "<account-name>",
287+
"fullTextQuery": {
288+
"state": "True"
289+
}
274290
}
275291
```
276292

0 commit comments

Comments
 (0)