|
| 1 | +--- |
| 2 | +title: Add or delete tables and columns in Azure Monitor Logs |
| 3 | +description: Create a table with a custom schema to collect logs from any data source. |
| 4 | +author: guywi-ms |
| 5 | +ms.author: guywild |
| 6 | +ms.reviewer: adi.biran |
| 7 | +ms.service: azure-monitor |
| 8 | +ms.topic: how-to |
| 9 | +ms.date: 11/09/2022 |
| 10 | + |
| 11 | +# Customer intent: As a Log Analytics workspace administrator, I want to create a table with a custom schema to store logs from an Azure or non-Azure data source. |
| 12 | +--- |
| 13 | + |
| 14 | +# Add or delete tables and columns in Azure Monitor Logs |
| 15 | + |
| 16 | +[Data collection rules](../essentials/data-collection-rule-overview.md) let you [filter and transform log data](../essentials/data-collection-transformations.md) before sending the data to an [Azure table or a custom table](../logs/manage-logs-tables.md#table-type). This article explains how to create custom tables and add custom columns to tables in your Log Analytics workspace. |
| 17 | + |
| 18 | +## Prerequisites |
| 19 | + |
| 20 | +To create a custom table, you need: |
| 21 | + |
| 22 | +- A Log Analytics workspace where you have at least [contributor rights](../logs/manage-access.md#azure-rbac). |
| 23 | +- A [data collection endpoint (DCE)](../essentials/data-collection-endpoint-overview.md). |
| 24 | +- A JSON file with the schema of your custom table in the following format: |
| 25 | + ```json |
| 26 | + [ |
| 27 | + { |
| 28 | + "TimeGenerated": "supported_datetime_format", |
| 29 | + "<column_name_1": "<column_name_1_value>", |
| 30 | + "<column_name_2": "<column_name_2_value>" |
| 31 | + } |
| 32 | + ] |
| 33 | + ``` |
| 34 | + |
| 35 | + For information about the `TimeGenerated` format, see [supported datetime formats](/azure/data-explorer/kusto/query/scalar-data-types/datetime#supported-formats). |
| 36 | +## Create a custom table |
| 37 | + |
| 38 | +Azure tables have predefined schemas. To store log data in a different schema, use data collection rules to define how to collect, transform, and send the data to a custom table in your Log Analytics workspace. |
| 39 | + |
| 40 | +> [!NOTE] |
| 41 | +> For information about creating a custom table for logs you ingest with the deprecated Log Analytics agent, also known as MMA or OMS, see [Collect text logs with the Log Analytics agent](../agents/data-sources-custom-logs.md#define-a-custom-log). |
| 42 | + |
| 43 | +### [Portal](#tab/portal-1) |
| 44 | + |
| 45 | +To create a custom table in the Azure portal: |
| 46 | + |
| 47 | +1. From the **Log Analytics workspaces** menu, select **Tables**. |
| 48 | + |
| 49 | + :::image type="content" source="media/manage-logs-tables/azure-monitor-logs-table-configuration.png" alt-text="Screenshot that shows the Tables screen for a Log Analytics workspace." lightbox="media/manage-logs-tables/azure-monitor-logs-table-configuration.png"::: |
| 50 | + |
| 51 | +1. Select **Create** and then **New custom log (DCR-based)**. |
| 52 | + |
| 53 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/new-custom-log.png" lightbox="media/tutorial-logs-ingestion-portal/new-custom-log.png" alt-text="Screenshot showing new DCR-based custom log."::: |
| 54 | + |
| 55 | +1. Specify a name and, optionally, a description for the table. You don't need to add the *_CL* suffix to the custom table's name - this is added automatically to the name you specify in the portal. |
| 56 | + |
| 57 | +1. Select an existing data collection rule from the **Data collection rule** dropdown, or select **Create a new data collection rule** and specify the **Subscription**, **Resource group**, and **Name** for the new data collection rule. |
| 58 | + |
| 59 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/new-data-collection-rule.png" lightbox="media/tutorial-logs-ingestion-portal/new-data-collection-rule.png" alt-text="Screenshot showing new data collection rule."::: |
| 60 | + |
| 61 | +4. Select a [data collection endpoint](../essentials/data-collection-endpoint-overview.md#create-data-collection-endpoint) and select **Next**. |
| 62 | + |
| 63 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/custom-log-table-name.png" lightbox="media/tutorial-logs-ingestion-portal/custom-log-table-name.png" alt-text="Screenshot showing custom log table name."::: |
| 64 | + |
| 65 | +1. Select **Browse for files** and locate the JSON file in which you defined the schema of your new table. |
| 66 | + |
| 67 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/custom-log-browse-files.png" lightbox="media/tutorial-logs-ingestion-portal/custom-log-browse-files.png" alt-text="Screenshot showing custom log browse for files."::: |
| 68 | + |
| 69 | + All log tables in Azure Monitor Logs must have a `TimeGenerated` column populated with the timestamp of the logged event. |
| 70 | + |
| 71 | +1. If you want to [transform log data before ingestion](../essentials//data-collection-transformations.md) into your table: |
| 72 | + |
| 73 | + 1. Select **Transformation editor**. |
| 74 | + |
| 75 | + The transformation editor lets you create a transformation for the incoming data stream. This is a KQL query that runs against each incoming record. Azure Monitor Logs stores the results of the query in the destination table. |
| 76 | + |
| 77 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/custom-log-data-preview.png" lightbox="media/tutorial-logs-ingestion-portal/custom-log-data-preview.png" alt-text="Screenshot showing custom log data preview."::: |
| 78 | + |
| 79 | + 1. Select **Run** to view the results. |
| 80 | + |
| 81 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/custom-log-query-01.png" lightbox="media/tutorial-logs-ingestion-portal/custom-log-query-01.png" alt-text="Screenshot showing initial custom log data query."::: |
| 82 | + |
| 83 | +1. Select **Apply** to save the transformation and view the schema of the table that's about to be created. Select **Next** to proceed. |
| 84 | + |
| 85 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/custom-log-final-schema.png" lightbox="media/tutorial-logs-ingestion-portal/custom-log-final-schema.png" alt-text="Screenshot showing custom log final schema."::: |
| 86 | + |
| 87 | +1. Verify the final details and select **Create** to save the custom log. |
| 88 | + |
| 89 | + :::image type="content" source="media/tutorial-logs-ingestion-portal/custom-log-create.png" lightbox="media/tutorial-logs-ingestion-portal/custom-log-create.png" alt-text="Screenshot showing custom log create."::: |
| 90 | + |
| 91 | +### [PowerShell](#tab/powershell-1) |
| 92 | + |
| 93 | +Use the [Tables - Update PATCH API](/rest/api/loganalytics/tables/update) to create a custom table with the PowerShell code below. This code creates a table called *MyTable_CL* with two columns. Modify this schema to collect a different table. |
| 94 | + |
| 95 | +> [!IMPORTANT] |
| 96 | +> Custom tables have a suffix of *_CL*; for example, *tablename_CL*. The *tablename_CL* in the DataFlows Streams must match the *tablename_CL* name in the Log Analytics workspace. |
| 97 | + |
| 98 | +1. Select the **Cloud Shell** button in the Azure portal and ensure the environment is set to **PowerShell**. |
| 99 | + |
| 100 | + :::image type="content" source="../logs/media/tutorial-workspace-transformations-api/open-cloud-shell.png" lightbox="../logs/media/tutorial-workspace-transformations-api/open-cloud-shell.png" alt-text="Screenshot of opening Cloud Shell in the Azure portal."::: |
| 101 | + |
| 102 | +2. Copy the following PowerShell code and replace the **Path** parameter with the appropriate values for your workspace in the `Invoke-AzRestMethod` command. Paste it into the Cloud Shell prompt to run it. |
| 103 | + |
| 104 | + ```PowerShell |
| 105 | + $tableParams = @' |
| 106 | + { |
| 107 | + "properties": { |
| 108 | + "schema": { |
| 109 | + "name": "MyTable_CL", |
| 110 | + "columns": [ |
| 111 | + { |
| 112 | + "name": "TimeGenerated", |
| 113 | + "type": "DateTime" |
| 114 | + }, |
| 115 | + { |
| 116 | + "name": "RawData", |
| 117 | + "type": "String" |
| 118 | + } |
| 119 | + ] |
| 120 | + } |
| 121 | + } |
| 122 | + } |
| 123 | + '@ |
| 124 | + |
| 125 | + Invoke-AzRestMethod -Path "/subscriptions/{subscription}/resourcegroups/{resourcegroup}/providers/microsoft.operationalinsights/workspaces/{workspace}/tables/MyTable_CL?api-version=2021-12-01-preview" -Method PUT -payload $tableParams |
| 126 | + ``` |
| 127 | + |
| 128 | +--- |
| 129 | + |
| 130 | +## Delete a table |
| 131 | + |
| 132 | +You can delete any table in your Log Analytics workspace that's not an [Azure table](../logs/manage-logs-tables.md#table-type). |
| 133 | + |
| 134 | +> [!NOTE] |
| 135 | +> Deleting a restored table doesn't delete the data in the source table. |
| 136 | + |
| 137 | +### [Portal](#tab/portal-2) |
| 138 | + |
| 139 | +To delete a table from the Azure portal: |
| 140 | + |
| 141 | +1. From the Log Analytics workspace menu, select **Tables**. |
| 142 | +1. Search for the tables you want to delete by name, or by selecting **Search results** in the **Type** field. |
| 143 | + |
| 144 | + :::image type="content" source="media/search-job/search-results-on-log-analytics-tables-screen.png" alt-text="Screenshot that shows the Tables screen for a Log Analytics workspace with the Filter by name and Type fields highlighted." lightbox="media/search-job/search-results-on-log-analytics-tables-screen.png"::: |
| 145 | + |
| 146 | +1. Select the table you want to delete, select the ellipsis ( **...** ) to the right of the table, select **Delete**, and confirm the deletion by typing **yes**. |
| 147 | + |
| 148 | + :::image type="content" source="media/search-job/delete-table.png" alt-text="Screenshot that shows the Delete Table screen for a table in a Log Analytics workspace." lightbox="media/search-job/delete-table.png"::: |
| 149 | + |
| 150 | +### [API](#tab/api-2) |
| 151 | + |
| 152 | +To delete a table, call the **Tables - Delete** API: |
| 153 | + |
| 154 | +```http |
| 155 | +DELETE https://management.azure.com/subscriptions/{subscriptionId}/resourcegroups/{resourceGroupName}/providers/Microsoft.OperationalInsights/workspaces/{workspaceName}/tables/<TableName>_SRCH?api-version=2021-12-01-preview |
| 156 | +``` |
| 157 | + |
| 158 | +### [CLI](#tab/cli-2) |
| 159 | + |
| 160 | +To delete a table, run the [az monitor log-analytics workspace table delete](/cli/azure/monitor/log-analytics/workspace/table#az-monitor-log-analytics-workspace-table-delete) command. |
| 161 | + |
| 162 | +For example: |
| 163 | + |
| 164 | +```azurecli |
| 165 | +az monitor log-analytics workspace table delete --subscription ContosoSID --resource-group ContosoRG --workspace-name ContosoWorkspace --name HeartbeatByIp_SRCH |
| 166 | +``` |
| 167 | + |
| 168 | +--- |
| 169 | +## Add or delete a custom column |
| 170 | + |
| 171 | +To add a custom column to a table in your Log Analytics workspace, or delete a column: |
| 172 | + |
| 173 | +1. From the **Log Analytics workspaces** menu, select **Tables**. |
| 174 | +1. Select the ellipsis ( **...** ) to the right of the table you want to edit and select **Edit schema**. |
| 175 | + This opens the **Schema Editor** screen. |
| 176 | +1. Scroll down to the **Custom Columns** section of the **Schema Editor** screen. |
| 177 | + |
| 178 | + :::image type="content" source="media/create-custom-table/add-or-delete-column-azure-monitor-logs.png" alt-text="Screenshot showing the Schema Editor screen with the Add a column and Delete buttons highlighted." lightbox="media/create-custom-table/add-or-delete-column-azure-monitor-logs.png"::: |
| 179 | + |
| 180 | +1. To add a new column: |
| 181 | + 1. Select **Add a column**. |
| 182 | + 1. Set the column name and description (optional), and select the expected value type from the **Type** dropdown. |
| 183 | + 1. Select **Save** to save the new column. |
| 184 | +1. To delete a column, select the **Delete** icon to the left of the column you want to delete. |
| 185 | + |
| 186 | +## Next steps |
| 187 | + |
| 188 | +Learn more about: |
| 189 | + |
| 190 | +- [Collecting logs with the Log Ingestion API](../logs/logs-ingestion-api-overview.md) |
| 191 | +- [Collecting logs with Azure Monitor Agent](../agents/agents-overview.md) |
| 192 | +- [Data collection rules](../essentials/data-collection-endpoint-overview.md) |
0 commit comments