Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
c7dd262
Move doc to Kusto docset, Updates to text
AndyKop Nov 4, 2024
b0ca6c6
Moniker pivots for Fabric exclusions
AndyKop Nov 5, 2024
dddfb32
Links updated
AndyKop Nov 5, 2024
d5aaf39
Monikers etc
AndyKop Nov 6, 2024
9569fe2
Fix example
AndyKop Nov 6, 2024
e43a072
added comment
AndyKop Nov 7, 2024
08a24a4
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
AndyKop Nov 7, 2024
75f1cf5
Mokiners for Fabric applicability
AndyKop Nov 7, 2024
b83c53d
Update TOC and redirect
AndyKop Nov 7, 2024
9d1be67
Tweaks
AndyKop Nov 7, 2024
2d04a13
Date
AndyKop Nov 7, 2024
6022b81
Acrolinx fixes
AndyKop Nov 7, 2024
29a1884
Build warning fixes
AndyKop Nov 7, 2024
1216da7
Build fixes
AndyKop Nov 7, 2024
f4f7653
Fixes to moniker
AndyKop Nov 7, 2024
b24f6ae
Build fix
AndyKop Nov 7, 2024
cc6e508
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
AndyKop Nov 10, 2024
4908be5
Build fixes, redirects fixed
AndyKop Nov 10, 2024
e8d17b4
Redirect fixes
AndyKop Nov 10, 2024
262dcaf
Review tweaks
AndyKop Nov 10, 2024
1fc8658
Text tweak
AndyKop Nov 10, 2024
d0661d9
Moniker fixes
AndyKop Nov 10, 2024
fdba0ee
Moniker fixes
AndyKop Nov 10, 2024
2b7d750
Bookmarks
AndyKop Nov 10, 2024
6365a6a
ToC updates
AndyKop Nov 10, 2024
2bcb19a
migrate debug python
mjosephym Nov 13, 2024
c659c79
images, edit
mjosephym Nov 13, 2024
5875200
edit
mjosephym Nov 13, 2024
ee88d54
edit
mjosephym Nov 13, 2024
b9f4bfe
edit
mjosephym Nov 13, 2024
4b5cb17
edit
mjosephym Nov 13, 2024
e0cde29
edit
mjosephym Nov 13, 2024
76546af
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
mjosephym Nov 14, 2024
06f32fa
edits per recommendations
mjosephym Nov 14, 2024
5ff5886
edits per recommendations
mjosephym Nov 14, 2024
52b90e3
Merge branch 'main' into wi-285701-kusto-deleta-data
AndyKop Nov 18, 2024
d4915a7
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
AndyKop Nov 18, 2024
6613c6a
Merge branch 'wi-285701-kusto-deleta-data' of https://github.com/Andy…
AndyKop Nov 18, 2024
c7c5037
build fix
AndyKop Nov 18, 2024
2f36420
Merge branch 'main' of https://github.com/MicrosoftDocs/dataexplorer-…
AndyKop Nov 19, 2024
dc8683c
added in deeplink examples and monikler pivots
AndyKop Nov 19, 2024
a9b3394
update deeplink
AndyKop Nov 19, 2024
62853e6
Merge pull request #6351 from AndyKop/wi-255766-pase-json-fix-example
PMEds28 Nov 20, 2024
8e64715
Merge pull request #6376 from mjosephym/343071-kql-migrate-debug-python
ttorble Nov 20, 2024
b9eda1b
Merge pull request #6344 from AndyKop/wi-285701-kusto-deleta-data
ttorble Nov 20, 2024
0eb3efc
Merge pull request #6395 from MicrosoftDocs/main
PMEds28 Nov 20, 2024
6203dfb
Merging changes synced from https://github.com/MicrosoftDocs/dataexpl…
Nov 20, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions data-explorer/.openpublishing.redirection.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,11 @@
"redirect_url": "/sql/azure-data-studio/notebooks/notebooks-kqlmagic?context=/azure/data-explorer/context/context",
"redirect_document_id": false
},
{
"source_path": "delete-data.md",
"redirect_url": "/kusto/concepts/delete-data?view=azure-data-explorer&preserve-view=true",
"redirect_document_id": false
},
{
"source_path": "connect-from-databricks.md",
"redirect_url": "/azure/data-explorer/spark-connector",
Expand Down Expand Up @@ -425,6 +430,11 @@
"redirect_url": "/kusto/set-timeout-limits?view=azure-data-explorer&preserve-view=true",
"redirect_document_id": false
},
{
"source_path": "debug-inline-python.md",
"redirect_url": "/kusto/debug-inline-python",
"redirect_document_id": false
},
{
"source_path": "excel-connector.md",
"redirect_url": "/azure/data-explorer/excel",
Expand Down
95 changes: 0 additions & 95 deletions data-explorer/debug-inline-python.md

This file was deleted.

79 changes: 0 additions & 79 deletions data-explorer/delete-data.md

This file was deleted.

4 changes: 2 additions & 2 deletions data-explorer/kusto/.openpublishing.redirection.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@
"redirect_document_id": false
},
{
"source_path": "concepts/delete-data.md",
"redirect_url": "/azure/data-explorer/delete-data",
"source_path": "data-explorer/delete-data.md",
"redirect_url": "/kusto/concepts/delete-data",
"redirect_document_id": false
},
{
Expand Down
4 changes: 2 additions & 2 deletions data-explorer/kusto/concepts/data-soft-delete.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,11 @@ description: This article describes data soft delete.
ms.reviewer: slneimer
ms.topic: reference
ms.date: 08/11/2024
monikerRange: "azure-data-explorer"
monikerRange: "azure-data-explorer || microsoft-fabric"
---
# Soft delete overview

> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)]
> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)]

The ability to delete individual records is supported. Record deletion is commonly achieved using one of the following methods:

Expand Down
95 changes: 95 additions & 0 deletions data-explorer/kusto/concepts/delete-data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
---
title: Delete data
description: This article describes delete scenarios, including purge, dropping extents and retention based deletes.
ms.reviewer: avneraa
ms.topic: how-to
ms.date: 10/11/2024
---
# Delete data

> [!INCLUDE [applies](../includes/applies-to-version/applies.md)] [!INCLUDE [fabric](../includes/applies-to-version/fabric.md)] [!INCLUDE [azure-data-explorer](../includes/applies-to-version/azure-data-explorer.md)]

Delete data from a table is supported in several ways. Use the following information to help you choose which deletion method is best for your use case.

:::moniker range="azure-data-explorer"
| Use case | Considerations | Method |
|--|--|--|
| Delete all data from a table. | | [Use the `.clear table data` command](#delete-all-data-in-a-table) |
| Routinely delete old data. | Use if you need an automated deletion solution. | [Use a retention policy](#delete-data-using-a-retention-policy) |
| Bulk delete specific data by extents. | Only use if you're an expert user. | [Use the `.drop extents` command](#delete-data-by-dropping-extents) |
| Delete records based on their content. | - Storage artifacts that contain the deleted records aren't necessarily deleted. <br /> - Deleted records can't be recovered (regardless of any retention or recoverability settings). <br />- Use if you need a quick way to delete records. | [Use soft delete](#soft-delete)|
| Delete records based on their content. | - Storage artifacts that contain the deleted records are deleted. <br /> - Deleted records can't be recovered (regardless of any retention or recoverability settings). <br />- Requires significant system resources and time to complete. | [Use purge](#purge) |
:::moniker-end

:::moniker range="microsoft-fabric"
| Use case | Considerations | Method |
|--|--|--|
| Delete all data from a table. | | [Use the `.clear table data` command](#delete-all-data-in-a-table) |
| Routinely delete old data. | Use if you need an automated deletion solution. | [Use a retention policy](#delete-data-using-a-retention-policy) |
| Bulk delete specific data by extents. | Only use if you're an expert user. | [Use the `.drop extents` command](#delete-data-by-dropping-extents) |
| Delete records based on their content. | - Storage artifacts that contain the deleted records aren't necessarily deleted.<br /> - Deleted records can't be recovered (regardless of any retention or recoverability settings).<br />- Use if you need a quick way to delete records. | [Use soft delete](#delete-individual-rows) |
:::moniker-end

The following sections describe the different deletion methods.

## Delete all data in a table

To delete all data in a table, use the [.clear table data](../management/clear-table-data-command.md) command. This command is the most efficient way to remove all data from a table.

Syntax:

```kusto
.clear table <TableName> data
```

## Delete data using a retention policy

Automatically delete data based on a [retention policy](../management/retention-policy.md). You can set the retention policy at the database or table level. There's no guarantee as to when the deletion occurs, but it will not be deleted before the retention period. This is an efficient and convenient way to remove old data.

Consider a database or table that is set for 90 days of retention. If only 60 days of data are needed, delete the older data as follows:

```kusto
.alter-merge database <DatabaseName> policy retention softdelete = 60d

.alter-merge table <TableName> policy retention softdelete = 60d
```

## Delete data by dropping extents

[Extent (data shard)](../management/extents-overview.md) is the internal structure where data is stored. Each extent can hold up to millions of records. Extents can be deleted individually or as a group using [drop extent(s) commands](../management/drop-extents.md).

### Examples

You can delete all rows in a table or just a specific extent.

- Delete all rows in a table:

```kusto
.drop extents from TestTable
```

- Delete a specific extent:

```kusto
.drop extent e9fac0d2-b6d5-4ce3-bdb4-dea052d13b42
```

## Delete individual rows

:::moniker range="azure-data-explorer"
Both purge and soft delete can be used for deleting individual rows. Soft delete doesn't necessarily delete the storage artifacts that contain records to delete, and purge does delete all such storage artifacts.

Both methods prevent deleted records from being recovered, regardless of any retention or recoverability settings. The deletion process is final and irreversible.

### Soft delete

With [soft delete](data-soft-delete.md), data isn't necessarily deleted from storage artifacts. This method marks all matching records as deleted, so that they'll be filtered out in queries, and doesn't require significant system resources.

### Purge

With [purge](data-purge.md), extents that have one or more records to be deleted, are replaced with new extents in which those records don't exist. This deletion process isn't immediate, requires significant system resources, and can take a whole day to complete.
::: moniker-end

:::moniker range="microsoft-fabric"
[Soft delete](data-soft-delete.md) can be used for deleting individual rows. Data isn't necessarily deleted from storage artifacts. Soft delete prevent deleted records from being recovered, regardless of any retention or recoverability settings. The deletion process is final and irreversible. This method marks all matching records as deleted, so that they'll be filtered out in queries, and doesn't require significant system resources.
::: moniker-end
Loading