You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: articles/hdinsight/hbase/troubleshoot-data-retention-issues-expired-data.md
+30-30Lines changed: 30 additions & 30 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -18,52 +18,52 @@ To prepare to follow the steps and commands below, open two ssh connections to H
18
18
* In one of the ssh sessions keep the default bash shell.
19
19
* In the second ssh session launch HBase shell by running the command below.
20
20
21
-
```
22
-
hbase shell
23
-
```
21
+
```
22
+
hbase shell
23
+
```
24
24
25
25
### Check if desired TTL is configured and if expired data is removed from query result
26
26
27
27
Follow the steps below to understand where is the issue. Start by checking if the behavior occurs for a specific table or for all the tables. If you're unsure whether the issue impacts all the tables or a specific table, just consider as example a specific table name for the start.
28
28
29
29
1. Check first that TTL has been configured for ColumnFamily for the target tables. Run the command below in the ssh session where you launched HBase shell and observe example and output below. One column family has TTL set to 50 seconds, the other ColumnFamily has no value configured for TTL, thus it appears as "FOREVER" (data in this column family isn't configured to expire).
30
-
```
31
-
describe 'table_name'
32
-
```
30
+
```
31
+
describe 'table_name'
32
+
```
33
33
34
34
1. If not configured, default TTL is set to 'FOREVER'. There are two possibilities why data is not expired as expected and removed from query result.
35
35
1. If TTL has any other value then 'FOREVER', observe the value for column family and note down the value in seconds(pay special attention to value correlated with the unit measure as cell TTL is in ms, but column family TTL is in seconds) to confirm if it is the expected one. If the observed value isn't correct, fix that first.
36
36
1. If TTL value is 'FOREVER' for all column families, configure TTL as first step and afterwards monitor if data is expired as expected.
37
37
1. If you establish that TTL is configured and has the correct value for the ColumnFamily, next step is to confirm that the expired data no longer shows up when doing table scans. When data expires, it should be removed and not show up in the scan table results. Run the below command in HBase shell to check.
38
-
```
39
-
scan 'table_name'
40
-
```
38
+
```
39
+
scan 'table_name'
40
+
```
41
41
### Check the number and size of StoreFiles per table per region to observe if any changes are visible after the compaction operation
42
42
43
43
1. Before moving to next step, from ssh session with bash shell, run the following command to check the current number of StoreFiles and size for each StoreFile currently showing up for the ColumnFamily for which the TTL has been configured. Note first the table and ColumnFamily for which you'll be doing the check, then run the following command in ssh session (bash).
1. Likely, there will be more results shown in the output, one result for each region ID that is part of the table and between 0 and more results for StoreFiles present under each region name, for the selected ColumnFamily. To count the overall number of rows in the result output above, run the following command.
### Check the number and size of StoreFiles per table per region after flush
55
55
56
56
1. Based on the TTL configured for each ColumnFamily and how much data is written in the table for the target ColumnFamily, part of the data may still exist in MemStore and isn't written as StoreFile to storage. Thus, to make sure that the data is written to storage as StoreFile, before the maximum configured MemStore size is reached, you can run the following command in HBase shell to write data from MemStore to StoreFile immediately.
57
57
58
-
```
59
-
flush 'table_name'
60
-
```
58
+
```
59
+
flush 'table_name'
60
+
```
61
61
62
62
1. Observe the result by running again in bash shell the command.
1. An additional store file is created compared to previous result output for each region where data is modified, the StoreFile will include current content of MemStore for that region.
1. To make sure expired data is also deleted from storage, we need to run a major compaction operation. The major compaction operation, when completed, will leave behind a single StoreFile per region. In HBase shell, run the command to execute a major compaction operation on the table:
77
77
78
-
```
79
-
major_compact 'table_name'
80
-
```
78
+
```
79
+
major_compact 'table_name'
80
+
```
81
81
82
82
1. Depending on the table size, major compaction operation can take some time. Use the command below in HBase shell to monitor progress. If the compaction is still running when you execute the command below, you'll see the output "MAJOR", but if the compaction is completed, you will see the output "NONE".
83
83
84
-
```
85
-
compaction_state 'table_name'
86
-
```
84
+
```
85
+
compaction_state 'table_name'
86
+
```
87
87
88
88
1. When the compaction status appears as "NONE" in hbase shell, if you switch quickly to bash and run command
You will notice that an extra StoreFile has been created in addition to previous ones per region per ColumnFamily and after several moments only the last created StoreFile is kept per region per column family.
94
94
95
95
1. For the example region above, once the extra moments elapse, we can notice that one single StoreFile remained and the size occupied by this file on the storage is reduced as major compaction occurred and at this point any expired data that has not been deleted before(by another major compaction), will be deleted after running current major compaction operation.
0 commit comments