Skip to content

Commit 47a2852

Browse files
committed
Added @UsualStatisticsSamplingPercent parameter and comparison to match. Also replaced version check with a column-existence check. Both of these were as Brent suggested.
This introduces the complexity of float comparison, since we now care about the float persisted_sample_percent column and have to compare with it. Also added a new check to distinguish the persisted sample rate being surprising and it being unsurprising.
1 parent 07c3fff commit 47a2852

File tree

3 files changed

+64
-16
lines changed

3 files changed

+64
-16
lines changed

Documentation/sp_BlitzIndex_Checks_by_Priority.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@ Before adding a new check, make sure to add a Github issue for it first, and hav
66

77
If you want to change anything about a check - the priority, finding, URL, or ID - open a Github issue first. The relevant scripts have to be updated too.
88

9-
CURRENT HIGH CHECKID: 125
10-
If you want to add a new check, start at 126.
9+
CURRENT HIGH CHECKID: 126
10+
If you want to add a new check, start at 127.
1111

1212
| Priority | FindingsGroup | Finding | URL | CheckID |
1313
| -------- | ----------------------- | --------------------------------------------------------------- | ---------------------------------------------------------------- | ------- |
@@ -25,7 +25,7 @@ If you want to add a new check, start at 126.
2525
| 80 | Abnormal Design Pattern | Filter Columns Not In Index Definition | https://www.brentozar.com/go/IndexFeatures | 34 |
2626
| 80 | Abnormal Design Pattern | History Table With NonClustered Index | https://sqlserverfast.com/blog/hugo/2023/09/an-update-on-merge/ | 124 |
2727
| 90 | Statistics Warnings | Low Sampling Rates | https://www.brentozar.com/go/stats | 91 |
28-
| 90 | Statistics Warnings | Persisted Sampling Rates | https://www.youtube.com/watch?v=V5illj_KOJg&t=758s | 125 |
28+
| 90 | Statistics Warnings | Persisted Sampling Rates (Unexpected) | `https://www.youtube.com/watch?v=V5illj_KOJg&t=758s` | 125 |
2929
| 90 | Statistics Warnings | Statistics Not Updated Recently | https://www.brentozar.com/go/stats | 90 |
3030
| 90 | Statistics Warnings | Statistics with NO RECOMPUTE | https://www.brentozar.com/go/stats | 92 |
3131
| 100 | Over-Indexing | NC index with High Writes:Reads | https://www.brentozar.com/go/IndexHoarder | 48 |
@@ -62,6 +62,7 @@ If you want to add a new check, start at 126.
6262
| 200 | Abnormal Design Pattern | Temporal Tables | https://www.brentozar.com/go/AbnormalPsychology | 110 |
6363
| 200 | Repeated Calculations | Computed Columns Not Persisted | https://www.brentozar.com/go/serialudf | 100 |
6464
| 200 | Statistics Warnings | Statistics With Filters | https://www.brentozar.com/go/stats | 93 |
65+
| 200 | Statistics Warnings | Persisted Sampling Rates (Expected) | `https://www.youtube.com/watch?v=V5illj_KOJg&t=758s` | 126 |
6566
| 200 | Over-Indexing | High Ratio of Nulls | https://www.brentozar.com/go/IndexHoarder | 25 |
6667
| 200 | Over-Indexing | High Ratio of Strings | https://www.brentozar.com/go/IndexHoarder | 27 |
6768
| 200 | Over-Indexing | Wide Tables: 35+ cols or > 2000 non-LOB bytes | https://www.brentozar.com/go/IndexHoarder | 26 |

README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,6 +288,7 @@ In addition to the [parameters common to many of the stored procedures](#paramet
288288

289289
* @SkipPartitions = 1 - add this if you want to analyze large partitioned tables. We skip these by default for performance reasons.
290290
* @SkipStatistics = 0 - right now, by default, we skip statistics analysis because we've had some performance issues on this.
291+
* @UsualStatisticsSamplingPercent = 100 (default) - By default, @SkipStatistics = 0 with either @Mode = 0 or @Mode = 4 does not inform you of persisted statistics sample rates if that rate is 100. Use a different float if you usually persist a different sample percentage and do not want to be warned about it. Use NULL if you want to hear about every persisted sample rate.
291292
* @Filter = 0 (default) - 1=No low-usage warnings for objects with 0 reads. 2=Only warn for objects >= 500MB
292293
* @OutputDatabaseName, @OutputSchemaName, @OutputTableName - these only work for @Mode = 2, index usage detail.
293294

sp_BlitzIndex.sql

Lines changed: 59 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,7 @@ ALTER PROCEDURE dbo.sp_BlitzIndex
2323
/*Note:@Filter doesn't do anything unless @Mode=0*/
2424
@SkipPartitions BIT = 0,
2525
@SkipStatistics BIT = 1,
26+
@UsualStatisticsSamplingPercent FLOAT = 100, /* FLOAT to match sys.dm_db_stats_properties. More detail later. 100 by default because Brent suggests that if people are persisting statistics at all, they are probably doing 100 in lots of places and not filtering that out would produce noise. */
2627
@GetAllDatabases BIT = 0,
2728
@ShowColumnstoreOnly BIT = 0, /* Will show only the Row Group and Segment details for a table with a columnstore index. */
2829
@BringThePain BIT = 0,
@@ -739,7 +740,7 @@ IF OBJECT_ID('tempdb..#dm_db_index_operational_stats') IS NOT NULL
739740
no_recompute BIT NULL,
740741
has_filter BIT NULL,
741742
filter_definition NVARCHAR(MAX) NULL,
742-
has_persisted_sample BIT NULL
743+
persisted_sample_percent FLOAT NULL
743744
);
744745

745746
CREATE TABLE #ComputedColumns
@@ -2280,7 +2281,7 @@ OPTION (RECOMPILE);';
22802281
INSERT #Statistics ( database_id, database_name, table_name, schema_name, index_name, column_names, statistics_name, last_statistics_update,
22812282
days_since_last_stats_update, rows, rows_sampled, percent_sampled, histogram_steps, modification_counter,
22822283
percent_modifications, modifications_before_auto_update, index_type_desc, table_create_date, table_modify_date,
2283-
no_recompute, has_filter, filter_definition, has_persisted_sample)
2284+
no_recompute, has_filter, filter_definition, persisted_sample_percent)
22842285
SELECT DB_ID(N' + QUOTENAME(@DatabaseName,'''') + N') AS [database_id],
22852286
@i_DatabaseName AS database_name,
22862287
obj.name AS table_name,
@@ -2309,9 +2310,15 @@ OPTION (RECOMPILE);';
23092310
s.has_filter,
23102311
s.filter_definition,
23112312
'
2312-
+ CASE WHEN (PARSENAME(@SQLServerProductVersion, 4) >= 15)
2313-
THEN N's.has_persisted_sample'
2314-
ELSE N'NULL AS has_persisted_sample' END
2313+
+ CASE WHEN EXISTS
2314+
(
2315+
/* We cannot trust checking version numbers, like we did above, because Azure disagrees. */
2316+
SELECT 1
2317+
FROM sys.all_columns AS all_cols
2318+
WHERE all_cols.[object_id] = OBJECT_ID(N'sys.dm_db_stats_properties', N'IF') AND all_cols.[name] = N'persisted_sample_percent'
2319+
)
2320+
THEN N'ddsp.persisted_sample_percent'
2321+
ELSE N'NULL AS persisted_sample_percent' END
23152322
+ N'
23162323
FROM ' + QUOTENAME(@DatabaseName) + N'.sys.stats AS s
23172324
JOIN ' + QUOTENAME(@DatabaseName) + N'.sys.objects obj
@@ -2360,7 +2367,7 @@ OPTION (RECOMPILE);';
23602367
INSERT #Statistics(database_id, database_name, table_name, schema_name, index_name, column_names, statistics_name,
23612368
last_statistics_update, days_since_last_stats_update, rows, modification_counter,
23622369
percent_modifications, modifications_before_auto_update, index_type_desc, table_create_date, table_modify_date,
2363-
no_recompute, has_filter, filter_definition, has_persisted_sample)
2370+
no_recompute, has_filter, filter_definition, persisted_sample_percent)
23642371
SELECT DB_ID(N' + QUOTENAME(@DatabaseName,'''') + N') AS [database_id],
23652372
@i_DatabaseName AS database_name,
23662373
obj.name AS table_name,
@@ -2388,8 +2395,8 @@ OPTION (RECOMPILE);';
23882395
s.filter_definition,'
23892396
ELSE N'NULL AS has_filter,
23902397
NULL AS filter_definition,' END
2391-
/* If we are on this branch, then we cannot have the has_persisted_sample column (it is a 2019+ column). */
2392-
+ N'NULL AS has_persisted_sample'
2398+
/* Certainly NULL. This branch does not even join on the table that this column comes from. */
2399+
+ N'NULL AS persisted_sample_percent'
23932400
+ N'
23942401
FROM ' + QUOTENAME(@DatabaseName) + N'.sys.stats AS s
23952402
INNER HASH JOIN ' + QUOTENAME(@DatabaseName) + N'.sys.sysindexes si
@@ -4462,7 +4469,7 @@ BEGIN
44624469
END;
44634470

44644471
----------------------------------------
4465-
--Statistics Info: Check_id 90-99, as well as 125
4472+
--Statistics Info: Check_id 90-99, as well as 125-126
44664473
----------------------------------------
44674474

44684475
RAISERROR(N'check_id 90: Outdated statistics', 0,1) WITH NOWAIT;
@@ -4511,22 +4518,61 @@ BEGIN
45114518
OR (s.rows > 1000000 AND s.percent_sampled < 1)
45124519
OPTION ( RECOMPILE );
45134520

4514-
RAISERROR(N'check_id 125: Statistics with a persisted sample rate', 0,1) WITH NOWAIT;
4521+
RAISERROR(N'check_id 125: Persisted Sampling Rates (Unexpected)', 0,1) WITH NOWAIT;
45154522
INSERT #BlitzIndexResults ( check_id, Priority, findings_group, finding, [database_name], URL, details, index_definition,
45164523
secret_columns, index_usage_summary, index_size_summary )
45174524
SELECT 125 AS check_id,
45184525
90 AS Priority,
45194526
'Statistics Warnings' AS findings_group,
4520-
'Persisted Sampling Rates',
4527+
'Persisted Sampling Rates (Unexpected)',
45214528
s.database_name,
45224529
'https://www.youtube.com/watch?v=V5illj_KOJg&t=758s' AS URL,
4523-
'The statistics sample rate/amount has been persisted here. ' + CONVERT(NVARCHAR(100), s.percent_sampled) + '% of the rows were sampled during the last statistics update. This may indicate that somebody is doing statistics rocket surgery. Perhaps you would be better off updating statistics more frequently?' AS details,
4530+
'The persisted statistics sample rate is ' + CONVERT(NVARCHAR(100), s.persisted_sample_percent) + '%'
4531+
+ CASE WHEN @UsualStatisticsSamplingPercent IS NOT NULL
4532+
THEN (N' rather than your expected @UsualStatisticsSamplingPercent value of ' + CONVERT(NVARCHAR(100), @UsualStatisticsSamplingPercent) + '%')
4533+
ELSE ''
4534+
END
4535+
+ N'. This may indicate that somebody is doing statistics rocket surgery. If not, consider updating statistics more frequently.' AS details,
45244536
QUOTENAME(database_name) + '.' + QUOTENAME(s.schema_name) + '.' + QUOTENAME(s.table_name) + '.' + QUOTENAME(s.index_name) + '.' + QUOTENAME(s.statistics_name) + '.' + QUOTENAME(s.column_names) AS index_definition,
45254537
'N/A' AS secret_columns,
45264538
'N/A' AS index_usage_summary,
45274539
'N/A' AS index_size_summary
45284540
FROM #Statistics AS s
4529-
WHERE s.has_persisted_sample = 1
4541+
/*
4542+
We have to do float comparison here, so it is time to explain why @UsualStatisticsSamplingPercent is a float.
4543+
The foremost reason is that it is a float because we are comparing it to the persisted_sample_percent column in sys.dm_db_stats_properties and that column is a float.
4544+
You may correctly object that CREATE STATISTICS with a decimal as your WITH SAMPLE [...] PERCENT is a syntax error and conclude that integers are enough.
4545+
However, `WITH SAMPLE [...] ROWS` is allowed with PERSIST_SAMPLE_PERCENT = ON and you can use that to persist a non-integer sample rate.
4546+
So, yes, we really have to use floats.
4547+
*/
4548+
WHERE
4549+
/* persisted_sample_percent is either zero or NULL when the statistic is not persisted. */
4550+
s.persisted_sample_percent > 0.0001
4551+
AND
4552+
(
4553+
ABS(@UsualStatisticsSamplingPercent - s.persisted_sample_percent) > 0.1
4554+
OR @UsualStatisticsSamplingPercent IS NULL
4555+
)
4556+
OPTION ( RECOMPILE );
4557+
4558+
RAISERROR(N'check_id 126: Persisted Sampling Rates (Expected)', 0,1) WITH NOWAIT;
4559+
INSERT #BlitzIndexResults ( check_id, Priority, findings_group, finding, [database_name], URL, details, index_definition,
4560+
secret_columns, index_usage_summary, index_size_summary )
4561+
SELECT 126 AS check_id,
4562+
200 AS Priority,
4563+
'Statistics Warnings' AS findings_group,
4564+
'Persisted Sampling Rates (Expected)',
4565+
s.database_name,
4566+
'https://www.youtube.com/watch?v=V5illj_KOJg&t=758s' AS URL,
4567+
CONVERT(NVARCHAR(100), COUNT(*)) + ' statistic(s) with a persisted sample rate matching your desired persisted sample rate, ' + CONVERT(NVARCHAR(100), @UsualStatisticsSamplingPercent) + N'%. Set @UsualStatisticsSamplingPercent to NULL if you want to see all of them in this result set. Its default value is 100.' AS details,
4568+
s.database_name + N' (Entire database)' AS index_definition,
4569+
'N/A' AS secret_columns,
4570+
'N/A' AS index_usage_summary,
4571+
'N/A' AS index_size_summary
4572+
FROM #Statistics AS s
4573+
WHERE ABS(@UsualStatisticsSamplingPercent - s.persisted_sample_percent) <= 0.1
4574+
AND @UsualStatisticsSamplingPercent IS NOT NULL
4575+
GROUP BY s.database_name
45304576
OPTION ( RECOMPILE );
45314577

45324578
RAISERROR(N'check_id 92: Statistics with NO RECOMPUTE', 0,1) WITH NOWAIT;

0 commit comments

Comments
 (0)